Just a suggestion, but have you tried an old fashioned image catalog? Image tiles are accessed as needed based on extent, I think the default is 9 tiles at a time but you can override that with layer properties...
It's a table with the fields:
IMAGE, XMIN, YMIN, XMAX, YMAX
IMAGE is the pathname, the rest is self-explanatory. I usually use this either for archived photos or as a quick temporary arrangement displaying new photos I haven't put into permanent storage or processed a mosaic for yet. It's fast and easy to generate with a script.