I'm using a layer with both from- and to-time fields. The timeFilterMode option for FeatureLayer seems to imply that there's an option with time-based layers to fetch all features up front, rather than by the incremental server-based approach:
Determines where features are filtered by time. By default features will be filtered by the server. If set to
'client' all features are loaded and filtered on the client before display.