You didn't specify whether you're using ArcMap or ArcGIS Pro. ArcMap is a 32-bit application, and consequently, you are much more likely to run into memory problems. If you're using ArcMap, please switch to ArcGIS Pro and try again. You may not encounter the same problem.
In general, the speed and memory requirements for Service Area depend on the number and size of the cutoffs as well as whether you're generating Generalized, Standard, or High Precision polygons. High Precision polygons are more accurate but take considerably longer to generate, so you'll have to weigh your performance requirements against your need for accuracy.
If you can write Python, the scripts and DevSummit presentation Dan pointed to are a great example of how to chunk and parallelize a large network analysis solve. The example there uses OD Cost Matrix, but the same principles apply for Service Area. However, you would have to make substantial modifications to the script to convert it to work for Service Area.
Another possibility is to use the Create Mobile Map Package tool to convert your network dataset to a transportation network in a sqlite geodatabase. Solves against this type of network dataset have some performance enhancements that may help. However, the network dataset is not editable once it's in this format. You can make edits to the original file geodatabase and then repackage it again.
