With the information provided, it is difficult to comment on whether changing your workflow to pandas would be more efficient. Are the feature classes in an enterprise geodatabase, file geodatabase, or something else? If file or mobile geodatabase, are they stored locally or on network share? Could you copy the feature classes into memory before processing? Are the fields you are querying indexed appropriately? There are likely more questions I could come up with if I spent a bit more time thinking about it.
Exporting a dataset into memory via pandas could definitely be faster, but not necessarily because there are lots of ways of processing data in pandas and some are faster than others. If the dataset is very large and it won't fit in memory, then having the Python process start paging will cause performance hit regardless of specific pandas workflow.
Given there are so many factors involved, the best approach is to do some testing oneself. Is it worth taking the time to test? Probably if you are here asking the question(s).