What about indexes, is the join field indexed in both data sets?
Even without indexes, I still can't get my head around why it would take some long. I have worked with some million-record or more data sets. Sure, they are slower than what most people are used to, but 6 hours to export? That just doesn't make sense to me.