Hi Travis,
It's hard to give specific advice without seeing your site. But my experience in doing something similar (I reproduced an ArcGIS JS API site in OpenLayers) was to isolate the parts of my code which were specific to ArcGIS, and replicate them in OpenLayers.
Like you I had a lot of non-ArcGIS code which I could re-use - the HTML and CSS were unchanged, and other functions like changing basemaps had an equivalent method in OpenLayers.
Once you're familiar with how the ArcGIS JS API works (by playing with the samples), you should have a better feel for the changes you'll need to make.
Eg, if your current site adds a polygon overlay to Google Maps, figure out how to do the same thing in ArcGIS. Same with changing basemaps, zooming to a bookmark, adding a KML layer, etc.
Cheers,
Steve