The checkout replica is created from an mxd with all of the data to be "checked-out" sourced to SDE. I'm guessing the mxd you're using right now is sourced to the file geodatabase.
You need to get the fgdb data into SDE first, then create your check-out replica. Consider the Extract Data tool for this: ArcGIS Desktop
You're in a little bit of a pickle. The immediate problem will be best solved with either Extract Data, or replacing the SDE feature classes with the fgdb feature classes, or using python to delete the existing SDE records and replace them with the fgdb feature class records. None are pretty, each have good and bad sides, but checkouts will help you after this step is done.
