I'm totally late for you, but maybe this can help someone else. I don't know why you have to set up the DLL this way, but:
You need to go into the bin folder of the FileGDB files you downloaded. Copy the 3 DLLs and paste them into the bin folder of your project in the solution explorer. If the bin folder in your project is not showing, click the "Show All Files" button at the top of the solution explorer. Then, expand the Debug folder under there and add those 3 DLLs if they are not there AND the Esri.FileGDBAPI.pdb from the FileGDB files bin folder you downloaded. Make sure all those files are next to each other in the debug folder. It should then load the files correctly.