I am also having this problem. My project's only ArcGIS-related reference is Esri.FileGDBAPI. Project is compiled for x86 under VS2010, targeting .NET 4. Using the 32-bit assemblies. Folder contains my EXE, Esri.FileGDBAPI.dll, and FileGDBAPI.dll. Using v1.2 of the API. Running in Windows 7 / 32-bit from the folder C:\Temp\Test by double-clicking the EXE from Windows Explorer. Also tried running from C:\Test and C:\Program Files\Test, as a normal user and "Run as administrator." In all cases, I get the message:
System.IO.FileNotFoundException: Could not load file or assembly 'Esri.FileGDBAPI.dll' or one of its dependencies. The specified module could not be found.
Any ideas?