Thank you for the repro case.
What you are seeing is not at all a bug. It is the expected behavior when you perform a large number of updates to your data. Additionally, the same thing happens in ArcGIS.
The reason that the files become very large when you do a great deal of updates is that the update process often creates internal fragmentation in the files. Usually, this happens when an updated record requires more storage than it did before. When this occurs, the record must be written out to a new location in the file, and the previous location is marked as deleted or free space within the file. All of these deleted records are kept track of in the "freelist" file.
There are a couple of ways that the free space can be recovered. You discovered one of them, which is deleting a field. When a field is deleted, the entire file is rewritten one record at a time, but the deleted records are skipped. The better way to recover free space is to run the Compact Database command in ArcGIS. You can do that from the database property page, or from a GP tool.
In the FileGDB API, we did not expose CompactDatabase functions in the original design. However, we are in the process of creating an updated release with many enhancements and bug fixes. We will add CompactDatabase functions as part of this new release. Our plan is to have the new release ready around the time of the Developer Summit in March.