I had a similar problem trying to delete files even after clearing the operational layers. Looking through the documentation, I found a note on GeoPackage.Close() saying that only after closing the geopackage can an underlying file be deleted/renamed etc. Adding a call to this got my system working. Hope this helps.
... View more