It seems that the following code does not delete the record in the .dbf file. Instead it is only marked for deletion:
await shapefileFeatureTable.DeleteFeatureAsync(feature);
shapefileFeatureTable.Close();
If you load the shapefile in ArcMap afterwards you'll get this warning message:
There was a problem drawing the shapefile. Drawing aborted.
If you look into the table you will still see the feature which was deleted in the ArcGIS Runtime app (there it isn't displayed anymore). I think what is missing is a kind of pack function call for the .dbf file during the closing of the ShapefileFeatureTable.
I found this behaviour in Version 100.9 (WPF) but I think it might have existed in the versions before. I also tested it in Version 100.11 (WPF) with the same result.