I did some testing and I have found the reason why these layers are not deleting.
Now I have no idea how I was able to do this, but the drawing that has the web layers present does not have a coordinate system assigned. Once I set a coordinate system and then remove and save I end up getting the correct behaviour.
Typically, AFA does not allow you to add web layers when a coordinate system does not exist.
![DerekBannon22_0-1678456445912.png DerekBannon22_0-1678456445912.png](https://community.esri.com/t5/image/serverpage/image-id/64895iF99F78699E4F1CAF/image-dimensions/762x198?v=v2)