Ok I found out. To delete a map tile package, you now have to go to the "Settings" page of the the package and hit that delete button. You get the following error, but I don't think there's anything to be scared of, as I've never kept the packages in AGOL before once they were published. I keep them on my computer and save the 1.2 credit/gb.
For "Map tile packages" delete, the difference is:
- If you delete a tile package (.tpk), the hosted tile layer published from it can no longer be taken offline.