Ali:
I have not seen that specific issue.
I have seen, and others have reported, ghost services left in Portal after they have been deleted.
In my case, it was in a dev portal so I just rebuilt the Portal.
And it was not on the home page.
The deleted service just kept showing up in my My Content lists.
I have taken to removing any item from any group membership before deleting it.
I don't recall seeing any specific solution for this but you would need to search Geonet to be sure.
If I were you, I would put in a support ticket and have Esri help work through this or create a bug report.
You have refreshed/deleted your browser cache, right?