There is only one text source for a Python script, but after an edit it is compressed into a zip bundle *.esriaddin using makeaddin.py
I am then supposed to double click on it to install it into the well known location in MyDocuments.
I assume this is different to addins created in java and .NET where the source is in an IDE. The Python wizard is not an integrated environment. So if you are creating a .NET addin you can edit your (non-python) script while ArcMap stays up?
It all works just fine if I close and reopen ArcMap, but what a pain!
It seems that the addin is loaded into memory when ArcMap starts and no amount of deleting refreshes the image of the addin.