That's true for any web app: it starts with a web map. I just re-read your o.p.: for an ArcGIS On Line map, you'll need to get your data from ArcGIS Server to AGOL. Is the Server data static or dynamic? There is a way to point AGOL to your server data, but I don't know the particulars on that off hand.
That should just about do it....