I get the impression that what this app does is create a placeholder website that just references (almost like an iFrame) a portal map stored at ESRI online. I'm REALLY hoping that this is NOT what is going on. I have downloaded the WebApp Builder, but haven't gotten it installed yet. My GIS group doesn't consider it mission critical so it's left to when I "have time" (which so far is never).
Could someone with more experience give me some feedback here?
Yes you have to Publish your map to your AGOL organizational account. The data can still be coming from your ArcGIS Server and you can control access using AGOL groups, etc. The WAB app just uses that WebMapId from AGOL to add the map.
That will not be necessary as I have been digging into the MapManager.js and have found that esri must be listening to all the complaints from users about being forced to us a WebMapID from AGOL and there is code in there to just use basemap and operational layers (like good old ArcGIS Viewer for Flex) in the config.json. They just have not exposed the ability to configure this in the WAB UI.
we have the same problem on that. We'd like to use WebApp Builder not only from AGOL.
So, how is it possible to do that as you say, by modifying MapManager.js and\or config.json?