Hey James,
Sadly I never was. We raised a support incident with our reseller (esri Netherlands) and although they were really helpful and even filmed the whole scenario, all they got back from esri Inc. was an "unable to reproduce".
We managed to work around it by forcing the builder to NOT be able to find subscriptions (by just emptying the primary directory...), so we could enter the tenant + subscription manually, but frankly: that feels horrible, which together with not being believed, gives a bit of a sour taste to cloudbuilder.
It actually got worse gradually, as we got further:
* Our staging/production environment, by our CSO's directive, needs to be deployed form a sub-account with higher rights. Cloudbuilder insists on using SSO to sign in. Naturally this is tough to combine... having a button "manual signin" instead of being over-friendly with SSO would be great. My work-around: go taksmanager; find the actual .exe for builder (as it's a clickonce application that's kind of obfuscated...), and execute "run as"... to get to the sub-account... Yay... I wish esri/microsoft had consulted with our CSO before building it this way
* Using keyvaults is broken. Once you do, in the final step, you're unable to export artefacts (message: illegal character). Similar tools using the same keyvault work. With keyvault on a deploy won't work either...
* If using a pre-peered network (after all, ArcGIS will have to fit in with the rest of the application landscape and ETL tools...), the tool outright refuses on occasion to deploy certificates to the webserver...
* A delete-environment whilst retaining a public IP (which our CSP provides) keeps all the tags on said IP. Cool, except it prevents you from being able to re-run the tool until those tags are removed. Those tags are where cloudbuilder stores its information about it being an ArcGIS deployment
What could be improved:
* When using Azure storage (blobs & tables, please start cosmosdb as well!), there's still dependance on the fileserver. I'd kill for that to go to fileshare, to potentially negate the fileserver altogether
* The device selection sometimes allows for VM & Storage sizes that are mismatched to be selected. Validate is only WAY WAY WAY in the actual deployment, causing a hard-crash. Even the web-portal on azure pre-validates before deploying now...
Honestly, I'm really a bit saddened on this tool. There's a huge reluctance to adopt a more cloud-native strategy and on paper the Azure Builder qualifies. It supports pre-made networks, keyvaults, etc, etc... but it's execution is lacking. What's worse: the way it is now, it's barely useable. Though in part due to our CSO office, off-course, it's largely in part as well due to wanting to be TOO user friendly. Don't be afraid to expose some advanced parameters (or heck... outright allowing us to create the post-config artefact out of the bat rather than dancing around in a GUI for ages).
What also struck me as saddening was how esri went and treated us. Both our sales rep and us said we'd be more than happy to have some sort of teams/remote session with esri inc to further clarify/demonstrate the problems. We even offered to help with the solution. Yet all we got was a "can't replicate, no problem" issue. That's a huge vote of "no trust" to not just ourselves, but also to their local colleagues.
As it stands now, I'd sadly accept the fact that you might end up having to build your own stuff/environments. You'll miss out on some really cool features like the ability to seperate content (which goes in azure storage) and functionality (which is the VM's for now), allowing for some really cool scaling/CI-CD scenario's...
