Hi Joe - I did finally get our stack to deploy successfully. My snag was with the aws secrets. For the arcgis user password there are pretty specific password requirements that if you are pointing to a aws secret arn in the template you will have no idea its wrong/non-conforming. It was only by putting passwords in as plain text that I was able to see the requirements for arcgis Windows users (which by the way is not documented anywhere in the Enterprise Cloudformation Template). Here are those requirements so you can double check:
Password must be at least eight characters in length and must contain characters from three of the following four categories: English uppercase characters (A through Z), English lowercase characters (a through z), digits (0 through 9), non-alphabetic characters (for example, !, $, #, %). Password must not contain backslashes (\) or quotation marks ("). Password must not contain the user's account name (arcgis) or parts of the user's full name (ArcGIS user account) that exceed two consecutive characters.
I also have an unconfirmed suspicion that the arcgis user password and your portal admin password cannot be the same as I had stack deploy fail even when I followed above guidelines for arcgis windows password and used the same password for portal admin. Once I made them different, the stack deployed successfully.
Another important note. I disabled rollback so that I could view the logs per esri guidance and then tried a couple Updates (which for aws means that it tried to pick up the stack build wherever it had left off). Update never worked even with all password requirements met. I had to do full rollback and redeploy stack from beginning with updated params for it to work.
As I mentioned above, I also had to edit the template json because the regex validation for domains prevents it from containing any numbers (even though our company name and all domains contain a number and by any web standard that is totally fine). I was then told multiple times by esri support that editing the template will mean that the resulting Enterprise deployment will be "unsupported." It's just hard to believe that this is maybe the 4th or 5th iteration of CloudFormation template and it is still riddled with gotchas that force users to hack the template and then be told by Esri support that doing so will result in an unsupported deployment.
Lastly and more to the point of helping you troubleshoot. If you EC2 instance is created, you can remote into it and check out chef logs. But confusingly, the cookbook is not longer chef but something called "cinc". If the ec2 is a windows server those should be here: C:\cinc\arcgis-enterprise-primary.log
Hope some of this is helpful.
