Ben,
Not having a 10.5.1 build to test on I tried adding the widget into the in built web appbuilder within portal 10.4.1, and I am experiencing the same issue of the widget not loading. After inspection of the network traffic I can see that some files that the configuration needs are not in the built in web appbuilder.
If your portal has been installed to the standard directories have a look in:
C:\Program Files\ArcGIS\Portal\apps\webappbuilder\stemapp\jimu.js\dijit
on 10.4.1 there is just a single folder called SymbolsInfo
yet in the same directory in web appbuilder developer contains, 105 items:
The parent folder tells a similar story (left portal, right web appbuilder developer):
As a test I copied over the missing files from the developer edition to portal, refreshed the browser and everything worked OK. I am not sure why the two environments differ so much in terms of the files contained within, maybe somebody from the web appbuilder team can explain (dlaw-esristaff junshan_liu-esristaff)
I think your best bet to get the widget working is to manually create the config file with the parameter I mentioned above and ignore the settings screen.
Regards
Anthony