Writing HTML and other < and > tagged content to a feature service
My ArcGIS Data Interoperability ETL process was going fine but suddenly its ArcGIS Online Feature Service writer threw an error which terminated my workspace, the error message looked like this:
The error code from the server was '400' and the message was: ''. Details: 'Field public_description has invalid html content.
The data is from a local government 311 site - the field public_description is where the public have captured the details of their service request. How did HTML get in there? What was invalid about it? How do I harden my translation to trap this kind of thing?
After a lot of investigation and making a note to self to bring writer rejection handling up with Safe again I found the offending data. The investigation reminded me of math class when we worked through Isaac Newton's regula falsi method, turning caching on and using two Sampler transformers to bracket the bad guy until I found it. It wasn't even HTML, it was a bracketed email: <somebody@hotmail.com>.
OK I thought, its reasonable to put an email in a free text field. Why is the feature service so finicky about it?
It turns out feature services have a setting (on by default) to cause potentially harmful content to be blocked. I have a few choices. If my data really did have HTML content I could percent-encode the field with the TextEncoder and HTML mode, I could manually encode brackets as < and >, I could remove anything in tags with StringReplacer and a regular expression <.*>, or I could switch the service property and let the data through. Let's say I want this last option.
I need to use an admin REST API call. The property I need to change is xssPreventionInfo.
To do this, in your browser and logged in as the service owner, go to the Content section in the Home app and navigate to the feature service item. At bottom right is a control that lets you View the item REST API:
Go there in your browser (your URL will of course be different).
To access the admin API insert the word 'admin' between 'rest' and 'services':
Then at the bottom of the page go to the UpdateDefinition endpoint:
Now search for the xssPreventionInfo property, set the value for xssInputRule to sanitizeInvalid:
Click the Update Service Definition button and check it completes OK. Now you can write HTML and things like <somebody@hotmail.com> to your feature service!