Skip navigation
All Places > Survey123 for ArcGIS > Blog > Authors JHasthorpe-esristaff

 

In the 2.3 release of Survey123 we made some changes to the way that surveys are published when working against ArcGIS Enterprise. These changes affect surveys using both hosted and federated feature services.

 

Background

 

When publishing hosted services to ArcGIS Enterprise, the service's data is stored in the Portal for ArcGIS Data Store. It is a requirement of the Data Store's underlying PostgreSQL database that all field names are lowercase, and this affects the service field names too. Although questions in Survey123 can be mixed case, if the app is to successfully write to these services, then the case used must match (i.e. be lowercase).

 

Previous versions of Survey123 would let you publish surveys with mixed case questions, as the field app would fix the case on submission. Unfortunately, this caused a problem when working with federated services (which can be mixed case), as forcing the field names to be lowercase resulted in miss matches in some circumstances.

 

Fixing the case on submission was a workaround we implemented to make it easier for the end user. However, in hindsight we should have ensured that survey was published using the correct case in the first place, and this is what we are doing now.

 

What has changed?

 

As result of the above, at 2.3 we have stopped the field app from fixing field cases on submission. Instead we have made Connect do two new checks when publishing a survey, that ensure the following:

 

  • Any survey published against the Portal for ArcGIS Data Store only has lowercase question names.
  • Any survey published against a federated service, has an exact case match between the questions and the corresponding field names in the feature service.

 

If either of the above is not true, the user will now be prompted to manually fix the question names before being able to publish.

 

How could this affect you?

 

If you have existing surveys published against the ArcGIS Data Store, with mixed case question names, and you have upgraded to the 2.3 version of the field app, you will no longer be able to submit data to any mixed case question/fields.

 

As previously mentioned, this is because the app is no longer changing field names to be lowercase on submission. While the survey will successfully submit in this situation, any mixed case fields in the service will be left blank. 

 

What do you need to do?

 

If you don't have un-submitted data on your field devices, all you need to do republish your surveys using the 2.3 version of Connect. The republishing process will identity any mixed case issues and prompt you to fix them (i.e. make all questions lowercase) before letting you re-publish. Remember that on top on changing question names, you will need to update any references to them, e.g. the calculation field or the form_id. But again you will not be able to re-publish the survey until all such issues are resolved.

Once republished, users in the field will be able to download the latest version of the survey and continue their work.

 

If you do have un-submitted data on field devices, you will need to get the .sqlite database off the field device and re-submit the data using a desktop version of the field app. There are two approaches that you can use for this.

 

Make sure you create a back up of your database before attempting the below.

 

Approach 1

Using version 2.3 of the field app (Windows, Mac of Linux) run through steps 1-9 in the second part of Recover data from a mobile device in Troubleshoot—Survey123 for ArcGIS | ArcGIS. However, before you complete step 5, you will need to fix the field cases in the database. This can be done by updating and executing the following statement for each affected question:

 

UPDATE Surveys SET data = REPLACE (data,'"Offences":', '"offences":')

 

The example above will set the Offences field to be lowercase for all surveys in the database. You will need to run an updated version of the statement for each mixed case question. Once complete, you can the proceed with steps 5-9.

 

 

Approach 2

Using a version prior to 2.3 of the field app (Windows, Mac of Linux) run through steps 1-5 in the first part of of Recover data from a mobile device in Troubleshoot—Survey123 for ArcGIS | ArcGIS. This will be the easier option if you have not yet upgraded Connect to 2.3.

 

After you have successfully re-submitted the surveys from the device, you should then go through and republish all affected surveys, as described above.

 

 

We hope that this change will not affect you too negatively and do regret that we did not provide a more advanced warning of the above. We are however planning to release some tools that will make the process of getting data out of the field app’s .sqlite database much easier – so please watch this space. In the meantime, if you have any questions, or need further advice or clarification please let us know. 

Tile Package Kreator

Tile Package Kreator -  the desktop utility used for the easy creation of Tile Packages (i.e. local basemaps that you can take offline in Survey123 for ArcGIS, Collector for ArcGIS and ArcGIS Runtime apps) has just been updated! Version 1.2 comes complete with a number of enhancements that will make the app easier and more intuitive to use. To find out how to use it - please see the following blog:Taking your maps offline with Tile Package Creator.

 

Esri Labs projects are developed by Esri employees and are inspired by our interactions with ArcGIS users like you. Esri Labs projects are free to use but are not official Esri products. These projects do not go through the rigorous software development cycle so they are not holistically tested, documented or supported by Esri technical support.

 

But for now let’s run through what’s new:

 

New functionality

You can now navigate to your desired location using a place name or address, and once there, you can draw polygons (as well as squares and polylines) to define your export extent. If you want to revisit the extent at later date, you can now save these drawn extents as spatial bookmarks (which can be exported to geojson too). TPK 1.2 will also enable you to interactively define the range of zoom levels you want to export – so if you want to export for example levels 4-12, this is now possible.

 

Improvements

To make life easier, you can now switch tile services/basemaps without having to return to the gallery, and any public tiled service that you added to app will be persisted (so no need to keep re-entering them). There have also been a number of bug fixes related to the UI and file management.

 

If you want to get your hands on the latest version of the app – please visit here: ArcGIS Marketplace

The latest version of the guide can be downloaded here: Guide

And finally - to provided feedback, please go here: github.com

Despite the best planning of your surveys, there may be circumstances when you need to change the data type of one or more of your questions. For example, you may have a question that will only accept integers, when you now need it to accept strings. The simplest way to do this is to make the change in XLSForm spreadsheet and republish the survey. However, the issue here is that Survey123 will need to republish the survey’s hosted feature service AND will delete your existing data in the process. If you want to do such a schema change BUT want to keep all your existing data, then keep on reading.....

 

Before we continue, there is a caveat: Not all field types can be successfully converted from one to another, and even if a conversion works – you could reduce the precision or integrity of your data. Please make sure that you consider the implications of any data type conversions before proceeding with this workflow.

 

High level workflow:

 

If you are comfortable with ArcGIS Online here is what you need to do (If not, I have attached a more detailed step by step document):

 

In the ArcGIS Online:

  • Back up the hosted feature service for the survey to a FGDB.

 

In the ArcGIS Online Map Viewer:

  • Add a temporary field to the hosted feature service. This should be of the new field type you are converting your question to.

  • Update the temporary field with the values from the original field.

  • Delete the original field

  • Create a new version of the original field, but give it the new data type that you require.

  • Update this new field with the values from the temporary field.

  • Delete the temporary field.

 

In Survey123 Connect:

  • In the XLSForm Spreadsheet, update the data type for the target question. This should now match the data type of the new field you created.

  • Republish the survey. You should receive the message that your data will not be affected.

  • Test the survey on the Survey123 field app.

  • Start collecting!