Hi community,
I have used survey123 form to collect forest attributes which have many nested columns and attachments fields. I found one of the column has error field name so I updated that and published the form. I updated that form in my tablet and continued to survey the forest plots. Now, the problem is I could not sync those data with server. All the data are in outbox.
I tried manually to retrieve those data using python code but could not retrieve all the data from lower field and also most important I could not get the information or link to attachments (e.g. Images) in the schema.
I need help either to upload to sync or export those data to excel.
Thank you in advance.
Solved! Go to Solution.
You’ve run into one of the more painful Survey123 “gotchas” — once you change the schema of a published form (renaming fields, adding/removing questions), the local database on the device no longer matches the hosted feature layer. That’s why your Outbox refuses to sync.
The SQLite database on the tablet (the Survey123 field app’s local store) was created with the old schema.
After you republished with a corrected field name, the server has the new schema.
Sync fails because ArcGIS can’t reconcile the column mismatch.
If possible, roll back:
In Survey123 Connect, rename the field back to the original error name.
Republish so the feature layer matches what the Outbox is expecting.
Then sync.
Once all outbox records are on the server, fix the schema again and migrate data with ArcGIS field calculator or Python.
➡️ This is usually the fastest/safest way if you can tolerate the “bad” field name temporarily.
On Android: navigate to Android/data/com.esri.survey123/files/ArcGIS/My Surveys/databases/
Each survey has an SQLite .sqlite database. Open it with DB Browser for SQLite or Python (sqlite3).
Tables:
main table has your attributes (with the old field names).
Related tables hold attachments. There you’ll find filenames or BLOBs. Attachments are stored under ArcGIS/My Surveys/Media/ on the device.
You can export to CSV/Excel manually or with Python (pandas.read_sql).
If you already extracted the local .sqlite, you can read:
import sqlite3, pandas as pd
conn = sqlite3.connect("path_to_survey.sqlite")
df = pd.read_sql("SELECT * FROM main", conn)
df.to_excel("survey_backup.xlsx", index=False)
For attachments:
Join main with attachments table on globalid.
Copy attachment files from the Media folder using the REL_GLOBALID.
✅ Recommended next step for you:
If you still have the old XLSForm (with the original “wrong” field name), republish that version, let the devices sync, then repair field names after everything is safely on the server.
Do you mean recover the data from the tablet? If so start here https://support.esri.com/en-us/knowledge-base/faq-what-are-the-ways-to-recover-arcgis-survey123-data...
This is a good article to check also https://resources.esri.ca/getting-technical/the-ultimate-guide-to-recovering-arcgis-survey123-data-f...
You’ve run into one of the more painful Survey123 “gotchas” — once you change the schema of a published form (renaming fields, adding/removing questions), the local database on the device no longer matches the hosted feature layer. That’s why your Outbox refuses to sync.
The SQLite database on the tablet (the Survey123 field app’s local store) was created with the old schema.
After you republished with a corrected field name, the server has the new schema.
Sync fails because ArcGIS can’t reconcile the column mismatch.
If possible, roll back:
In Survey123 Connect, rename the field back to the original error name.
Republish so the feature layer matches what the Outbox is expecting.
Then sync.
Once all outbox records are on the server, fix the schema again and migrate data with ArcGIS field calculator or Python.
➡️ This is usually the fastest/safest way if you can tolerate the “bad” field name temporarily.
On Android: navigate to Android/data/com.esri.survey123/files/ArcGIS/My Surveys/databases/
Each survey has an SQLite .sqlite database. Open it with DB Browser for SQLite or Python (sqlite3).
Tables:
main table has your attributes (with the old field names).
Related tables hold attachments. There you’ll find filenames or BLOBs. Attachments are stored under ArcGIS/My Surveys/Media/ on the device.
You can export to CSV/Excel manually or with Python (pandas.read_sql).
If you already extracted the local .sqlite, you can read:
import sqlite3, pandas as pd
conn = sqlite3.connect("path_to_survey.sqlite")
df = pd.read_sql("SELECT * FROM main", conn)
df.to_excel("survey_backup.xlsx", index=False)
For attachments:
Join main with attachments table on globalid.
Copy attachment files from the Media folder using the REL_GLOBALID.
✅ Recommended next step for you:
If you still have the old XLSForm (with the original “wrong” field name), republish that version, let the devices sync, then repair field names after everything is safely on the server.
Thank you for the solution. However, I could not uploaded with the old form since I could not afford losing those data and it still pop-up sameG:\ForDis\Demo_field_data.json error. Unfortunately, I mistakenly use "." in creating the field (e.g. fuel.t1.10hrs). Attached is my demo data.
I am writing a python code to export those data to excel with temparing it.
Finally, I would like to appreciate your effort and time.
Hi,
Just a quick question, if I could update all the field (e.g. using DB browser) same as the server that I updated latest, will there be chance to be synced.
That’s a really good thought but unfortunately, simply editing the local SQLite (with DB Browser, etc.) to match your new server schema won’t “trick” Survey123 into syncing.
Thanks for accepting the solution.
Regards,
Venkat