Survey123 version 3.12.274 Pulldata() functionality changes?

[BamJam]

Hello Survey123 Community and @JamesTedrick or @IsmaelChivite 

We have a form that has been in use for months that utilizes the pull data functionality to populate some fields in the form based on a barcode scan. 

We have had several users call that are not getting data populated in those fields after a barcode scan.  All our users are using iOS devices.  We have isolated this issue to users that have version 3.12.274 of Survey123.  We've verified that the necessary .csv file is in the media folder within the Portal item and that the data is properly populated in the file itself.  Furthermore, users that are still on 3.11.164 are able to scan and get the necessary fields populated...even after updating the form itself.  So, it seems to be an issue with version 3.12.274.

I have not been able to find any information on version 3.12.274.  Is there a change in the pulldata() functionality? 

Any help would be appreciated.

Thanks,
Brandon

[ZacharySutherby]

Hello @KaitlynAbrahamson, 

The reason for the issue is because column N in the CSV is missing content. If you move the contents from column O (areaformula) into column N then the CSV should work properly. 

This is related to the original issue where empty column headers prevent the CSV from being imported into the database. 

Thank you, 

Zach

[KaitlynAbrahamson]

Hi!  It looks like that helped for some of my pulldata functions (thanks!), but not all of them.  I'm not positive why that's the case, but it is definitely the more important part of my survey.

[DustyJordan]

Sorry to dredge up an old topic, but I am having issues with the pulldata function once again. It seems every time I try to update one of my surveys it re-breaks the pulldata functionality. 

I've tried reformatting columns, resaving csv files, etc. and I just cannot get this to work anymore. It functions fine in Survey123 Connect, but will not function in the mobile app. I've attached a copy of the csv I'm having issues with. 

Any help is appreciated. 

[ZacharySutherby]

Hello @DustyJordan, 

Thank you for passing along the CSV! If you open the CSV in a text editor like Notepad or Notepad++ you can see that there are a number of extra commas (indicating empty columns) in the CSV. 

I had opened the CSV and deleted empty columns N-X which removed the extra commas from the CSV. When the CSV was updated and the extra commas were removed after republishing the survey it worked as expected. 

I was also able to reproduce the behavior in Connect 3.12, just to confirm are you using the 3.12 version of Survey123 Connect? 

Thank you, 

Zach

[DustyJordan]

Hi Zach,

Thank you for your help on this, sorry it's taken me so long to get back to you. 

I tried as you suggested and deleted the extra columns in my CSV. This seems to have...sort of worked, sort of not. Let me explain. 

In my form there is a null field where the user inputs a number (e.g., 9) and the pulldata function then searches the table and returns a 4-digit number formatted as text (e.g., 0009). This 4-digit number format is important for further pulldata and concatenate functions in the form. In Survey123 Connect, this still seems to function properly, as "0009" is returned when I enter "9", and further fields like Latitude and Longitude are then also able to be pulled from the table. Up until recently this was also working in the mobile app.

However, currently on the mobile app it appears that the pulldata function is ignoring the leading zeroes and returning a single digit number, which kind of breaks the rest of the form. So while the extra columns were surely causing issues, there seems to be an underlying issue as well. I've ran into this before and resolved it by making sure the field in Excel was formatted as text before saving to the CSV, which retained the 4-digit format. This seems to no longer work. (I confirmed by checking the CSV in notepad, where the leading zeroes were present).

Any further recommendations on resolving this new problem?

Thanks,

-Dusty

P.S. I am using Survey123 Connect v3.12.

[ZacharySutherby]

Hello @DustyJordan ,

We have a defect logged for this behavior. The defect is BUG-000138199, the workaround for the defect is if there is a set number of leading 0's a character can be added to the prefix of the value (like ` for example) and the substr() function can be used to filter it out. 

Alternatively a comparison of the returned length can be done and zeros can be added where necessary. Something like: 

if(string-length(${r_code}) = 1, concat("000", string(${r_code})), if(string-length(${r_code}) = 2, concat("00", string(${r_code},....) 

Thank you, 

Zach 

[AndrewPadilla]

I am also having issues with the pulldata function and select_one_from_file type since the update to 3.12. My external files does not have extra columns or rows in the data, the column names are database appropriate, no commas in the file, and no special characters. My original post is here: https://community.esri.com/t5/arcgis-survey123-blog/what-s-new-in-arcgis-survey123-february-2021/bc-... in the comment section. I am currently working with an ESRI technician but do not have a solution yet.

I might suggest, when changes are made to functionality of functions such as pulldata, that the "What's new in ArcGIS Survey123 " blogs include a description of these changes. It might also be beneficial for users to have an updated guide on the use and limitations of pulldata and external data. 

[ZacharySutherby]

Hello @AndrewPadilla, 

Thank you for reaching out! I have taken a look at our Tech Support cases and believe I have found the case you had logged. 

Assuming the case I found is the correct one the issue with the Form 1 survey that was passed along is the pulldata() function was being used incorrectly. The XLSForm that was passed along has 5 values specified in the function when the function only takes 4. Please use these links for more information on the pulldata() function: https://doc.arcgis.com/en/survey123/desktop/create-surveys/xlsformformulas.htm#ESRI_SECTION1_A3C2F35...

https://community.esri.com/t5/arcgis-survey123-blog/use-existing-data-in-your-survey-the-pulldata-fu...

I have attached the modified sample form that is working on my end. 

As for the original survey that was attached to the case there was only 1 CSV file attached to the case, although it looks like the survey uses more. In the CSV file there are two columns with the same name "name", columns B and R. Once I had updated the column name for column R to Full_Name then the CSV file started working as expected. 

As for the issue with the JavaScript function I did not see a JS file attached to the case, but please feel free to reach out to me directly ZSutherby@esri.com and I'll be happy to look into it. 

We are currently working on documenting best practices when using CSV's for external choices and pulldata() functions. 

Thank you, 

Zach

[AndrewPadilla]

Hi Zach, 

Thank you for you quick response! I read your previous post and had found the duplicate column names: "name" and "Name". As you suggested, I renamed one "FullName" and the survey is functioning correctly. Woohoo!! Sometimes the simplest fixes take the longest to figure out. The additional media files are attached.  I can't seem to load a .js so here is what is in the file:

function HasDups (myArray){

return new Set(myArray).size !== myArray.length;

}

function getLast(questionInRepeat){

return conditionsArray[questionInRepeat.length-1];

}

The JavaScript issue may have been related to the duplicate "Name" columns. I appreciate that you are working on a best practices for using external *.csv and thank you for taking the time to find a solution for pulldata! Happy Friday.

[ZacharySutherby]

Hello @AndrewPadilla, 

Thank you for passing along the JS file and the other CSV files. I have tested on my end and they all seem to be working. Please let me know if there are any issues or errors encountered and I will be happy to take a look into them. 

Thank you, 

Zach