Performance lag for field app for repeats

473
10
10-18-2021 04:58 AM
Cyprus_Forest_Department
New Contributor III

Hi I am currently facing an issue, maybe the last field app update brought this up, where the repeats take too long to populate the external csv files via the pull data function.

When I click my relevant question to make the repeat table appear there is a 10" lag, and when i click the repeat record to move to the next one there is another 15" lag.

This was not the case before.

What troubles me is that users try to hit other buttons while the device is trying to run the request, causing the application to freeze more, or even crash.

I have attached my xlsx file, highlighting with yellow colour the repeat table that is facing this issue.

Thanks in advance for the support.

10 Replies
apericak
New Contributor II

Adding a +1 here; I've been noticing the exact same performance lag over the last few days for a survey I've been developing, also leading to crashes (or, on a relatively old iPhone SE, not even loading the survey at all). And like @Cyprus_Forest_Department's survey, mine also uses csv pulldata() to initially populate the repeat_count column for the repeat, has a relevant function to decide whether the repeat should appear, and uses additional csv pulldata() calls to other csv files within the repeat. 

iOS app 3.13.234; Survey123 Connect 3.12.232

0 Kudos
Cyprus_Forest_Department
New Contributor III

Maybe an option to downgrade the field app would be of use...I have already replicated the issue to my phone with a Mediatek Helio G95 (Octa-core (2x2.05 GHz Cortex-A76 & 6x2.0 GHz Cortex-A55)). I have tried also stripping down a few calculations (saw in another post that calculations can be cumbersome for the performance of the form), but to no avail.

My pull data csv files for that repeat table have each approximately 40.000 entries. I don't know if breaking up the data with relevant questions to lets say 20.000 entries, would speed things up...

Lastly, my feature layer is involved in integromat scenarios, in webapps and dashboards. It would be a pity if a republishing would break these things :( 

0 Kudos
DougBrowning
MVP Frequent Contributor

If you remove repeat count does is it still slow?  I just posted here that repeat count has caused a lot of issues for me.  See here for an idea https://community.esri.com/t5/arcgis-survey123-questions/survey-123-controls-not-working-when-launch... 

0 Kudos
apericak
New Contributor II

Thanks @DougBrowning for the idea and the neat repeat count workaround you came up with.

In my case, removing the repeat count doesn't seem to improve the delay. I'm still testing everything out piece by piece, and the only thing I've found so far that brings the load times back to manageable levels is deleting the number of pulldata() calls within the repeat to only a few, but of course that means removing survey questions so that's not workable.

The problem seems somewhat related to memory or processing power; I had a field worker with a much newer phone test it out and it was slow for them as well, but faster than for me. But still, minutes of time to load a repeat page. 

Cyprus_Forest_Department
New Contributor III

Hi, did my tests as well. Created several copies of the surveys and tested various scenarios with the same tablet that our field workers carry out the tree inventory.

Removing the Repeat Count doesn't have any effect on the response time of the form.

Removing some calculations that used the pulldata function, brought down the time of moving to the next record in the repeat table, down by 10".

However I think it is an update issue of the field app, because I can't remember the form lagging before.

For sure CPU power has a tremendous effect on performance, not so RAM I believe. My phone loads snappier the form than the tablet.

0 Kudos
apericak
New Contributor II

I've made some progress to minimizing this load time. I turned on logging and tried to load my survey; the logs revealed hundreds of repeated error messages saying "QML XFormNoteControl: Binding loop detected for property "calculatedValue"". From what I could tell in the log, these messages appeared once the software was attempting to parse a row in my XLSForm that contained a position(..) function within a repeat.

I was using position(..) to get the repeat index, which I then concatenated onto a composite key; that [composite key + repeat index] is my lookup value for the CSV files that I'm accessing through pulldata(). In other words, my calculation to attain the [composite key + repeat index] was: concat(${composite_key}, "_", position(..)) And then I would use pulldata() to find that [composite key + repeat index] value in the CSV table and retrieve the desired attribute.

I used @DougBrowning's suggestion (thank you!!) to find the repeat index using once(count({$any_repeat_field})) instead of position(..). Removing position(..) from this calculation drastically, but not completely, sped up my survey load time; in my case, down to about 50 seconds. 

I then just tried removing a lot of rows from my CSVs (some of them are hundreds of thousands of rows long), and that dropped the load times back to what I had been seeing before, about 3-5 seconds. I was getting similarly quick load times with these big CSVs before the recent app updates, so I'm still not convinced this is totally the solution, but at least for me position(..) seemed to be the main culprit (or perhaps how I was using it in the calculation?)

DougBrowning
MVP Frequent Contributor

Do you have nested repeats?  If so I had to use position({repeatname}).  That may help.  My lists are over 6,000 I have not seen any speed change.

apericak
New Contributor II

Thanks; no nested repeats for me (at least in this survey!) 

0 Kudos
Cyprus_Forest_Department
New Contributor III

The issue most probably was introduced after an update of the field app. The form did not change from our part. The performance drop was noticed among all people working with their tablets. There should be an option to downgrade the field app. I also noticed that apart from the initial "Optimizing Survey" message, the  "Loading Survey" message lasts longer. This screen appears every time you launch the form from the application.

0 Kudos