Max surveys? Load test options?

778
6
Jump to solution
03-27-2018 02:40 PM
CJH
by
New Contributor II

What is the maximum number of submitted surveys? I have a simple 5 question damage assessment survey. We will have 70 inspectors and 400k property damage assessments to complete, in only 3 weeks. Can Survey123 and ArcGIS Online handle this load? Will the Analysis tab graph results for all 400k surveys? Will Data tab allow download of all 400k surveys to Excel? Can you provide successful use cases demonstrating this should work fine for me?

Any suggestions on how to do a load/stress test to demonstrate this works, without manually submitting 400k surveys?

For this case would you advise upgrading to AGOL 'Premium' data storage to handle the load?

THANKS!

0 Kudos
1 Solution

Accepted Solutions
JamesTedrick
Esri Esteemed Contributor

Hi CJ,

One of the test surveys used by the Survey123 team is on the same order you describe  and performs adequately with standard data storage. As you might expect, rendering of the Analyze and Data pages is a bit slower, but still adequate.  One thing you didn't mention is the communications environment - will the surveys be sent as filled in over cellular, or will they need to be stored on device then sent in bulk?  If the latter, I would be curios to know what the comms situation to the internet is at the sending location - if all 70 are sending simultaneously and the connection is relatively thin/congested, I could foresee some degradation.

That being said, I'm a little concerned on the basic logistics you present: 400,000 inspections to be done by 70 inspectors over a 21 day period (assuming working weekends).  This is ~5700 inspections per inspector for the period, or ~270 inspections per inspector per day.  Assuming a 12 hour work shift, that is about 2:36 (2 minutes 36 seconds) per form, not allowing for transport or breaks.  Even with a simple, 5 question survey, this seems to me somewhat aggressive considering if the assessment needs to take place at the damage site (if we're talking about inspection from aerial photography, this becomes a bit more reasonable). 

View solution in original post

6 Replies
JamesTedrick
Esri Esteemed Contributor

Hi CJ,

One of the test surveys used by the Survey123 team is on the same order you describe  and performs adequately with standard data storage. As you might expect, rendering of the Analyze and Data pages is a bit slower, but still adequate.  One thing you didn't mention is the communications environment - will the surveys be sent as filled in over cellular, or will they need to be stored on device then sent in bulk?  If the latter, I would be curios to know what the comms situation to the internet is at the sending location - if all 70 are sending simultaneously and the connection is relatively thin/congested, I could foresee some degradation.

That being said, I'm a little concerned on the basic logistics you present: 400,000 inspections to be done by 70 inspectors over a 21 day period (assuming working weekends).  This is ~5700 inspections per inspector for the period, or ~270 inspections per inspector per day.  Assuming a 12 hour work shift, that is about 2:36 (2 minutes 36 seconds) per form, not allowing for transport or breaks.  Even with a simple, 5 question survey, this seems to me somewhat aggressive considering if the assessment needs to take place at the damage site (if we're talking about inspection from aerial photography, this becomes a bit more reasonable). 

CJH
by
New Contributor II

Thanks very much, James. It's helpful to know you've tested and proven this volume will work. We'll be using cellular, regularly submitting. Three weeks is the ideal outcome, but we understand a variety of factors could make it closer to 2 months. Thanks!

0 Kudos
IsmaelChivite
Esri Notable Contributor

Adding a bit more to this, here is a short video showing how the website behaves against a survevy with over 600K surveys (5 questions each). Survey600K_WebsitePerformance - YouTube  

If you plan on downloading the data, you have multiple options including a direct export from the Survey123 website or alternatively you an also use ArcGIS Pro.  In our testing, we  have  found no issues downloading datasets of this size (but takes a while...).

I totally agree that the network conditions when submitting data will most likely be your weakest point.

Understanding the peak rate  of submits will be important to understand the ultimate load on ArcGIS Online  (from the discussion above it does not seem clear how surveys will be submitted).

CJH
by
New Contributor II

Thank you very much for the demo video Ismael.  Nice to see it works. How could I do a similar test, where I create a simulated survey with 400k records without me manually submitting them one at a time?

The plan is for the inspectors to complete and send surveys one at a time. We have discussed in the cases where cell signal is weak, to save completed surveys to their devices and send them after moving to an area with greater signal.

Can you please advise me how to test different loads? I would appreciate suggestions to help prepare.

Thanks!

0 Kudos
IsmaelChivite
Esri Notable Contributor

If you would like to artificially add data into your survey for testing purposes, I suggest you have a look at the ArcGIS API for Python | ArcGIS for Developers  With the Python API you can create random values and insert them into your survey feature layer.  You can also append data to a feature layer using ArcGIS Pro. I feel very confident you will not run into any performance issues when it comes to exploring with the Survey123 website your 400K surveys. 

CJH
by
New Contributor II

Thank you very much, Ismael. 

0 Kudos