GeoAnalytic Server Error -Copy data to Data store

[AnjithaSenarath3]

We are trying to load 4 Million Records to Spatiotemporal Data store (2 servers nods) using GeoAnalytic servers (1 Clustered withg 2 servers). All Servers federated using Portal +Hosting server. 

All servers 10.6.1 

We use ArcGIS Pro Desktop to load data using GeoAnalytic Tools > Copy data to Data store and select Spatiotemporal data store to save data. This process keep failing... Is there any data limit on this process as I can see process work with small amount of data like 1000 records. 

GeoAnalytic server cluster  keep showing below error :

Any idea why this is happening ? 

Error executing tool. CopyToDataStore Job ID: jce19aec132484df681b96a3a246ac443 : {"messageCode":"BD_101030","message":"Failed to execute CopyToDataStore. Please contact your administrator with job ID 'jce19aec132484df681b96a3a246ac443'.","params":{"toolName":"CopyToDataStore","jobID":"jce19aec132484df681b96a3a246ac443"}} Failed to execute (CopyToDataStore).

System/GeoAnalyticsTools.GPServer

[SarahAmbrose]

Hi Anjitha Senarath‌,

What is the input of your data? When you are copying from ArcGIS Pro to GeoAnalytics it's best to have your data local to your server. This means as either a hosted feature layer, or as a big data file share. If you're using other inputs, such as dataset local to your Pro machine, the data will be copied from Pro to Server before the GeoAnalytics tool even runs, which can take a very long time. If your data is a big data file share or a hosted layer and you're seeing this error message - I would recommend calling into support. They will be able to turn on more informative error messages and help diagnose the issue.

Please let me know if you have any follow up questions.

Thanks,

Sarah Ambrose

Product Engineer, GeoAnalytics Team

[AnjithaSenarath3]

Thank you very much for detailed guide. for small amount data we can create a feature Service. How can we create a feature service for 4 million records ? Can ArcGIS Server return entire data set ? There is a limitation for records returned by server.  (1000 by default). 

Our GeoAnalytic servers (2) in a cluster hence ArcGIS server directory in UNC paths. Loading data also loading from data folder in same UNC directory with register folder path for GeoAnalytic server cluster. 

System is working for 1.5 million data but It is not working for 4 million data. Only option to load by batch process using append data which is giving errors for same schema/same projection same Geometry type data. I dont know why. I cant even append 10 records. Serious bugs in  ArcGIS GeoAnalytic server 10.6.1. 

we contact ESRI support and they said to restart platform services in GeoAnalytic servers which is standard procedure. ( Every time I check them before run an Analytic process ( Standard procedure of mine) ) . ESRI support is not helpful at the moment unfortunately. 

AppendData Error log as this : 

Error executing tool. AppendData Job ID: j1d5f0b9bc40e4860bc8d16de5a20f4de : {"messageCode":"BD_101030","message":"Failed to execute AppendData. Please contact your administrator with job ID 'j1d5f0b9bc40e4860bc8d16de5a20f4de'.","params":{"toolName":"AppendData","jobID":"j1d5f0b9bc40e4860bc8d16de5a20f4de"}} Failed to execute (AppendData).

Why we need logs like this :  Failed to execute (AppendData) ???? 

We cant see the real error in logs? seriously ESRI is hiding real errors ?? 

Any Solution please 

[SarahAmbrose]

Thanks Anjitha Senarath‌,

Are you able to send me the case number?

Can you please clarify what your input data is? (big data file share registered with GeoAnalytics, something else?)

The 1000 limit from server will not impact your results at all and you are able to create a feature service with 4 million points. I will point out that you don't need to create a feature service to run GeoAnalytics tools though, you can keep the data where it is, and register it as a big data file share and use that as input to any tool.

- Sarah

[Anonymous User]

Hi Sarah Ambrose‌,

I registered the data as a big data file share and I used it as an input for Aggregate Points tool. The tool asks me to provide the result layer name to generate a new feature layer. This feature layer will be shared with others persons in the organization. The data is updated regularly and I should execute the aggregate points tool after each update. However, when I do so it only allows me to create a new feature layer. Is there I way to update the first generated feature layer?

Thanks

Zakaria

[SarahAmbrose]

Hi Zakaria Douiri‌,

Can you please clarify your question. Are you trying to run a tool with a big data file share as input? And running into issues?  Thank you for updating the question.

Since you question is different than the original, can you please ask a separate question so it gets it's own thread.

Thanks,

Sarah

[Anonymous User]

Ok, I'll ask my question in a new thread.

Thanks