Latest Contributions by TimMinter

I'm attempting to use ArcGIS Pro 3.3.1 to read a non-geodatabase PostgreSQL 15.8 database table with ~860 million rows, then write those rows to a geodatabase table.  I'm using the Append GP tool to map the input table columns to the output geodatabase table fields and load the records.  When I have tried with an enterprise geodatabase target table in SQL Server 2019 and 2022, the process slows to a terrible crawl after ~262 million records are loaded.  Same thing when I try with a file geodatabase target table, but it looks more like the load stopped instead of slowing at ~262 million records.  Storage is fine, RAM looks happy, CPU says ArcGIS Pro is doing something noticeable.  Disk reads and writes drop off for the local attempts and network transport drops off for the remote server attempts when the load stalls.  I ensured that the only index on the target table columns was the ArcGIS ObjectID index, and that it's not clustered in order to eliminate the SQL Server "load slows when unsorted data is loaded to a table with a clustered index" problem.  I'm using the 32-bit ObjectID because I'm not getting too close to the 2.14 billion limit that would bump me up into 64-bit ObjectID territory. Technically, the documentation and discussions in the community suggest that the file geodatabase itself can handle "unlimited" records.  Any thoughts on why ArcGIS Pro is getting stuck at ~262m? cheers, tim ... View more

In a December 2022 "Sharing Content to ArcGIS Enterprise" Esri course, the instructor noted that when I share an ArcGIS Pro layer to Portal as a hosted feature layer, if I ensure that editing capability is not enabled, then AGE-Portal persists the layer's data as a file geodatabase instead of as an enterprise geodatabase (PostgreSQL, probably).  The instructor said that the file geodatabase is the highest READ performance data structure that AGE-Portal uses for its hosted feature layers.  I want to use the highest READ performance data structure. I've only now come around to actually make use of this information, and everything changes all the time.  When peeking under the AGE-Portal hood, I'm not seeing (yet, I hope not to dive into the dark box on this) that it's persisting hosted feature layers without editing enabled in anything other than its managed PostgreSQL enterprise geodatabase data store. So, my questions:  If, in early 2024 I use ArcGIS Pro 3.2.1 to share a layer to AGE-Portal 11.1.0 as a hosted feature layer without editing enabled, in what data structure/format is the data persisted? What is the highest performance data structure/format I can convince AGE-Portal to manage for me? I have a time-enabled point feature class with ~53m rows that I need to persist in an AGE-Portal data store.  Typically, only about 800 active records are queried and rendered for the full spatial envelope of the data set.  Scale dependencies are set for ~1.5m active records, so they get queried and subsetted to reasonable quantities when zooming in and rendering. There are numerous other performance, gotcha, bug, and version discrepancy red flags in my questions and examples, and I'm pretty much aware of and controlling for all of them.  I'm specifically interested in understanding what controls I have over AGE-Portal's decisions on what data structures/formats to persist my data that it manages.  I'll be selecting for highest possible READ performance. Ok, well, I also want to know whether or not AGE-Portal respects and includes all of my performance configurations (field indexes, coordinate reference system, and the like) when it ingests my data into its data store.  Maybe it does some of its own?  I have noticed that it has not been entirely respectful of things like field order and field naming choices, so maybe it's happy to jack up or ignore my other configurations. Cheers & thanks, tim edit:  the only way to see typos is to glance at the post after posting it. ... View more