topic Re: Create large GDB? in Data Management Questions

Create large GDB?

MichaelLenore1 — Thu, 23 Feb 2017 20:14:29 GMT

I am trying to create a geodatabase with many (15000+) shapefiles in it. I have a folder with all the .shps, and I have an empty gdb. I have tried the feature class to geodatabase tool, but it hangs forever - over 45 minutes last time I tried it. What is the fastest way to do this?

Thanks!

Re: Create large GDB?

JoeBorgione — Thu, 23 Feb 2017 20:28:51 GMT

15,000 feature classes in one geodatabase? Not trying to bust your chops but I doubt you would find that to be any where near best practices. I'm trying to imagine what it would be like to find on particular feature class out of 15,000. Wait... I can't...

My suggestion would be to re-think your approach. Can you combine some of the features? Would multiple databases make sense? Especially if each contained related features? (Think Transportation, Facilities, etc each in their respective geodatabase...)

Re: Create large GDB?

George_Thompson — Thu, 23 Feb 2017 20:37:36 GMT

There are a lot of unknowns in your issue to give advice. Like jborgion‌ said 15,000 FC in a GDB is alot.

What type of geodatabase is this being loaded into? File or Enterprise?

If Enterprise, what RDBMS?

What is the geometry storage?

How large are the feature classes being loaded?

https://community.esri.com/groups/geodatabase?sr=search&searchId=487754c6-73c4-4347-9d0d-465e3c8cf636&searchIndex=0‌ https://community.esri.com/community/gis/enterprise-gis?sr=search&searchId=739a9998-8de1-4d76-abea-17fbf3a4138e&searchIndex=0‌

Re: Create large GDB?

MichaelLenore1 — Thu, 23 Feb 2017 20:39:02 GMT

Yes, it's a lot. I made a model that iterates through a gdb and performs a calculation on each feature class. In this case, the gdb I'm trying to create will have every Census block group in NY in it. I can't think of a way around creating the huge gdb.

Re: Create large GDB?

MichaelLenore1 — Thu, 23 Feb 2017 20:41:59 GMT

It's a file geodatabase. The FCs are polygons, and are small - about 40kb each.

Re: Create large GDB?

JoeBorgione — Thu, 23 Feb 2017 20:55:57 GMT

So why not have 1 geodatabase with 1 feature class of the Block Groups? What is the point of have them all their own feature class?

I just downloaded the New York Block Groups and they come as one shapefile: Cartographic Boundary Shapefiles - Block Groups - Geography - U.S. Census Bureau

Re: Create large GDB?

MichaelLenore1 — Thu, 23 Feb 2017 21:01:59 GMT

In order for the model iterator to work, each block group needs to be its own feature class; the output I need consists of a calculation for each block group.

Re: Create large GDB?

JoeBorgione — Thu, 23 Feb 2017 21:14:39 GMT

What is it you are trying to accomplish? Perhaps there is a better way than iterating through 15,000 feature classes. Python Cursors come to mind.

UpdateCursor—Help | ArcGIS for Desktop

Re: Create large GDB?

MichaelLenore1 — Thu, 23 Feb 2017 21:20:47 GMT

Maybe! I am calculating the percentage of each block group that is residential in use, using the statewide tax parcel layer’s coding. The model I made iterates through each block group, calculating various land use types, and then makes a calculation and spits out a new shapefile for each block group, appended with residential data. I finally merge them back together and export the table.

Re: Create large GDB?

JoeBorgione — Thu, 23 Feb 2017 21:32:45 GMT

Seems more like a geoprocessing exercise to me. Take a look at: https://community.esri.com/thread/190750-find-the-percentage-of-polygon-that-overlaps-other-polygon from a couple days ago.

You could combine ( intersect ) your block group polygons with the parcel polygons if that's what you mean by statewide tax parcel layer's coding.