topic Re: Re-Organization question in Data Management Questions

Re-Organization question

KristenHook — Fri, 12 Jun 2015 20:29:10 GMT

So I just got hired as the only planner in years to have GIS training in my local planning office. The GIS folders on the sever are a mess. I'm trying to re organize the system before I start major projects. My biggest issue is that if i start moving shape files around the ArcMap files will lose the data and i will have to spend forever reconnecting each layer. Is there anyway around this issue?

Thank you

Re: Re-Organization question

DanPatterson_Retired — Fri, 12 Jun 2015 20:37:30 GMT

wow...don't move anything unless you have relative paths set in each project...then you would have to move the project and the shapefiles with the same folder/path structure to a new location. This is easy if you are moving a folder and its subfolders to a new location, but if you want to consolidate shapefiles in a new location relative to the project(s) you will have to rebuild the projects by re-establishing paths. Not too onerous unless you decide to do something drastic. Perhaps you might want to provide a current structure and indicate what you want to do before code suggestions are put forward

Re: Re-Organization question

SepheFox — Fri, 12 Jun 2015 20:45:02 GMT

Can someone move this question to Managing Data?

Re: Re-Organization question

DanPatterson_Retired — Fri, 12 Jun 2015 21:18:05 GMT

Dont' know who runs this but if it is a private group, the owner will have to

EDIT

It is an open user group so either the owners of the group or the original poster has permissions to do so, it is out of the hands of Moderators.

Re: Re-Organization question

RebeccaStrauch__GISP — Sat, 13 Jun 2015 03:33:28 GMT

Kristen, I'm working on a set of tools to due just that for our organization and will share once I finish my python addin and figure out how to clearly document it for others to use. My issue is, not only are we moving from mapped drives to UNC references, I'm also working on changing all the users profile connections to a common location, and the trickier part is getting shapes to gdb, coverages to gdb, etc. it'll be a week before I'm back at it, but getting closer. Anyway, I'll be happy to share once I get at least most scenarios covered. In any case, probably can't do 100%, but with thousands of mxds, and starting with about 4000 unique broken links, if I can simplify 80% for my agency, that is a great saving.

bottom line, do planning....and keep a list of original path, new path....and old type and new type if moving shapes etc into fgdb. Oh, and I suggest keeping the old structure and creating the new structure.....force a broken-link by renaming the old folder....point to the new.....all before you delete the old structure. Lots of things can be in play when you take this on.....but with technology and best practices changing over the years, what worked great years ago can be improved just be reorganizing into a more logical structure, especially for those that may not have the history to understand why it was done like that in the past. My opinion.

oh, and I think Frank Garofalo is the owner of this group. He maybe can move it to a better location, if Kristen doesn't.

Re: Re-Organization question

DanPatterson_Retired — Sat, 13 Jun 2015 03:40:52 GMT

just glad I work in a USB stick environment

Re: Re-Organization question

KristenHook — Mon, 15 Jun 2015 12:22:22 GMT

Currently I decided to make a complete second GIS folder on our server and just called it GIS 2. There Ive been creating copy's of all the Shape files from the old system and placing them into more recognized folders. For example a data set containing all the rivers for NYS was stuck in various folders multiple times in the old system. In my new folder I have it once under NYS data. My only plan at the moment is to re-link all the Map-projects to the correctly organized files and then delete the old data but its a startlingly large task.

Re: Re-Organization question

RebeccaStrauch__GISP — Mon, 15 Jun 2015 12:39:54 GMT

Hi Kristen, make sure you compare at least a couple of the river data sets to make sure they are equivalent....same name unfortunately doesn't always mean same data....especially if the GIS system has been around for years. Minor tweaks may have been done in a copy for a particular project ....at least that has been my experience. But you will have to weigh how much any tweaks may be worth (time to investigate) vs putting the latest and greatest in and dealing with it in the future, if and when you ever need the project again.

versions of data....especially when it comes to formats like shape files can be a pain, even if there is one person dealing with it. Trying to deal with projects others have worked on is a challenge, but in the long run, cleaning up you structure like you are doing is well worth it.

Re: Re-Organization question

KristenHook — Mon, 15 Jun 2015 12:53:12 GMT

Thanks Rebecca,

So in your opinion there no other way around this? I'm not the most experienced user, particularly when it comes to data management. In fact, your comment below went slightly over my head. But I keep wondering if there's any way to speed up the process. I've already pointed to the same boundary file (that is commonly used) about a dozen times in various projects. Is there anyway to 'teach' ArcMap the new location and then send that to all the mxds files?

Re: Re-Organization question

RebeccaStrauch__GISP — Mon, 15 Jun 2015 13:27:19 GMT

Short answer, yes there is an easier way. I have a program wriitten to do just that for my many broken links....it will work easier for yours. I'm out if the office this week but can send you a preliminary addin ne t week with some explanation. i need to handle many data formats, and each needs to be handled a little differently (tables, rasters, ArcSDE...ETC) which is overkill for you.

But, besides reorganizing, once you gave that done, I would create a file geodatabbases (fgdb) for each group, or however you want to organize it, then import the shapes into the gdb. You can then fix the projects with the shapes to get up and running and make sure things are correct. But long term you can then change them to point to the fgdb once my program is ready.

THe programs will use the "walk" feature so it can process all mxd in a folder and sub folders, and will be able to read in a comma delimited (csv) file to read in old paths, new paths .... And old format, new format.

if it is just changing paths to shapes only, the process can be pretty straightforward, so if you can't wait until next week, others can help with the walk thru folders. And if you search geonet, there are many threads for replace Workspace, etc. here is the help page

ArcGIS Help (10.2, 10.2.1, and 10.2.2)

Re: Re-Organization question

KristenHook — Mon, 15 Jun 2015 13:32:00 GMT

Thank You!

Ill try to get a simplified version of what your doing up and running. Ill check back next Monday if It fails but hopefully not considering the file mess is simpler then yours

Re: Re-Organization question

WesMiller — Mon, 15 Jun 2015 13:36:50 GMT

As you identify\download\create the most current data begin loading the data into Local Government Information Model | ArcGIS for Local Government then start pointing your most used projects and any new projects you create to the LGIM database. As a side note i think you may have to have an advanced license to create this data structure, but it's agreat place to start.

Re: Re-Organization question

RebeccaStrauch__GISP — Wed, 24 Jun 2015 20:17:58 GMT

Hi Kristen, I'm back in the office but working thru a few end of reg year deadlines this before getting back on my project. I was wondering how your project was coming along. I "followed" you on Geonet, so if you want to follow me and send me a direct message, I can see if I can explain what I have that might help you. It may get too much for this list until I have things cleaned up some. It's been a process for me to make exceptions for all the different types of data I have. Your stuff may be much simpler.