IT and justification for large geospatial datasets

[LaurieWilliams]

Hi All,

My IT department thinks the DPW GIS staff are using too much network space with our LiDAR and other large spatial datasets. Does anyone have a manager-level document that shows these datasets are the norm, and increasingly so? I'm finding some decent stuff on the internet but much is too involved, and in some cases, too old. I could synthesize what I'm finding, but I'd prefer an independent journal article or research paper to back up my case: that we are not space hogs, that the industry is rapidly changing with increasing use of large datasets, and that IT needs to adjust and serve our needs. Oh, and also that my county is behind the times and needs to catch up!

Thanks,
Laurie

[JasonVerachtert]

I do not recall seeing any kind of documentation that can be used for justification for disk usage, but the way we manage it in our county is we simply include funds in our budget to help our IT department purchase extra hard drive space. 

We have over 20 years of orthophotos, about 19 years of historic parcel maps as PDFs, 10 years of ArcSDE archived cadastral data sets and countless other large collections of files we have created or purchased over the years including LiDAR.  We are using almost 9 TB of disk space and as long as we continue to help pay for our share of new disk space IT does not care how much we fill up. 

This way we just need to be able to continue to show our county board how useful the all of the data is to the rest of the county so they do not cut that line item in our budget.

[KenCarrier]

Many times the way GIS people think about storage is completely different than how an IT person thinks of storage.

IT people tend to think of storage as "oh my backups are going to take forever now", whereas GIS people think of it as well when I need it, it will be there and it is going to be backed up.

You might want to consider trying to understand what is about the large amount of data that they have a problem with. If it is backups, possibly devising a strategy for those network locations that need active backups versus passive might help ease some of that tension.

Another option many IT departments are going to is external drives for large amounts of data storage with a direct connect to the server. This keeps it off of disk and usually is not backed up like most other folders or drives.

I agree with the previous post that showing the benefits of the data is always key in any discussion. Although many times an IT person does not use the data directly and therefore has difficulty in understanding why the data is still so large.

If you can get down to the real reason of why storage is a problem I think it will make it easier to make a case with IT. In all honesty storage does not cost that much anymore unless you are talking about SSD(Solid State Hard Drives).

If you determine the real reason post back and I am sure others including myself could help you structure an educated discussion with your IT staff that might change their opinions.

With imagery and lidar, are you considering storing these using raster mosaic and las datasets inside of your geodatabase? Is the data currently in SDE or just on the network somewhere?