topic Re: ArcSDE Raster moving is Hard Disk Drive Space enough? in Data Management Questions

ArcSDE Raster moving is Hard Disk Drive Space enough?

NancyGnanicys — Thu, 23 Aug 2012 19:30:51 GMT

Here is what I am trying to accomplish:
All raster aerial imagery and vector files are in one SDE geodatabase currently on the C Drive. Since this is a back up issue, I decided it'd be best to move all imagery to a separate SDE geodatabase. I have already created a new sde gdb and put an aerial photo on there. However, the C Drive where both SDE gdb are located is getting full.
1-Would there be web application/desktop user performance issues if I put gdb on two different drives?
2-Is there server license limitations on multip gdb on different drives?
3-Would I run a post install to move my raster gdb to the new drive space?

I have 3 Hard Disk Drives on my GIS server with the following space:
C Drive: Used 31.1 GB Free 17.4 GB
E Drive : Used 426 GB Free 891 GB
F Drive: Used 1.12 GB Free 28.1 GB

SQL Server 2003
ArcSDE 10.0
Windows XP

Rather new with SDE server technology for Arcgis, and still trying to understand the correlations amongst it all.

Re: ArcSDE Raster moving is Hard Disk Drive Space enough?

VinceAngelo — Thu, 23 Aug 2012 22:48:49 GMT

1) The answer to performance questions usually depends on unprovided information.
Let me try to answer by referring to a system I do have all the information on:

My home PC has four internal disks. I doubt I could get into your current fix, because
I limit the C: drive to the operating system and non-GIS, non-database software with
a raw allocation of 30Gb (26Gb formatted). The remainder of the 1000Gb disk is broken
into a number of partitions (including two Linux installs). Two more 1000Gb (7ms seek)
15Krpm ESATA disks are configured as a striped pair, and the majority of the database
files go on them. Finally, I've got a 64Gb SSD (0.7ms seek), where I park temporary
files and the temp tablespace (and maybe some index tablespaces, as time goes on).

First of all, I generally use all the physical disks I have, but I do so with an eye toward
performance -- I'm not using my 3Tb (15ms seek) backup disk for database operations
(yet), and doubt I would until I had exhausted a lot more space.

But I generally won't use more than one partition on any one physical disk (but since
I don't use the C: volume for software, 😧 is a safe location). There is usually a strong
*positive* impact of placing datafiles on different disks, but only if the volumes are
on different spindles (you can degrade performance splitting blocks across a single
controller [artificial fragmentation])

2) Disk configuration has no impact on licensing, though ArcSDE Personal and Workgroup
are subject to size limitations (total), which is governed by Microsoft's limitations on
SQL-Server Express.

3) There are a number of ways to divide data, but first you have to determine which
physical disk is the fastest, and where all your current partitions reside. I'm not as
familiar with non-Enterprise class databases, but SQL-Server stores databases in
filegroups, and you can take the database offline (detach), move the files, and reattach
without changing anything (except taking ArcSDE offline). If you do need a new database,
it would require post-install (and some DBTUNE configuration to get the best possible
storage). In many ways, you'd be best off moving the non-raster data into a different
database as well (and evetually deleting the original), just to make sure you can recover
the storage allocated to the filegroup.

Finally, I wouldn't recommend clogging up a new database with raster files. ArcGIS Server
(and ArcGIS Desktop) work much faster with GeoTIFFs in Mosaic Datasets. Yes it's a new
technology, but if you need to learn something anyway, better to learn the replacement, eh?
You probably want your pyramid images on the fastest part (inner spindle) of the fastest disk,
so that will probably give you some juggling to do.

- V

BTW: The SQL-Server release after 2000 was 2005, and after that 2008 -- there was no 2003.

Re: ArcSDE Raster moving is Hard Disk Drive Space enough?

NancyGnanicys — Fri, 24 Aug 2012 12:13:02 GMT

I got the two mixed up, it is SQL Server 2005, and windows server 2003. Thank you for the suggestions and advice Vangelo. I will have to read up more about your answers to my question 1, I'm not quite there yet knowledge base wise :(. Your suggestions to move vector as well makes sense. So, if I separate out the vector to it's own GDB after I move the raster files (and the current aerial imagery on there is a mosaicked dataset), I would have two copies of all my data temporarily until I delete the old geodatabase right?

Would I just use SQL server management software to delete the old geodatabase or is there some kind of "reverse post install process" to do so?

Where could I look to find out which disk is the fastest?

I'm getting very nervous right now as this is one of the first tasks I'm trying to accomplish on SDE and don't want our users to be out of GIS that they rely on everyday, talk about alot of phone calls that day!

Re: ArcSDE Raster moving is Hard Disk Drive Space enough?

VinceAngelo — Fri, 24 Aug 2012 13:03:51 GMT

Frankly, the disk sizes involved scare me. You have what appears to be an 80Gb disk
(in two partitions) and a 1.5Tb disk. Given the age of the system, it seems likely that the
large disk is also a slow one, possibly as much as 3x the seek time. Folks are probably
going to notice if you move the data to 20ms seek disk.

You'd need to take the machine offline, open the case, and get part numbers off the disks
(probably by removing them) to determine the hardware specs. You could probably
find a disk performance test suite to confirm the disk characteristics.

Transferring the files to the big disk would probably be trivial (stop arcsde service, detach
database, copy the datafiles to the big disk, reattach, start service), which would give
you a way to confirm the performance change. But that will put you in a quandry -- there's
enough space to restructure the database back onto the original disk, but it doesn't help
with the running out of space issue.

You could get a new fast disk, but there may not be space or power (or a compatible disk
transfer protocol) on the system, then you'd need a new card for the eSATA, and OS drivers,...
In a perfect world, you'd be able to spend ~$1K for a *really* nice 64-bit host with new fast
disks, and be able to transfer the data there, but then you'd need to update the database,
and that can become an all-encompassing task as well.

The first things you need to do are to determine the hardware specs, volume-disk layout,
and actual storage used on the disks by your database. If the big disk is slow, and the
other volumes are on one spindle, then you probably don't really have enough resources
to accomplish your task. 😞

- V