topic Re: workspace as network drive in Python Questions

workspace as network drive

AmyKlug — Tue, 30 Jul 2013 19:57:19 GMT

Is it possible to set the the workspace to a network drive and if so how would you write it?

When using arcpy.da.walk and workspace = "U:/" , it froze up my computer...or it was just really slow.

Re: workspace as network drive

RhettZufelt — Sat, 11 Dec 2021 22:58:56 GMT

Sure, I have done it a couple ways:

Assuming the networked computer is named "server1" and has a share named "U" and/or is mapped to drive U:

import arcpy

workspace = arcpy.env.workspace = r"U:"    ## don't put the slash at the end.....
workspace = arcpy.env.workspace = r"\\server1\U"

either works.

Keep in mind, though, setting it to the base level, by default, will drill down into all directories in there as well, so it can take a long time if there is much there. Better to set it to a working folder on the drive. Mine is still running as it is categorizing my entire drive.

R_

When topdown is True, the dirnames list can be modified in-place, and Walk() will only recurse into the subworkspaces whose names remain in dirnames. This can be used to limit the search, impose a specific order of visiting, or even to inform Walk() about directories the caller creates or renames before it resumes Walk() again. Modifying dirnames when topdown is False is ineffective, because in bottom-up mode the workspaces in dirnames are generated before dirpath itself is generated.

Re: workspace as network drive

JamesGirvan — Wed, 31 Jul 2013 13:56:35 GMT

I am interested in doing something similar. However, my reference needs to be to a SQL Enterprise GDB feature class within a feature dataset. The server is somcty06, FDS is named Cadastre and the FC is Parcels. I am trying to create a parcel search tool available to all of my users that will require entering a municipal name from a drop down list then freely entering text for the block and lot numbers. The tool should then zoom to the selected parcel or return No Parcel Found message.

Re: workspace as network drive

RhettZufelt — Sat, 11 Dec 2021 22:58:58 GMT

James,

You would go into ArcCatalog, Database Connections, Add Database Connection and set it up to your database.

Then you can reference as such:

indatabase = "Database Connections\\Connection to RCES_P.sde\\SIS_ADM.IMAGES"  # can use the database connection

indatabase = "C:\\Users\\rkzufelt\\AppData\\Roaming\\ESRI\\Desktop10.1\\ArcCatalog\\Connection to RCES_P.sde\\ARCUPDATE.MAPS"  #or the database connection file

Re: workspace as network drive

AmyKlug — Sat, 11 Dec 2021 22:59:01 GMT

Sure, I have done it a couple ways:

Assuming the networked computer is named "server1" and has a share named "U" and/or is mapped to drive U:
import arcpy

workspace = arcpy.env.workspace = r"U:"    ## don't put the slash at the end.....
workspace = arcpy.env.workspace = r"\\server1\U"
either works.

Keep in mind, though, setting it to the base level, by default, will drill down into all directories in there as well, so it can take a long time if there is much there. Better to set it to a working folder on the drive. Mine is still running as it is categorizing my entire drive.

R_

Thanks, Yes..Unfortunaltey I will have to drill down into all directories.

Another question..... I have been trying to write my results to a geodatabase table and keep getting an error that the size of the cell and path form my list aren't the same size. I have had sucess writing to csv.....but I'm pretty sure there should be a way to write to GDB table?

Re: workspace as network drive

RhettZufelt — Wed, 31 Jul 2013 16:53:03 GMT

To write to the FGDB table you can do it several ways. If you are modifying data in an existing row, you could use updateCursor, or calculatefield.
If you are adding new rows to the table, you want InsertCursor
http://resources.arcgis.com/en/help/main/10.1/index.html#//018w00000014000000
http://resources.arcgis.com/en/help/main/10.1/index.html#//00170000004m000000

R_

Also, keep in mind that the da.walk with topdown set as true, you can remove any of the directories from the walk list. you say you need to drill down, but if you don't need to drill into ALL subdirs, this can be done with da.walk.

The bottom example here http://resources.arcgis.com/en/help/main/10.1/index.html#//018w00000023000000 shows that with the dirnames.remove statement.

Re: workspace as network drive

AmyKlug — Sat, 11 Dec 2021 22:59:04 GMT

Forgot to tell you I am using the insert cursor method and was getting this message:

"TypeError: sequence size must match size of the row"

I just got it to work, I needed to use arcpy.Describe before appending to my list

>>> import arcpy
>>> import os
>>> workspace = r"H:/LIBTest"
>>> newtable = r"H:/TableList.gdb/GISLibrary"
>>> list = []
>>> for dirpath, dirnames, filenames in arcpy.da.Walk(workspace):
...     for filename in filenames:
...         descr = arcpy.Describe(os.path.join(dirpath, filename))
...         list.append([descr.catalogPath, descr.name])
...         
>>> with arcpy.da.InsertCursor(newtable, ['Path', 'FileName']) as insert:
...     for f in list:
...         insert.insertRow(f)
...

Re: workspace as network drive

AmyKlug — Wed, 31 Jul 2013 18:52:55 GMT

So I keep getting errors that certain tools in a toolbox don't exist. I can't get past the first one which is the "con" tool in the spatial analyst toolbox so I made sure I had spatial analyst extension checked to no avail. Doesn't seem like that should make a difference?

Re: workspace as network drive

RhettZufelt — Wed, 31 Jul 2013 19:34:38 GMT

No, you would just get a not licesed error if that were the case.

If you have too much security, you will get unable to run errors as they rely on ActiveX being enabled now.

If it is not finding it, even after a re-boot of the computer, you may need a re-install.

Does it not find them in ArcMap or ArcCatalog?

That is a weird one, as if it is showing you the list under Conditional, then it has to be seeing the Spatial Analyst Tools.tbx...

R_

Re: workspace as network drive

AmyKlug — Wed, 31 Jul 2013 20:17:44 GMT

I just ran the tool on my C drive and it's telling me another file does not exist (a table in a access .mdb). Is there a way to write in the code for it to continue on if it can't find the file?

EDIT: The access table was linked and does not exist in the parent database

I am going to try running in ArcCatalog next. The folder the code is failing on is an archive folder and contains toolboxes from Arc 9.3 but "con" exists in both versions so I don't know. I would still like to skip those errors, not sure if try/except would work?

Re: workspace as network drive

RhettZufelt — Wed, 31 Jul 2013 20:39:48 GMT

First, are you running the toolbox tool, or running code?

If coding, there is normally a way to tell it to move on if it can't find the file. However, since you have to tell it what file(s) to use, normally, you just don't pass a non-existent file to it.

also, not sure why you are getting that error as the only valid inputs for the con tool are rasters, so it should never be looking for a table, yet alone in an mdb.

R_

Re: workspace as network drive

AmyKlug — Wed, 31 Jul 2013 21:15:59 GMT

I am creating a table of filenames, paths and data type using arcpy.da.Walk in a script tool. When it gets to the "con" tool it says it can't find it (even though it's there and usable) and the code quits. I just tried it in ArcCatalog and same result. Then copied the "con" tool from the 10.1 system toolbox into another toolbox, ran the script...... it failed. When I deleted the "con" tool from the toolbox, the script worked fine. I tested it with some other spatial analyst tools and had no problems. I would really like to have my code skip the data it can't find.

arcpy.da.Walk thinks the MDB's are persoanl GDB's I guess. I edited my post above that the table it could not find was linked and the tool could not find it because it had been deleted from the original database.

Re: workspace as network drive

RhettZufelt — Sat, 11 Dec 2021 22:59:06 GMT

Could put something like this in there:

if not arcpy.Exists(filename):
     continue

this way, if the filename doesn't exist, it will move on to the next iteration.

also, keep in mind that when using da.walk, the actuall path to and including the filename is

os.path.join(dirpath,filename) not just "filename", especially important if the file is not in your set env.workspace.

there are some posts here that also show how to code for ( if not in something, then do something) http://forums.arcgis.com/threads/89638-List-features-in-multiple-workspaces-then-convert-to-point-BUT-ignore-some-paths that might come in handy.

as far as the script tool issues, can't help you there, I do all my stuff stand alone.

R_

Re: workspace as network drive

AmyKlug — Wed, 31 Jul 2013 22:06:32 GMT

Code gets hung up using stand-alone too, it just doesn't recognize the "con" tool for some reason. I will try .Exists

Re: workspace as network drive

RhettZufelt — Wed, 31 Jul 2013 22:16:38 GMT

Like I said, I don't make script tools, but I do use con now and then, I just use arcpy for it.

http://resources.arcgis.com/en/help/main/10.1/index.html#//009z00000005000000

R_

Re: workspace as network drive

AmyKlug — Sat, 11 Dec 2021 22:59:09 GMT

Your suggestion to use .Exist worked perfectly.........

Still very curious why it errored that "con" does not exist, we may never know

The almost there product:

import arcpy
import os
workspace = r"U:"
newtable = r"H:/TableList.gdb/GISLibrary"
list = []
for dirpath, dirnames, filenames in arcpy.da.Walk(workspace):
    if not arcpy.Exists(filenames):
        continue
        for filename in filenames:
            descr = arcpy.Describe(os.path.join(dirpath, filename))
            list.append([descr.catalogPath, descr.name, descr.dataType])
with arcpy.da.InsertCursor(newtable, ['Path', 'FileName', 'Type']) as insert:
    for f in list:
        insert.insertRow(f)

Re: workspace as network drive

AmyKlug — Sat, 11 Dec 2021 22:59:12 GMT

Your suggestion to use .Exist worked perfectly.........

Still very curious why it errored that "con" does not exist, we may never know

The final product:

import arcpy
import os
workspace = r"U:"
newtable = r"H:/TableList.gdb/GISLibrary"
list = []
for dirpath, dirnames, filenames in arcpy.da.Walk(workspace):
    if not arcpy.Exists(filenames):
        continue
        for filename in filenames:
            descr = arcpy.Describe(os.path.join(dirpath, filename))
            list.append([descr.catalogPath, descr.name, descr.dataType])
with arcpy.da.InsertCursor(newtable, ['Path', 'FileName', 'Type']) as insert:
    for f in list:
        insert.insertRow(f)

The code above actually did not work the next day....but this should be correct (although this code is now saying all tools don't exist, might be I need to use something different than arcpy.Exists(filename) or how I defined the environment/workspace or setting my parameter as a workspace but still working that out) :

import arcpy
import os
p = arcpy.GetParameterAsText(0)
arcpy.env.workspace = p
newtable = r"H:/TableList.gdb/GISLibrary"
list = []
for dirpath, dirnames, filenames in arcpy.da.Walk(p):
    for filename in filenames:
        if arcpy.Exists(filename):
            arcpy.AddMessage("adding file to list")
            descr = arcpy.Describe(os.path.join(dirpath, filename))
            list.append([descr.catalogPath, descr.name, descr.dataType])
        else:
           arcpy.AddMessage('File   ' + filename + 'does not exist')
        continue 
with arcpy.da.InsertCursor(newtable, ['Path', 'FileName', 'Type']) as insert:
    for f in list:
        insert.insertRow(f)

EDIT: this code only works when i chose a geodatabase as input p. if I used a folder (that contained geodatabases) as input all of my feature classes were tagged as "do not exist)

Re: workspace as network drive

RhettZufelt — Thu, 01 Aug 2013 18:03:46 GMT

import arcpy import os p = arcpy.GetParameterAsText(0) arcpy.env.workspace = p newtable = r"H:/TableList.gdb/GISLibrary" list = [] for dirpath, dirnames, filenames in arcpy.da.Walk(p):     for filename in filenames:         if arcpy.Exists(os.path.join(dirpath, filename)):             arcpy.AddMessage("adding file to list")             descr = arcpy.Describe(os.path.join(dirpath, filename))             list.append([descr.catalogPath, descr.name, descr.dataType])         else:            arcpy.AddMessage('File   ' + filename + 'does not exist')         continue  with arcpy.da.InsertCursor(newtable, ['Path', 'FileName', 'Type']) as insert:     for f in list:         insert.insertRow(f)

EDIT: this code only works when i chose a geodatabase as input p. if I used a folder (that contained geodatabases) as input all of my feature classes were tagged as "do not exist)

Suspect the p variable is not getting passed correctly as the da.walk works fine for FGDB or the folder. Especially since the way you have it, it will only evaluate to False if you end up with an empty filenames list, which means that either p is the wrong path, or there is NODATA in there at all. Basically, da.walk makes a list, then you iterate through it and say "This file exists, so I put it in my list, if this file doesn't exist, then: so, the only way this would not be true is if the list is empty, so deleting that line here would have the same effect (minus the error of course on empty list). Also, the continue statement you have is not needed. Basically, it is the last line in the for loop. So, you are telling it, that when it is finished will all the code for the first item in the list, continue to the next. this is the default behaviour. Don't think it would make a difference, but possibly telling it to skip one?

perhaps you need to set p = to:
p = str(arcpy.GetParameterAsText(0)) or something. Like i said, don't tool script tools, so not sure how a single parameter gets passed.

can print p
and type(p) to make sure it is a string and the value is that actual path/name to the folder you want to walk through.

The code above actually did not work the next day....but this should be correct (although this code is now saying all tools don't exist, might be I need to use something different than arcpy.Exists(filename) or how I defined the environment/workspace or setting my parameter as a workspace but still working that out)

for arcpy.Exists:

 Tests for the existence of feature classes, tables, datasets, shapefiles, workspaces, layers, and files in the current workspace

Don't think it was intended to look for "Tools", of course, a toolbox is a .tbx "file" so it should find them. However, I'm a little confused here. I don't do script tools, but I know that you have to tell the script what tool to use, where the toolbox is, etc. and I see no reference what so ever in your script to any tools or toolboxes. where are you defining them, and where are you calling the con tool that is giving you the error?

The spatial analyst toolbox should be here:
c:\program files\arcgis\desktop10.1\ArcToolbox\Toolboxes\Spatial Analyst Tools.tbx

R_

Shoot, just saw something else going on here. Try the change in red above. you are checking to see if a file exists, da.walk reports filenames and directories sepratly, so you need to pass the path and name to locate the file. Just like in your describe statement.

Re: workspace as network drive

AmyKlug — Sat, 11 Dec 2021 22:59:15 GMT

I tried using:

if arcpy.Exists(os.path.join(dirpath, filename)):

In an earlier version before the code was this far along. But this REALLY did the trick. It only errors the "con" tool now (very strange). Using a parameter with a script tool I can just select the network drive.....might have to do more than one.
Thanks for all your help!

Here it is:

import arcpy
import os
p = arcpy.GetParameterAsText(0)
arcpy.env.workspace = p
newtable = r"H:/TableList.gdb/GISLibrary"
list = []
for dirpath, dirnames, filenames in arcpy.da.Walk(p):
    for filename in filenames:
        if arcpy.Exists(os.path.join(dirpath, filename)):
            arcpy.AddMessage('adding ' + os.path.join(dirpath, filename)  + ' to list............')
            descr = arcpy.Describe(os.path.join(dirpath, filename))
            list.append([descr.catalogPath, descr.name, descr.dataType])
        else:
           arcpy.AddMessage('ERROR: ' + os.path.join(dirpath, filename) + ' does not exist')
        continue 
with arcpy.da.InsertCursor(newtable, ['Path', 'FileName', 'Type']) as insert:
    for f in list:
        insert.insertRow(f)

Re: workspace as network drive

AmyKlug — Thu, 01 Aug 2013 19:53:47 GMT

Don't think it was intended to look for "Tools", of course, a toolbox is a .tbx "file" so it should find them. However, I'm a little confused here. I don't do script tools, but I know that you have to tell the script what tool to use, where the toolbox is, etc. and I see no reference what so ever in your script to any tools or toolboxes. where are you defining them, and where are you calling the con tool that is giving you the error?

someone had made a copy of another server and put it on the U drive of the server I am cataloging, so we have a toolbox in the U(current server):/otherserver/......../program files/arcgis/........./toolbox. arcpy.da.walk does not recognize that tool for some reason. I tried the code on other toolboxes and same result. What is happening is that it is not writing the CON tool to my library table like it does with the other tools, i am not trying to use the con tool.