ExportXYV with standalone script doesn't work

[JeffreyPardus]

I'm so close - it writes the txt file or csv, writes the field names, but doesn't write any of the field data.  Where am I going wrong?  These are coverages, and it works using arctoolbox from within

print "Connecting to your project, please wait..."
import arcpy
from arcpy import env

mxdPath = r'C:\Projects\test\test-03112014.mxd'
md = arcpy.mapping.MapDocument(mxdPath)
ws = "C:\\Projects\\test\\tables\\"
df = arcpy.mapping.ListDataFrames(md)[0]
targetGroupLayer = arcpy.mapping.ListLayers(md, "interchange_bolivar", df)[0]
tables = arcpy.ListTables()
for tables in targetGroupLayer:
    fieldname = [f.name for f in arcpy.ListFields(tables.dataSource)]
    td = tables.dataSource
    ext = ".txt"
    filename = (ws + tables.name + ext)
    try:
         arcpy.ExportXYv_stats(td, fieldname, "SPACE", filename, "ADD_FIELD_NAMES")
    except:
         print arcpy.GetMessages()
del md

Here is the error message:

Failed to execute (ExportXYv).
Failed at Wed Apr 02 15:24:01 2014 (Elapsed Time: 1.00 seconds)
Executing: ExportXYv C:\Projects\test\Mine_Maps\BMR_DATA\Bolivar\data\db\db\region.r110 FID;Shape;AREA;PERIMETER;R110#;R110-ID SPACE "db region.r110.txt" ADD_FIELD_NAMES
Start Time: Wed Apr 02 15:24:01 2014
Running script ExportXYv...
Writing Results to ASCII File....

Traceback (most recent call last):
  File "c:\program files (x86)\arcgis\desktop10.1\ArcToolbox\Scripts\ExportXYV.py", line 159, in <module>
    export = setupXYV()
  File "c:\program files (x86)\arcgis\desktop10.1\ArcToolbox\Scripts\ExportXYV.py", line 49, in setupXYV
    outFieldNames = outFieldNames)
  File "c:\program files (x86)\arcgis\desktop10.1\ArcToolbox\Scripts\ExportXYV.py", line 129, in exportXYV
    formatValue = LOCALE.format(localeDict[field], value)
  File "C:\Python27\ArcGIS10.1\lib\locale.py", line 190, in format
    return _format(percent, value, grouping, monetary, *additional)
  File "C:\Python27\ArcGIS10.1\lib\locale.py", line 196, in _format
    formatted = percent % value
TypeError: not all arguments converted during string formatting

Here's the output of TD, fieldname, and filename:

TD - [u'FID', u'Shape', u'AREA', u'PERIMETER', u'R104#', u'R104-ID']
fieldname - C:\Projects\test\Mine_Maps\BMR_DATA\Bolivar\data\a\a\region.r104
filename - a region.r104.txt

[T__WayneWhitley]

Have to filter out the geometry Shape field - that is what it's tripping on.  Maybe the error doesn't indicate that and it's because if you look closely at the script (right-click on the original script tool, click Edit to view the source code in your default editor) you'll see the initial inputFields parameter (before the DA.SearchCursor is opened) is defined as:

inputFields = [ssdo.oidName, "SHAPE@XY"] + fieldList

The important thing to note is the geometry field is NOT fed in directly, but as you can see the numerical values for the XY coords are fetched instead.  So what is fieldList?...from the tool interface it is a filter of the available fields and SHAPE is disallowed; in other words, you cannot feed in SHAPE and indeed if you make a copy of the tool into your own tool directory (so that you can see the parameter setups), you can click on the Value Field (field data type) parameter and investigate the Parameter Properties.  Clicking on the ellipsis, a field filter is applied where the type, Geometry (Shape), is not checked (then, filtered out).

Hope that makes sense now.  And, yes, out of curiosity I changed the Field Filter in my toolbox copy (thankfully you cannot muck-up the system tool setting!), included the Geometry field, executed the tool with the Shape field selected, and I get the same failure message.

The allowed field types are (by the Field Filter list applied):
Short, Long, Float, Double, Text, Date, OID, GlobalID

...disallowed:
Geometry, Blob, Raster, GUID, XML

Wayne

[JeffreyPardus]

Thank you Wayne, that took me one step closer.

fieldname = [f.name for f in arcpy.ListFields(tables.dataSource,)] became fieldname = [f.name for f in arcpy.ListFields(tables.dataSource, "*-ID")] and now it lists the data.  Three more questions:

1.  Is there any way to eliminate the X / Y coordinates from being written?
2.  This exports all of the attribute tables for all of the coverages.  I have a selection created based off of any of these coverages within or touching a shapefile - how would I write just those?
3.  How can I write these all to one file?

Thanks again for your time.

Jeffrey

[T__WayneWhitley]

I guess I don't see why you need to 'backward engineer' a solution when there are other ways you can more easily do what I think you want to do.... Did you pick the spatial utilities tool to have access to the write to txt aspect of it?  Because Python will readily do this without hacking at the ExportXYv script tool.  Since now you want to 'strip off' the coordinate fields added by the tool, did you know you can simply use one of the tools listed below that does not generate the XY coordinates in the first place?...also, if you want to eliminate other fields, make a Table View of an fc selection with a 'view' of only your desired fields to export or append selected rows to a common destination table, if that is what you're aiming for?  You can loop on your various tables (I guess they're loaded with selections in your mxd?)

Allow me to make a few suggestions (see links below) from the web help.  It may make more sense to look at some of their py examples at those pages.  I don't think these will convert directly to a txt file, but no matter - you can easily make the jump to txt format from an intermediate output.  At 10.2.x there are Excel tools too, if that helps.

Table to Table (Conversion)
Desktop » Geoprocessing » Tool reference » Conversion toolbox
http://resources.arcgis.com/en/help/main/10.2/index.html#//001200000027000000

Copy Rows (Data Management)
Desktop » Geoprocessing » Tool reference » Data Management toolbox
http://resources.arcgis.com/en/help/main/10.2/index.html#//0017000000n4000000

Table To Excel (Conversion)
Desktop » Geoprocessing » Tool reference » Conversion toolbox
http://resources.arcgis.com/en/help/main/10.2/index.html#/Table_To_Excel/001200000054000000/


...and, if this makes for an easier scripting experience, here's the help on Make Table View (see below), works on either fc or tbl, with a good example on how to filter other fields as desired (also, a parameter is provided in order to create subset selections for export as well).  This doesn't do the export, but set's up the view of the preprocessed output, so to speak.

Make Table View (Data Management)
Desktop » Geoprocessing » Tool reference » Data Management toolbox
http://resources.arcgis.com/en/help/main/10.2/index.html#//00170000006v000000


...or if you're just curious about Python and writing to txt, see this:
https://docs.python.org/2/tutorial/inputoutput.html#reading-and-writing-files

[JeffreyPardus]

Maybe I'm going about it all wrong.  I have a shapefile that I create.  I need to get just the selected attribute information from multiple layers as a txt or csv.  The selected information would be anything within 1000 feet of the shapefile.

The way i'm looking at it is this:

Combine all the feature classes I need to reference into a shapefile. (Merge)
Then I'd do a selection from the merged shapefile to the shapefile I created earlier (select layer by location)
finally I'd export the information (copy rows)

Am I overcomplicating this?

[T__WayneWhitley]

Without knowing the details or final objective of your task, it sounds like you are back on track.  You aren't required to merge but if that makes it easier for you, it isn't too processing-intensive for your computer, and you don't mind the 'extra' intermediate data, then go for it.

By the way, if I remember what I've read about the ExportXYv utility, I think it is geared for the purpose of 'interoperability' --- in other words, a means to an end purpose of generating output for another app outside of ArcGIS.  It's a way to make the data more sensible in a 'lowest common denominator' form.  (If I haven't overly complicated this myself.)

Good luck,
Wayne

[JeffreyPardus]

It's sloppy, and I probably have redundant code in it, but I finally got it to work.

Step 1.  Check the group layer "BMR" for group layers that start with "interchange".  If they do (and they will), check the sublayers and export them to individual shapefiles.

Step 2. Add fields for the coal seam and the unique ID that goes along with the attribute table.  I tried the new Alter Field, but it has to be in a GDB for it to work.  Since they're shapefiles at this point I AddField and add the file name (which is the coal seam name) and add a field for the unique index.  Also the twist is that field is named differently per shapefile, so I have it look for the unique identifier (*_ID) and use CalculateField to enter that information.

Step 3. Combine all of those shapefiles into a single shapefile.

Step 4. Create a buffer around the permit boundary (which was created prior) and combine those two into a shapefile.

Step 5. Create a GDB for the two shapefiles that were created.

Step 6. MakeFeatureLayer for the two shapefiles, do a select layer by location, copy those features into the GDB that was created, and finally use ExportXYV_stats to export to a text file that can be read by Access in relation to a table for referencing.

There's 2 pieces that are bashed out, I have to relink some things when I bring it over since they were lyr files.  This should give you an idea on what's what, sorta.  Like I said, it's ugly, but I know what the variables are, and it works flawlessly.

import arcpy
import string
import os
import fnmatch
from arcpy import env
# Set the workspace
print "Fixing annotation links, please wait..."
ws = r'c:\projects\test\shapefiles\testshp\\'
arcpy.env.workspace = ws
mxdPath = r'C:\Projects\test\test-03112014.mxd'
md = arcpy.mapping.MapDocument(mxdPath)
df = arcpy.mapping.ListDataFrames(md)[0]
targetGroupLayer = arcpy.mapping.ListLayers(md, "BMR", df)[0]
for layer in targetGroupLayer:
    if layer.isGroupLayer:
        if layer.name.startswith("annotation"):
            anno, quadname = layer.name.split("_")
    if layer.name.endswith(".wpa"):
        s = layer.name
        sanno = s[s.find("-")+1:s.find(".")]
        bmrpath = r'C:\projects\test\mine_maps\bmr_data\\'
        oldpath = bmrpath + quadname + "\maps_" + quadname
        newpath = bmrpath + quadname + "\data\\" + sanno 
        #layer.findAndReplaceWorkspacePath(oldpath, newpath, True)
        #print "Updating annotation from " + sanno + " to " + newpath
print "making interchanges into shapefiles, please wait..."
for layer in targetGroupLayer:
    if layer.isGroupLayer:
        if layer.name.startswith("interchange"):
            interchange, quadname = layer.name.split("_")
            for subLayer in layer:
                coalseam = subLayer.name.split(" ", 1)[0]
                fn = ws + quadname + "_" + coalseam + '.shp'
                arcpy.Merge_management(subLayer,fn)
                print "Creating " + fn
md.save()
del md
print "fixing shapefiles for proper format, please wait..."
fcs = arcpy.ListFeatureClasses()
for shp in fcs:
    seam = shp[shp.find("_")+1:shp.find(".")]
    arcpy.AddField_management(shp, "seam_wID", "text")
    arcpy.AddField_management(shp, "Cards_wID", "text")
    arcpy.CalculateField_management(shp, "seam_wID", "'%s'" % (seam), "PYTHON",)
    print(seam), "processed"
    tables = arcpy.ListFields(shp,"*_ID")
    for table in tables:
        oldid = table.name
        arcpy.CalculateField_management(shp, "Cards_wID", "!"+oldid+"!", "PYTHON",)
print "coal seams added and cards info fixed"
print "creating massive shapefile for interchanges, please wait..."
ws = r'c:\projects\test\shapefiles'
arcpy.env.workspace = r'c:\projects\test\shapefiles\testshp'
fcs = arcpy.ListFeatureClasses()
sfn = r'c:\projects\test\shapefiles\testshp\interchange.shp'
arcpy.Merge_management(fcs,sfn)
arcpy.env.workspace = ws
print "creating shapefile of buffer, please wait..."
arcpy.Buffer_analysis("TomsRun.shp", "TomsRunBuffer1000.shp", "1000 Feet", "Outside_Only", "ROUND", "NONE", "")
print "combining permit buffer and boundary please wait..."
ps = "TomsRun.shp"
bs = "TomsRunBuffer1000.shp"
os = os.path.join(ws,"\\testshp\Supershape.shp")
arcpy.Merge_management([ps,bs],os)
testws = r'c:\projects\test\shapefiles\testshp'
arcpy.env.workspace = testws
fGDBpath = testws + "\\"
gdbname = "shpfGDB.gdb"
arcpy.CreateFileGDB_management(fGDBpath, gdbname)
shpfGDB = r'C:\projects\test\shapefiles\testshp\shpfGDB.gdb'
arcpy.MakeFeatureLayer_management("Supershape.shp", "permit_lyr")
arcpy.MakeFeatureLayer_management("interchange.shp", "interchange_lyr")
arcpy.SelectLayerByLocation_management("interchange_lyr", "INTERSECT", "permit_lyr", "", "NEW_SELECTION")
matchcount = int(arcpy.GetCount_management('interchange_lyr').getOutput(0)) 
if matchcount == 0:
    print('no features matched spatial and attribute criteria')
else:
    touchingpermit = shpfGDB + "\\touchingpermit"
    arcpy.CopyFeatures_management('interchange_lyr', touchingpermit)
    print('{0} interchanges that matched criteria written to {0}' .format(matchcount, touchingpermit))
featureclass = touchingpermit
filename = "C:/Projects/test/tables/interchanges.txt"
fields = [f.name for f in arcpy.ListFields(featureclass, "*_wID")]
arcpy.ExportXYv_stats(featureclass, fields, "SPACE", filename, "ADD_FIELD_NAMES")
print "Exported to a txt file"