How do I import shapefiles to GDB in arcpy script with "if" statement and date range?

[AdamKarczynski]

I am trying to import network shapefiles to a local file GDB and I am having some trouble writing the script. The shapefiles have a consistent naming syntax as follows:

bob_[date as 8-digit YYYYMMDD]

I would also like to add a counter so that as each file transfers, the user will know how many have been transferred and how many are left to go. I had some ideas on it, but am interested to see how you all might solve that small addition.

Here is the sample script:

# Import system modules
import arcpy, sys, string, os, os.path
from arcpy import env

# Set environment / variables
sourceWorkspace = "C:/Users/jim/Documents/ArcGIS/bob_Shapefiles/"
env.workspace = "C:/Users/jim/Documents/ArcGIS/bob_Shapefiles"

targetWorkspace = "C:/Users/jim/Documents/ArcGIS/Park/bob_Original.gdb"

arcpy.env.overwriteOutput = True

dateBegin = 20000101
dateEnd = 20000103

shapes = arcpy.ListFeatureClasses()

# Loop to process each shapefile in sourceWorkspace
for shape in shapes:
      dateName = shape[-8:]

# Set conditional statement to transfer only the desired date range.

      if (dateName >= dateBegin) and (dateName <= dateEnd):
            outFeature = shape

            arcpy.FeatureClassToGeodatabase_conversion(outFeature, targetWorkspace)
            print 'Transferring : ' + dateName

      else:
            print 'No files in date range to transfer.'
            quit()

[BlakeTerhune]

First, I want to point out there's a nice syntax highlighting feature in GeoNet for posting code blocks.

Posting code with Syntax Highlighting on GeoNet 

Are you getting an error when running your code or are you just looking for advice on how to add the counter you mentioned? If it isn't doing what you want, please describe what is wrong.

[DanPatterson_Retired]

Blake's suggestion and its companion... /blogs/dan_patterson/2016/08/14/script-formatting would indeed be useful

[XanderBakker]

You are comparing a string "dateName" with a numeric value. 

If I do this:

dateBegin = 20000101
dateEnd = 20000103

dateName = '20000102'

# Set conditional statement to transfer only the desired date range.
if (dateName >= dateBegin) and (dateName <= dateEnd):
    print "in range"

else:
    print "not in range"‍‍‍‍‍‍‍‍‍‍‍

... it will yield "not in range".

If I change line 4 in this snippet to:

dateName = int('20000102')‍

... it will yield "in range".

Comparing dates as numeric values will only succeed if you specify them in the "yyyymmdd" format.

It could be to change all the dates to dates.

[AdamKarczynski]

I did this form your suggestion:

# Loop to process each shapefile in sourceWorkspace
    for shape in shapes:
    
        dateName = shape[-8:]
        dateFile = int(shape[-8:])

# Set conditional statement to transfer only the desired date range.

        if (dateFile >= dateStart) and (dateFile <= dateEnd):
            outFeature = shape
            arcpy.FeatureClassToGeodatabase_conversion(outFeature, targetWorkspace)
            print 'Transferring : ' + dateName
            
        else:
            pass‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

This fixed the issue. Thanks!

As a secondary question:

Do you have any suggestions for coding a running counter of the total number of files to be transferred and a  how many have been transferred?

[XanderBakker]

You could do something like this:

# Import system modules
import arcpy, sys, string, os, os.path
from arcpy import env

# Set environment / variables
sourceWorkspace = "C:/Users/jim/Documents/ArcGIS/bob_Shapefiles/"
env.workspace = "C:/Users/jim/Documents/ArcGIS/bob_Shapefiles"

targetWorkspace = "C:/Users/jim/Documents/ArcGIS/Park/bob_Original.gdb"
arcpy.env.overwriteOutput = True

dateBegin = 20000101
dateEnd = 20000103

shapes = arcpy.ListFeatureClasses()

# Loop to process each shapefile in sourceWorkspace
cnt_shp, cnt_inrange = 0, 0
for shape in shapes:
    cnt_shp += 1
    dateName = shape[-8:]
    dateFile = int(shape[-8:])

    if cnt_shp % 100 == 0:
        print "Processing shape:", cnt_shp, "   # in range:", cnt_inrange, "({0}%)".format(round((cnt_inrange) * 100.0 / float(cnt_shp), 2))

    # Set conditional statement to transfer only the desired date range.
    if (dateFile >= dateStart) and (dateFile <= dateEnd):
        cnt_inrange += 1
        outFeature = shape
        arcpy.FeatureClassToGeodatabase_conversion(outFeature, targetWorkspace)
        # print 'Transferring : ' + dateName
    else:
        pass

print "Finished processing shape:", cnt_shp, "   # in range:", cnt_inrange, "({0}%)".format(round((cnt_inrange) * 100.0 / float(cnt_shp), 2))‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

[AdamKarczynski]

Is this better? Sorry first time posting script to GeoNet.

# Import system modules
import arcpy, sys, string, os, os.path
from arcpy import env
 
# Set environment / variables
sourceWorkspace = "C:/Users/jim/Documents/ArcGIS/bob_Shapefiles/"
env.workspace = "C:/Users/jim/Documents/ArcGIS/bob_Shapefiles"
 
targetWorkspace = "C:/Users/jim/Documents/ArcGIS/Park/bob_Original.gdb"
arcpy.env.overwriteOutput = True

dateBegin = 20000101
dateEnd = 20000103

shapes = arcpy.ListFeatureClasses()

# Loop to process each shapefile in sourceWorkspace
for shape in shapes:
      dateName = shape[-8:]

# Set conditional statement to transfer only the desired date range.
      if (dateName >= dateBegin) and (dateName <= dateEnd):
            outFeature = shape
            arcpy.FeatureClassToGeodatabase_conversion(outFeature, targetWorkspace)
            print 'Transferring : ' + dateName

      else:
            print 'No files in date range to transfer.'
            quit()‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

The files in the sourceWorkspace and the set environment do exist and follow the format I mention at the top. I can process them individually through ArcMap and Catalog but due to the size of the folder (over 5,000 shape files) doing so requires a large amount of extra time just to open the folder.

No "error" messages are popping up, but the files are not transferring and the only output is "No files in date range to transfer."