Export Table to Table Conversion with specific fields example

[ClintonCooper1]

I found the reference python code for a table to table conversion at: http://resources.arcgis.com/en/help/main/10.2/index.html#//001200000027000000 with this code:

import arcpy from arcpy import env env.workspace = "C:/data" arcpy.TableToTable_conversion("vegtable.dbf", "C:/output/output.gdb", "vegtable")

I would like to export just a few specific fields from a .txt file.  I have done an exhaustive search as well as  tried many different iterations, but I have not been able to get this to work successfully. Does someone have a quick example of how I would set up exporting like 3 or 4 specific fields from the first table into the second table?  Thanks in advance!

Clinton

[MathewCoyle]

Using table to table you would have to use field mappings to setup the conversion from source field to destination field. Alternatively you could use an insert cursor on a template with the field restrictions you want. You can find examples on these forums of field mapping.

Here's an example of me working through some issues I was having with field mapping. Should give you all the pieces you need for that route.
http://forums.arcgis.com/threads/72269-field-map-problems

[curtvprice · Accepted solution]

There is a easier way, if you don't need to get fancy with the nifty functions of field mappings:

If you create a table for output and add just the fields you want and use Append_management with the NO_TEST option, only the fields with matching names get copied.

[ClintonCooper1]

There is a easier way, if you don't need to get fancy with the nifty functions of field mappings:

If you create a table for output and add just the fields you want and use Append_management with the NO_TEST option, only the fields with matching names get copied.

I am dealing with 7+ million records, so performance is a must.  mzcoyle's idea of insert cursor will take too long, and I made a script that worked based off of his example.  What kind of performance will I get with the append management?  The files that I am starting with are .txt files, and they are split into 2 (roughly 3.5 million per).  So I am also trying to figure out the fastest way to get them both together into one file into a file geodatabase.

[ClintonCooper1]

Using table to table you would have to use field mappings to setup the conversion from source field to destination field. Alternatively you could use an insert cursor on a template with the field restrictions you want. You can find examples on these forums of field mapping.

Here's an example of me working through some issues I was having with field mapping. Should give you all the pieces you need for that route.
http://forums.arcgis.com/threads/72269-field-map-problems

Is there way to set field length as well?  I didn't see it in the help page.  Most of the fields I am exporting are of the text variety, and the 255 length eats up way too much space and slows down other operations.  Thanks!!

[MathewCoyle]

I am dealing with 7+ million records, so performance is a must.  mzcoyle's idea of insert cursor will take too long, and I made a script that worked based off of his example.  What kind of performance will I get with the append management?  The files that I am starting with are .txt files, and they are split into 2 (roughly 3.5 million per).  So I am also trying to figure out the fastest way to get them both together into one file into a file geodatabase.

An insert cursor would be your best performance for this kind of operation. Can you post the code you have where you aren't getting the performance you require?

Is there way to set field length as well?  I didn't see it in the help page.  Most of the fields I am exporting are of the text variety, and the 255 length eats up way too much space and slows down other operations.  Thanks!!

Yes, length should be an editable property of a field object in field mapping.

field.length = 10

[ClintonCooper1]

An insert cursor would be your best performance for this kind of operation. Can you post the code you have where you aren't getting the performance you require?

Sorry let me explain, I created an example using field mappings.  It worked great, and had great performance.  My test file is 70,000 records (1/100th of my larger set), and it took around 10 seconds to export. 

I also tried the append method, and I have decided actually to go with that route, as 1.  the code is simpler (and allows to easily set the field length), 2.  performance is exactly the same, 3. allows me to take the two files that my data comes in, and can easily merge them together into one.

You were saying insert cursor would still be faster then both these methods?  Do you have an example that I could look at to play around with?  Thanks!