topic Re: Export Table to Table Conversion with specific fields example in Python Questions

Export Table to Table Conversion with specific fields example

ClintonCooper1 — Mon, 28 Oct 2013 18:53:00 GMT

I found the reference python code for a table to table conversion at: http://resources.arcgis.com/en/help/main/10.2/index.html#//001200000027000000 with this code:

import arcpy from arcpy import env env.workspace = "C:/data" arcpy.TableToTable_conversion("vegtable.dbf", "C:/output/output.gdb", "vegtable")

I would like to export just a few specific fields from a .txt file. I have done an exhaustive search as well as tried many different iterations, but I have not been able to get this to work successfully. Does someone have a quick example of how I would set up exporting like 3 or 4 specific fields from the first table into the second table? Thanks in advance!

Clinton

Re: Export Table to Table Conversion with specific fields example

MathewCoyle — Mon, 28 Oct 2013 19:48:16 GMT

Using table to table you would have to use field mappings to setup the conversion from source field to destination field. Alternatively you could use an insert cursor on a template with the field restrictions you want. You can find examples on these forums of field mapping.

Here's an example of me working through some issues I was having with field mapping. Should give you all the pieces you need for that route.
http://forums.arcgis.com/threads/72269-field-map-problems

Re: Export Table to Table Conversion with specific fields example

curtvprice — Mon, 28 Oct 2013 21:27:41 GMT

There is a easier way, if you don't need to get fancy with the nifty functions of field mappings:

If you create a table for output and add just the fields you want and use Append_management with the NO_TEST option, only the fields with matching names get copied.

Re: Export Table to Table Conversion with specific fields example

ClintonCooper1 — Tue, 29 Oct 2013 12:16:13 GMT

There is a easier way, if you don't need to get fancy with the nifty functions of field mappings:

If you create a table for output and add just the fields you want and use Append_management with the NO_TEST option, only the fields with matching names get copied.

I am dealing with 7+ million records, so performance is a must. mzcoyle's idea of insert cursor will take too long, and I made a script that worked based off of his example. What kind of performance will I get with the append management? The files that I am starting with are .txt files, and they are split into 2 (roughly 3.5 million per). So I am also trying to figure out the fastest way to get them both together into one file into a file geodatabase.

Re: Export Table to Table Conversion with specific fields example

ClintonCooper1 — Tue, 29 Oct 2013 12:18:44 GMT

Using table to table you would have to use field mappings to setup the conversion from source field to destination field. Alternatively you could use an insert cursor on a template with the field restrictions you want. You can find examples on these forums of field mapping.

Here's an example of me working through some issues I was having with field mapping. Should give you all the pieces you need for that route.
http://forums.arcgis.com/threads/72269-field-map-problems

Is there way to set field length as well? I didn't see it in the help page. Most of the fields I am exporting are of the text variety, and the 255 length eats up way too much space and slows down other operations. Thanks!!

Re: Export Table to Table Conversion with specific fields example

MathewCoyle — Tue, 29 Oct 2013 12:30:59 GMT

I am dealing with 7+ million records, so performance is a must. mzcoyle's idea of insert cursor will take too long, and I made a script that worked based off of his example. What kind of performance will I get with the append management? The files that I am starting with are .txt files, and they are split into 2 (roughly 3.5 million per). So I am also trying to figure out the fastest way to get them both together into one file into a file geodatabase.

An insert cursor would be your best performance for this kind of operation. Can you post the code you have where you aren't getting the performance you require?

Is there way to set field length as well? I didn't see it in the help page. Most of the fields I am exporting are of the text variety, and the 255 length eats up way too much space and slows down other operations. Thanks!!

Yes, length should be an editable property of a field object in field mapping.

field.length = 10

Re: Export Table to Table Conversion with specific fields example

ClintonCooper1 — Tue, 29 Oct 2013 12:52:17 GMT

An insert cursor would be your best performance for this kind of operation. Can you post the code you have where you aren't getting the performance you require?

Sorry let me explain, I created an example using field mappings. It worked great, and had great performance. My test file is 70,000 records (1/100th of my larger set), and it took around 10 seconds to export.

I also tried the append method, and I have decided actually to go with that route, as 1. the code is simpler (and allows to easily set the field length), 2. performance is exactly the same, 3. allows me to take the two files that my data comes in, and can easily merge them together into one.

You were saying insert cursor would still be faster then both these methods? Do you have an example that I could look at to play around with? Thanks!