nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Export to CSV with Override and NO Header Row - Python

ModernGIS

I've built a Pythons script the organizes a dataset that is used for an external database and exported it from a File Geodatabase to a .CSV file. The last part of the workflow is to manually delete the header row and copy the CSV to a network drive and override the existing .CSV file. Now, its been requested to make the workflow 100% automatic and have it built into my current Batch File collection to run each night. Just (2) pieces of the script I cannot figure out how to write:

1) Export File GDB Table to .CSV file WithOut the Header Row

2) When the .CSV file is exported, it Overrides the existing file in it's place with the same name.

Any assistance, would be appreciated.

Thank You

Encuentra más publicaciones etiquetadas con

Accepted answers

MehdiPira1

Modern Electric‌

You're using Python 2.x which doesn't support 'newline'.

Remove this

newline=''

then add the following at the end of the previous code. It should work now.

file_object <SPAN class="operator token">=</SPAN> open<SPAN class="punctuation token">(</SPAN>outfile<SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'r'</SPAN><SPAN class="punctuation token">)</SPAN>
lines <SPAN class="operator token">=</SPAN> csv<SPAN class="punctuation token">.</SPAN>reader<SPAN class="punctuation token">(</SPAN>file_object<SPAN class="punctuation token">,</SPAN> delimiter<SPAN class="operator token">=</SPAN><SPAN class="string token">','</SPAN><SPAN class="punctuation token">,</SPAN> quotechar<SPAN class="operator token">=</SPAN><SPAN class="string token">'"'</SPAN><SPAN class="punctuation token">)</SPAN>
flag <SPAN class="operator token">=</SPAN> <SPAN class="number token">0</SPAN>
data<SPAN class="operator token">=</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">]</SPAN>
<SPAN class="keyword token">for</SPAN> line <SPAN class="keyword token">in</SPAN> lines<SPAN class="punctuation token">:</SPAN>
    <SPAN class="keyword token">if</SPAN> line <SPAN class="operator token">==</SPAN> <SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">:</SPAN>
        flag <SPAN class="operator token">=</SPAN><SPAN class="number token">1</SPAN>
        <SPAN class="keyword token">continue</SPAN>
    <SPAN class="keyword token">else</SPAN><SPAN class="punctuation token">:</SPAN>
        data<SPAN class="punctuation token">.</SPAN>append<SPAN class="punctuation token">(</SPAN>line<SPAN class="punctuation token">)</SPAN>
file_object<SPAN class="punctuation token">.</SPAN>close<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>
<SPAN class="keyword token">if</SPAN> flag <SPAN class="operator token">==</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">:</SPAN> <SPAN class="comment token">#if blank line is present in file</SPAN>
    file_object <SPAN class="operator token">=</SPAN> open<SPAN class="punctuation token">(</SPAN>outfile<SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'w'</SPAN><SPAN class="punctuation token">)</SPAN>
    <SPAN class="keyword token">for</SPAN> line <SPAN class="keyword token">in</SPAN> data<SPAN class="punctuation token">:</SPAN>
        str1 <SPAN class="operator token">=</SPAN> <SPAN class="string token">','</SPAN><SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>line<SPAN class="punctuation token">)</SPAN>
        file_object<SPAN class="punctuation token">.</SPAN>write<SPAN class="punctuation token">(</SPAN>str1<SPAN class="operator token">+</SPAN><SPAN class="string token">"\n"</SPAN><SPAN class="punctuation token">)</SPAN>
    file_object<SPAN class="punctuation token">.</SPAN>close<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

All comments

ModernGIS

BINGO!!

Only one thing I changed in the code:

if flag ==0

This made the spaced go away. When there was a "1", it kept including the spaces.

Appreciate your help

MehdiPira1

Modern Electric‌

You're using Python 2.x which doesn't support 'newline'.

Remove this

newline=''

then add the following at the end of the previous code. It should work now.

file_object <SPAN class="operator token">=</SPAN> open<SPAN class="punctuation token">(</SPAN>outfile<SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'r'</SPAN><SPAN class="punctuation token">)</SPAN>
lines <SPAN class="operator token">=</SPAN> csv<SPAN class="punctuation token">.</SPAN>reader<SPAN class="punctuation token">(</SPAN>file_object<SPAN class="punctuation token">,</SPAN> delimiter<SPAN class="operator token">=</SPAN><SPAN class="string token">','</SPAN><SPAN class="punctuation token">,</SPAN> quotechar<SPAN class="operator token">=</SPAN><SPAN class="string token">'"'</SPAN><SPAN class="punctuation token">)</SPAN>
flag <SPAN class="operator token">=</SPAN> <SPAN class="number token">0</SPAN>
data<SPAN class="operator token">=</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">]</SPAN>
<SPAN class="keyword token">for</SPAN> line <SPAN class="keyword token">in</SPAN> lines<SPAN class="punctuation token">:</SPAN>
    <SPAN class="keyword token">if</SPAN> line <SPAN class="operator token">==</SPAN> <SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">:</SPAN>
        flag <SPAN class="operator token">=</SPAN><SPAN class="number token">1</SPAN>
        <SPAN class="keyword token">continue</SPAN>
    <SPAN class="keyword token">else</SPAN><SPAN class="punctuation token">:</SPAN>
        data<SPAN class="punctuation token">.</SPAN>append<SPAN class="punctuation token">(</SPAN>line<SPAN class="punctuation token">)</SPAN>
file_object<SPAN class="punctuation token">.</SPAN>close<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>
<SPAN class="keyword token">if</SPAN> flag <SPAN class="operator token">==</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">:</SPAN> <SPAN class="comment token">#if blank line is present in file</SPAN>
    file_object <SPAN class="operator token">=</SPAN> open<SPAN class="punctuation token">(</SPAN>outfile<SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'w'</SPAN><SPAN class="punctuation token">)</SPAN>
    <SPAN class="keyword token">for</SPAN> line <SPAN class="keyword token">in</SPAN> data<SPAN class="punctuation token">:</SPAN>
        str1 <SPAN class="operator token">=</SPAN> <SPAN class="string token">','</SPAN><SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>line<SPAN class="punctuation token">)</SPAN>
        file_object<SPAN class="punctuation token">.</SPAN>write<SPAN class="punctuation token">(</SPAN>str1<SPAN class="operator token">+</SPAN><SPAN class="string token">"\n"</SPAN><SPAN class="punctuation token">)</SPAN>
    file_object<SPAN class="punctuation token">.</SPAN>close<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

ModernGIS

Thank You Mehdi:

Looks like we are almost there, however:

with open(outfile,'w', newline='') as f:<SPAN class="line-numbers-rows"><SPAN>‍</SPAN></SPAN>

is giving me this:

TypeError: 'newline' is an invalid keyword argument for this function

Not sure what that means.

If I replace this line with what you had in the original script:

with open(outfile,'w') as f:<SPAN class="line-numbers-rows"><SPAN>‍</SPAN></SPAN>

The script works, BUT still has spaces between the rows.

MehdiPira1

Modern Electric‌

Try this one, it removes blank rows and you can specify which fields to output in the csv file:

<SPAN class="keyword token">import</SPAN> arcpy
<SPAN class="keyword token">import</SPAN> csv<SPAN class="punctuation token">,</SPAN> os

arcpy<SPAN class="punctuation token">.</SPAN>env<SPAN class="punctuation token">.</SPAN>overwriteOutput <SPAN class="operator token">=</SPAN> <SPAN class="token boolean">True</SPAN>

workspace <SPAN class="operator token">=</SPAN> r<SPAN class="string token">"working path"</SPAN>
gdb_fc   <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>workspace<SPAN class="punctuation token">,</SPAN> r<SPAN class="string token">"path to GDB Feature Class"</SPAN><SPAN class="punctuation token">)</SPAN>
outfile <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>workspace<SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"output_data.csv"</SPAN><SPAN class="punctuation token">)</SPAN>      

fields <SPAN class="operator token">=</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>ListFields<SPAN class="punctuation token">(</SPAN>gdb_fc<SPAN class="punctuation token">)</SPAN>
<SPAN class="comment token"># for instance, you want to output just the following fields</SPAN>
fieldsToKeep <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN><SPAN class="string token">'Tree_Type'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'Latitude'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'Longitude'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'Tree_Height'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'Note'</SPAN><SPAN class="punctuation token">]</SPAN>
<SPAN class="comment token"># field_names = [field.name for field in fields if field.name in fieldsToKeep]</SPAN>

<SPAN class="keyword token">with</SPAN> open<SPAN class="punctuation token">(</SPAN>outfile<SPAN class="punctuation token">,</SPAN><SPAN class="string token">'w'</SPAN><SPAN class="punctuation token">,</SPAN> newline<SPAN class="operator token">=</SPAN><SPAN class="string token">''</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="keyword token">as</SPAN> f<SPAN class="punctuation token">:</SPAN>
    w <SPAN class="operator token">=</SPAN> csv<SPAN class="punctuation token">.</SPAN>writer<SPAN class="punctuation token">(</SPAN>f<SPAN class="punctuation token">)</SPAN>
<SPAN class="comment token">#     w.writerow(field_names) **This is commented out to skip writing a header**</SPAN>
    <SPAN class="keyword token">for</SPAN> row <SPAN class="keyword token">in</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>SearchCursor<SPAN class="punctuation token">(</SPAN>gdb_fc<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
        field_vals <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN>row<SPAN class="punctuation token">.</SPAN>getValue<SPAN class="punctuation token">(</SPAN>field<SPAN class="punctuation token">.</SPAN>name<SPAN class="punctuation token">)</SPAN> <SPAN class="keyword token">for</SPAN> field <SPAN class="keyword token">in</SPAN> fields <SPAN class="keyword token">if</SPAN> field<SPAN class="punctuation token">.</SPAN>name <SPAN class="keyword token">in</SPAN> fieldsToKeep<SPAN class="punctuation token">]</SPAN>
        w<SPAN class="punctuation token">.</SPAN>writerow<SPAN class="punctuation token">(</SPAN>field_vals<SPAN class="punctuation token">)</SPAN>
    <SPAN class="keyword token">del</SPAN> row
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍<SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

ModernGIS

Mehdi:

Thank You for the assistance. Looks like we are almost there. Here are (2) issues that I am having. I hate to be picky but if the .CSV file is not 100% correct, the database that loads it will fail the upload process.

When I open the .CSV in Microsoft Excel, the export adds a separate field, I assume its the ESRI automatic OID number (might be mistaken on the field name).

The other problem is between each line, there is a space. I know for a fact having the space between each row will force the upload process to fail.

Here is a clip of what the .CSV file looks like after opening it in Excel.

Any other advice on these (2) items and I should be in good shape.

Thank You again.

JoshuaBixby

This issue is much simpler to solve, and I would argue more robust, by addressing it on the import side rather than export side. It is so much easier to simply skip a line at the start of a file when reading it than removing the first line after it is created. Once the file is created, most or all techniques for removing the line involves reading and re-writing the file. One approach to not creating the header line in the first place would be to export the data set to a pandas dataframe and then export to csv from there.

Can the import process be modified to inspect the first line and then decide whether to skip it?

MehdiPira1

Hi Modern Electric‌

Here's the code that exports a File GDB Table to a .CSV file WithOut the Header Row and it also overrides the existing file in it's place with the same name.

<SPAN class="keyword token">import</SPAN> arcpy
<SPAN class="keyword token">import</SPAN> csv<SPAN class="punctuation token">,</SPAN> os

arcpy<SPAN class="punctuation token">.</SPAN>env<SPAN class="punctuation token">.</SPAN>overwriteOutput <SPAN class="operator token">=</SPAN> <SPAN class="token boolean">True</SPAN>

workspace <SPAN class="operator token">=</SPAN> r<SPAN class="string token">"working path"</SPAN>
gdb_fc   <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>workspace<SPAN class="punctuation token">,</SPAN> r<SPAN class="string token">"path to GDB Feature Class"</SPAN><SPAN class="punctuation token">)</SPAN>
outfile <SPAN class="operator token">=</SPAN> os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>workspace<SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"output_data.csv"</SPAN><SPAN class="punctuation token">)</SPAN>      

fields <SPAN class="operator token">=</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>ListFields<SPAN class="punctuation token">(</SPAN>gdb_fc<SPAN class="punctuation token">)</SPAN>
field_names <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN>field<SPAN class="punctuation token">.</SPAN>name <SPAN class="keyword token">for</SPAN> field <SPAN class="keyword token">in</SPAN> fields<SPAN class="punctuation token">]</SPAN>

<SPAN class="keyword token">with</SPAN> open<SPAN class="punctuation token">(</SPAN>outfile<SPAN class="punctuation token">,</SPAN><SPAN class="string token">'w'</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="keyword token">as</SPAN> f<SPAN class="punctuation token">:</SPAN>
    w <SPAN class="operator token">=</SPAN> csv<SPAN class="punctuation token">.</SPAN>writer<SPAN class="punctuation token">(</SPAN>f<SPAN class="punctuation token">)</SPAN>
<SPAN class="comment token">#     w.writerow(field_names) **This is commented out to skip writing a header**</SPAN>
    <SPAN class="keyword token">for</SPAN> row <SPAN class="keyword token">in</SPAN> arcpy<SPAN class="punctuation token">.</SPAN>SearchCursor<SPAN class="punctuation token">(</SPAN>gdb_fc<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
        field_vals <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN>row<SPAN class="punctuation token">.</SPAN>getValue<SPAN class="punctuation token">(</SPAN>field<SPAN class="punctuation token">.</SPAN>name<SPAN class="punctuation token">)</SPAN> <SPAN class="keyword token">for</SPAN> field <SPAN class="keyword token">in</SPAN> fields<SPAN class="punctuation token">]</SPAN>
        w<SPAN class="punctuation token">.</SPAN>writerow<SPAN class="punctuation token">(</SPAN>field_vals<SPAN class="punctuation token">)</SPAN>
    <SPAN class="keyword token">del</SPAN> row<SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

Únete a la Conversación

Los miembros registrados pueden publicar, seguir actualizaciones y más. ¿Nuevo aquí? Regístrate gratis.

Enlaces Rápidos

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.