topic CSV Field Types in Python Questions

CSV Field Types

MarkPaulson — Sat, 30 Mar 2013 14:08:40 GMT

Newbie to Python, but have written a script to concatenate multiple CSV files into one file and add a couple of computed fields. I was wondering if there is a way specify field types in python where the finished CSV file would not require a schema.ini file to import fields in the correct format such as: date, Long, Double and also control the field width.

Re: CSV Field Types

curtvprice — Sat, 30 Mar 2013 23:08:24 GMT

... is there a way specify field types in python where the finished CSV file would not require a schema.ini file to import fields in the correct format such as: date, Long, Double and also control the field width.

If the field properties you are getting by default aren't correct you must create a schema.ini file - or edit an existing one in that folder - to get the field types you want.

Alhough writing schema.ini in Python is not that difficult (the format is not that complicated) there is a another approach:

The Python standard library includes the csv module, which you may be helpful to you as it will not convert data on input. You could read and append your csv data using Python lists, create a table with the schema you want, and then add the data to the table using an insert cursor. In this way you could bypass all the schema.ini jazz.

Re: CSV Field Types

MarkPaulson — Wed, 03 Apr 2013 21:35:33 GMT

The Python standard library includes the csv module, which you may be helpful to you as it will not convert data on input. You could read and append your csv data using Python lists, create a table with the schema you want, and then add the data to the table using an insert cursor. In this way you could bypass all the schema.ini jazz.

Curt

Thanks for the reply. I am using the csv module to both read and write the multiple csv files into a merged file with some processing in between the merge. I played with the csv module, but I am not able to create the fields as I need them. I am using a schema.ini file, but it seems rather a kludge to have to keep track of it. Being new to this I am not following the "create a table" that you mentioned above.

Mark