I'm looking for assistance to writing a dictionary to a .csv file. First, set a "gdata2" variable: taburl = hostedFeatureService + '/{}/query/0' where = '1=1' query = "?where={}&outFields={}&returnGeometry=true&f=json&token={}" . format ( where , fieldsM , token ) queryfsURL = taburl + query greq = urllib2 . Request ( queryfsURL ) gresp = urllib2 . urlopen ( greq ) gdata = json . load ( gresp ) gdata2 = gdata [ 'features' ] print gdata2 This codeblock gives me the gdata2 variable result as: [ { u 'geometry' : { u 'y' : 26.67525376665167 , u 'x' : - 80.09297701660482 } , u 'attributes' : { u 'Ingress1Arrive' : 1519133580000L , u 'SurveyDate' : 1519102800000L , u 'globalid' : u 'fdd28a97-9a28-400d-8284-5ce91868f26e' , u 'objectid' : 3 } } , { u 'geometry' : { u 'y' : 26.67525546250736 , u 'x' : - 80.0929301851895 } , u 'attributes' : { u 'Ingress1Arrive' : 1519136280000L , u 'SurveyDate' : 1519102800000L , u 'globalid' : u 'cc83f1b4-6335-4659-8463-9d6d50d82bd7' , u 'objectid' : 4 } } , { u 'geometry' : { u 'y' : 26.675340745164966 , u 'x' : - 80.09289801628141 } , u 'attributes' : { u 'Ingress1Arrive' : 1519227780000L , u 'SurveyDate' : 1519189200000L , u 'globalid' : u '99ed6b91-6823-4702-868a-2bb4dbd32cbf' , u 'objectid' : 5 } } ] From this I am attempting to write this result to a .csv file. It actually works great however I am having an issue with date values (Ingress1Arrive and SurveyDate) as they are epoch formatted values because this source is a hosted feature service. The DictWriter proc: with open ( 'C:temp\{}.csv' . format ( sheetname ) , 'wb' ) as outf : dw = csv . DictWriter ( outf , delimiter = "," , quotechar = "|" , fieldnames = [ 'objectid' , 'globalid' , 'SurveyDate' , 'Ingress1Arrive' ] ) headers = { } for n in dw . fieldnames : headers [ n ] = n dw . writerow ( headers ) for row in gdata2 : dw . writerow ( row [ 'attributes' ] ) Any recommendations on how/where to perform a conversion prior to writing the final .csv output so that the date format is something that makes sense?

I haven't played around much with DictWriter. I usually use the following when I processing my AGOL data; it does show how I process the date/time data when given an epoch date. Code is just snippets, but it shows the basics. f = open ( 'certification.xls' , 'w' ) jsonResponse = urllib . urlopen ( URL , urllib . urlencode ( query_dict ) ) features = json . loads ( jsonResponse . read ( ) . decode ( "utf-8-sig" ) . encode ( "utf-8" ) , object_pairs_hook = collections . OrderedDict ) [ u 'features' ] # write header f . write ( "{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\n" . format ( "OBJECTID" , . . . "Date Certified" , . . . "EditDate" , . . . for feature in features : # AGOL uses GMT/UTC - convert to local time if feature [ 'attributes' ] [ 'CertDate' ] is not None : certTime = time . strftime ( '%c' , time . localtime ( feature [ 'attributes' ] [ 'CertDate' ] / 1000 ) ) else : certTime = None editTime = time . strftime ( '%c' , time . localtime ( feature [ 'attributes' ] [ 'EditDate' ] / 1000 ) ) f . write ( "{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\n" . format ( feature [ 'attributes' ] [ 'OBJECTID' ] , . . . certTime if certTime is not None else "" , . . . editTime , . . .

nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

csv.DictWriter Help convert epoch dates

JamesCrandall

I'm looking for assistance to writing a dictionary to a .csv file. First, set a "gdata2" variable:

taburl <SPAN class="operator token">=</SPAN>&nbsp; hostedFeatureService <SPAN class="operator token">+</SPAN> <SPAN class="string token">'/{}/query/0'</SPAN>
where <SPAN class="operator token">=</SPAN> <SPAN class="string token">'1=1'</SPAN>
query <SPAN class="operator token">=</SPAN> <SPAN class="string token">"?where={}&outFields={}&returnGeometry=true&f=json&token={}"</SPAN><SPAN class="punctuation token">.</SPAN>format<SPAN class="punctuation token">(</SPAN>where<SPAN class="punctuation token">,</SPAN> fieldsM<SPAN class="punctuation token">,</SPAN> token<SPAN class="punctuation token">)</SPAN>
queryfsURL <SPAN class="operator token">=</SPAN> taburl <SPAN class="operator token">+</SPAN> query
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 
greq <SPAN class="operator token">=</SPAN> urllib2<SPAN class="punctuation token">.</SPAN>Request<SPAN class="punctuation token">(</SPAN>queryfsURL<SPAN class="punctuation token">)</SPAN>
gresp <SPAN class="operator token">=</SPAN> urllib2<SPAN class="punctuation token">.</SPAN>urlopen<SPAN class="punctuation token">(</SPAN>greq<SPAN class="punctuation token">)</SPAN>
gdata <SPAN class="operator token">=</SPAN> json<SPAN class="punctuation token">.</SPAN>load<SPAN class="punctuation token">(</SPAN>gresp<SPAN class="punctuation token">)</SPAN>
gdata2 <SPAN class="operator token">=</SPAN> gdata<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'features'</SPAN><SPAN class="punctuation token">]</SPAN>
<SPAN class="keyword token">print</SPAN> gdata2‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍<SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

This codeblock gives me the gdata2 variable result as:

<SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">{</SPAN>u<SPAN class="string token">'geometry'</SPAN><SPAN class="punctuation token">:</SPAN> <SPAN class="punctuation token">{</SPAN>u<SPAN class="string token">'y'</SPAN><SPAN class="punctuation token">:</SPAN> <SPAN class="number token">26.67525376665167</SPAN><SPAN class="punctuation token">,</SPAN> u<SPAN class="string token">'x'</SPAN><SPAN class="punctuation token">:</SPAN> <SPAN class="operator token">-</SPAN><SPAN class="number token">80.09297701660482</SPAN><SPAN class="punctuation token">}</SPAN><SPAN class="punctuation token">,</SPAN> u<SPAN class="string token">'attributes'</SPAN><SPAN class="punctuation token">:</SPAN> <SPAN class="punctuation token">{</SPAN>u<SPAN class="string token">'Ingress1Arrive'</SPAN><SPAN class="punctuation token">:</SPAN> 1519133580000L<SPAN class="punctuation token">,</SPAN> u<SPAN class="string token">'SurveyDate'</SPAN><SPAN class="punctuation token">:</SPAN> 1519102800000L<SPAN class="punctuation token">,</SPAN> u<SPAN class="string token">'globalid'</SPAN><SPAN class="punctuation token">:</SPAN> u<SPAN class="string token">'fdd28a97-9a28-400d-8284-5ce91868f26e'</SPAN><SPAN class="punctuation token">,</SPAN> u<SPAN class="string token">'objectid'</SPAN><SPAN class="punctuation token">:</SPAN> <SPAN class="number token">3</SPAN><SPAN class="punctuation token">}</SPAN><SPAN class="punctuation token">}</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="punctuation token">{</SPAN>u<SPAN class="string token">'geometry'</SPAN><SPAN class="punctuation token">:</SPAN> <SPAN class="punctuation token">{</SPAN>u<SPAN class="string token">'y'</SPAN><SPAN class="punctuation token">:</SPAN> <SPAN class="number token">26.67525546250736</SPAN><SPAN class="punctuation token">,</SPAN> u<SPAN class="string token">'x'</SPAN><SPAN class="punctuation token">:</SPAN> <SPAN class="operator token">-</SPAN><SPAN class="number token">80.0929301851895</SPAN><SPAN class="punctuation token">}</SPAN><SPAN class="punctuation token">,</SPAN> u<SPAN class="string token">'attributes'</SPAN><SPAN class="punctuation token">:</SPAN> <SPAN class="punctuation token">{</SPAN>u<SPAN class="string token">'Ingress1Arrive'</SPAN><SPAN class="punctuation token">:</SPAN> 1519136280000L<SPAN class="punctuation token">,</SPAN> u<SPAN class="string token">'SurveyDate'</SPAN><SPAN class="punctuation token">:</SPAN> 1519102800000L<SPAN class="punctuation token">,</SPAN> u<SPAN class="string token">'globalid'</SPAN><SPAN class="punctuation token">:</SPAN> u<SPAN class="string token">'cc83f1b4-6335-4659-8463-9d6d50d82bd7'</SPAN><SPAN class="punctuation token">,</SPAN> u<SPAN class="string token">'objectid'</SPAN><SPAN class="punctuation token">:</SPAN> <SPAN class="number token">4</SPAN><SPAN class="punctuation token">}</SPAN><SPAN class="punctuation token">}</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="punctuation token">{</SPAN>u<SPAN class="string token">'geometry'</SPAN><SPAN class="punctuation token">:</SPAN> <SPAN class="punctuation token">{</SPAN>u<SPAN class="string token">'y'</SPAN><SPAN class="punctuation token">:</SPAN> <SPAN class="number token">26.675340745164966</SPAN><SPAN class="punctuation token">,</SPAN> u<SPAN class="string token">'x'</SPAN><SPAN class="punctuation token">:</SPAN> <SPAN class="operator token">-</SPAN><SPAN class="number token">80.09289801628141</SPAN><SPAN class="punctuation token">}</SPAN><SPAN class="punctuation token">,</SPAN> u<SPAN class="string token">'attributes'</SPAN><SPAN class="punctuation token">:</SPAN> <SPAN class="punctuation token">{</SPAN>u<SPAN class="string token">'Ingress1Arrive'</SPAN><SPAN class="punctuation token">:</SPAN> 1519227780000L<SPAN class="punctuation token">,</SPAN> u<SPAN class="string token">'SurveyDate'</SPAN><SPAN class="punctuation token">:</SPAN> 1519189200000L<SPAN class="punctuation token">,</SPAN> u<SPAN class="string token">'globalid'</SPAN><SPAN class="punctuation token">:</SPAN> u<SPAN class="string token">'99ed6b91-6823-4702-868a-2bb4dbd32cbf'</SPAN><SPAN class="punctuation token">,</SPAN> u<SPAN class="string token">'objectid'</SPAN><SPAN class="punctuation token">:</SPAN> <SPAN class="number token">5</SPAN><SPAN class="punctuation token">}</SPAN><SPAN class="punctuation token">}</SPAN><SPAN class="punctuation token">]</SPAN>‍‍<SPAN class="line-numbers-rows"><SPAN>‍</SPAN></SPAN>

From this I am attempting to write this result to a .csv file. It actually works great however I am having an issue with date values (Ingress1Arrive and SurveyDate) as they are epoch formatted values because this source is a hosted feature service.

The DictWriter proc:

<SPAN class="keyword token">with</SPAN> open<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'C:temp\{}.csv'</SPAN><SPAN class="punctuation token">.</SPAN>format<SPAN class="punctuation token">(</SPAN>sheetname<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'wb'</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="keyword token">as</SPAN> outf<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; dw <SPAN class="operator token">=</SPAN> csv<SPAN class="punctuation token">.</SPAN>DictWriter<SPAN class="punctuation token">(</SPAN>outf<SPAN class="punctuation token">,</SPAN>&nbsp; delimiter<SPAN class="operator token">=</SPAN><SPAN class="string token">","</SPAN><SPAN class="punctuation token">,</SPAN> quotechar<SPAN class="operator token">=</SPAN><SPAN class="string token">"|"</SPAN><SPAN class="punctuation token">,</SPAN> fieldnames<SPAN class="operator token">=</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'objectid'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'globalid'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'SurveyDate'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'Ingress1Arrive'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; headers <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">{</SPAN><SPAN class="punctuation token">}</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> n <SPAN class="keyword token">in</SPAN> dw<SPAN class="punctuation token">.</SPAN>fieldnames<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; headers<SPAN class="punctuation token">[</SPAN>n<SPAN class="punctuation token">]</SPAN> <SPAN class="operator token">=</SPAN> n
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; dw<SPAN class="punctuation token">.</SPAN>writerow<SPAN class="punctuation token">(</SPAN>headers<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">for</SPAN> row <SPAN class="keyword token">in</SPAN> gdata2<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; dw<SPAN class="punctuation token">.</SPAN>writerow<SPAN class="punctuation token">(</SPAN>row<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'attributes'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN>‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍<SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

Any recommendations on how/where to perform a conversion prior to writing the final .csv output so that the date format is something that makes sense?

Find more posts tagged with

Accepted answers

RandyBurton

I haven't played around much with DictWriter. I usually use the following when I processing my AGOL data; it does show how I process the date/time data when given an epoch date. Code is just snippets, but it shows the basics.

f <SPAN class="operator token">=</SPAN> open<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'certification.xls'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'w'</SPAN><SPAN class="punctuation token">)</SPAN>

jsonResponse <SPAN class="operator token">=</SPAN> urllib<SPAN class="punctuation token">.</SPAN>urlopen<SPAN class="punctuation token">(</SPAN>URL<SPAN class="punctuation token">,</SPAN> urllib<SPAN class="punctuation token">.</SPAN>urlencode<SPAN class="punctuation token">(</SPAN>query_dict<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
features <SPAN class="operator token">=</SPAN> json<SPAN class="punctuation token">.</SPAN>loads<SPAN class="punctuation token">(</SPAN>jsonResponse<SPAN class="punctuation token">.</SPAN>read<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">.</SPAN>decode<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"utf-8-sig"</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">.</SPAN>encode<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"utf-8"</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; object_pairs_hook<SPAN class="operator token">=</SPAN>collections<SPAN class="punctuation token">.</SPAN>OrderedDict<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">[</SPAN>u<SPAN class="string token">'features'</SPAN><SPAN class="punctuation token">]</SPAN>

<SPAN class="comment token"># write header</SPAN>
f<SPAN class="punctuation token">.</SPAN>write<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\n"</SPAN><SPAN class="punctuation token">.</SPAN>format<SPAN class="punctuation token">(</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="string token">"OBJECTID"</SPAN><SPAN class="punctuation token">,</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="string token">"Date Certified"</SPAN><SPAN class="punctuation token">,</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="string token">"EditDate"</SPAN><SPAN class="punctuation token">,</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN>

<SPAN class="keyword token">for</SPAN> feature <SPAN class="keyword token">in</SPAN> features<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="comment token"># AGOL uses GMT/UTC - convert to local time</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> feature<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'attributes'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'CertDate'</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="keyword token">is</SPAN> <SPAN class="operator token">not</SPAN> None<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; certTime <SPAN class="operator token">=</SPAN> time<SPAN class="punctuation token">.</SPAN>strftime<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'%c'</SPAN><SPAN class="punctuation token">,</SPAN> time<SPAN class="punctuation token">.</SPAN>localtime<SPAN class="punctuation token">(</SPAN>feature<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'attributes'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'CertDate'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="operator token">/</SPAN><SPAN class="number token">1000</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">else</SPAN> <SPAN class="punctuation token">:</SPAN> certTime <SPAN class="operator token">=</SPAN> None
&nbsp;&nbsp;&nbsp; editTime <SPAN class="operator token">=</SPAN> time<SPAN class="punctuation token">.</SPAN>strftime<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'%c'</SPAN><SPAN class="punctuation token">,</SPAN> time<SPAN class="punctuation token">.</SPAN>localtime<SPAN class="punctuation token">(</SPAN>feature<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'attributes'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'EditDate'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="operator token">/</SPAN><SPAN class="number token">1000</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>

&nbsp;&nbsp;&nbsp; f<SPAN class="punctuation token">.</SPAN>write<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\n"</SPAN><SPAN class="punctuation token">.</SPAN>format<SPAN class="punctuation token">(</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; feature<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'attributes'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'OBJECTID'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">,</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; certTime <SPAN class="keyword token">if</SPAN> certTime <SPAN class="keyword token">is</SPAN> <SPAN class="operator token">not</SPAN> None <SPAN class="keyword token">else</SPAN> <SPAN class="string token">""</SPAN><SPAN class="punctuation token">,</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; editTime<SPAN class="punctuation token">,</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN>
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍<SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

All comments

JamesCrandall

Randy,

Here's an alternative method I was able to get from StackOveflow! Pretty slick way to just update the dictionary and then write header/row over the rest.

https://stackoverflow.com/questions/48935250/csv-dictwriter-convert-epoch-values-before-writing/48935816#48935816

RandyBurton

I can see how DictWriter would be helpful with lots of columns. I suppose you could loop through gdata2 and edit

u'SurveyDate': 1519102800000L

so that it contains the time/date data in the desired format before passing it to DictWriter.

JamesCrandall

Thanks Randy. I wasn't sure about if/how to use the .format() so this helps. ugh, this isn't going to be very nice with FS's with tons of columns!

RandyBurton

f <SPAN class="operator token">=</SPAN> open<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'certification.xls'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'w'</SPAN><SPAN class="punctuation token">)</SPAN>

jsonResponse <SPAN class="operator token">=</SPAN> urllib<SPAN class="punctuation token">.</SPAN>urlopen<SPAN class="punctuation token">(</SPAN>URL<SPAN class="punctuation token">,</SPAN> urllib<SPAN class="punctuation token">.</SPAN>urlencode<SPAN class="punctuation token">(</SPAN>query_dict<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
features <SPAN class="operator token">=</SPAN> json<SPAN class="punctuation token">.</SPAN>loads<SPAN class="punctuation token">(</SPAN>jsonResponse<SPAN class="punctuation token">.</SPAN>read<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">.</SPAN>decode<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"utf-8-sig"</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">.</SPAN>encode<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"utf-8"</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; object_pairs_hook<SPAN class="operator token">=</SPAN>collections<SPAN class="punctuation token">.</SPAN>OrderedDict<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">[</SPAN>u<SPAN class="string token">'features'</SPAN><SPAN class="punctuation token">]</SPAN>

<SPAN class="comment token"># write header</SPAN>
f<SPAN class="punctuation token">.</SPAN>write<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\n"</SPAN><SPAN class="punctuation token">.</SPAN>format<SPAN class="punctuation token">(</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="string token">"OBJECTID"</SPAN><SPAN class="punctuation token">,</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="string token">"Date Certified"</SPAN><SPAN class="punctuation token">,</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="string token">"EditDate"</SPAN><SPAN class="punctuation token">,</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN>

<SPAN class="keyword token">for</SPAN> feature <SPAN class="keyword token">in</SPAN> features<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="comment token"># AGOL uses GMT/UTC - convert to local time</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">if</SPAN> feature<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'attributes'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'CertDate'</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="keyword token">is</SPAN> <SPAN class="operator token">not</SPAN> None<SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; certTime <SPAN class="operator token">=</SPAN> time<SPAN class="punctuation token">.</SPAN>strftime<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'%c'</SPAN><SPAN class="punctuation token">,</SPAN> time<SPAN class="punctuation token">.</SPAN>localtime<SPAN class="punctuation token">(</SPAN>feature<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'attributes'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'CertDate'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="operator token">/</SPAN><SPAN class="number token">1000</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">else</SPAN> <SPAN class="punctuation token">:</SPAN> certTime <SPAN class="operator token">=</SPAN> None
&nbsp;&nbsp;&nbsp; editTime <SPAN class="operator token">=</SPAN> time<SPAN class="punctuation token">.</SPAN>strftime<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'%c'</SPAN><SPAN class="punctuation token">,</SPAN> time<SPAN class="punctuation token">.</SPAN>localtime<SPAN class="punctuation token">(</SPAN>feature<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'attributes'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'EditDate'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="operator token">/</SPAN><SPAN class="number token">1000</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>

&nbsp;&nbsp;&nbsp; f<SPAN class="punctuation token">.</SPAN>write<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\t{}\n"</SPAN><SPAN class="punctuation token">.</SPAN>format<SPAN class="punctuation token">(</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; feature<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'attributes'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="string token">'OBJECTID'</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">,</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; certTime <SPAN class="keyword token">if</SPAN> certTime <SPAN class="keyword token">is</SPAN> <SPAN class="operator token">not</SPAN> None <SPAN class="keyword token">else</SPAN> <SPAN class="string token">""</SPAN><SPAN class="punctuation token">,</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; editTime<SPAN class="punctuation token">,</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN>
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍<SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

Join the Conversation

Signed in members can post, follow updates, and more. New here? Register a free account.

Quick Links

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.