Hi - I have a web request that returns JSON that I'm trying to get into a file geodatabase table. When I execute arcpy.da.NumPyArrayToTable, I get the following error: RuntimeError: create table This doesn't give me much to go on, so I'm not sure what to try next. Here is my code: from pathlib import Path import arcpy import numpy import pandas import requests proxyDict = { "http" : " " , "https" : " " } url = 'https://api.census.gov/data/2018/acs/acs5?get=NAME,group(B18102)&for=county:*&in=state:53' fgdb = Path ( 'C:/Temp/ACS2018/ACS2018.gdb' ) table = fgdb / 'test_table' r = requests . get ( url ) # r = requests.get(url, proxies=proxyDict) # if you have a proxy server df = pandas . read_json ( r . text ) headers = df [ 0 : 1 ] # first row is headers header_list = headers . values . tolist ( ) [ 0 ] # convert to list header_list [ 0 ] = 'FULL_NAME' # de-duplicate "NAME" new_columns = dict ( zip ( list ( range ( 0 , len ( header_list ) ) ) , header_list ) ) #make dict df2 = df [ 1 : ] # data in rows 2+ df3 = df2 . rename ( columns = new_columns ) # add column names back in # Convert to numpy array, then to table numpy_array = numpy . array ( numpy . rec . fromrecords ( df3 . values ) ) names = df3 . dtypes . index . tolist ( ) numpy_array . dtype . names = tuple ( names ) arcpy . da . NumPyArrayToTable ( numpy_array , str ( table ) ) I also tried df3.to_numpy(), but got the same result.

nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

NumPyArrayToTable error "RuntimeError: create table"

davedoesgis

Hi - I have a web request that returns JSON that I'm trying to get into a file geodatabase table. When I execute arcpy.da.NumPyArrayToTable, I get the following error:

RuntimeError: create table

This doesn't give me much to go on, so I'm not sure what to try next. Here is my code:

<SPAN class="keyword token">from</SPAN> pathlib <SPAN class="keyword token">import</SPAN> Path
<SPAN class="keyword token">import</SPAN> arcpy 
<SPAN class="keyword token">import</SPAN> numpy
<SPAN class="keyword token">import</SPAN> pandas
<SPAN class="keyword token">import</SPAN> requests

proxyDict <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">{</SPAN>
    <SPAN class="string token">"http"</SPAN><SPAN class="punctuation token">:</SPAN><SPAN class="string token">"<http_URL>"</SPAN><SPAN class="punctuation token">,</SPAN>
    <SPAN class="string token">"https"</SPAN><SPAN class="punctuation token">:</SPAN><SPAN class="string token">"<https_URL>"</SPAN>
<SPAN class="punctuation token">}</SPAN>
url <SPAN class="operator token">=</SPAN> <SPAN class="string token">'https://api.census.gov/data/2018/acs/acs5?get=NAME,group(B18102)&for=county:*&in=state:53'</SPAN>
fgdb <SPAN class="operator token">=</SPAN> Path<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'C:/Temp/ACS2018/ACS2018.gdb'</SPAN><SPAN class="punctuation token">)</SPAN>
table <SPAN class="operator token">=</SPAN> fgdb <SPAN class="operator token">/</SPAN> <SPAN class="string token">'test_table'</SPAN>

r <SPAN class="operator token">=</SPAN> requests<SPAN class="punctuation token">.</SPAN>get<SPAN class="punctuation token">(</SPAN>url<SPAN class="punctuation token">)</SPAN>
<SPAN class="comment token"># r = requests.get(url, proxies=proxyDict) # if you have a proxy server</SPAN>
df <SPAN class="operator token">=</SPAN> pandas<SPAN class="punctuation token">.</SPAN>read_json<SPAN class="punctuation token">(</SPAN>r<SPAN class="punctuation token">.</SPAN>text<SPAN class="punctuation token">)</SPAN>

headers <SPAN class="operator token">=</SPAN> df<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">:</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="comment token"># first row is headers</SPAN>
header_list <SPAN class="operator token">=</SPAN> headers<SPAN class="punctuation token">.</SPAN>values<SPAN class="punctuation token">.</SPAN>tolist<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="comment token"># convert to list</SPAN>
header_list<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="operator token">=</SPAN> <SPAN class="string token">'FULL_NAME'</SPAN> <SPAN class="comment token"># de-duplicate "NAME"</SPAN>
new_columns <SPAN class="operator token">=</SPAN> dict<SPAN class="punctuation token">(</SPAN>zip<SPAN class="punctuation token">(</SPAN>list<SPAN class="punctuation token">(</SPAN>range<SPAN class="punctuation token">(</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">,</SPAN>len<SPAN class="punctuation token">(</SPAN>header_list<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> header_list<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="comment token">#make dict</SPAN>

df2 <SPAN class="operator token">=</SPAN> df<SPAN class="punctuation token">[</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">:</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="comment token"># data in rows 2+</SPAN>
df3 <SPAN class="operator token">=</SPAN> df2<SPAN class="punctuation token">.</SPAN>rename<SPAN class="punctuation token">(</SPAN>columns<SPAN class="operator token">=</SPAN>new_columns<SPAN class="punctuation token">)</SPAN> <SPAN class="comment token"># add column names back in</SPAN>

<SPAN class="comment token"># Convert to numpy array, then to table</SPAN>
numpy_array <SPAN class="operator token">=</SPAN> numpy<SPAN class="punctuation token">.</SPAN>array<SPAN class="punctuation token">(</SPAN>numpy<SPAN class="punctuation token">.</SPAN>rec<SPAN class="punctuation token">.</SPAN>fromrecords<SPAN class="punctuation token">(</SPAN>df3<SPAN class="punctuation token">.</SPAN>values<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
names <SPAN class="operator token">=</SPAN> df3<SPAN class="punctuation token">.</SPAN>dtypes<SPAN class="punctuation token">.</SPAN>index<SPAN class="punctuation token">.</SPAN>tolist<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>
numpy_array<SPAN class="punctuation token">.</SPAN>dtype<SPAN class="punctuation token">.</SPAN>names <SPAN class="operator token">=</SPAN> tuple<SPAN class="punctuation token">(</SPAN>names<SPAN class="punctuation token">)</SPAN>
arcpy<SPAN class="punctuation token">.</SPAN>da<SPAN class="punctuation token">.</SPAN>NumPyArrayToTable<SPAN class="punctuation token">(</SPAN>numpy_array<SPAN class="punctuation token">,</SPAN> str<SPAN class="punctuation token">(</SPAN>table<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍<SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

I also tried df3.to_numpy(), but got the same result.

Encuentra más publicaciones etiquetadas con

Accepted answers

DanPatterson_Retired

You better add some print statements to check that your array is properly formed...

You have passed on the dtype names, but there is no indication of the type format (ie text, numbers etc)

Here is an example.

d<SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">:</SPAN><SPAN class="number token">3</SPAN><SPAN class="punctuation token">]</SPAN>
 
array<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="string token">'alabaster'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'0.7.12'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'py_0'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'python'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN>
       <SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN> snip <SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN> <SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN>
       <SPAN class="punctuation token">(</SPAN><SPAN class="string token">'argh'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'0.26.2'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'py36_0'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'python'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">,</SPAN>
      dtype<SPAN class="operator token">=</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="string token">'Name'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'<U30'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="punctuation token">(</SPAN><SPAN class="string token">'Version'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'<U15'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="punctuation token">(</SPAN><SPAN class="string token">'Build'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'<U15'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN>
             <SPAN class="punctuation token">(</SPAN><SPAN class="string token">'Requires'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'<U293'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

Note that Uxx is for unicode (aka, text fields), and you can use

('Float_fld', '<f8') for double/float fields and

('Int_fld', '<i4') for integer fields (or '<i8' if you need large integers)

The help topic uses a dictionary format to form the dtype, which is also permissable, but I prefer explicit and you can use the short forms to cast the text/numeric format

NumPyArrayToTable—Data Access module | Documentation

PS, I have tons of blog posts on numpy

All comments

VHolubec

Dear Joshua Bixby‌,

thank you, it was the case!

danretired

The whole array will be of object dtype, hence string as Joshua suggested if the structure is simple, otherwise, you have to use a compound dtype specifying the sequence of dtypes within it.

Numpy can use compound dtypes as in the following example

dt <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="string token">'OID_'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'<i4'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="punctuation token">(</SPAN><SPAN class="string token">'XYZ'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="string token">'X'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'<f8'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="punctuation token">(</SPAN><SPAN class="string token">'Y'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'<f8'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="string token">'Z'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'<f8'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="line-numbers-rows"><SPAN>‍</SPAN></SPAN>

So instead of X, Y and Z being separate entities, you can the Y values using

XYZ['Y'] (structured array) or

XYZ.Y (recarray)

To simplify your data structure you can convert the compound nature of the above to separate 'fields'

dt <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="string token">'OID_'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'<i4'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="punctuation token">(</SPAN><SPAN class="string token">'X'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'<f8'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="punctuation token">(</SPAN><SPAN class="string token">'Y'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'<f8'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="punctuation token">(</SPAN><SPAN class="string token">'Z'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'<f8'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">]</SPAN>‍<SPAN class="line-numbers-rows"><SPAN>‍</SPAN></SPAN>

bixb0012

It is likely choking on the Object type of 'Popis'. Assuming Popis is a string field, you will need to convert it to string before passing it to Esri to create a table or feature class.

VHolubec

Dear Dan Patterson‌,

Thank you for your post. I have the same problem. Everything seems fine in my settings. I just don´t know if some of the dtype couldn´t case the problem.

dtype=[('MistoMEreniID', '<i4'), ('Provozovatel', '<U3'), ('Datum', '<U10'), ('MaximalniHodnota', '<f8'), ('PrumernaHodnota', '<f8'), ('MistoMereni', '<U10'), ('Popis', 'O')])

Are all these types supported?

Thank you for any idea.

Regards,

Vladimir

davedoesgis

I did a LOT of tinkering to get this working, so I'm not 100% sure what fixed the problem, but just to close the loop on this, here is the Jupyter notebook I got working exported to Python. Notes:

I think the solution was setting the columns as int, but I also removed a bunch of columns I wasn't interested in.
It writes to both a table and does a join to an existing feature class at the end.
This is just some prototype code I was thrashing on and my next task is to clean this up, so you've been warned, it's a bit rough, but it runs (for me, anyhow).
I did update to the latest Pandas using Pip.
There's some places I took out my environment settings. Should be pretty obvious.
As long as the Census keeps this URL working and you have their County feature class with field GEOID, this should work with very little modification. It's requesting data on language ability (group B16005) and sums up certain columns (a.k.a. names) that I'm interested in.

<SPAN class="comment token">#!/usr/bin/env python</SPAN>
<SPAN class="comment token"># coding: utf-8</SPAN>
<SPAN class="comment token"># In[222]:</SPAN>

<SPAN class="keyword token">from</SPAN> pathlib <SPAN class="keyword token">import</SPAN> Path
<SPAN class="keyword token">import</SPAN> arcpy 
<SPAN class="keyword token">import</SPAN> numpy
<SPAN class="keyword token">import</SPAN> pandas
<SPAN class="keyword token">import</SPAN> requests

<SPAN class="comment token"># In[223]:</SPAN>

proxyDict <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">{</SPAN>
    <SPAN class="string token">"http"</SPAN><SPAN class="punctuation token">:</SPAN><SPAN class="string token">"<url>"</SPAN><SPAN class="punctuation token">,</SPAN>
    <SPAN class="string token">"https"</SPAN><SPAN class="punctuation token">:</SPAN><SPAN class="string token">"<url>"</SPAN>
<SPAN class="punctuation token">}</SPAN>
url <SPAN class="operator token">=</SPAN> <SPAN class="string token">'https://api.census.gov/data/2018/acs/acs5?get=NAME,group(B16005)&for=county:*&in=state:*'</SPAN>
sum_names <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN><SPAN class="string token">'B16005_007E'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'B16005_008E'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'B16005_012E'</SPAN><SPAN class="punctuation token">,</SPAN> 
       <SPAN class="string token">'B16005_013E'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'B16005_017E'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'B16005_018E'</SPAN><SPAN class="punctuation token">,</SPAN>
       <SPAN class="string token">'B16005_022E'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'B16005_023E'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'B16005_029E'</SPAN><SPAN class="punctuation token">,</SPAN>
       <SPAN class="string token">'B16005_030E'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'B16005_034E'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'B16005_035E'</SPAN><SPAN class="punctuation token">,</SPAN>
       <SPAN class="string token">'B16005_039E'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'B16005_040E'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'B16005_044E'</SPAN><SPAN class="punctuation token">,</SPAN>
       <SPAN class="string token">'B16005_045E'</SPAN><SPAN class="punctuation token">]</SPAN>
fgdb <SPAN class="operator token">=</SPAN> Path<SPAN class="punctuation token">(</SPAN><SPAN class="string token">'C:/<folder1>/<folder2>/<your_FGDB>.gdb'</SPAN><SPAN class="punctuation token">)</SPAN>
table <SPAN class="operator token">=</SPAN> fgdb <SPAN class="operator token">/</SPAN> <SPAN class="string token">'test_table'</SPAN>
counties <SPAN class="operator token">=</SPAN> fgdb <SPAN class="operator token">/</SPAN> <SPAN class="string token">'County'</SPAN>
<SPAN class="keyword token">print</SPAN><SPAN class="punctuation token">(</SPAN>sum_names<SPAN class="punctuation token">)</SPAN>

<SPAN class="comment token"># In[224]:</SPAN>

r <SPAN class="operator token">=</SPAN> requests<SPAN class="punctuation token">.</SPAN>get<SPAN class="punctuation token">(</SPAN>url<SPAN class="punctuation token">)</SPAN>
<SPAN class="comment token"># r = requests.get(url, proxies=proxyDict) # if you have a proxy server</SPAN>
<SPAN class="keyword token">print</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="string token">'response received'</SPAN><SPAN class="punctuation token">)</SPAN>
df <SPAN class="operator token">=</SPAN> pandas<SPAN class="punctuation token">.</SPAN>read_json<SPAN class="punctuation token">(</SPAN>r<SPAN class="punctuation token">.</SPAN>text<SPAN class="punctuation token">)</SPAN>
df

<SPAN class="comment token"># In[225]:</SPAN>

headers <SPAN class="operator token">=</SPAN> df<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">:</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="comment token"># first row is headers</SPAN>
header_list <SPAN class="operator token">=</SPAN> headers<SPAN class="punctuation token">.</SPAN>values<SPAN class="punctuation token">.</SPAN>tolist<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="comment token"># convert to list</SPAN>
header_list<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="operator token">=</SPAN> <SPAN class="string token">'FULL_NAME'</SPAN> <SPAN class="comment token"># de-duplicate "NAME"</SPAN>
new_columns <SPAN class="operator token">=</SPAN> dict<SPAN class="punctuation token">(</SPAN>zip<SPAN class="punctuation token">(</SPAN>list<SPAN class="punctuation token">(</SPAN>range<SPAN class="punctuation token">(</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">,</SPAN>len<SPAN class="punctuation token">(</SPAN>header_list<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> header_list<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="comment token">#make dict</SPAN>
new_columns

<SPAN class="comment token"># In[230]:</SPAN>

df2 <SPAN class="operator token">=</SPAN> df<SPAN class="punctuation token">[</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">:</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="comment token"># data in rows 2+</SPAN>
df3 <SPAN class="operator token">=</SPAN> df2<SPAN class="punctuation token">.</SPAN>rename<SPAN class="punctuation token">(</SPAN>columns<SPAN class="operator token">=</SPAN>new_columns<SPAN class="punctuation token">)</SPAN> <SPAN class="comment token"># add column names back in</SPAN>
df3<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'fips_append'</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="operator token">=</SPAN> df3<SPAN class="punctuation token">[</SPAN><SPAN class="string token">"state"</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="operator token">+</SPAN> df3<SPAN class="punctuation token">[</SPAN><SPAN class="string token">"county"</SPAN><SPAN class="punctuation token">]</SPAN>
<SPAN class="keyword token">print</SPAN><SPAN class="punctuation token">(</SPAN>len<SPAN class="punctuation token">(</SPAN>df<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
<SPAN class="keyword token">print</SPAN><SPAN class="punctuation token">(</SPAN>len<SPAN class="punctuation token">(</SPAN>df3<SPAN class="punctuation token">.</SPAN>columns<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
df3<SPAN class="punctuation token">[</SPAN>sum_names<SPAN class="punctuation token">]</SPAN> <SPAN class="operator token">=</SPAN> df3<SPAN class="punctuation token">[</SPAN>sum_names<SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">.</SPAN>astype<SPAN class="punctuation token">(</SPAN>int<SPAN class="punctuation token">)</SPAN>
<SPAN class="comment token"># verify type set</SPAN>
<SPAN class="comment token">#for name in sum_names:</SPAN>
<SPAN class="comment token">#    print(name, df3.dtypes[name])</SPAN>
<SPAN class="comment token">#print(df3.dtypes) # prints all, but truncated</SPAN>
df3<SPAN class="punctuation token">[</SPAN><SPAN class="string token">'sumvalue'</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="operator token">=</SPAN> df3<SPAN class="punctuation token">[</SPAN>sum_names<SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">.</SPAN>sum<SPAN class="punctuation token">(</SPAN>axis<SPAN class="operator token">=</SPAN><SPAN class="number token">1</SPAN><SPAN class="punctuation token">)</SPAN>
out_cols <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN><SPAN class="string token">'fips_append'</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="operator token">+</SPAN> sum_names <SPAN class="operator token">+</SPAN> <SPAN class="punctuation token">[</SPAN><SPAN class="string token">'sumvalue'</SPAN><SPAN class="punctuation token">]</SPAN>
<SPAN class="keyword token">print</SPAN><SPAN class="punctuation token">(</SPAN>out_cols<SPAN class="punctuation token">)</SPAN>
df3 <SPAN class="operator token">=</SPAN> df3<SPAN class="punctuation token">[</SPAN>out_cols<SPAN class="punctuation token">]</SPAN>
df3

<SPAN class="comment token"># In[227]:</SPAN>

numpy_array <SPAN class="operator token">=</SPAN> numpy<SPAN class="punctuation token">.</SPAN>array<SPAN class="punctuation token">(</SPAN>numpy<SPAN class="punctuation token">.</SPAN>rec<SPAN class="punctuation token">.</SPAN>fromrecords<SPAN class="punctuation token">(</SPAN>df3<SPAN class="punctuation token">.</SPAN>values<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
names <SPAN class="operator token">=</SPAN> df3<SPAN class="punctuation token">.</SPAN>dtypes<SPAN class="punctuation token">.</SPAN>index<SPAN class="punctuation token">.</SPAN>tolist<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>
numpy_array<SPAN class="punctuation token">.</SPAN>dtype<SPAN class="punctuation token">.</SPAN>names <SPAN class="operator token">=</SPAN> tuple<SPAN class="punctuation token">(</SPAN>names<SPAN class="punctuation token">)</SPAN>
numpy_array

<SPAN class="comment token"># In[228]:</SPAN>

arcpy<SPAN class="punctuation token">.</SPAN>da<SPAN class="punctuation token">.</SPAN>NumPyArrayToTable<SPAN class="punctuation token">(</SPAN>numpy_array<SPAN class="punctuation token">,</SPAN> str<SPAN class="punctuation token">(</SPAN>table<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>

<SPAN class="comment token"># In[229]:</SPAN>

arcpy<SPAN class="punctuation token">.</SPAN>da<SPAN class="punctuation token">.</SPAN>ExtendTable<SPAN class="punctuation token">(</SPAN>str<SPAN class="punctuation token">(</SPAN>counties<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'GEOID'</SPAN><SPAN class="punctuation token">,</SPAN> numpy_array<SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'fips_append'</SPAN><SPAN class="punctuation token">,</SPAN> append_only<SPAN class="operator token">=</SPAN><SPAN class="token boolean">False</SPAN><SPAN class="punctuation token">)</SPAN>

<SPAN class="comment token"># In[ ]:</SPAN>


‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍<SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

davedoesgis

Hi Dan - Regarding print statements, I've been developing these in a notebook, and my code blocks end by executing variable names that show what the value. I normally develop in an IDE (Eclipse/PyDev) with debugging breakpoints, and also use logging, but I was giving Jupyter a try for this. It's new to me, but I really like it for noodling on new things.

Anyhow, I created a list of numeric fields 'sum_names' and cast them all to int in Pandas:

df3[sum_names] = df3[sum_names].astype(int)‍

I'm not sure why that helped, because I still have other fields that are the default 'object' type, but the error went away.

After creating the table, I was going to join it to a feature class. In all my Googling, I also found arcpy.da.ExtendTable, which is really what I needed to do all in one step. Interestingly, that was partially working, in that it added the fields to my table, but all the values were NULL and it returned another cryptic error. When I got NumPyArrayToTable working, the ExtendTable also started working.

Are your blog posts here on GeoNet or somewhere else? I'm loving the power of numpy, but find Pandas a bit more intuitive.

DanPatterson_Retired

You better add some print statements to check that your array is properly formed...

You have passed on the dtype names, but there is no indication of the type format (ie text, numbers etc)

Here is an example.

d<SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">:</SPAN><SPAN class="number token">3</SPAN><SPAN class="punctuation token">]</SPAN>
 
array<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="string token">'alabaster'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'0.7.12'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'py_0'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'python'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN>
       <SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN> snip <SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN><SPAN class="punctuation token">.</SPAN> <SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN>
       <SPAN class="punctuation token">(</SPAN><SPAN class="string token">'argh'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'0.26.2'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'py36_0'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'python'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">,</SPAN>
      dtype<SPAN class="operator token">=</SPAN><SPAN class="punctuation token">[</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="string token">'Name'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'<U30'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="punctuation token">(</SPAN><SPAN class="string token">'Version'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'<U15'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="punctuation token">(</SPAN><SPAN class="string token">'Build'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'<U15'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN>
             <SPAN class="punctuation token">(</SPAN><SPAN class="string token">'Requires'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'<U293'</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">]</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

Note that Uxx is for unicode (aka, text fields), and you can use

('Float_fld', '<f8') for double/float fields and

('Int_fld', '<i4') for integer fields (or '<i8' if you need large integers)

The help topic uses a dictionary format to form the dtype, which is also permissable, but I prefer explicit and you can use the short forms to cast the text/numeric format

NumPyArrayToTable—Data Access module | Documentation

PS, I have tons of blog posts on numpy

Únete a la Conversación

Los miembros registrados pueden publicar, seguir actualizaciones y más. ¿Nuevo aquí? Regístrate gratis.

Enlaces Rápidos

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.