topic edit_features null values becoming 0s in Python Questions

edit_features null values becoming 0s

MackWood — Fri, 12 Mar 2021 23:54:58 GMT

I am trying to use edit_features to update a hosted table with data from a pandas dataframe. The process runs without error, but the null values in the dataframe are automatically converted to 0s in the table as shown below. Is there a way to prevent this because 0s in these fields have an inherent meaning and they should be missing, not 0.

Thanks!

Re: edit_features null values becoming 0s

DanPatterson — Sat, 13 Mar 2021 00:42:22 GMT

strange, but the second table looks like the values are a text representation of integers (left justified, with the floats converted to integer).

NaN only applies to floats, there is no NaN for integers, the normal procedure is to assign an integer that can't exist in the data set (smallest negative or positive integers are often used).

So in short, if conversion to integer was done (inadvertantly or not), ... 0 .... is the substitute in the absence or other rules of

Re: edit_features null values becoming 0s

MackWood — Mon, 15 Mar 2021 15:57:57 GMT

This is the output I am trying to get. When I append data manually using a csv file I am able to get the nulls. These values are stored as integers in the hosted table.

Re: edit_features null values becoming 0s

DanPatterson — Mon, 15 Mar 2021 16:01:53 GMT

As indicated, there is no "null" for integers, they are converted to "0" by default, so my suggestion stands about using a value that can't possibly exist in your data set (eg -9999999 or whatever), then deal with the null-replicants as you see fit.

Re: edit_features null values becoming 0s

MackWood — Mon, 15 Mar 2021 16:04:39 GMT

How does the current hosted table have nulls as integers?

Re: edit_features null values becoming 0s

DanPatterson — Mon, 15 Mar 2021 16:14:37 GMT

they are handled internally ... masking if you will, just like they are in ArcGIS Pro. There are real NaNs for floats, just not integers. Your first table has the columns as floats in the dataframe, hence NaNs, your second table is integers hence no NaNs (NaInt not an integer doesn't exist)

So either make sure that they are floats on transfer, deal with the zeros or exclude the records that are NaN is pretty well your options

NaN - Wikipedia

Re: edit_features null values becoming 0s

MackWood — Mon, 15 Mar 2021 16:18:14 GMT

Gotcha, I will try to figure out how to have the python script mask them as they are handled when manually updating then. Because the maps that are pointing to this data use the missing values as a specific symbology for census tracts with no data and I'm trying to not have to mess with the map too much.