I've got a few scripts that I run every now and then to update some of our hosted layers in place. I prefer it to overwriting and truncate/append, as the datasets in question have tens of thousands of features, but only a handful might change from week to week.
Currently, I use pandas to pull updated features from the source database as a spatially-enabled dataframe, then use the compare and merge functions to see which features and attributes have been updated. I then push updates, adds, and deletes to the hosted layer based on the results. I much prefer this method, as it is possible to update my layers in a matter of seconds rather than minutes.
During the development of these scripts, I've noticed something. Suppose I submit a query like this:
service.layers[1].query(where="this_field is like '0%'",
out_fields=['this_field'],
return_geometry=False,
as_df=True)
I might get a response like this:
| | objectid | this_field |
| 0 | 14 | 01 |
| 1 | 19 | 07 |
| 2 | 33 | 02 |
But if I submit a query like this:
service.layers[1].query(where="1=0",
out_fields=['this_field'],
return_geometry=False,
as_df=True)
I get the following:
| | objectid | this_field | that_field | the_other | SHAPE | SHAPE__Area | SHAPE__Length |
Where this trips me up is when this happens to an intermediate output used in a merge, and a later sort, drop, rename, etc., fails because the overlapping columns get new names.
I can deal with this easily enough with some basic if... else... logic, and sometimes by defining custom suffixes on merges. But still, I'd rather the "out_field" parameter be honored regardless of the output.
