Unable to publish item using Item ID

JaredPilbeam2 · ‎04-08-2021

I want to publish a CSV to Enterprise Portal. So far, I have been unsuccessful. First, I use the content manager to add the CSV to the Portal. Then I try publishing using the Item ID of that newly added item. The item gets added, as I used a print statement to verify the Item ID. The exception is telling me it already exists? Does it not have to first exist as an item in order to publish?

My CSV is attached. Here's my script followed by the error.

from arcgis import GIS, features
#Connection to ArcGIS Enterprise Portal running 10.8
gisE = GIS(url='', username='', password='', verify_cert=False, trust_env=False)

cm = gisE.content
#Add CSV to ArcGIS Enterprise
item = cm.add(item_properties={
    'title' : 'VaccineCurrent2',
    'tags' : 'test',
    'type' : 'CSV'}, data=csv_file)
item_id = item.id
print(item_id)

#Publish
pitem = item.publish(item_id=item_id, file_type='csv')

4b872c25baf7448f845e7c15912f4c9a ##<---this is the ouput of print statement
Traceback (most recent call last):
  File "\\Python Scripts\ArcGISPro\test.py", line 47, in <module>
    pitem = item.publish(item_id=item_id, file_type='csv')
  File "C:\Users\jpilbeam\Miniconda3\envs\env1\lib\site-packages\arcgis\gis\__init__.py", line 10340, in publish
    folder, buildInitialCache, item_id=item_id)
  File "C:\Users\jpilbeam\Miniconda3\envs\env1\lib\site-packages\arcgis\gis\_impl\_portalpy.py", line 408, in publish_item
    resp = self.con.post(path, postdata, files)
  File "C:\Users\jpilbeam\Miniconda3\envs\env1\lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 720, in post
    force_bytes=kwargs.pop('force_bytes', False))
  File "C:\Users\jpilbeam\Miniconda3\envs\env1\lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 514, in _handle_response
    self._handle_json_error(data['error'], errorcode)
  File "C:\Users\jpilbeam\Miniconda3\envs\env1\lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 536, in _handle_json_error
    raise Exception(errormessage)
Exception: Unable to publish item.
Item with Id '4b872c25baf7448f845e7c15912f4c9a already exists.
(Error Code: 400)

EDIT: when I run the analyze() call on the CSV I get JSONDecodeError.

analyze_csv = cm.analyze(item=item, file_type='csv')

Traceback (most recent call last):
  File "\\ArcGISPro\test.py", line 41, in <module>
    analyze_csv = cm.analyze(item=item, file_type='csv')
  File "C:\Users\jpilbeam\Miniconda3\envs\env1\lib\site-packages\arcgis\gis\__init__.py", line 4381, in analyze
    return gis._con.post(path=surl, postdata=params, files=files)
  File "C:\Users\jpilbeam\Miniconda3\envs\env1\lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 720, in post
    force_bytes=kwargs.pop('force_bytes', False))
  File "C:\Users\jpilbeam\Miniconda3\envs\env1\lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 504, in _handle_response
    data = resp.json()
  File "C:\Users\jpilbeam\Miniconda3\envs\env1\lib\site-packages\requests\models.py", line 900, in json
    return complexjson.loads(self.text, **kwargs)
  File "C:\Users\jpilbeam\Miniconda3\envs\env1\lib\json\__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "C:\Users\jpilbeam\Miniconda3\envs\env1\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Users\jpilbeam\Miniconda3\envs\env1\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

FrédéricPRALLY · ‎04-12-2021

Hi Jared,

First check if "Allow data to be copied to the site when publishing services" is actived under server/manager/site.html > data stores > settings.

Second use following modified python code:

from arcgis import GIS, features
#Connection to ArcGIS Enterprise Portal running 10.8
gisE = GIS(url='', username='', password='', verify_cert=False, trust_env=False)

cm = gisE.content
#Add CSV to ArcGIS Enterprise
item = cm.add(item_properties={
    'title' : 'VaccineCurrent2',
    'tags' : 'test',
    'type' : 'CSV'}, data=csv_file)
item_id = item.id
print(item_id)

publish_params = {
  'type': 'csv',
  'latitudeFieldName': 'Latitude',
  'longitudeFieldName': 'Longitude',
  'locationType': 'coordinates'
  }

#Publish
pitem = item.publish(publish_parameters=publish_params)

Hope this can you,

Best regards,

Fred

FrédéricPRALLY · ‎04-13-2021

Hi Jared,

Can you say me if modified python script solve this problem?

Best regards,

Fred

JaredPilbeam2 · ‎04-14-2021

Hi Fred,

Apologies for not answering sooner. I received an error that I've yet to see once I included your snippets. I should note I'm running this in a Conda environment. If I run it in a ArcGIS Pro environment I get the same error.

Here's what I ran:

import pandas as pd
from arcgis import GIS, features
from datetime import datetime as dt
import requests

now = dt.now()
dStr = now.strftime('_%m_%d_%Y')

#Connection to ArcGIS Enterprise Portal running 10.8
gisE = GIS(url='', username='', password='', verify_cert=False, trust_env=False)
#Url to data
j = r'https://idph.illinois.gov/DPHPublicInformation/api/covidVaccine/getVaccineAdministrationCurrent'
#this print statement makes sure it's valid json
print(requests.get(j).json()['VaccineAdministration'])

r = requests.get(j).json()['VaccineAdministration']
csvData = fr'C:/Users/jpilbeam/Test/Test{dStr}.csv'

#Create DataFrame from the json
df = pd.json_normalize(r)
csv_file = df.to_csv(csvData, index=False)

cm = gisE.content
#Add CSV to ArcGIS Enterprise
item = cm.add(item_properties={
    'title' : 'VaccineCurrent2',
    'tags' : 'test',
    'type' : 'CSV'}, data=csv_file)
item_id = item.id
print(item_id)

publish_params = {
  'type': 'csv',
  'latitudeFieldName': 'Latitude',
  'longitudeFieldName': 'Longitude',
  'locationType': 'coordinates'
  }

#Publish
pitem = item.publish(publish_parameters=publish_params)

##I removed the print statement verifying the JSON is valid because it's too lengthy...
8d7744baf55f4c4287486dd830e42280 ##<--print output of item_id
Traceback (most recent call last):
  File "\\pathtopythonfile\test.py", line 48, in <module>
    pitem = item.publish(publish_parameters=publish_params)
  File "C:\Users\jpilbeam\Miniconda3\envs\env1\lib\site-packages\arcgis\gis\__init__.py", line 10332, in publish
    res = self._portal.con.post(path, postdata)
  File "C:\Users\jpilbeam\Miniconda3\envs\env1\lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 720, in post
    force_bytes=kwargs.pop('force_bytes', False))
  File "C:\Users\jpilbeam\Miniconda3\envs\env1\lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 514, in _handle_response
    self._handle_json_error(data['error'], errorcode)
  File "C:\Users\jpilbeam\Miniconda3\envs\env1\lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 536, in _handle_json_error
    raise Exception(errormessage)
Exception: Analyze Service error: Server tool execution failed : ERROR 001506: Failed to download the data for the portal item. Failed to execute (Analyze Features for Portal). Failed. Failed to execute (Analyze Features for Portal). Failed.
(Error Code: 0)

EDIT: The box is checked that allows data to be copied to the Portal when publishing.

FrédéricPRALLY · ‎04-16-2021

Hi Jared,

It's strange because from ArcGIS Pro 2.7.2 on my environment there is no problem.

See result below:

JaredPilbeam2 · ‎04-16-2021

Yes, that seems to be a pattern. I have different versions of the script above and they all seem to work on someone outside of our office's machine/environment. One of these scripts I gave to my colleague to try on his machine--we both have accounts on the same Portal and we both are using the same Pro Python API environment--he gets the same errors as me.

JaredPilbeam2 · ‎04-21-2021

@FrédéricPRALLY

Any idea how to troubleshoot that error? The only help I've found on it was this. Based off this doc, I made sure the folder (which is local) where the *.csv was saved was shared to our organization's admin user account. That didn't solve the error.

Also, when referencing the publish method in the docs it gives you a link for publish parameters: https://developers.arcgis.com/rest/#/Publish_Item/02r300000080000000/

But, this just brings you to the landing page of the ArcGIS REST API. Is this a mistake?