publish() to Enterprise Portal in stand alone script

[JaredPilbeam2]

Is it possible to use the API publish() method in a stand alone script? The following script runs from a Pro Notebook using version 1.8.4 of the API. But, if I run the same script as a stand alone I get a JSONDecodeError. I'm trying to publish a CSV to Enterprise Portal. When I put it in a try and except block it catches the error, but nothing publishes

Script that I'm running in Pro:

from arcgis.gis import GIS
import os

gis_dest = GIS('Home',verify_cert=False, trust_env=True)
# data
fp = r'C:\data.csv' #path to CSV
# content manager
cm = gis_dest.content

# 1. add
item = cm.add(item_properties={
    'type' : 'CSV',
    'title' : 'c19_Vaccine_Current_test3',
    'tags' : 'tags',
    'typeKeywords' : "CSV"
    },
       data=fp)

# 2. analyze
analyze_csv = cm.analyze(item=item, file_type='csv')
pp = analyze_csv['publishParameters']

# 3.  Modify the publish parameters then publish
#     For tables ensure the `locationType` is None
pp['locationType'] = 'none'
pitem = item.publish(publish_parameters=pp, file_type='csv')

Line 17 it's referring to is this line:

data=fp)

Traceback (most recent call last):
  File "pathto\test.py", line 17, in <module>
    data=fp)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\__init__.py", line 4265, in add
    owner_name, folder)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\_impl\_portalpy.py", line 355, in add_item
    resp = self.con.post(path, postdata, files)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 720, in post
    force_bytes=kwargs.pop('force_bytes', False))
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\arcgis\gis\_impl\_con\_connection.py", line 504, in _handle_response
    data = resp.json()
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\requests\models.py", line 900, in json
    return complexjson.loads(self.text, **kwargs)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\json\__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[JaredPilbeam2]

I'm posting my whole script in case it helps to have the whole picture. The first part (which writes requested JSON to a CSV) works fine. It only fails where I pointed out in the above post. NOTE: this is easily reproducible.

import json,requests
from arcgis.gis import GIS
import os,sys
import csv
from datetime import datetime

##-------------this part writes requested JSON to a  CSV-----------

#file name
csv_file = 'blankcsv'
#the directory you want the csv written to
file_path = r'C:\Users\jpilbeam\Test'
##idph url goes here
idph_data = 'https://idph.illinois.gov/DPHPublicInformation/api/covidVaccine/getVaccineAdministrationCurrent'

full_file = os.path.join(file_path, csv_file) + str(datetime.now().strftime('_%m_%d_%Y_%H%M')) + '.csv'

#get json data from idph
response = requests.get(idph_data,verify=True)
#read the json response and keep the VaccineAdministration part
data = response.json()['VaccineAdministration']

#write to file
with open(full_file, 'w', newline='', encoding='UTF-8') as csvfile:
    f = csv.writer(csvfile) 
    #write the headers of the csv file
    f.writerow(['County','AdminCount','AdminCountChange', 'RollAvg', 'AllocDoses', 'FullyVaccinated', 'FullyVaccinatedChange', 'ReportDate', 'Pop', 'PctVaccinated', 'LHDInventory', 'CommInventory','TotalInventory', 'InventoryDate'])
    for elem in data:
        #get the values for all the keys (i.e. CountyName, AdministeredCount, etc...)
        f.writerow([elem['CountyName'], elem['AdministeredCount'], elem['AdministeredCountChange'], elem['AdministeredCountRollAvg'], elem['AllocatedDoses'], elem['PersonsFullyVaccinated'], elem['PersonsFullyVaccinatedChange'], elem['Report_Date'], elem['Population'], elem['PctVaccinatedPopulation'], elem['LHDReportedInventory'], elem['CommunityReportedInventory'], elem['TotalReportedInventory'], elem['InventoryReportDate']])

##---------This part (is supposed to) adds then publishes to Portal---------

#provide Enterprise Portal login info here
gis_dest = GIS('portalURL', 'user', 'pass', verify_cert=False, trust_env=True)
#content manager
cm = gis_dest.content

#add the CSV file
item = cm.add(item_properties={
    'type' : 'CSV',
    'title' : 'VaccineCurrent',
    'tags' : 'test',
    'typeKeywords' : "CSV"
    },
       data=full_file)

#analyze the CSV file to autogenerate the publish parameters
analyze_csv = cm.analyze(item=item, file_type='csv')
pp = analyze_csv['publishParameters']
#Modify the publish parameters 
#for tables ensure the `locationType` is None
pp['locationType'] = 'none'

#Publish
pitem = item.publish(publish_parameters=pp, overwrite=True, file_type='csv')

print("---done---")

[MarkKinnaman]

I saw your post on github (and the other post here) and was wondering if you got this to work. I am having the same issue when publishing to portal, but not when publishing to AGOL.

What enterprise/portal version are you trying to publish to? I am currently using 10.6.1.

I took a slightly different approach to you, but got the same result.

import pandas as pd
import json
from arcgis import GIS

j = 'https://idph.illinois.gov/DPHPublicInformation/api/covidVaccine/getVaccineAdministrationCurrent'

r = requests.get(j).json()['VaccineAdministration']

df = pd.json_normalize(r)

#AGOL Connection
gis = GIS(url='URL', username= 'username', password='password')

#Enterprise Portal Connection
gisE = GIS(url='URL', username='username', password='password')

cm = gisE.content

item = cm.add(item_properties={
    'type' : 'CSV',
    'title' : 'VaccineCurrent',
    'tags' : 'test',
    'typeKeywords' : 'CSV'}, data='/media/WDB/Backup/Projects/data/IDataa.csv')

cm.analyze(item=item, file_type='csv', location_type='none')

[JaredPilbeam2]

Hi Mark,

I still don't have a surefire way. I'm using Enterprise Portal v. 10.81. I have a case going with ESRI Support and will report back whenever I find something out.

[JaredPilbeam2]

You are not passing in the f=json parameter needed to get a JSON response from the REST API.

I got some help on my GitHub post, but I don't know enough to know exactly where the f=json parameter is supposed to go? It is in response to the JSONDecodeError on this line:

data=full_file)

[MarkKinnaman]

Alright I will post on Github as well.

If you look at my script below, the json parameter is being passed as a method. This is the same as passing the parameter 'f=json'. You are accomplishing the same thing by using 'response.json()'. At least that is how I understand it.

r = requests.get(j).json()['VaccineAdministration']