topic Re: Not all scripts copy to server when publishing GP service in ArcGIS Pro Questions

Not all scripts copy to server when publishing GP service

DarrenConly — Tue, 08 Mar 2022 00:14:25 GMT

Hi All,

Here's what's happening

In ArcGIS Pro, make a toolbox script and run it successfully.
Publish as a web tool successfully
BUT, when I run the web tool, I get the following error

I look in the server machine where published tools are hosted and I see that not all of the scripts used by the tool are uploaded to the server. Comparing the two photos below, the photo with more stuff in it is my dev folder, while the one with less stuff is the folder on the server machine. As you can see, when I publish, not all the scripts make it on to the server.

Scripts on dev/desktop

Scripts that actually get published to server

As a workaround, if I manually copy over the scripts that didn't publish, then restart the web service, all works well, but this seems like an inconvenient workaround. Why aren't all scripts publishing? Do I need to tweak my code? Here is how I'm importing modules in the main script--highlighted lines are for modules that did not import*:

* I have an untested theory that if I specify from <module> import <thing> that it will result in the module not uploading. Is this the case? Do I instead need to write import <module> then in my code specify <module>.<thing>?

Re: Not all scripts copy to server when publishing GP service

Luke_Pinner — Tue, 08 Mar 2022 03:49:31 GMT

Your theory is likely correct, this has been noticed previously (see 2nd comment below this GIS-SE answer).

You can also use import <module>.<thing> as <thing>.

Re: Not all scripts copy to server when publishing GP service

DarrenConly — Tue, 08 Mar 2022 16:30:47 GMT

Thanks Luke,

Progress but still no victory yet. Here's what happens:

Rejiggered code to the import <module>.<thing> format as discussed above.
Successfully ran in Pro
Successfully published (after mysteriously getting a few 00068 errors)
Checked in server. As you'll see below, all of the stuff in the main folder got published (yay! Progress!), but the utils folder still didn't publish, even though one of the main scripts (accessibility_calcs.py) calls a script from within the utils folder using the import command import utils.utils as ut.

Re: Not all scripts copy to server when publishing GP service

Luke_Pinner — Wed, 09 Mar 2022 09:47:21 GMT

Why am I not surprised...

I'm assuming since you can import utils.utils then it is a proper python package with a utils/__init__.py and then the other submodules like utils.py etc.

Possible dodgy workaround (I can't test publishing) - the docs only discuss importing modules not packages (other docs discuss how to install external 3rd party packages, but that's not relevant). But there is a section in there that talks about sys.path.append:

Another technique for referencing modules to import is to use the sys.path.append method. This allows you to set a path to a folder containing scripts that you need to import.
...
note that the sys.path.append method requires a folder as an argument. Since r'e:\Warehousing\Scripts' is a folder, the entire contents of the folder will be consolidated.

I paraphrased the quote slightly, but it makes me think you might be able to throw in sys.path.append("utils") and the utils package and all its submodules would get consolidated and published.

I don't think there would be any issue with the import utils.utils call after appending the utils dir to sys.path, I just tried in a simple python script on my local drive and it worked. But if there is, you could always sys.path.pop() or del sys.path[-1] directly after the append.

Re: Not all scripts copy to server when publishing GP service

DarrenConly — Fri, 11 Mar 2022 19:17:27 GMT

Thanks for the tip Luke. I was trying the sys.path.append("utils") approach and for some reason it was giving me a "module not found error". I then tried adding an __init__.py script to the utils folder, but that didn't fix it, but then commenting out the sys.path.append("utils") did seem to fix it. At the same time my Arc Pro has been inconsistent this morning--I'll try the .append() approach perhaps on a different day after restarting my machine.

Just too bad that the utils folder doesn't just import!