Publishing a geoprocessing toolBOX

[KevinGooss]

I see how i can publish a gp tool by creating the tool, running it in arcmap, and then selecting the results and clicking Share As...
That creates an sd file that i can then use in ags manager to publish my tool.
The problem is that i have a toolbox with 30 tools in it. I don't want to run all 30 and go through this process. Can't i just publish the entire toolbox somehow?

Also, why the change in publishing gp services? I liked being able to go into catalog, connect to my ags, then right click and add new service based on a python scripted tool.
that seemed like a great workflow because i could then edit the python script in the background without having to reup my service (unless i changed params).
It appears that under the new workflow I cannot modify my python script without having to open arcmap and rerun everything.
This will add much time to my dev process.

[MattMoyles]

I've got to agree with Kevin Gross, and Aaron Barkhurt. I hope some ESRI employees read this thread because you guys really seem to have dropped the ball on 10.1 when it comes to publishing.

My first complaint with publishing is that you cannot publish an arcobjects executable as a GPService on an ArcServer. This is a major regression. I don't believe it's your place to say if it's a security vulnerability or not. That's up to our team. I wasted 2 expensive support tickets to find out that you can't publish an executable. I ended up having to wrap it in a python script calling my executable as a sub-process. It feels hacky to say the least.

I also agree with everything that Aaron Barkhurst said about publishing python GP scripts. Changing my code at publish time is an insult to the developer. You guys seem to be catering to a crowd who is not interested in learning arcpy or programming in general at the expense of power users.

Model Builder is a useless tool. It may look good on your powerpoint slides at the ESRI conference but it promotes not learning how to program. Organizations that use model builder can be found using tools like Entity Framework and DevExpress. Your code is likely very weakly tested and buggy. Enforcing the publishing successful of results is just catering to this crowd, again, at the expense of power users.

We have a ton of python classes and libraries and encapsulating them into their own working directory at publish time just does not work, especially when publishing to the ArcServer is rewriting my code and changing paths and directories. We ended up doing the same thing listed above. Which is to just publish a dummy python script and then going into the ArcServer system folders and copying in the real script.

We are now tasked with trying to implement a continuous integration pipeline from source control to published gp services on our development ArcServer. If it's able to be done it's going to be very hacky.

So I will end by saying this. ESRI should be giving developers tools that empower them and make their lives easier, and not boxing them in because some people are too lazy to learn.

10.1 has got me interested in looking into PostGIS.

[KeG]

I am in the same boat here. Horrible experience just trying to publish a simple toolbox with three tools in it. Analysis takes 30 minutes. Publish takes an hour. And that is for ANY change i make in ANY script in those 3 tools.
How is this possibly a working solution?
I could create separate services for each tool but that is overkill. These tools are related and should go in a toolbox.
Sp1 didn't improve upon anything. The script solution adds many, many more steps into the process. I have to save out results, create an sddraft, turn that into and sd, take that and then publish it? Come on.

[NikolasStevenson-Molnar]

I'd like to add by 0.02 and provide a workaround for those struggling with this problem.

First, I agree with many of the previous posters that the process for deploying GP tools to server is extremely kludgy and unnecessarily burdensome. It's un-intuitive (publish from the result of a tool run?), and makes a number of problematic assumptions: namely that you're deploying from a machine with ArcGIS Desktop, that you're deploying individual tools, not toolboxes (unless I've missed an option?), and that the tools are even intended to run in a Desktop environment. Additionally, the extra burden to deploy tools does very little to ensure tools are fully functional before deployment. Any tool with even a bit of complexity will have multiple execution paths and what may succeed with one set of inputs may fail with another. The only way to ensure the functionality of all execution paths is to write good tests, and frankly you can't force people to do that; they either will or they won't.

Now, for a workaround. You can by-pass the whole run->puslish-from-result process by using the ArcGIS Server REST API. If your sources are already present on the target server (e.g., my deployment process involves pulling sources from a Mercurial repository onto the target server), then you can POST to the "createService" admin endpoint with JSON-formatted service info to create the service (http://resources.arcgis.com/en/help/server-admin-api/createService.html) and then POST to the "startService" admin endpoint (http://resources.arcgis.com/en/help/server-admin-api/startService.html) to start the previously created service. Wrap this all up into a deploy script and you'll be set.

If your sources aren't already on the server, I imagine you could use the upload API to get them to the server, though I haven't tried this approach.

Server is sort of finicky about creating and starting services, so be prepared for a bit of trial and error. Often, if you get something wrong, it will let you create the service, but you won't be able to start the service. If your service fails to start, check the server log for clues about why. I found that the "outputDir" and "virtualOutputDirectories" parameters are actually required despite being reported as "optional" by the API docs. If you create your service without those parameters, then it will fail to start.

Cheers, and happy deployments!
_Nik

[DaveHighness]

I agree with these posts. Publishing Python GP Tasks has become a big pain under 10.1. I now routinely create two scripts, the real one and a dummy one with just the input and output parameters. I publish the dummy and replace it with the real one once published. This saves me some time in publishing but I still can't publish a toolbox at once like I did in 10.0. It shouldn't have to work that way. Why can't I be trusted to test and debug my own scripts? Please give those of us who know what we are doing to ability to force publish a toolbox as is!

Thanks, Dave

[KeG]

Couldn't agree more with the last two posts.
You made some great points _Nik and thank you very, very much for those tips.
between your method and Dave's that's two new ways around this issue.

I have resorted to going back to the old tbx method and am not using the pyt python toolbox that i was so looking forward to.
I could never get the calls out to resources to work the way i wanted.
I have 45 gp tools that i need to publish and the pyt file would be enormous if i used it as anything other than just a placeholder for params and a pointer to py files.

To publish those 45 tools took me over 5 days! That is cost to me and to my clients. Now i have a (precious) mxd with successful results for every one of them. However, i had to publish them all as separate services because i couldn't just publish a toolbox. This is clutter in my services directory.

ESRI needs to undo this in my opinion. I think they were going for a level of handholding that will allow casual desktop users to start cranking out services. But that is not the way i use the software.
And there is a marked difference between a gp script that is designed for desktop and one designed for ags publishing. they need to recognize that and split the workflows - not try to incorporate them.

[LakshmananVenkatesan]

All,

I am scared of reading all the threads. We have developed custom geoprocessing tools using ArcObjects by extending
Geoprocessing framework. We have 70+ tools with wide variety of inputs. This tool box has been published as
Geoprocessing service in ArcGIS Server 10.0. This was simple and straightforward exercise. 

But I dont think this can be done easily in ArcGIS Server 10.2 version. I am still figuring out the way to achieve this. if any one has some experience please point me on this. Our plan to migrate to 10.2 is mainly depend on this. I cannot run a each tool and generate a result and share them as service is NOt possible, because every tool input might be different based on work flow. I dont see this as a alternative.

Please throw some light on this.

[AndreasRuloffs1]

Hi,
I don't like the new deployment either.
Our normal workflow is that we develop the tools in our system and later publish the well ready developed and well tested at our customer. Especially when deploying at the customer a too complicated and too long workflow does not make us look good.

But I have another ugly thing with publishing. Our Scripts normally contain several files, the script itself, a config script and a general helper script. The last ones are imported in the first one.
Publishing the script copies only the script itself but not the other files, so after publishing executing the script fails.
I have to manually copy the files into the path Kevin mentioned earlier.
I have registered the path where the scripts came from as data store, but this does not prevent the copy.

Bye,
Andreas

[DanielGarcia]

I agree with these post, but I think that Nikolas give us the best option. Actually I use the "createService" rest service method on the root or desire ArcGIS Server folder. Like this...

http://servername:6080/arcgis/admin/services/foldername/createService

I use the attach json code to publish a tbx file. This tbx has a lot of tools that reference to many others python scripts. All of this are located into a registered folder to ensure that ArcGIS Server has permissions.

This process give me the best way to publish tbx.

Regards

Dani