10.1 ExtractData.py script not the same as embedded Extract Data tool script

[GerardoGarza]

I have been creating services in 10.1 using geoprocessing tools created in 10.0.
One tool was based on the Extract Data Task in the Server Tools toolbox.
I ran the tool which completed successfully, but when I tried to publish results as geoprocessing service, I received the following:

Error - Code 68 - Script Extract Data contains broken project data source: gdb
Warning - Code 24032 - Data source used by Script Extract Data is not registered with the server and will be copied to the server: gdb

I then used the 10.1 ExtractData.py script to refactor the 10.0 script and received the same error.

When I ran the Extract Data Task directly from the Server Tools toolbox, it published successfully.
When I copied the Extract Data Task and script to a new toolbox, publishing failed again with the above error.
After some more research and testing, I realized that the Server Tools Extract Data was using an embedded script which was different from the Extract Data.py script in the ArcGIS\Desktop10.1\ArcToolbox\Scripts folder.

Question:  Is it possible to get copy of the functional embedded ExtractData.py script?

[SteffenSchillinger]

Hi, I'm having the same problem and contacted ESRI support about it because I think it's a bug. Could you solve it or has anyone sent you some hints? Why do you think the tools run embedded scripts that are different from those in ArcGIS\Desktop10.1\ArcToolbox\Scripts folder? I tested this with the ExtractData.py script and added a message command to the original code and it shows up when I execute the original Server Toolbox model.

Best,

Steffen

steffen.schillinger@gmx.net

[KevinHibma]

This shouldn't have anything to do with the ExtractData script, nor matter if its embedded or not.
The analyzer error 00068 says that the packaging/publishing process cannot find required data to complete publishing. (broken project data required for the service)

I've never seen that error pop up in regards to the Extract Data task.
We'll bang on it here to see if we can reproduce it, but I'm sort of at a loss as why thats happening for you.
Instead of publishing as a service, can you save it as a geoprocessing package - same error, or no?
Can you very clearly spell out your workflow? (maybe you're doing something I'm not...?)

thanks

[SteffenSchillinger]

Dear Kevin, thanks for answering. The error also pops up when trying to publish a package. The workflow follows the example at http://resources.arcgis.com/en/help/main/10.1/index.html#/Clip_and_ship/005700000073000000:

1. Copy Extract Data Task from System Toolboxes\Server Tools\Data Extraction to MyToolboxes\New Toolbox
2. Edit model and set layers to clip and other parameters as in example
3. Execute Model -> works
4. Publish Model as Geoprocessing service -> works, no errors when analyzing (underlying data of MXD have already been registered with the server and been published as map service)
5. Copy Extract Data Script tool from Server Tools\Data Extraction to MyToolboxes\New Toolbox
6. Copy ExtractData.py from ArcGIS\Desktop10.1\ArcToolbox\Scripts to different folder on local drive (i.e. c:\work)
7. Set source property of MyToolboxes\New Toolbox\Extract Data model script to c:\work\ExtractData.py
8. Delete Extract Data model tool from MyToolboxes\New Toolbox\Extract Data Task model
9. Drag and drop MyToolboxes\New Toolbox\Extract Data to model and reassign/define parameters and links
10. Execute model -> works
11 Publish model as geoproc service -> error pops up when analyzing

Resume: same model script without any modifications causes this error when replacing original script tool with it in Extract Data Task model


regards and thanks,

Steffen

[KevinHibma]

Steffen,

Thanks for the detailed steps.
Using steps 6-11, on a ArcGIS 10.1 final machine I can reproduce the issue.
Using a 10.1 SP1 machine (we havent released SP1, we're finalizing it right now), I cannot reproduce the issue - so we've fixed it.
My apologies for the inconvenience.

A question though - why do you need to copy the script and model tool to a custom toolbox?
If you want to control parameters (for example, you want to remove options from the feature choice list, or you dont want to expose the raster format parameter) you can run the tool from the system toolbox, and then during publishing modify those parameters inside the service editor. Just click the parameter name on the left hand side and change the Input Mode, or configure the Choice List options. Unless you're adding new values to the data extract format task, everything else you want to do should be configurable through the service editor.

I hope this will provide a suitable workaround till SP1.

[SteffenSchillinger]

Dear Kevin, thanks a lot for checking this so quickly. I was afraid I had to wait for SP1 - unfortunately, I have another incident pending (Silverlight Viewer can't display image service as basemap) and support told me that SP1, which would be released later this month, will fix this, too. Isn't there a way through EDN or so to get a beta version of SP1? My task requires to customize the Python source code of ExtractData.py and I thought best way to start is using a copy of the original file. Do you think I could publish a geoproc service building the model and model script from scratch?

In any case, many thanks for clarifying this. You were faster than ESRI support 😉

Cheers,


Steffen.

[KevinHibma]

No, we dont do "beta" service packs, only the final release of them.

If you absolutely require a custom version of the extract data task, do you still need feature parameter input? This is the problem, if you can either hardcode or change that parameter, you can probably get around the analyzer problem.

The tool does a name match, so its possible (i havent tried this though) to play with the name inside the parameter and then change the code in the tool. Of course if you dont need the feature type parameter, the best solution would be to hardcode it in the script and not expose that parameter. Again, I think this will work, but no guarantees.

[SteffenSchillinger]

Hi again,

you mean the Value List in the Feature Format parameter causes the problem? Because it uses the string ".gdb"? That's why it's looking for a data source named "gdb"? Well, thanks for the hint! I can hardcode-limit this to .shp only until SP1 is available. I'll give it a try and post my results.

Steffen.

[KevinHibma]

Hi again,

you mean the Value List in the Feature Format parameter causes the problem? Because it uses the string ".gdb"? That's why it's looking for a data source named "gdb"? Well, thanks for the hint! I can hardcode-limit this to .shp only until SP1 is available. I'll give it a try and post my results.

Steffen.

exactyl! 🙂

[SteffenSchillinger]

No, we dont do "beta" service packs, only the final release of them.

If you absolutely require a custom version of the extract data task, do you still need feature parameter input? This is the problem, if you can either hardcode or change that parameter, you can probably get around the analyzer problem.

The tool does a name match, so its possible (i havent tried this though) to play with the name inside the parameter and then change the code in the tool. Of course if you dont need the feature type parameter, the best solution would be to hardcode it in the script and not expose that parameter. Again, I think this will work, but no guarantees.

Hi Kevin, I tried this but without success. I even completely deleted the two format type variables from the model and hard-coded the getparameter string to shapefile. The model runs and does its job but the analyzer error remains.

Besides, why would this error occur only when the script is not in it's original location? Seems a bit odd to me. SP1 addresses this error for example

NIM081276 - Sharing some script tools that use Network Analyst tools as a geoprocessing service incorrectly raises analyzer error "Results contains
broken project data source".

, and since this seems to fix the before mentioned error, too, I guess there are things wrong in the model builder itself.


Best,

Steffen