Pythonw.exe Error

418
5
05-03-2018 12:48 PM
JoshuaBrengel
New Contributor III

Hello,

I keep getting the following error when I run my script in IDLE:

While I believe it is hard to isolate the specific error that causes Python to crash like this (any tips doing that are appreciated!), I've isolated the line of code where the Python crash seems to happen:

arcpy.MosaicToNewRaster_management([stream_mask,sqr_mask],strm_rd_combo,mosaic,"","64_BIT","3.2","1","SUM")

I cannot share the entire code.  The inputs are masked rasters representing elevation data with "sqr_mask" purposely having very large data values.  Also, they are raster objects which have been saved to disk.  All other input parameters seemingly line up with the input rasters' properties.

The funny thing is I've run other similar datasets from different geographic areas/extents and the code completes without any errors.  When I try running the datasets that caused Python to crash in ArcMap instead, sometimes the Mosaic to New Raster tool just stops working and doesn't progress/freezes.  It causes me to restart ArcMap.  Once I restart, I try the tool again using the same rasters and it will complete successfully.

I've tried the following in my script and in ArcMap to avoid getting the script error or tool freezing to no avail:

1) Using Int on the input rasters to reduce potential memory issues

2) Adjusting input parameters- especially mosaic type and pixel type

3) Making the extent of one of the input rasters the extent of the mosaic to new raster tool in the environment settings in script.

4) Switching the order of the input rasters.

5) Restarting and shutting down my machine.

My thinking is there may be a more significant issue with my Python installation.  Maybe some .dll file not installed properly or a path variable that is missing/included that's causing the crash.

As this has been extremely frustrating, any help or pointing in the right direction would be very much appreciated.

- Josh

0 Kudos
5 Replies
DanPatterson_Retired
MVP Esteemed Contributor

The problem usually resides with file and folder constructs.  Examine these

The output location and file type WITH extension are important as are the other 'optional' parameters.  Optional means that you don't get to make the choice, which you should.

MosaicToNewRaster_management (input_rasters, output_location, raster_dataset_name_with_extension, {coordinate_system_for_the_raster}, {pixel_type}, {cellsize}, number_of_bands, {mosaic_method}, {mosaic_colormap_mode})

If you suspect a bad installation, go to Control Panel, Add/remove programs and do a repair.

Try the process in Pro, although the help topics and seemingly the functionality are the same, Pro often gets stuff cone that *Map can't because it is true 64 bit

http://pro.arcgis.com/en/pro-app/tool-reference/data-management/mosaic-to-new-raster.htm

JoshuaBrengel
New Contributor III

Thanks for the reply Dan.  Interesting thought about the file and folder constructs.  Raster objects include the file path and don’t have extensions if they are stored in file geodatabases, right?  My general data structure strategy was to already have file geodatabases created for each step in my script that generated output data.  Then, as I move along in my script, I use arcpy.env.workspace to change the workspace to the file geodatabase where I save the raster object output.  It was my attempt at keeping very organized in my definition of path variables early on in my script. May be a bad strategy but without rewriting my whole script not sure how to avoid it at this point...

Do you suggest that I include all optional parameters even those I don’t want to use? In my case, I suppose would have “ “ for the mosaic colormap mode parameter.

I will try the repair as a more last resort option but thanks for the heads up. Thanks again!

0 Kudos
DanPatterson_Retired
MVP Esteemed Contributor

Joshua... I tend to organize my projects within a folder structure with the aprx in the root of the folder, then a folder for geodatabase(s), generic data, rasters, numpy arrays etc etc.  I either store my rasters as *.tif (now recommended by esri at some level) or *.npy/*.npz (numpy data structures) so I can use them in other programs.  Storing rasters in geodatabases I don't do since it would mean that I wouldn't have access to them outside of and arc* environment.  I just use geodatabases for vector storage and if I need something outside, I keep shapefiles in a designated folder.  When I need to move stuff or share stuff... zip the folder and everything goes along for the ride.

As for where you are storing rasters and the properties that they have depends on what and where you are storing them and a good read (if you follow all the branches), is

http://pro.arcgis.com/en/pro-app/tool-reference/spatial-analyst/analysis-environments-and-spatial-an...

As for keeping everything organized, I do that with respect to the script... 

import sys
script = sys.argv[0]
proj_folder = "/".join(script.split("/")[:-1])
proj_folder

'C:/Git_Dan/arraytools'
So I can reference other folders and/or geodatabases, files etc with respect to the aprx which is located up one level from the script location
JoshuaBrengel
New Contributor III

Thanks Dan!  Your post included data storage concepts and recommendations I've not heard of before.  I'm using ArcGIS for Desktop, but I assume the organizational concepts are transferable.  In any case, having a clean and Pythonic "friendly" file structure is something I'm learning is vital for script performance.  Hard to always know how to build file structures within Arc without issues/crashes, so the post is helpful!  

I've typically operated under the impression that all data types (both raster or vector) should be stored in some type of native Esri storage database.  From experience, I know vector data is often better stored in fgdb, but haven't done extensive Pythonic raster analysis in Arc until now.  Since I will likely share the raster data and want to generally get better at Pythonic raster analysis inside and outside of ArcGIS products, knowing that .tif or numpy array storage is likely a superior storage format is helpful.   Thanks for tips! 

Unfortunately, I don't think I can quite mark as correct, since I'd need to rewrite my script and transform all my data entirely to confirm if file structure/data format is the root cause of my original problem. But, I happily mark as helpful and I think others who have a similar crash can likely avoid hard to debug headaches by using your organizational strategy!  

JoshuaBrengel
New Contributor III

Also, another thought I had that maybe caused this error was the length of my Esri grid raster object names or paths.   I would've anticipated a "000878" or "000161" error if that were the case though.  Plus, most of the names of my outputs in all my other fgdb's are longer than 13 characters and I had no issues/crashes except at this mosaic to new raster step. Is it possible I just got lucky with some of my successful iterations?  In any case, here's a good reminder on Esri GRID formats and names: Output raster formats and names—ArcGIS Help | ArcGIS Desktop 

0 Kudos