arcpy AsciiToRaster cannot convert ascii files that contain a lot of 'nodata (-9999)'

[RYokochi]

I have 570 ascii files with headers (example shown below). About 400 of them opens as ascii or I can convert to raster using arcpy 'AsciiToRaster'. But I have problem with coastal cells containing relatively large fraction of 'Nodata_value' cells.

I first thought it could be the format of nodata_values, but some files contain smaller number of nodata-values works, so this is probably not the case. The coastal cells I can open does not seem to have exact matching coastal line with the baseman. The coordinate system of the ascii data is JGD2011.  

Anyone could advise what the problem is and how I could fix? 

-----

NCOLS 225
NROWS 150
XLLCORNER 130.3
YLLCORNER 32.675
CELLSIZE 5.555555333335368e-05
NODATA_VALUE -9999

[DanPatterson]

Is there an error message?

How are you opening them?

[RYokochi]

Thank you for asking! 

The error message is: Error 010067: Error in executing grid expression

Failed to execute (ASCIIToRaster) 

I tried both ways (from python and from ArcGISPro Tool). 

I attach the file I cannot open. Ascii files are not supported so it was zipped. 

[DanPatterson]

Sounds like you aren't specifying a *.tif as the output format and/or your destination workspace has spaces in it.

It you are doing this through a script ensure that you are using "raw" encoding for your path, make your path short and simple and use *tif files as the output raster type

Also, no matter how I try every line has a line break after every number so it doesn't comply with the header, so compare the "worked" vs the "didn't work"

NCOLS 225
NROWS 150
XLLCORNER 130.125
YLLCORNER 32.683333333
CELLSIZE 5.555555999999721e-05
NODATA_VALUE -9999
3.40
15.10
17.20
21.80
27.30
29.20
28.10
27.30
28.60
30.10
31.30
30.30
27.90
24.40
21.20
18.20
15.10
12.00

[RYokochi]

Thank you for checking!!!
I processed the files that worked in the exact same way, and a large fraction of them get converted to rasters. The files that worked is attached this time. The ones that does NOT work are along the coast lines with a lot of -9999. But everything else is the same (or at least to me it seems).

I also attach a screenshot of the map that contains all the patches that worked. As you see, the ones that did not work are all by the coast and have smaller file size (a lot of -9999). Some of the working files do have -9999 but only sparsely. 

[Luke_Pinner]

---

@DanPatterson wrote:

Also, no matter how I try every line has a line break after every number so it doesn't comply with the header, so compare the "worked" vs the "didn't work"

---

The use of space or newline separators doesn't matter as long as the number of values = NCOLS x NROWS.

[Luke_Pinner · Accepted solution]

Your A01-20-DEM5A.asc is corrupt. The header specifies NCOLS 225 and NROWS 150 which means you should have exactly 33750 (225x150) values in your asc file, but there are only 16063 values.

If I pad out the file so there are 33750 values, then the file will convert (but obviously is missing lots of actual data).

See below (I've used black to represent -9999, blue for data values and green for the padding values that I added just to demonstrate).

[RYokochi]

Thank you so much for pointing this out. I converted the original file provided as xml (attached) and it does say 225x150, but you are correct, and the number of values is lower than anticipated in the original file, too. 

It seems that the file with more -9999 have smaller number of values. I was looking at wrong problem. 

As I tried more things, it turned out that some of the ascii files can open directly in ArcGISPro if I wait for a long time (like 30 min or 1 hour). I also noticed that the cells with its LLcorner in water seems to be the ones that are problematic, sometimes showing up as a strangely cut mosaic.   

This problem turned out to be caused by the original file and not the ArcGIS, so I close this for now. But I will post if I ever find what to do with these files in case someone else works on Japanese DEM database:  
https://fgd.gsi.go.jp/download/mapGis.php?tab=dem

P.S. I am new here. If anything missing, please let me know. 

[RYokochi]

Going back to the .xml file, I realized that there is 

<gml:startPoint>16 78</gml:startPoint>

I suspect that the data starts from there [col,row]=[16,78]

This will not make exactly 16063, but closer than 33750. 

The files that work do not always have 33750, and I suspect that this was the reason. 

[Luke_Pinner · Accepted solution]

Have a look at "demtool" for converting Japanese GML DEMs:

• https://github.com/tmizu23/demtool
• https://www.ecoris.co.jp/contents/demtool.html