Aster Data

[GregDowning]

I have been trying to use ASTER Data to use as a DEM in CityEngine. I also converted it to a geotiff in Globalmapper just to make sure it got the coordinate data. I found it surprising that CityEngine is prompting me for a coordinate system. I thought this would be embeded in the GeoTiff. None of the selections I have made have worked, they produce obviously wrong terrain. Anyone know what I should be choosing for this?

[MatthiasBuehler1]

Hi Greg.




If importing 32bit GeoTIF files, you need the additional metadata files (that data is not embedded) :
- .prj (the actual projection)
- .twf ('world file', defining the extent)

If you work with any georeferenced data, the first thing you want to do is set the SCS, the Scene Corrdinate System. That is usually done in a new empty scene.
E.g. if you have created an empty scene and drag-n-drop the GeoTIF into the viewport, the import dialog will ask you to select the SCS ( read the window title : Select Scene Coordinate System ). This will not come up if it was set before, e.g. by importing footprint data.

So the very best to do is to ( if necessary ) reproject all GIS data you have ( ESPECIALLY the DEM ) to the target SCS you'll use in CityEngine. There's no way around that when using CE when using georeferenced data.

Here's an other informative thread to read :
http://forums.arcgis.com/threads/54072-textures-raster-files-concepts-in-CityEngine-ArcGIS ( especially note point 5] )

Btw. did you already see the tutorial videos ? 😮
http://forums.arcgis.com/threads/64843-CityEngine-Collection-RESOURCES-FAQ-HELP


Cheers and let me know if you have more questions .. I guess so. 🙂

Matt

[GregDowning]

None of the ASTER files I have downloaded from usgs http://earthexplorer.usgs.gov/ have these files. However, I did find this in the attached PDF that came with the files. Is this the info I need?
- 5 -
[INDENT]
GDEM PACKAGE
The basic unit of the ASTER GDEM is the 1 x 1 tile. Each GDEM tile includes a DEM (.dem), a quality assessment (QA, .num), and this readme file (.pdf), all of which are compressed into a single �??.zip�?�. Both data files (.dem and .num) have dimensions of 3,601 samples by 3,601 lines, corresponding to the 1 x 1 tile area.
- 6 -
Figure 1 ASTER GDEM File Structure
The names of individual data tiles refer to the latitude and longitude at the geometric center of the lower-left (southwest) corner pixel. For example, the coordinates of the lower-left corner of the tile ASTGTM_N00E006 tile are 0 degrees north latitude and 6 degrees east longitude. ASTGTM_N00E006_dem and ASTGTM_N00E006_num files accommodate DEM and QA data, respectively. The rows at the north and south edges, as well as the columns at the east and west edges, of each tile overlap and are identical to the edge row and column in the adjacent tile.[/INDENT]

[MatthiasBuehler1]

Hey Greg.


What's the specific reason you're going for ASTER data ? Additional precision ?

Note that in CityEngine it's recommended not to use textures ( e.g. such dem 'raster files' ) bigger than 4K. Somewhere between 6 and 8 K they'll not even be displayed since the files get too big.

I've seen that the necessary metadata is not available on the website you mentioned.

I have an other site, recommended by my coworker. I also have a workflow how to process the data afterwards to bring it to CE.

Do you have a copy of ArcGIS ( ArcMap ? ) accessible ?


Let me know ..

Matt

[GregDowning]

No I was mainly using Aster as it was the highest res (I think it is 30m) and maybe only DEM that I could find for Athens. The files are not so large, 3100 pixels. If you know of a better source I would be happy to use something else.

Sadly I do not have ArcGIS or ArcMap. The only GIS like app I have is GlobalMapper.

[MatthiasBuehler1]

Hi .. Here's the steps for use in ArcMap ( a part of ArcGIS ). I'll mention more things below.

Ok ?

Matt

1]
- http://glcfapp.glcf.umd.edu:8080/esdi/index.jsp
- > Map Search
- Left : Elevation Data : check 'SRTM, WRS2 Tiles'
- zoom in, use mouse cursor with the arrow and a '+' icon to select a tile ( gets red )
- preview & download
- choose under attr list, the best ID :
-- 'filled finished' ( filled missing data, processed data, not raw data )
-- 1 or 3 arcseconds precision
-- note that the dems are projected in UTM
- click on id, download
- download files (get the tif file)

2]
unzip file

3]
- Open ArcMap, load image (create pyramids)
- RMB on layer
- Export Data > Downsample or clip, [leave default extent usually unless it's bigger than 6K on sides]
- Export to new GeoTif file
- > gives .tfw 'world' file ( writes a file of metadata which is GeoTif)

3]
- Open new empty CE scene
- Find UTM zone, e.g. http://www.okono.com/art/maps/utmZones.gif
- Set UTM zone in preferences/scene : Scene Coordinate System (SCS) ( use wild card search *UTM*34*, WGS 84, North )
- drag-n-drop file into viewport
- check correct position by finding some footprints of the city.

Notes :
----------
1]
both 'NaN' and '0' values of a GeoTif are mapped (in CE 2012) to the elevation range max of the terrain, producing a spike. Processed data should have no spikes, otherwise, you'll need to fix the 16bit or 32bit dem file. (There's tools for this in ArcGIS too.)

2]
The reason why you can not use that DEM directly is that CityEngine needs 2 things for each GeoTif :
- it's .tfw file which defines the terrain's extens. This is actually in the GeoTif, but CityEngine only reads .tfw files and does not check the metadata of the file.
- it's .prj file. Though this is actually ONLY used if it's the first georeferenced data you import in a blank scene: To set the SCS. If the SCS is already set, you don't need the .prj file.

3]
CityEngine can only RE-PROJECT vector data such as polygonal shapes (e.g. footprints) and polylines ( street center lines ) on import if the imported data's projection is different than the SCS. CityEngine CAN NOT reproject 'raster files' ( textures, such as GeoTifs ) on import.
==> Thus I highly recommend to project all data to your target SCS projection prior to importing. This way, it will not depend on which file you import first in CityEngine in a blank scene.

4]
Editing Dems in Photoshop or such will most probably fuck up the float values, especially if you're resizing images or going from bit level to bit level. imageJ gives you a simple way to sample precise float values, even on 32 bit images : http://rsb.info.nih.gov/ij/

For now, I guess I can help you out with producing e.g. the tfw files if you can not do that withing Globemapper so you don't have to buy ArcMap now.

Let me know if this all makes sense to you ! 😮

Matt

[GregDowning]

I downloaded the files from the Earth Science Data Interface site you recommended I noticed that there were two files for each area that was "filled and finished" One named SRTM_ff03_p183r034.tif and the other SRTM_ffB03_p183r034.tif in Global Mapper the image that does not have the "B" in it's name is red and purple with the land in red and the other is more traditional false color.
[ATTACH=CONFIG]22652[/ATTACH]

What is the difference between these? Can either be used? Sadly the SRTM for this area is lower res than the ASTER, only 90 meter is available.

I was able to produce the .tfw file w/ Globalmapper, but not the .proj file. If I were to select the "NGO 1948 UTMZone 34 N ESRI:102134" would this have the same result?

does the .tfw file go into the maps directory w/ the Geotiff?

So far I have no results. Thank you for offering to convert the SRTM for me, it was too large so I have temporarily put it here, http://www.xrez.com/SRTM_ff03_p183r034.tif.gz and http://www.xrez.com/SRTM_ffB03_p183r034.tif.gz I also have the ASTR if you would like to check that as well since it is higher resolution http://www.xrez.com/ASTGTM2_N38E023.zip .

[MatthiasBuehler1]

Hi,


The difference seems to be that 'B' is better than 'A' ( or nothing ).
See here :
http://glcf.umd.edu/data/srtm/

You you always want to go for something like 'Filled Finished-B'.
The version with the holes seems also to have a slight different projection than the 'B' version.
[there's about 4'000 projections, so it's a real mess to understand]

* * *

Do you know http://www.openstreetmap.org ?
That's a free source of footprints, road center lines and such. You can import this too into your scene.

OSM data is unprojected ( has big coordinate values ), so you can import them directly into a georeferenced scene.

My coworker got me this link where you can find all osm data for greece :
http://download.geofabrik.de/europe.html

In this link, you also find the .shp equivalent, which I used to test if the DEMs are properly aligned.


* * *

I played around with all the files:

I put all the files in ArcMap, see map_1.png to map_4.png.

Note that I see the basic raw data in ArcMap, so GlobalMapper itself makes these wrong-color-ramps to display the values, that's not the actual information in the file.


Indeed, your ASTER DEM is better quality, since it's extent is much smaller, with a bit more resolution than the SRTM files.

So what I did is RE-project your ASTER dem in ArcMap to 'WGS-84 UTM 34 N', which is a widely used projection system and also of course supported in CityEngine. This new GeoTif file comes with the .tfw 'world' file and since we know the projection, we can set the SCS directly in a new scene in CityEngine.

> means that you either have a .prj file or you set the projection in CityEngine. The result's the same. Again, CE can not reproject images ( raster files ), so this has to be done manually. Importing the osm data to check the position in CityEngine is also done easily.

NOW :
I've seen that 1 spike gets created in CityEngine (see Spike.JPG), most probably due to a reprojection error (if I import your file without georeferencing info, there's no spike). Not sure how to isolate that sucker. Also, the center of the 'most important city' in the area is off the edge of your DEM. So I'm not sure if you're happy with that DEM tile or want to choose 1 tile further south.

Let me know and I can send you the reprojected DEM and upload to your ftp. (maybe send me the access details via PM)


Cheers !

Matt

[GregDowning]

Wow, thanks Matt! I owe you a few beers next time I see you.