Latest Contributions by MarcoBoeringa

‎05-28-2020

Natalia Gutierrez‌. Am I right you used the "*.osm.bz2" file directly as input? If so, that would explain the problem, because you need to "UN-bz2" (unzip / decompress) the file. Unfortunately, this cannot be done by the default unzip functionality of Windows, because *.bz2 files are in a different compression format. I use the free 7-Zip tool for that, other generic compress / decompress utilities may be able to do it as well if they support the format. After decompression, you should have a file with only the *.osm extension, which is in fact a plain text XML file, just with a different extension. If the file is small though (< 50MB), you can simply open it in Wordpad and see it's XML content as plain text. It is this *.osm file that needs to be used as input for the tool, not the bz2 zipped one. By the way, I am actually surprised you were able to sort of run the tool. I would expect the tool to error out immediately with a compressed file as input...

‎05-21-2020

And one other last thing, you are aware of this recent ESRI blog post?: https://www.esri.com/arcgis-blog/products/arcgis-living-atlas/mapping/live-openstreetmap-data-in-arcgis/ It is probably not going to offer you a solution for almost continental size processing as you desire with the US, but it is good to be aware it is available.

‎05-21-2020

There is one more option that I should have mentioned, and that you may be able to run from ArcGIS Pro: use the Data Interoperability extension. Personally, I have had mixed experiences with this tool for importing OSM data, but it does support PBF directly also if I remember well. The output of the data is not the same as that of "OSM File Loader (Load Only)" tool: it will already break up the data in thematic layers (e.g. your desired buildings!), instead of outputting a Point/Line/Polygon feature class containing all data. This may actually be desirable for you though. Try it out and see if it works for you. One BIG warning though: once you enable Data Interoperability, ArcGIS will start to support "direct read" of OSM XML and PBF files in the Catalog window. That is: it will show apparent Feature Classes in the Catalog window, that you can select and add to your map. While this sounds great, it is a nightmare when selecting a huge extract like the US: the Data Interoperability "direct read" will start automatic dynamic indexing and reading the entire huge XML or PBF file once you click it in the Catalog window, which is very likely to block your application for ever (hours at least), while it attempts to dynamically index and read the data. This just won't work for large dataset. So: never select or click on large OSM extracts in the Catalog window once Data Interoperability is enabled (it does kind of work with tiny extracts, still I think this is a bad non-scalable implementation).

‎05-21-2020

If you download the ArcGIS Editor for OpenStreetMap extension (which currently only exists for up to version 10.7.x of ArcMap, and not for ArcGIS Pro. The 10.8 version for ArcMap still needs releasing), you can use the "OSM File Loader (Load Only)" tool to import an OSM XML file (not PBF!) and create a File Geodatabase. Geofabrik also still offers XML as download format, although PBF is preferred due to size and speed, but cannot be handled by this import tool, so is of no use to you. Note that this import process will be a lengthy process for something the size of the entire US. Even though the tool is multi-core enabled via the Parallel Processing Factor environment setting for some stages of the processing, and the tool is thus quite capable, it is still not nearly as fast as some of the other specialized open source tools out there as developed by the OpenStreetMap community (osm2pgsql, imposm3). However, the nice thing about the Editor is that it is entirely "ArcGIS", and doesn't require setting up a PostgreSQL database, nor running a Linux environment to call command line tools, so it is easy and familiar to get "up and running". Note also that this tool will load ALL information in the OSM XML file, including non-building data, which will significantly extent the time needed to import the data. If you are only interested in buildings, you should filter the OSM XML file and write a new XML file with only buildings from the Geofabrik extract using, as your likely best bet for this kind of filtering and re-writing work, the command line Osmium Tool. After writing a new, much smaller OSM XML file only containing buildings, you can then use the "OSM File Loader (Load Only)" tool to import the data into the File Geodatabase. After having imported the data, you can simply access the Feature Classes in ArcGIS Pro, no problem. By the way, I would strongly discourage you to use the "Download, Extract and Symbolize OSM Data" tool to attempt to download the entire US. It will fail. It is not designed to handle large data download loads. Only the "OSM File Loader (Load Only)" tool has the power to handle larger country sized extracts.

‎05-21-2020

The first step is to review the permissions / priviliges your current login / user has, to check if you are allowed to create roles. If you have, then it may be time to contact ESRI support.

‎05-21-2020

Maybe a stupid question, but do you have database privileges to create roles? Enterprise databases like Oracle, PostgreSQL and SQL Server offer fine grained access to privileges for doing things on the database. Unless your DBA permitted creating roles, the Create Role tool might fail. This is something out of control of ESRI and must be handled by your DBA.

‎05-15-2020

Although I cannot speak for ESRI as a fellow user like you, I think the blog post ESRI made about the feature layers, at least partly explains it: Live OpenStreetMap data in ArcGIS There are two things mentioned there: the suggestion they used the Geofabrik regional extracts to do the initial load for their databases, and that, in order to support minutely updates, they had to work with these continental sized extracts instead of the whole planet to make it work with the current infrastructure they have available ("In order to support timely updates of the data..."). I think they may have run into some technical limitations here. As I have processed data myself for the whole of Europe, I know some of the challenges involved. All of Europe's buildings already amounts to close to 180M features! And some 35 GB disk size when exported to File Geodatabase from my PostgreSQL instance. It can be handled from one database, but there may be limitations regarding AGOL (I personally have little experience with using AGOL except accessing already available layers there).

‎05-14-2020

You can't do this in ArcMap using Python, but I am pretty sure you can in ArcGIS Pro using the new Python CIM access. It will take a bit of time though to figure out exactly where in the JSON object model the information is, and you can then use arcpy.mp and the getDefinition() method on a layer object to access it.

‎05-14-2020

As I said, as far as I know, ESRI hasn't released any global vector based contour dataset that could be used for custom styling. If there are contours in a vector tile layer visible, it is usually baked into the raster hillshade that accompanies the vector layers. So you will probably have to generate and host such dataset yourself if you have a need. There are several sources of global DEM available, but it will be quite a bit of work and hassle to create a vector contour dataset of it on a global scale. Such global vector contour datasets are huge by the way. I think I read that the developer of OpenTopoMap had to create a vector contour data dataset somewhere in the range of 1TB or so, to cover the globe.

‎05-10-2020

The error message includes a warning about an emtpy geometry, indicating the integrity of the geometries in your input features class may be compromised. I recommend running the Repair Geometry—Data Management toolbox | Documentation tool to check and repair any invalid geometries before attempting to load the data using the Stage Data Loading tool.

‎05-09-2020

Yes, just make sure that one of the requirements is at least 95% sRGB coverage for the screen of the laptop. sRGB is a color space based on the traditional kathode ray computer screen of the '90s and so, and pretty much the standard for anything that needs to display well on the internet or a screen. Don't get mislead by resolution or IPS. My laptop actually has a slightly better screen than the TN panel with 50% sRGB coverage I linked. It is an IPS panel with wide viewing angle and 1920x1080 resolution, but still has a horrible 65% sRGB coverage or so, far to little for color sensitive design work.

‎05-07-2020

AFAIK, ESRI hasn't yet released any vector based global elevation contour dataset (at least, I haven't been able to find it on ArcGIS Online). The contours you see in the base topographic map are baked into the raster layer, and can thus not be separately styled. It would be nice to have access to a global vector tile elevation contour dataset for styling purposes, but I think ESRI fears the load this would put on their systems if usage soared, as vector contour datasets are heavy stuff in terms of file sizes and vertex complexity.

‎05-06-2020

Hi Matthew Beal, No, I don't think there is a really good way to look this up in Windows settings. You really need to find out the exact make of the panel, and then find specs or some online review mentioning it. But as to the particular color shift you are seeing, it very much looks like what I experienced. I also noticed it first, and especially, in the subtle beige/greyish/light brown color range being distorted. I think they deliberately sacrificed the subtle colors in favor for bright saturated ones with these 18bit panels. So I really wouldn't be surprised if you have a similar type of panel on your laptop.

‎04-29-2020

Not a direct solution to your problem, but I have found DBeaver database IDE a really nice interface to manage my PostgreSQL database including views. Added benefit is has syntax highlighting and automatic formatting of SQL, something missing in ArcCatalog.

‎04-29-2020

Be aware that some laptops are still sold with poor 18-bit color TN panels as a price/performance compromise, and they aren't necessarily the cheapest ones, that can only display about 50% of the sRGB color space. E.g. my own Acer Aspire VX15 gaming laptop, which is mid to high range in terms of price and high on performance (runs Pro decently on 32 GB RAM and 4GB video card), has such a screen. In the shop it looked OK running some music video with bright colors. However, for any color sensitive work, it just sucks, the colors are way off due to the 18-bit limitation, versus 24/32 bit color on proper screens. I plug it into a proper desktop screen for any real work regarding styling of maps. E.g., see this review of a similar Acer laptop having such a screen: Acer Aspire V15 V5-591G review - solid specs and great price, but...

Online Status	Offline
Date Last Visited	‎05-29-2026 03:22 AM

My Ideas

Latest Contributions by MarcoBoeringa

Re: Can I download OSM data and save as a file geodatabase?

Re: Can I download OSM data and save as a file geodatabase?

Re: Can I download OSM data and save as a file geodatabase?

Re: Can I download OSM data and save as a file geodatabase?

Re: Create Role Error

Re: Create Role Error

Re: Why is there no worldwide OSM building coverage in LAOW?

Re: Label Expression Parser

Re: Can we make the Topo Layer lines stand out more? bold? darker?

Re: Stage Data Loading Failed

Re: ArcGIS Pro on my work laptop is has horrible color display. Is there anything I can do to fix it?

Re: Can we make the Topo Layer lines stand out more? bold? darker?

Re: ArcGIS Pro on my work laptop is has horrible color display. Is there anything I can do to fix it?

Re: ArcGIS Pro should allow you to see the SQL used to create a database view

Re: ArcGIS Pro on my work laptop is has horrible color display. Is there anything I can do to fix it?

Re: Hinduja Swiss (Switzerland) IT-Professional-Ho...

Re: Issues with Editors using the new threading op...

Re: Issues with Editors using the new threading op...

Re: Issues with Editors using the new threading op...

Re: Issues with Editors using the new threading op...