Latest Contributions by MarcoBoeringa

Cannot answer your question as I don't have such a setup, but I do know that the distinction between traditional RDBMS and NoSQL databases is blurry these days... E.g. PostgreSQL can store and access unstructured data as well, either being key/value data stored in something like an "hstore" or "jsonb" column, or non-traditional table / unstructured data accessed through Foreign Data Wrappers (FDW). In addition, there are variants of PostgreSQL that run over multiple computation nodes / servers, and PostgreSQL itself is quite capable of using multiple cores and large datasets (I currently have the whole of OpenStreetMap's Europe data in a single PostgreSQL database, with one of the spatial tables having > 218M records). So, I wouldn't be surprised if the spatialtemporal database is just a PostgreSQL database at its core. ... View more

It is slightly unclear what exactly it is you try to achieve: a single Feature Service based on a database JOIN that combines data from tables in multiple (enterprise geo-)databases, or multiple Feature Services in a single webapplication, each showing data from a different (enterprise geo-)database? Anyway, if trying to join data from multiple datasources, then using a MATERIALIZED VIEW might be an option. Oracle, SQL Server and PostgreSQL all support creating such views. Don't forget to index the view, including the spatial column. You can then either: - Register the view with your geodatabase, making it fully "native" to ArcGIS - Or create a Query Layer to access the view's data in ArcGIS. To be honest, I do not know if you can easily incorporate a Query Layer in a web feature service in ArcGIS, you will have to look that up yourself. Note that PostgreSQL also has a lesser known nifty feature called Foreign Data Wrappers (FDWs), that allows you to transparently access a wide range of (unstructured) datasources and use them like ordinary tables in PostgreSQL. So if you have exotic datasources you would like to make accessible, FDW might be the way to go. FDW also seems to include the capability to treat remote tables in a certain PostgreSQL database as native / local to another PostgreSQL database, according to the link I posted, which would avoid replication. ... View more

I think the Geoprocessing Help is quite clear about this. The scratch geodatabase, if used as a geoprocessing environment variable, so called as: arcpy.env.scratchGDB is guaranteed to be there. ArcGIS will automatically create it, even if it is not there. See here: Scratch GDB (Environment setting)—Geoprocessing | Documentation  This makes it safe to write to this environment / workspace even in portable applications / tools. ... View more

marc alfsen‌, did you ever resolve this issue? And if so, what did you do to get it resolved? I am now in a situation where ArcGIS Pro is totally unusable on my home laptop (it runs fine on my desktop). All maps and layouts are permanently stuck on the pause display and Pro reporting the "hardware graphics change" warning. Closing and re-opening the project and maps / layouts doesn't help, clicking on the "Pause" icon in the lower right of the display does nothing to resolve the issue. Upgrading to the latest NVidia driver neither helps. It has an GTX 1050 / 4GB laptop videocard. I was already in a situation where I couldn't run Pro properly on the laptop itself (screen stuck), but could run it on an external screen connected via HDMI (screen normally usable). However, now, after upgrading to Windows 10 v2004, ArcGIS Pro on the external screen is stuck as well on a paused display. I already contacted ESRI support here in the Netherlands before about the issue (which wasn't resolved then, and since I had other things to do, I left it at that at some point since the laptop could still be used on the external screen and I wasn't in a dire need for mobility with this home laptop). All other software, including e.g. ArcGIS Earth, runs fine on the laptop. It is just Pro getting stuck with the map display. Seems I will need to re-open this issue with ESRI... ... View more

Although you do not mention it explicitly, I assume you have ArcGIS for Desktop's "64 bit Background Geoprocessing" installed? Otherwise, of course, everything will run in 32 bit by default. If so, and accessing Oracle through ODBC, you likely also need an Oracle 64 bit ODBC driver, something like this may need to be installed: https://www.oracle.com/database/technologies/odac-downloads.html ... View more

Dappere poging! Zal niet makkelijk zijn om van zoiets als een echte topografische kaart een volledige vector styling te maken binnen de toch wel forse beperkingen van zoiets als Mapbox vector tiles (waarop ook ArcGIS vector tiles gebaseerd zijn), en JSON styles en style editors. Ik vertel je hoogstwaarschijnlijk niets nieuws, maar ik zie voornamelijk nog forse issues met fonts / symbolen die niet goed overkomen doordat het font wellicht niet beschikbaar is in ArcGIS Online, en niet goed schalende symbolen voor shields en embankments / taluds, en de verkeerde verticale stapeling daarvan. ... View more

You know about ArcGIS Query Layers? You can either create a (materialized) database view, if the aggregation is heavy to execute, and add the resulting view as Query Layer, or add the SQL defining the aggregation directly as the Query Layer definition. Just make sure you have unique identifier in the result as well, as that is required for a spatial layer. If you create a table instead of view of the aggregation results, you can use the Add Incrementing ID Field—Data Management toolbox | Documentation tool to add such field. ... View more

Natalia Gutierrez‌ What is your baseline for saying "This tool is taking a very long time even to load very small testing areas" (it shouldn't!), what do you compare it to, and do you have concrete figures for this (x MB *.osm file imports in y minutes)? Yes, there are considerably faster tools, like the command line tools reliant on PostGIS like osm2pgsql and imposm 3, but those are not "ArcGIS friendly", and require quite a bit of IT proficiency and familiarity with Linux to be able to work with them. By the way, you haven't forgot to set the Parallel Processing Factor geoprocessing environment setting when running the tool? Only when you set it, will the tool be able to use multiple cores. Also, be aware that I consider slow HDD based network drives utterly dead for use with OpenStreetMap. To import OSM data, you need fast random access to data on drive. This requires SSD storage. If you have slow network drives, and only an HDD on your local computer, I strongly recommend to get a fast USB 3.x SSD based external drive, and use that to import the data. After import, you can copy the File Geodatabase to your slow network drive for usage. Why do you need fast random access? OpenStreetMap data has its own geometry datamodel, that resembles classic CAD files (or if you have been around long enough, ESRI's ArcInfo "coverage" format), where an explicit Polygon data type does not exist. You may be astounded to hear this, but OSM at its basis only knows about nodes (points / vertex) and ways (lines that may or may not form a closed loop). Polygons are only secondarily defined by 1) either ways forming a closed loop and having been tagged with a tag that is generally recognized to be a polygon (e.g. building=x, a closed way with the tag building=house will be imported as Polygon by every tool I know of), 2) closed ways additionally tagged with an explicit area=yes, and 3) if the way is part of a multipolygon relation, otherwise the way is recognized and imported as Line geometry. There a Pro and Cons to this datamodel: - Pro is that it is a very flexible and efficient data format: the nodes / vertexes of any shared line between polygons are only stored once, and the two ways bordering each other, simply reference them. This makes for a very efficient storage. - Con is that any way / Line, or way / Polygon needs to be reconstructed from its referenced nodes and ways. It is this latter Con that is causing the need for fast random access to data. For any way or multipolygon relation, the import tools need to look up the corresponding nodes and ways that make it up. Only then, is the tool able to create a Simple Features compliant geometry (Line / Polygon), that can be stored in a File Geodatabase feature class, shapefile or PostGIS database spatial column. ... View more

Natalia Gutierrez‌ Be aware that actual building height information is only scarcely available. Some cities like e.g. New York may have pretty extensive building height information, and in some other cases, there may be information about the number of stories / floors / levels of a building, but certainly DO NOT expect anywhere near global consistent coverage of this information in OpenStreetMap. It is a volunteer project after all, and only the things users find interesting or have available source for, will be added. To get some impression if your area of interest has actual building heights, you can visit the osmbuildings or F4map websites, that display OpenStreetMap 3D data based on actual OSM derived data. Note though that F4Map also adds in some more advanced 3D models including texture, that is not in OSM. osmbuildings is in that respect more faithful to the original data. Note that, in order see specific attributes that are not part of default set of fields added, you can run the "OSM Attribute Selector" tool to add new fields. This tool is part of the toolbox. Note that this tool may be relatively slow though on really large datasets, as it needs to extract the information from a binary tag storage field that is added by default. The other option is to specify a custom set of field on initial import using the "OSM File Loader (Load Only)" tool. For your specific question, you probably need to think about these OpenStreetMap keys: - height=x - levels=x - building:levels=x As documented e.g. here: Simple 3D buildings - OpenStreetMap Wiki  ... View more

Natalia Gutierrez‌. Am I right you used the "*.osm.bz2" file directly as input? If so, that would explain the problem, because you need to "UN-bz2" (unzip / decompress) the file. Unfortunately, this cannot be done by the default unzip functionality of Windows, because *.bz2 files are in a different compression format. I use the free 7-Zip tool for that, other generic compress / decompress utilities may be able to do it as well if they support the format. After decompression, you should have a file with only the *.osm extension, which is in fact a plain text XML file, just with a different extension. If the file is small though (< 50MB), you can simply open it in Wordpad and see it's XML content as plain text. It is this *.osm file that needs to be used as input for the tool, not the bz2 zipped one. By the way, I am actually surprised you were able to sort of run the tool. I would expect the tool to error out immediately with a compressed file as input... ... View more

And one other last thing, you are aware of this recent ESRI blog post?: https://www.esri.com/arcgis-blog/products/arcgis-living-atlas/mapping/live-openstreetmap-data-in-arcgis/  It is probably not going to offer you a solution for almost continental size processing as you desire with the US, but it is good to be aware it is available. ... View more

There is one more option that I should have mentioned, and that you may be able to run from ArcGIS Pro: use the Data Interoperability extension. Personally, I have had mixed experiences with this tool for importing OSM data, but it does support PBF directly also if I remember well. The output of the data is not the same as that of "OSM File Loader (Load Only)" tool: it will already break up the data in thematic layers (e.g. your desired buildings!), instead of outputting a Point/Line/Polygon feature class containing all data.   This may actually be desirable for you though. Try it out and see if it works for you. One BIG warning though: once you enable Data Interoperability, ArcGIS will start to support "direct read" of OSM XML and PBF files in the Catalog window. That is: it will show apparent Feature Classes in the Catalog window, that you can select and add to your map. While this sounds great, it is a nightmare when selecting a huge extract like the US: the Data Interoperability "direct read" will start automatic dynamic indexing and reading the entire huge XML or PBF file once you click it in the Catalog window, which is very likely to block your application for ever (hours at least), while it attempts to dynamically index and read the data. This just won't work for large dataset. So: never select or click on large OSM extracts in the Catalog window once Data Interoperability is enabled (it does kind of work with tiny extracts, still I think this is a bad non-scalable implementation). ... View more

The first step is to review the permissions / priviliges your current login / user has, to check if you are allowed to create roles. If you have, then it may be time to contact ESRI support. ... View more

Maybe a stupid question, but do you have database privileges to create roles? Enterprise databases like Oracle, PostgreSQL and SQL Server offer fine grained access to privileges for doing things on the database. Unless your DBA permitted creating roles, the Create Role tool might fail. This is something out of control of ESRI and must be handled by your DBA. ... View more