Latest Contributions by MarcoBoeringa

Although I cannot speak for ESRI as a fellow user like you, I think the blog post ESRI made about the feature layers, at least partly explains it: Live OpenStreetMap data in ArcGIS  There are two things mentioned there: the suggestion they used the Geofabrik regional extracts to do the initial load for their databases, and that, in order to support minutely updates, they had to work with these continental sized extracts instead of the whole planet to make it work with the current infrastructure they have available ("In order to support timely updates of the data..."). I think they may have run into some technical limitations here. As I have processed data myself for the whole of Europe, I know some of the challenges involved. All of Europe's buildings already amounts to close to 180M features! And some 35 GB disk size when exported to File Geodatabase from my PostgreSQL instance. It can be handled from one database, but there may be limitations regarding AGOL (I personally have little experience with using AGOL except accessing already available layers there). ... View more

You can't do this in ArcMap using Python, but I am pretty sure you can in ArcGIS Pro using the new Python CIM access. It will take a bit of time though to figure out exactly where in the JSON object model the information is, and you can then use arcpy.mp and the getDefinition() method on a layer object to access it. ... View more

As I said, as far as I know, ESRI hasn't released any global vector based contour dataset that could be used for custom styling. If there are contours in a vector tile layer visible, it is usually baked into the raster hillshade that accompanies the vector layers. So you will probably have to generate and host such dataset yourself if you have a need. There are several sources of global DEM available, but it will be quite a bit of work and hassle to create a vector contour dataset of it on a global scale. Such global vector contour datasets are huge by the way. I think I read that the developer of OpenTopoMap had to create a vector contour data dataset somewhere in the range of 1TB or so, to cover the globe. ... View more

The error message includes a warning about an emtpy geometry, indicating the integrity of the geometries in your input features class may be compromised. I recommend running the Repair Geometry—Data Management toolbox | Documentation tool to check and repair any invalid geometries before attempting to load the data using the Stage Data Loading tool. ... View more

Yes, just make sure that one of the requirements is at least 95% sRGB coverage for the screen of the laptop. sRGB is a color space based on the traditional kathode ray computer screen of the '90s and so, and pretty much the standard for anything that needs to display well on the internet or a screen. Don't get mislead by resolution or IPS. My laptop actually has a slightly better screen than the TN panel with 50% sRGB coverage I linked. It is an IPS panel with wide viewing angle and 1920x1080 resolution, but still has a horrible 65% sRGB coverage or so, far to little for color sensitive design work. ... View more

AFAIK, ESRI hasn't yet released any vector based global elevation contour dataset (at least, I haven't been able to find it on ArcGIS Online). The contours you see in the base topographic map are baked into the raster layer, and can thus not be separately styled. It would be nice to have access to a global vector tile elevation contour dataset for styling purposes, but I think ESRI fears the load this would put on their systems if usage soared, as vector contour datasets are heavy stuff in terms of file sizes and vertex complexity. ... View more

Hi Matthew Beal, No, I don't think there is a really good way to look this up in Windows settings. You really need to find out the exact make of the panel, and then find specs or some online review mentioning it. But as to the particular color shift you are seeing, it very much looks like what I experienced. I also noticed it first, and especially, in the subtle beige/greyish/light brown color range being distorted. I think they deliberately sacrificed the subtle colors in favor for bright saturated ones with these 18bit panels. So I really wouldn't be surprised if you have a similar type of panel on your laptop. ... View more

Not a direct solution to your problem, but I have found DBeaver database IDE a really nice interface to manage my PostgreSQL database including views. Added benefit is has syntax highlighting and automatic formatting of SQL, something missing in ArcCatalog. ... View more

Be aware that some laptops are still sold with poor 18-bit color TN panels as a price/performance compromise, and they aren't necessarily the cheapest ones, that can only display about 50% of the sRGB color space. E.g. my own Acer Aspire VX15 gaming laptop, which is mid to high range in terms of price and high on performance (runs Pro decently on 32 GB RAM and 4GB video card), has such a screen. In the shop it looked OK running some music video with bright colors. However, for any color sensitive work, it just sucks, the colors are way off due to the 18-bit limitation, versus 24/32 bit color on proper screens. I plug it into a proper desktop screen for any real work regarding styling of maps. E.g., see this review of a similar Acer laptop having such a screen: Acer Aspire V15 V5-591G review - solid specs and great price, but...  ... View more

Robert, As someone who is active in the OpenStreetMap community, but not working for ESRI, I do have a couple of remarks regarding your concerns. Note that I speak fully on personal terms here as a community member! I am also not a legal representative of the OpenStreetMap community: ESRI only copies / displays the existing data, OpenStreetMap data content is not curated or owned by ESRI (or for that matter anyone). It is a community project residing under the OSMF (OpenStreetMap Foundation). So for any questions regarding OpenStreetMap content, you likely should not be contacting ESRI, but use one of the communication channels of OpenStreetMap (forums, mailing lists etc., e.g. OpenStreetMap Help Forum , OpenStreetMap: Geo News you can use! ,  GIS - OpenStreetMap). If you have concrete evidence that transmission lines, substations or other infrastructure in your region was directly copied / imported from data that does not conform to the Open Database License (ODbL) of OpenStreetMap, then you should contact the Data Working Group of the OSMF (OpenStreetMap Foundation). They take care of illegal imports not conforming to the ODbL. That said, although I do not know the details, "There is an indication that one of our lines was added 12 years ago." seems a claim that needs much better substantiation. There is a lot of highly detailed data in OpenStreetMap, often carefully crafted from detailed aerial imagery, that may seem similar to other datasets, but has been independently digitized by community members. In addition, legal imports do take place after consent of data owners and if the data's license is ODbL compatible. See the OpenStreetMap Import Guidelines for the recommended practice. On the one hand, I understand your concern regarding Homeland Security, but on the other hand, it is not that you can hide powerline infrastructure. Even if it does not appear on the map, high voltage lines and substations are visible from miles away due to their size, and can easily be found on the ground or identified from aerial imagery as available almost everywhere. So, as a personal opinion, I don't think removing power infrastructure from OpenStreetMap, would add any security to these vital infrastructure elements. In addition, there are many countries around the world, e.g. here in the Netherlands where I live, where powerlines and substations are a formal part of official government produced topographic maps, and have no special status. OpenStreetMap is a community project, it can only thrive in an atmosphere where people accept that there may be local country specific differences in content added to the map (actually, the underlying database), but that any rendering in a map has to be a neutral, not country specific, representation, and may include something like powerline infrastructure. While there are a few known exceptions, like Google showing different content and specifically country borders to users in some countries, it is virtually impossible to create a styling that takes care of all sensitivities in each country. It is already hugely complex to render a map as detailed as OpenStreetMap is, let alone take care of >200 country specific styling wishes. The only realistic way that the visual map on www.openstreetmap.org therefor can differ in content, is if the local - country specific - community has broad consensus to not map specific content. I actually personally only know of one such case: In Israel, there seems to be broad consensus among the local community, to not map any military infrastructure (which is being mapped in other countries), and remove anything added by community members not aware of this local consensus. In your case, if you feel action is needed, you would need to convince the local US chapter of the OpenStreetMap community, to have consensus about not mapping powerline infrastructure in the US in OpenStreetMap. That will be a tough call I think, but is the only realistic way. ... View more

Late response, and you probably solved your issues by now, but what is "lots of vertices"? Thousands, millions? If you get over 100k vertices per polygon or line geometry, things can slow down dramatically. E.g. in a custom Python scripting and generalizing workflow, I've been processing data with records of up to 1.7M vertices per polygon in a PostGIS database generalizing them. I discovered by monitoring pgAdmin, the database management software of PostgreSQL, that it could take half a day for a single record to be processed when dealing with geometries > 1-2M vertices. I don't know if it is acceptable for your workflow, but by dicing them to 100k vertex limit, processing times were dramatically improved. E.g. for one medium long running dataset (although not containing that 1.7M monster polygon), it reduced processing times from 1h23m to just over 4-5 minutes. Dicing may not always be acceptable, but in my case it was, and it helped solving the performance issue. ... View more

Hi Megan, My remarks were more of general note and warning to anyone reading this thread, not to specifically criticize your work. I understand the difficulties with this type of work, and also the cost involved with analysis, and how all needs to be balanced. Sorry to have side stepped the question a bit, and not actually help you out with your problem, but good to read you managed to solve it yourself. ... View more

Actually, one of the first things I did when receiving that data set of Radon measurements, was to simply load it in ArcView, and set a graduated color legend. That simple act of "exploratory data analysis" already showed me the kind of random nature of the measurements of the dataset, with a confetti of light and dark colors abounding. One thing to keep in mind as well is the importance of a sense of "scale" of the phenomena you are trying to measure. E.g. if you wanted to interpolate a DEM based on height measurements in a coastal dune area, than clearly sampling one point per square kilometer isn't going to deliver results, and you just get random noise. Dunes generally vary on scales of dozens of meters (some exceptions like the ultra large coastal dunes in the Namib desert), hence you need to sample on a much finer scale. Ground surface height is also generally not a phenomena well suited for something like (kriging) interpolation, as ground surface can vary abruptly as well (ridges, canyons etc.). In general, it is the smoothly varying data like groundwater levels in sandy soils, pollution spreading in aquifers, ocean or the atmosphere, that shows strong spatial autocorrelation and is most suitable for interpolation. That said, there have been some interesting developments in the past 20 years or so, also with the incorporation of new interpolation techniques in Geostatistical Analyst like "Emperical Bayesian kriging", "Indicator kriging", and "Aerial interpolation", that allow you to deal with less "standard type" of datasets (An introduction to interpolation methods—Help | Documentation ) ... View more

As a nice anecdote about how things can go wrong if you forget that some data sets are unsuitable for interpolation: About 15 years ago, due to my Kriging Interpolator extension being available on ArcScripts, I was contacted by a fellow working for an environmental agency in one of the Nordic countries here in Europe. They had this fantastic data set with more than 25000 measurements of Radon gas in basements of houses across the country, including detailed data about the age, construction and condition of the house, and on what soils it was located if I remember it all well. All-in-all a treasure trove for a statistician. Radioactive Radon gas is a health thread, and they wanted to have a country wide map showing the regions with high and low radon in houses. Ergo, they wanted to interpolate the data, at least, that was their first idea of solving this issue. So why did he contact me then?... Turned out the data showed almost 90-100% nugget variance, and hence virtually no spatial auto-correlation, which they didn't understand. They had this fantastically detailed data set, and now they couldn't interpolate it?! After receiving the data, I could confirm their observation, virtually no spatial autocorrelation between Radon measurements. Even measurements in houses separated by only a small distance, could vary wildly in Radon. So, this set me thinking, why is this? The problem is, Radon is highly correlated with the specific condition of the house: on what soil type is it located, what building materials have been use, how is the ventilation status of the basement? None of these factors are ones that necessarily have high spatial autocorrelation, with smoothly varying changes. In fact, things like soil type and construction often vary abruptly: a soil or rock type is often abruptly changing from one spot to another, when geological processes like sedimentation, uplifting, volcanic activity etc. have changed the origin of the underground layers. The same holds true for human influenced factors like building construction. As a consequence, Radon in houses can vary abruptly and wildly as well in any random selection of buildings, even in the same town or even the same street. All of this precluded interpolation as the scientific and statistical method to analyze the Radon data! And the 90-100% nugget variance showed it. Yet the fellow that contacted me, had a hard time digesting this: a fantastic data set of Radon measurements across the country, and now they couldn't interpolate it to a country wide map? I ultimately suggested to him, to use traditional statistical methods with software like R and SPSS, to establish correlations between all the factors and conditions they had so zealously collected, and the corresponding Radon measurements in each house. After all, this huge data set with all its measurements and ancillary data, still was a treasure trove for a statistician! After establishing correlations, they could then use this knowledge to assess risk factors and regions at risk based on things like a geological soil map. If one or more soil types showed to be highly correlated with high Radon levels, simply classifying a geological soil map based on the statistically determined relations, might give a coarse, but statistically sound, indication of regions "at risk". The same could be done for building construction: if some types of buildings were shown to be at risk, classifying the countries buildings based on the statistically determined risk factors, could give them an indication of the status of all buildings in the country. ... View more