New 3D basemaps solution

[Gert_van_Maren]

Today we released our new 3D basemaps solution. This solution includes a series of workflows that can be used by mapping technicians to create and maintain a 3D basemap for their organization, leveraging existing data such as lidar, building footprints and underground utility features. The output of the workflows is a collection of 3D layers which can be used to serve as a 3D basemap for desktop, mobile and web mapping applications.

Leveraging lidar to create a 3D basemap.

This new release is a major enhancement of the existing local government 3D basemaps solution which has been moved to mature support. New in the 3D basemaps solution is the ability to:

• create 3D bridges using classified lidar and procedural rules 
• create 3D power lines using existing GIS data such as power pole locations
• create underground pipes from existing utility data

We also improved the task workflows and structured them around a layer based output. For more information, check out the "Getting to know 3D basemaps" storymap or visit the ArcGIS Solutions website. 

[Anonymous User]

Hello, I'm having trouble using this tool, I think it has something to do with my point clouds coming from a Photometric program (Pix4d) and not from Lidar. I tried to do it in several ways, but it's either does not work or it marks different kinds of errors. Does anyone know how to fix this problem? 

The first image is showing how when using it to extract the building footprint it turns out empty. 

Building Footprint extraction empty

These other two images show how when using it to extract the elevation surfaces from the LAS dataset it kind of detects the buildings but still has lots of problems defining a polygon on the dtm, it processes the dsm totally empty and the dsm as just one big square polygon.

Error on extracting elevation surfaces from LAS Dataset (theZoom in

[Gert_van_Maren]

Hi Raul,

I had a look at your data.

The problem I think is that there are not enough ground points in this dataset so that ArcGIS can create a proper dtm. Also it seems a number of points are misclassified as ground because they seem part of a roof?

Gert

[GabrieleVozza]

Hi, congratulations for your work with 3D basemaps.

Unfortunately, I have a problem with a tasks:
- Segmenting building footprints using elevation

In "Segmenting building footprints using elevation" I get this error:

Error on line 1023
Error 000854

Here you can see an explanatory image.



How could this error be resolved?

I also encountered another error in: "Create footprints from raster".

Here an image.



I solved this last error/problem by using the following geoprocessing tools:
-LAS Point Statistics as raster
-Elevation Void Fill
-Con
-Raster to Polygon
-Regularize Building Footprint.

I don't know if the layer generated in this way could be a problem for "Segmenting building footprints using elevation".  
Alternatively, is there a method for "Segmenting building footprints using elevation" using geoprocessing tools?

I am currently testing the solution with your example data.

[Gertvan_Maren1]

Hi,

It seems odd that this happens with the sample data. Did you make a selection of a smaller area? In any case, it should work so I really would need to know exact steps (with all input data and input parameters) to be able to reproduce this error. Maybe you can make a small video that shows the steps taken with the sample data?

Regards Gert

[GabrieleVozza]

Hi Gert, Thank you for your reply!

I have prepared 3 videos. As they are a bit heavy (in terms of GB), I have included the links. If you want I can send you the videos privately.

In the first video, I process sample data. 
I carry out all the steps described above.
Video 1

In the other two videos, I developed my own small dataset (only one building). 

I have encountered the problem that the layer processed with "Create Footprint raster" is empty. I have solved the problem with the path described above, but I don't know how correct it is. In addition, the segmentation via DSM is still not produced.
Video 2

Video3

I would like to add that by processing the data with a building footprint taken from open street maps (for example) I do not experience any problems in extracting the roofs.

Regards,

Gabriele

[Gertvan_Maren1]

Hi Gabriele,

Thanks for the videos. I managed to reproduce this and it has to do with the regional settings. I made some fixes in the code and it would be great if you could test this before our next release in June.

If you send me your email address, then I will send you a link to the updated solution.

Please use: gvanmaren@esri.com

Thanks Gert

[Mokwena9]

@Gert_van_Maren I have used the 3D basemaps solution to create 3D Building Multipatch, but my buildings fall into the ground  WHEN I include any elevation source? I discovered that the reason could be the elevation source is using different units (FEET) compared to the Building Multipatch Layer(METRES), If I select a single unit for all the layers, the buildings drop into the ground

[Gertvan_Maren1]

Hi,

Can you share a small sample area so I can test?

Thanks gert

[Mokwena9]

@Gert_van_Maren how should I share the data? Because I sent an email to gavnmaren@esri.com and it was not delivered

[Gertvan_Maren1]

Hi,

My email is gvanmaren@esri.com

Gert

[Gertvan_Maren1]

Hi,

2 things:

• You need to modify the dtm around your buildings. These buildings are on a slope and when creating, they get the lowest dtm value as base elevation. You can use the ‘Modify ground Surface task in the 3D basemaps solution to modify the terrain in the location of the building.

• Secondly, more segmentation of the building footprints will help placing buildings on sloping terrain. If automatic segmentation does not improve it, then you can do manual segmentation also supported in the 3D basempas solution.

You didn’t send me the dsm. If you send that I can see if I can improve the auto segmentation.

Does this make sense?

Gert

[VHolubec]

Hello,

I have a problem with this Solution on Pro 2.9 with sample data as well as with custom data. 

With sample data:

Error in file name: C:\Users\vholubec\Desktop\3DBasemapsNew\scripts\bm_common_lib.py
with error message: TypeError: cannot unpack non-iterable NoneType object
Failed script Extract Elevation From LAS Dataset....

With custom data (non-classified): 

Looking for class codes:
Class codes detected:
Unhandled exception: invalid literal for int() with base 10: ''
Error on line 293
Error in file name: C:\Users\vholubec\Desktop\3DBasemapsNew\scripts\bm_common_lib.py
with error message: TypeError: cannot unpack non-iterable NoneType object
Failed script Extract Elevation From LAS Dataset...

Have you encounter the same problem? 

Thank you.

[Gertvan_Maren1]

Hi,

I haven't seen this before with testing the sample data on various machines and configurations. Where did you place the sample data? Is it on a local drive with a simple path, like c:\temp? If not can you try this?

Custom data: the las data needs to be classified (ground classification is minimum). You can use: 'Classify LAS Ground' in Pro.

https://pro.arcgis.com/en/pro-app/2.8/tool-reference/3d-analyst/classify-las-ground.htm

Let me know if this works.

Gert

[VHolubec]

Hi Gert,

thank you very much. 

It seems, that there was some problem with Las Dataset name (the name was 03258575.lasd). When created new one EsriSampledata.lasd and import the smae las file, it has started to work. 

After classification of my custom data it has worked fine too.

Once again thank you very much!

Vladimir

[JayHodny]

Hi Gert,  I have deployed the 3D Basemaps solution and worked through the tasks using the sample data.  For the most part I was successful in developing the 3D products.  I did have a couple of errors, which I did not capture well, but pressed on.  I then worked through the "Extract 3D buildings from lidar data" learn.arcgis tutorial.  This one went well.  Now I am starting the "Extract roof forms for municipal development" learn.arcgis tutorial.  This tutorial instructs one to download the 3D Basemaps solution and deploy it, I assume so that I have an empty Pro project to start the exercise with.  Do I simply download and install the solution again, creating multiple folders in AGOL?  Will this overwrite my original folder/work on my computer?  I struggle to understand how I can tap into and reuse the solution tasks in other 3D practice attempts if this makes sense?  I have some lidar data from 2014 that I want to run the solution steps as practice.  I am not sure how to proceed to preserve my work so far, while continuing on with the learning.  Strange questions I fear, but many thanks for any advice.  Regards, Jay

[Gertvan_Maren1]

Hi,

The Extract 3D buildings from lidar data learn lesson focusses on just buildings and uses a different methodology than the 3D Basemaps solution. You can compare the 2 and see which works better on your data.

The "Extract roof forms for municipal development" learn.arcgis tutorial leverages the Create Buildings task from the 3D basemaps solution so you do not need to repeat this i.e. it is the same workflow. Typically I copy an empty 3D basemaps project to a separate folder on my desktop if I start to work on a different project / location. In AGOL, i have separate folders for each project but I share layers between projects if this makes sense. So I have a 3D basemaps folder with buildings, trees, etc. and those layers gets shared in other projects.

It would be great though if you could capture the errors in the 3D Basemaps solution so we can fix these.

Regards Gert 

[JayHodny]

Hi Gert,

Thanks for the reply, this was helpful.  I had this error while publishing the power line work:

I had the same error when I went to publish two of the three underground sanitary sewer layers, the manhole layer published ok.  Same error as with the power line publishing.

The trench step failed as well:

I hope this helps.  Thanks, Jay

[Gertvan_Maren1]

Hi Jay,

Did you convert the underground lines and power lines to multipatch features? It is part of the tasks and is necessary to publish.

Gert

[Mokwena9]

I have created 3D Buildings Multipatch using the 3D basemaps solution (left image), but it fails to create tall/block of flats in some areas. @Gert_van_Maren @Gertvan_Maren1 Can you kindly assist on what I can do to improve this?

[Gertvan_Maren1]

Hi, 

You will need to manually segment these building footprints according to areas of equal height. Then you can run the Extract Roof Form tool again,

Regards Gert

[fahmiresali]

Hi @Gert_van_Maren 

I have problem with this solution for Creating Powerlines using customized data. I getting error JunctionPoints_Points does not exist or is not supported
Failed to execute (CopyFeatures).

Full error message as below:

Failed to execute. Parameters are not valid.
ERROR 000732: Input Features: Dataset C:\Users\Training\Sample3DROW\Intermediate.gdb\JunctionPoints_Points does not exist or is not supported
Failed to execute (CopyFeatures).
Error on line 379
Error in file name: C:\Users\Training\Sample3DROW\scripts\bm_common_lib.py
with error message: TypeError: cannot unpack non-iterable NoneType object
Error on line 527
Error in file name: C:\Users\Training\Sample3DROW\scripts\bm_common_lib.py
with error message: TypeError: cannot unpack non-iterable NoneType object
Failed script Create 3D Transmission lines...
Failed to execute (Create3DTransmissionlines).
Failed at Thursday, 23 June, 2022 5:00:19 PM (Elapsed Time: 23.20 seconds)

Have you encounter same problem? I believe it need to be checking on the python scripts as it is using customized data (different kV) but I have checked and not sure where to fix it.

Thanks

[viewpropartner]

Hello @Gert_van_Maren @fahmiresali 

I am currently working on ArcGIS Pro 3.0.0 version and recently started a new project to create 3D buildings using the ArcGIS Solutions (3D Basemap).

While performing two following tasks, I am getting errors that I am not so sure how to solve. 

Segment Roof Parts

Extract Roof Form

 

Can you please offer suggestions on how to resolve this?

Can it be related to the newer Pro version for which 3D Basemap Solution is not compatible? If yes, can you retrofit the 3D Basemap solution to work with the most current Pro version 3.0.0?

I would appreciate you looking into this promptly.

[Gert_van_Maren]

Hi,
Please download the lates release of the 3D basemaps solution which has support for Pro 3.0:
https://doc.arcgis.com/en/arcgis-solutions/latest/reference/introduction-to-3d-basemaps.htm
Regards Gert

[mfresali]

Hi @Gert_van_Maren 

Thanks for your reply, another issue I'm facing is the error compiling when using cga rules from CityEngine 2021.1, I have copied the exact similar cga code from the solution (which is version 2018) and put it on my CityEngine 2021.1 and compile it as .rpk, when I'm trying to generate transmission line using the .rpk it keep giving me the error. Is it related to version compatibility issue?

[Gertvan_Maren1]

Hi,

Can you share the rule package? That would make it easier to find out what is causing the issue.

Gert

[mfresali]

Hi @Gertvan_Maren1 I have sent the rule package on your email.

Thanks

[Wiama99]

Hello @Gert_van_Maren 

I would like to know if you need a special type of license to deploy the solution ? I'm novice to arcgis solutions and even when i activate another portal the solution is still not accessible (see screen below) 

[Gertvan_Maren1]

Hi,

You can download the solution on ArcGIS Onlien or Enterprise portal:

https://doc.arcgis.com/en/arcgis-solutions/latest/reference/introduction-to-3d-basemaps.htm

https://doc.arcgis.com/en/arcgis-solutions/11.0/reference/introduction-to-3d-basemaps.htm

Regards Gert

[Wiama99]

Thank you @Gert_van_Maren 

[WillySimons1]

Hi Gert,

I downloaded the latest 3D basemaps solution which supports ArcGIS 3.0 but the Create Buildings script fails to extract the height from the elevation surfaces. I have tried it with  the following 2 ArcGIS lessons: 

• Extract 3D buildings using photogrammetry

• Extract roof forms for municipal development

No problems with the first lesson on ArcGIS Pro 2.9, but I obviously don't want to go back to an earlier version.

Kind regards,

Willy Simons

[Gertvan_Maren1]

Hi Willy,

Can you send the error log please?

Gert

[WillySimons1]

Hi Gert,

I sent you the log via Direct Message. I had another look at the problem and it appears that the script doesn't apply the procedural fill as it did in ArcGIS Pro 2.9. Should be an easy fix?

Regards, Willy

[Gertvan_Maren1]

Hi Willy,

I never received a direct message. Please email to gvanmaren at esri.com

Thanks Gert

[VHolubec]

Hello,

why there is in the 3D basemap solution the same tool for building creating twice?

Once as an Extract roof form and then as Create Buildings - both pointing to the same Python script?

Thank you for clarification.

Vladimir

[VHolubec]

Hello,

one more question to 3D Basemaps process - when running thru Python, it returns quite a lot of  errors, but at the it seems that all is done.

Is there any issue with, or this is something we shouldn´t been concerned?

Thank you,

Vladimir

[EBrayshaw]

Hi !

Thanks for the solution.

I'm currently practicing with the Portland, Oregon sample data following the 'Working with the ArcGIS Solution for 3D Basemaps' ArcGIS lab.

I'm getting issues whenever there is a task that has a 'previous_[insert]' tab to fill in. The first was with the task: 'Extract Elevation from LAS Dataset'.

There is no information on what this means, just that it's 'required'. I run the tool by inserting what I think best (there are no drop-down options only type in), it runs and outputs nothing. On the ArcGIS lab there is no mention of it whatsoever, nor is it seen on any video I've watched that goes through the solution. I was able to get around this by making the DTM, DSM, and nDSM manually.

However, I'm now trying to run the 'Segment Roof Parts' where I'm getting the same issue.  The tool runs and seemingly is successful but I get no output.

The only similarity I can see is that the corresponding task to the tool is 'Extract roof form information' which has a 'previous_layer' section, again with no information on what need to be inputted.

I've tried this multiple times by starting from scratch in case there was anything wrong in my data paths but had no luck unfortunately. I eventually need to do these tasks with my own data but it seems strange that it's not working as should with the sample data and following the ArcGIS lab tutorials. Also that the tasks do not match the documentation.

Any help would be appreciated. Thanks!

[Gertvan_Maren1]

Hi,

Please connect with @NicholasPopovich . He owns the solution now and should be to help you out.

REgards Gert

[Allison_Hockey]

Hello,

Wondering if I can get some assistance with an issue I am experiencing with a task in the 3D basemaps solution.  I'm trying to run the 'extract tree points using cluster analysis' task and am having a coordinate system issue for my LASD. Here's a picture of the error: 

And here's a picture of the spatial reference information of the lasd (the individual tiles are all the same):

To me, it looks like all the units are in feet for the Projected CS, the Geographic CS, and the vertical CS. There's something big I'm missing and would be so grateful if someone could teach me what I need to know. Is there something that doesn't jive between NAVD88 and NAD83? Where should I be looking to see that the 'vertical and linear units' aren't the same? Any guidance is very much appreciated.

@NicholasPopovich 

[VHolubec]

Hello @NicholasPopovich,

I am encountering errors with tool Segment footprints using elevation - the log is full of: 

Segment Mean Shift failed for batch grid 12. Original polygons will be used for this batch.

 Please could you provide more information about this to we know how solve these?

Thank you.

Vladimir

[Gertvan_Maren1]

Hi Allison,

Looking at the error message, it seems 1 .las file fails. Could you have a look at the units for this file specifically?

Gert

[Gertvan_Maren1]

Hi Vladimir, 

It could be that for this batch it has encountered corrupt original polygons and is unable to do any segmentation.

Gert

[VHolubec]

Thank you very much for an idea @Gert_van_Maren ! I will check my polygons. 

[Allison_Hockey]

Hi @Gert_van_Maren 

thanks for responding to my inquiry. The script is throwing an error on the first las file it grabs. All of the las files have the same CS. I tried removing that specific file from the lasd and it does the same thing. I used the extract las tool and removed the vertical CS from the lasd and it threw a different error (no features were found, I believe.) 

Here is the CS information on that specific las file:

This could certainly be an issue of the user not knowing what she doesn't know and if so, I'd like to learn!

To get around this, and in the interest of time, I created the trees running  surface analysis and didn't have this issue. I would certainly prefer to do it with cluster analysis if possible. Looking forward to some resolution 🙂 

Thanks again

[Gert_van_Maren]

Hi,
Could you share 1 las file so we can test?
Gert

[Allison_Hockey]

Hey Gert, 

Here's a link to the file in my one drive:

https://1drv.ms/u/s!AoKN8A92GtmCk89lenYDoXS_EsWtAw?e=MF2Ddt

let me know if you get it. Thanks again for the assist!

[Gert_van_Maren]

Hi,
I can't access the link, unfortunately...
Gert

[Allison_Hockey]

I've sent you an email Gert. Thanks for sticking with me on this! I will certainly post a follow up to my issue here once I learn 🙂

@Gert_van_Maren 

[Gertvan_Maren1]

Hi Allison,

Couple of things:

- I removed the VCS using the 'Extract LAS GP tool and this gets around the first error

- when running cluster analysis, you need to make sure the 'Use return numer 1' toggle is on. The las only has return value of 1.

Then the tool runs but because the lidar is so dense, it will take a long time to finish. Better to run the Vegetation from surface task or run it on a smaller dataset.

Gert

[GBacon]

Hello. ESRI tech support requested that I post my issue here. In an AGO web scene, I'm seeing an outline beneath 3D buildings I created using the 3D Basemap solution. Link to community post:

https://community.esri.com/t5/arcgis-online-questions/selecting-feature-in-web-scene-layer-shows/m-p...

Has anyone else experienced (or fixed) this? Thanks.