idea ArcGIS Pro crashes when running Create Integrated Mesh Scene Layer Package tool a half, could this tool continue with the last result and not start from scratch? in ArcGIS Pro Ideas

ArcGIS Pro crashes when running Create Integrated Mesh Scene Layer Package tool a half, could this tool continue with the last result and not start from scratch?

XinyiZhu1 — Fri, 03 Apr 2020 02:23:38 GMT

I get very big osgb data about 600G from Institue of surveying and mapping, then convert this big osgb data using Create Integrated Mesh Scene Layer Package tool in Pro 2.5, but when the tool runs a half(it may have taken many hours or several days), pro crashes because of the quality of the osgb data or pro closes suddenly because of some unexpected circumstances such as power lost or be closed by mistakes.

Then I check and repair this big osgb data, and I want to run this tool continue with the last result and not start form scrach. I know Pro has no this function in current version, but could pro add this function in the future version? Because this function is very important for dealing with very big osgb data, and it may take very very long time before the tool fails. Thank you!

Re: ArcGIS Pro crashes when running Create Integrated Mesh Scene Layer Package tool a half, could this tool continue with the last result and not start from scratch?

DanPatterson_Retired — Fri, 03 Apr 2020 02:42:59 GMT

supposedly fixed in 2.3, if this is the error

BUG-000117586: ArcGIS Pro crashes when running the Create Integrate..

there are others

https://support.esri.com/en/Search-Results#search?q=Create%20integrated%20scene%20layer&content-type=Bugs

just sort from Newest to Oldest

Re: ArcGIS Pro crashes when running Create Integrated Mesh Scene Layer Package tool a half, could this tool continue with the last result and not start from scratch?

XinyiZhu1 — Fri, 03 Apr 2020 02:54:30 GMT

Thanks for the bugs and the user wants to run this tool continue with his last result. Because this tool may have taken many hours or several day. If he must run this tool form scratch, he think it is inefficient and unacceptable. Thanks again.

Re: ArcGIS Pro crashes when running Create Integrated Mesh Scene Layer Package tool a half, could this tool continue with the last result and not start from scratch?

DanPatterson_Retired — Fri, 03 Apr 2020 03:04:57 GMT

Tech Support would be my suggestion, there may be underlying issues for the failure, making restarting the process a secondary concern

Re: ArcGIS Pro crashes when running Create Integrated Mesh Scene Layer Package tool a half, could this tool continue with the last result and not start from scratch?

XinyiZhu1 — Fri, 03 Apr 2020 03:15:25 GMT

Yes, it may be quality of the osgb data, but the user wants to deal with the left osgb data after he repairs and don't want to execute all the osgb data again. Thank you.

Re: ArcGIS Pro crashes when running Create Integrated Mesh Scene Layer Package tool a half, could this tool continue with the last result and not start from scratch?

Andrew--Johnson — Mon, 06 Apr 2020 14:09:15 GMT

Hi,

I think we need to get to the bottom of why this is crashing ArcGIS Pro. We cannot pick up a half baked scene layer package after the tool has already failed. Is the user inputting the folder of OSGB files or are they inputting the root OSGB file? My recommendation would be to input the root OSGB file as the tool will then read the children nodes from that root file but if you input the entire folder then the tool will attempt to read all OSGB files in that directory. We have seen cases where there are OSGB files in a folder that should not be there so please try inputting the root file and let me know if that resolves the crash.

thanks,

Andrew

Re: ArcGIS Pro crashes when running Create Integrated Mesh Scene Layer Package tool a half, could this tool continue with the last result and not start from scratch?

XinyiZhu1 — Tue, 07 Apr 2020 02:53:41 GMT

Hi Andrew,

Thanks for your recommendation, it is work when I convert a small osgb files which results Pro crashes before. I will tell the user to try this. I will update you if he answers me.

Thanks so much

Xinying

Re: ArcGIS Pro crashes when running Create Integrated Mesh Scene Layer Package tool a half, could this tool continue with the last result and not start from scratch?

XinyiZhu1 — Thu, 09 Apr 2020 09:29:51 GMT

Hi Andrew,

The user has converted the osgb data which about 10G, the tool runs successfully. But he has a question for only input the root osgb files, that is, the source osgb data has 585 subfolders but the number of root osgb file only has 432, if only input these 432 root osgb file, whether it will result the lost of data? Or say, these subfolders which lack of root osgb file are all the bad data and don't need to put these subfolders as the input dataset?

Addtionally, the user is testing the big 320G osgb data, if he completes, I will update to you. Thanks so much, your recommendation is very helpful!

Regards,

Xinying

Re: ArcGIS Pro crashes when running Create Integrated Mesh Scene Layer Package tool a half, could this tool continue with the last result and not start from scratch?

Andrew--Johnson — Thu, 09 Apr 2020 17:29:27 GMT

Hi,

Glad to hear it worked! It's hard to really say if there's lost data or not. They're going to have to analyze the data to understand whether it's all there or not. I've seen cases where there are osgb files that are from a different dataset but somehow they got put in that directory so that could be the case here. What I would recommend and viewing some of those osgb files in a viewer (e.g. Acute 3D viewer) to see if they do match. They can also write out the osgb file to an ASCII file (osgLab utility) to see if the nodes are referenced properly within the file.

osgLAB – The Scene Baker

Re: ArcGIS Pro crashes when running Create Integrated Mesh Scene Layer Package tool a half, could this tool continue with the last result and not start from scratch?

XinyiZhu1 — Fri, 10 Apr 2020 09:50:42 GMT

Hi Andrew,

Thanks for your recommendation again, it is very valuable to me, I will test these osgb files.

The user has answered me that it is failed when dealing with 320G osgb data. The error as the follow picture:

The output slpk file has already written 450G when this tool failed. Do you know what's the possible reasons with this error or is there any other methods to run these osgb data successfully? I'm grateful to have your help.

Thanks

Xinying

Re: ArcGIS Pro crashes when running Create Integrated Mesh Scene Layer Package tool a half, could this tool continue with the last result and not start from scratch?

Andrew--Johnson — Fri, 10 Apr 2020 19:23:50 GMT

Hi,

Unfortunately with that error message it could be anything. It could be the process being stopped prematurely by the machine going to sleep, or the D drive running out of space etc. It also doesn't seem feasible to get that much data to try and reproduce on our end so the only thing I can recommend are:

1. Run the tool again to see if it fails at the exact same time.

2. Try on a different machine.

3. Split up the job so that you have multiple slpks instead of one large slpk

Let me know if any of those options are feasible.

thank you,

Andrew

Re: ArcGIS Pro crashes when running Create Integrated Mesh Scene Layer Package tool a half, could this tool continue with the last result and not start from scratch?

XinyiZhu1 — Mon, 13 Apr 2020 03:25:07 GMT

Hi Andrew,

The user selects a part of 320G osgb files, about 69 subfolders, and pro fails again, as the following picture:

I think this may be the problem of the source osgb data, Could you give me some recommendation on how to troubleshoot the osgb data problems?

Addtionally, if split up the job and generate multiple slpks instead of one large slpk, how to merge these small splks into one large slpk? Do you have any methods or tools to merge these small slpks? Because the user wants to have one scene service layer at final. It is very kind of you to reply me, I'm grateful to have your help.

Thank you!

Best Regards

Xinying

Re: ArcGIS Pro crashes when running Create Integrated Mesh Scene Layer Package tool a half, could this tool continue with the last result and not start from scratch?

Andrew--Johnson — Mon, 13 Apr 2020 22:33:18 GMT

Hi,

Okay so that provides a little more information. It looks like there is a root node missing from one of the LODs. We know previously that the tool did work when we input just the root node for a subset so my recommendation would be to input each root node from each input folder and when you run the tool it will work down the node tree from each root node and create the slpk. In order for the tool to run properly we need the root node to build the node hierarchy.

an example below. I have these LODs and within each LOD folder I select the root node

Re: ArcGIS Pro crashes when running Create Integrated Mesh Scene Layer Package tool a half, could this tool continue with the last result and not start from scratch?

XinyiZhu1 — Wed, 15 Apr 2020 04:33:31 GMT

Hi Andrew,

The user is using root osgb files as the input dataset when this error appears, so I don't know what's the problem with this osgb data.

So I think it is need to make a tool to check the correct of source osgb files before run this tool. Could you give me some suggest for realizing this function?

Re: ArcGIS Pro crashes when running Create Integrated Mesh Scene Layer Package tool a half, could this tool continue with the last result and not start from scratch?

Andrew--Johnson — Wed, 15 Apr 2020 12:52:20 GMT

I think the quickest way might be to run each individual LOD and the one that fails remove that entire folder. In my example above let's assume Tile_+003_+001 is the problem. Once that one fails let's remove that folder and then input the root folder and rerun the tool. How does that sound? If you are able to narrow it down to a specific folder can you share that folder along with the anchor point so we can debug further on our end?

thanks,

Andrew

Re: ArcGIS Pro crashes when running Create Integrated Mesh Scene Layer Package tool a half, could this tool continue with the last result and not start from scratch?

XinyiZhu1 — Mon, 08 Jun 2020 02:22:48 GMT

Hi Andrew,

Sorry to bother you again. I have tested this 320G osgb in Pro 2.5 again and this tool failed when generated 450G slpk, then I tested this 320G osgb in Pro 2.6 beta1 version, this tool failed when generated 471G slpk(because the slpk generated by Pro 2.6 beta1 is a little bigger than Pro 2.5, so I think it is the same reason) , the error is "Error : Failed to encode image 375989 using method dds Error : Failed to encode image 375990 using method dds", so I think it is due to the image of source osgb file when converting to dds format. So I want to know what's the problem about this error and how I check and get these error osgb files? I would appreciate if you could provide some clues about this error.

Furthermore, I have a little suggestion, that is, provide an option which let the user select what kind of texture format he wants, for example, jpg or dds or both. Because some users may only want the jpg files, but if the geneated slpk must contain dds format, it will increase the size, the time spent and error probability when converting osgb to slpk using Create Integrated Mesh Scene Layer Package tool.

Thanks so much.

Regards,

Xinying

Re: ArcGIS Pro crashes when running Create Integrated Mesh Scene Layer Package tool a half, could this tool continue with the last result and not start from scratch?

Andrew--Johnson — Mon, 08 Jun 2020 15:48:46 GMT

Hi Xinyi,

Can you tell me what setting you have used for maximum texture size? Have you tried adjusting this? Also if you just run one of the OSGB files through the tool can you reproduce or is it too difficult to find the problematic OSGB file with this much data? Additionally, you need the DXT textures if you are going to publish the scene layer to your portal. This was requested by Esri China in the previous release due to performance issues you were seeing in the Scene Viewer. For the next release (2.7) we are adding the texture optimization parameter that will allow you to choose what kind of textures you want. We are also planning on adding support for Basis textures which will write only 1 texture file to the scene layer reducing file storage but allowing optimizations in both desktop, mobile and web clients.

thanks,

Andrew

Re: ArcGIS Pro crashes when running Create Integrated Mesh Scene Layer Package tool a half, could this tool continue with the last result and not start from scratch?

XinyiZhu1 — Tue, 09 Jun 2020 07:20:56 GMT

Hi Andrew,

Thanks for your response. I set maximum texture size as '4096' in my python scripts. Before I runing this tool, I have checked all source osgb files and no texture size(width and height) exceed 4096, so I think the problem may not the texture size. I have run this tool dealing with a single osgb folder such as "Tile_+000_004", it is success. Because the number of root osgb files is 3867, if I create 3867 slpk it will be time consuming. So it too difficult to find the problematic OSGB file with this much data.

I think if the image of source osgb file can not be converted to .dds file, Could this tool output the path of the error osgb files(or skip these error osgb files) and go on executing and generte the final slpk? Thanks!

Have a wonderful day!

Regards,

Xinying

Re: ArcGIS Pro crashes when running Create Integrated Mesh Scene Layer Package tool a half, could this tool continue with the last result and not start from scratch?

Andrew--Johnson — Tue, 09 Jun 2020 15:00:31 GMT

Hi Xinyi,

Yes, we've already implemented that when we encounter a large texture but we can add additional messaging for this issue as well.

thanks,

Andrew

Re: ArcGIS Pro crashes when running Create Integrated Mesh Scene Layer Package tool a half, could this tool continue with the last result and not start from scratch? - Status changed to: Under Consideration

Andrew--Johnson — Tue, 17 Nov 2020 18:14:37 GMT