WEBGISDR Logs Issue

[DeanHowell1]

Hello, we have been noticing an issue in the webgisdr log where it is letting giving us the following message:

During the export operation, some items could not be exported because they appear to be missing from the content directory. Check the logs and review the Why do I receive an error about a missing item when creating a backup of my portal? section in the Common Problems and Solutions documentation for more information.

I couldn't find any information in the logs, so it was recommend to increase the logging for the webgisdr to 'DEBUG'.

The webgisdr kicks off at 1am each morning and around 6am we ran out of diskspace as the logging was recording the transfer of the backup data from the file system to the AWS S3 bucket. The logging was creating 50MB log files every 10 seconds. I was able to delete the log files, monitor and delete each new log file during the day until the process completed.

I have since set the logging back to WARNING which created more manageable log files but no real information about the issues.

My questions, is there a way to get more meaningful logs without killing the system with file transfer information?

[JonathanQuinn]

You should see a WARNING message with a list of the items that were in the database but not the content directory:

Ex.
<Msg time="2021-01-06T14:12:20,700" type="WARNING" code="209129" source="Portal Admin" process="16660" thread="1" methodName="" machine="machine.domain.com" user="" elapsed="" requestID="">During the export operation, some items could not be exported because they appear to be missing from the content directory. Review the information under the "Why do I receive an error about a missing item when creating a backup of my portal?" section in the Common Problems and Solutions documentation to resolve the problem. The item ids for the items with the problem are: 7c39d8c29abe480ab9a3781dcc1eb9d5, 1170bbf0d6174e068e8898ffd6ac75df, 827a622e4faf4cdd96a8569761733d26,...</Msg>

[DeanHowell1]

Thanks @JonathanQuinn, I couldn't see any warnings in the webgisdr logs, hence why I tried to increase the logging info, but could see what you are taking about if I query the logs from the portal admin tools i.e. https://xxxxx/arcgis/portaladmin/logs.

I guess note for Esri doco and developers would be to advise people not to increase logging to DEBUG for webgisdr with very large portals as the log files will kill the system.

[JonathanQuinn]

We can update the error to indicate that you should check the portal logs:

Check the portal logs and review the Why do I receive an error about a missing item when creating a backup of my portal?

Where in the documentation are you seeing suggestions to increase the logging to DEBUG?

[DeanHowell1]

Thanks @JonathanQuinn , I agree a more targeted messaging, even referencing the path to the logs would be beneficial, as there are so many log files in the overall operation, multiple logs for webgisdir, logs for portal, server, datastore, etc. It is hard to know which logs to monitor and check.

Rather than put anyone specific under the bus, the change to the logging was suggested by Esri Support 🙂

[lvargas]

Hello,

This is an old post... but it helped me find the solution.

I have a similar message using webgisdr.

Its possible identify the items that have a problem from the Portal log (like Jonathan indicates). Once I know the item with the problem, I follow this guide.

It consists first in creating an item, it can be a web map and I save it, I write down the ID.

Then I go to the installation folder of the portal (example c:\arcgisportal\content\items ) and look for the folder with the ID of the item I created, then I make copies of the folder (equal to the amount of items with problems).
I rename the copied folders one by one so that each one has the ID of the problem item.
It is important, that inside the new modified folder must be edited a file without extension that is in the folder (if the folder is called 1bc123 the file inside it must be called 1bc123), also inside the folder esriinfo I must edit a file called item.pkinfo, inside it has an ID that must be equal to the name of the folder (... <ID>1bc123</ID>).

If the process is carried out correctly, it will be possible to enter the item in the portal using the ID and once it appears in the portal you can go and delete it normally.

Without the orphaned items, the webgisdr is done and you do not have the message with the alert.

I hope this data is useful.

[gahe]

Basically this error occurs when portal items exist for example a webmap but folder doesn't exist in the content directory. You'll have to clean up the mismatch before the webgisdr tool will run correctly.

I used your guide to delete the problem webmap and it fixed this error for us. Thanks.

Also, originally the logs will not show anything about the problem item ids until I ran a Reindex in the ArcGIS Server Administrator Directory. The log finally showed the problem ID and that's how I identify the bad webmaps in portal.

[Anonymous User]

@DeanHowell1 

Good morning Dean, I hope your day is going well. One thing that could help get some more information in the logs for the Portal is just to go to the Portal's /portaladmin endpoint and use the "Export Site" operation that's available.

This will just take a backup of your Portal's site and not the entire ArcGIS Enterprise so hopefully it will be faster, smaller in size, and provide more granular logs.

[DeanHowell1]

Thanks @Anonymous User  I appreciate that suggestion and will look into that further. 

[JeffPace]

Sorry to dig up an old message.  We do this (export from the portal site instead of giswebdr) due to the size of our federated infrastructure (was taking up to 24 hours to backup).  However, what do we do about the logfile accumulation? Do we need to manually clear them?  @Anonymous User