Portal 10.5 Index Mismatch (search)

[FraserHand1]

Hi,

Has anyone come across a situation where the Portal Search index drops? This mean things like symbology, web app templates etc start to vanish from Portal. Note that doing a reindex still does not result in a full match. I have tried the go from an empty index and reindex from here:

http://support.esri.com/technical-article/000012663

and have run the indexer manually on the portal machine from here

C:\ArcGIS\Portal\tools\indexer>Indexer.bat -f -i F:\arcgisportal\index

Still end up with a mismatch on the Search Index 

Note watching the index output I see stuff like

INFO: Query returned 7167 records.

0 [WARN] GWContent: - 'extent' parameter in item card is invalid

977 [WARN] GWContent: - 'extent' parameter in item card is invalid

2480 [WARN] GWContent: - 'extent' parameter in item card is invalid

6991 [WARN] GWContent: - 'extent' parameter in item card is invalid

which looks like the items not being indexed? I had though maybe it was permissions but procmon didn't turn anything up.

Running ArcGIS enterprise 10.5

Portal is single machine with federated Server.

Thanks,

Fraser

[MichaelRobb]

I have received this as well.

Unknown cause or unknown fix.

[BingLu]

We ran into the same problem. Last week our ArcGIS Portal's server ran out of space, which caused some index issues. So we had to use the "empty index" folder to regenerate the index. After that process was complete, for the "search" index, "Count in store" was 5797, and "Count in index" was 5790. But all other indexes match. 

Any suggestions to fix this issue would be appreciated

[PhilipHughes]

Hi

Did you ever resolve this, have the same problem at 10.5.1

[FraserHand1]

Hi,

Yes I did - the first issue we had was the index service port was blocked on the second portal server. The index still did drop and bounce around though. The best way I found to fix it was to shut down the secondary portal server, rerun a full index and check, and then bring the second portal back up. This was 10.5 though.

Thanks,

Fraser

[KierenTinning2]

Fraser,

Our ports are open, but indexes are out of sync.  Did you shut down the entire server or just the service? We have shut down the service and it didn't correct the issue.

Running portal 10.5.1

Thanks,

Kieren

[RobertDriessen2]

Is there any further information on this issue.  Reindex seems to stop with search count in store set to 7529 and search count in index set to 7503. 

Have started reindex  again with same result.  Can anyone please confirm that the users' count has no direct relationship with the number of users.  We don't have 64 users.  I assume that means 64 user indexes?

We have a pair of portal VMs  so I will attempt Frasers' solution by shutting down the secondary server.

Many of the items in our portal became orphans (users could not see them).  However they were visible in the groups but not manageable.  After indexing they also did not appear in the groups.  Hopefully indexing will fix this.

Thanks, Rob

[JonathanQuinn]

What version are you using? Before 10.6 or 10.6.1, the index returned system accounts and groups, so the counts are a bit inflated.

If you're at 10.5.1, there's a bug that causes a "split-brain" problem, where both portal machines can think the component that controls the index is the master node. This causes index synchronization problems.

[RobertDriessen2]

Hi Jonathan,  thanks fir your quick response.

We are at 10.5.1.  Stopping the secondary portal service, as suggested above, did not help so I still have an index problem.  Sounds  like upgrading to 10.6.1 will be the solution.

Incidentally, shutting down the secondary virtual machine rendered the portal inoperative.I have since read elsewhere (one of your posts) that this is expected behavior.  That is, both machines need to be up for the portal to work in high availability setup however the portal service does not need to be running on one of the machines.  Am I understanding that correctly?  Is there any documentation on this? 

Thanks again, Rob

[JonathanQuinn]

I don't think 10.6.1 will fix that problem if there's an issue with the underlying index schema or any other files. I'll follow up with a message.

You should definitely be able to shut down either machine and still have Portal function. If you shut down the primary, the standby should be promote to primary. If you shut down the standby, the primary should still function. If you start the VM, does everything work? If so, can you sign into Portaladmin and check the Machines list to see which machine is primary? If the machine you shut down is, but the standby is never promoted, then it's likely a recovery.done file exists in the db directory of the standby. That tells the internal DB that it already has failed over, so nothing happens.

In my other post, I may have been referring to shutdown order, which can certainly be a problem depending on how you shut down the machines. For example, if you shut down standby, shut down primary, and start standby, the standby won't assume the role of primary as it hasn't been able to get the latest data from the primary.