Error: The index service seems to be unavaliable. Please check the index service.

10611
17
07-24-2018 09:57 AM
TobiasFimpel
Occasional Contributor

I get this error intermittently when attempting to create and delete accounts. The intallation was done with Enterprise Builder on a virtual machine.

When I go to portaladmin I can see in teh Index Status the users "Count in store" is greater than "Count in index". Re-indexing does not solve the problem. I assume this symptom is related to the error message ("Error: The index service seems to be unavaliable. Please check the index service.)"

Software version is 10.6

Thanks for any help or suggestions.

Tags (1)
0 Kudos
17 Replies
JonathanQuinn
Esri Notable Contributor

Index related errors can be difficult to solve, so you may want to reach out to Support. Do you see a java.exe process running on the machine? That's the index service, so if you don't see it, then that's a problem. I assume you've tried to restart the service? If you were to reindex with the Portal logging DEBUG logs, do you see any errors in the logs?

0 Kudos
TobiasFimpel
Occasional Contributor

Hi Jonathan,

thanks for your reply. I do have a tech support case open but so far to no avail. Please see my answers to your questions below. I appreciate any insights or suggestions.

Do you see a java.exe process running on the machine? Yes, several. But the machine runs ArcGIS Server as well so some of them are likely from ArcGIS Server and not from Portal.

 I assume you've tried to restart the service? Yes.

 If you were to reindex with the Portal logging DEBUG logs, do you see any errors in the logs?

Yes, here is what I see in the logs:

Indexer response. Error occurred during initialization of VM Unable to allocate 262144KB bitmaps for parallel garbage collection for the requested 8388608KB heap. Error: Could not create the Java Virtual Machine. Error: A fatal exception has occurred. Program will exit.

Simultaneously Windows gives me the warning "Your computer is low on memory. To restore enough memory for programs to work correctly, save your files and then close or restart all open programs." But as you can see in below screenshot the total memory used is only at 59%. I see the same warning sometimes when I run Python scripts or ArcGIS Pro on that VM.

Is there anything my IT department should configure differently on the VM? What pointers can I give them so they know where to look? Anybody else experience with this?

0 Kudos
TobiasFimpel
Occasional Contributor

Update: we changed the Windows pagefile size from automatically managed to min:20GB, max:20GB; now re-indexing actually changes the numbers reported by the Index Status (so I can tell it is ding something) and I don't get the java heap size error, BUT the numbers are still off ("Count in store" is greater than "Count in index"). 

AaronDorsey1
New Contributor III

Were you ever able to get the indexes to match? I am able to create and modify new accounts, but get "the index service seems to be unavailable" message in the portal logs when trying to delete older accounts. I restarted the Portal service first but no change. I finally gave in and restarted the machine but still no change. I wonder if there is a way to just remove the index and start from scratch.

After I reindex, the user store count (54) is still greater than the user index count (46)...

, 2019 5:19:18 PM com.esri.arcgis.portal.Indexer e INFO: Updating the user indexes. 
, 2019 5:19:18 PM com.esri.arcgis.portal.Indexer a INFO: Querying the database for table 'gw_users' and column 'username'. 
, 2019 5:19:18 PM com.esri.arcgis.portal.Indexer a INFO: Query returned 56 records. Executing new bulk composed of56actions 
, 2019 5:19:18 PM com.esri.arcgis.portal.Indexer e INFO: Total users indexed: 56 in 0 seconds. 
0 Kudos
TobiasFimpel1
Occasional Contributor III

Hi Aaron,

no I was never able to solve the problem. Esri tech support tried a lot of things...I can't remember if they ever tried deleting and re-creating the indices completely from scratch. After a lot of troubleshooting I concluded that it would be be easier for me to re-build on a new VM than to continue to troubleshoot. Since then I have always made sure that the machine has plenty of resources available because I believe having an under-resourced machine is how I caused the problem in the first place.

0 Kudos
AaronDorsey1
New Contributor III

I see there's a way to manually delete users (Cannot Delete Portal Users), but even after a re-index, the store and index counts for the users type still do not match. And no other tables have a record count of 46.

0 Kudos
JonathanQuinn
Esri Notable Contributor

You may have to contact Support so they can walk you through re-creating the indices, (which shouldn't be confused with reindexing). The index files under <default portal directory>\index may be corrupt.

0 Kudos
DanCamboia
New Contributor

I have a call into support for this issue right now and I'm waiting to here back from them. While waiting I came across this and would like to know if you have any guidance to re-creating the indices or should I just wait for support?

0 Kudos
JonathanQuinn
Esri Notable Contributor

It's best to wait for support as it requires modifying files on disk. We plan on adding a health check/repair type functionality to handle these situations automatically.