ArcGIS Server on Linux Fails to Recognize Oracle Client on NAS

[arahman_mdmajid]

I'm encountering an unusual issue while adding a Linux-based ArcGIS Server machine to an existing site. The server is unable to recognize the Oracle client, which is located on a mounted NAS storage path.

Here are the details:

• The same Oracle client path on NAS is used in two other environments without any issues.

• The primary machine in this environment successfully validates database connections using the exact same client path and configuration as specified in init_user_param.sh.

• However, this particular machine fails during the join process with the error:

"Client software failed to load. Be sure the database client software is installed and configured correctly."

• I have verified that the NAS storage is correctly mounted on this machine.

• Permissions and environment variable definitions in init_user_param.sh appear identical to the working machines.

Has anyone encountered a similar issue? Any tips on what else can I check?

[JoshuaBixby]

Make sure all the existing registered data stores validate.  Any registered data store that fails to validate on the site will prevent new machines from joining a site. I have seen red-herring messages in the past when joining machines to a site where the error points to one thing when that isn't the issue.  Can you use the database client on the newer machine outside of ArcGIS?

[arahman_mdmajid]

The primary machine is validating all the databases and folders. However, when the secondary machine is being joined, it displays an error stating that the registered databases are unable to validate, and the site operation fails after some time. Our client is one the NAS storage and all other server machines can use this client. We also attempted to install the server component on another machine with its client configured and then tried to perform a join, but the same database validation error occurred. Is it possible that the primary machine is the culprit and is preventing the other machines from joining?

I will try to use the client on the machine and try to connect to the geodatabase, and will get back to you on that.

[arahman_mdmajid]

So when I am trying to run the client on the machine that is having an issue, it giving the following error

./sqlplus: error while loading shared libraries: libnsl.so.1: cannot open shared object file: No such file or directory

The oracle paths and varaible are all set in the .cshrc file running source on it is not giving any errors but sqlplus command itself is giving the above error. Any ideas what could be the issue?

[arahman_mdmajid]

New info:

So I installed ArcGIS Server on a new VM where the sqlplus command and the Oracle client is working, and we were also able to connect to the geodatabase from it. When I am trying to join the site, the log for this machine are showing the following errors

Failed to update the data store connections file. Registration of database '/enterpriseDatabases/AGSDataStore_' is incomplete. Failed to update the data store connections file. Using ArcGIS Server Administrator Directory check if System/PublishingTools GP service is started and re-register the database.

[JoshuaBixby]

Although I stated above ArcGIS Server can give red-herring or false error messages, sometimes the error messages are completely accurate.  I have definitely seen cases where the PublishingTools GP service is either not running or corrupted/broken, and it prevents data store registration or validation.  Have you tried what the message suggests, i.e., restart the service and reregister the data store in question?

[JoshuaBixby]

This is why I always test a database client outside of ArcGIS software on a machine.  As to why the database client isn't working on this machine, I don't work with Linux enough to hazard an informed guess.

[TimWestern]

Is there a possibility that the client in question can't actually route to the datasource?  (A routing issue of some kind)