Workflow Manager Configuration (JXL) Deployment

2812
5
09-27-2017 08:22 AM
ApurvDanke
Occasional Contributor

Hi,

The workflow manager configuration deployment is becoming a big challenge for us. We have many different custom job types, Maps(MXD's), Workflows, Notifications, Queries, step types, job filters which we're using in our Workflow Manager configuration. Initially we used to export the entire Workflow Manager configuration as a JXL file and then this file would get installed into target environments such as testing and production environments. After installing, we would need to import this JXL file into Workflow Manager Administrator, but we would need to do a lot of post installation activities, such as setting the target data workspace, adding/removing users according to target environments, setting the data workspaces for job types, subscribing users to notifications etc. These post installation steps would take a lot of time and there is always a chance that some mistakes would be committed during these steps.

However we found that we can also export individual entities like job types, workflows, notifications, queries etc. as JXL files and import these in target environment as part of deployment. This approach works fine for new additions in the WFM configuration such as new workflow, new job type, new step type etc. However if we need to make changes to existing entity such as a workflow, or a job type, then we cannot overwrite the existing entity in WFM administrator by choosing to import the JXL file. For example if we need to make changes to an existing job type, then in the target environment we first need to delete the existing job type and then import the JXL for the updated job type. However at the time of deleting the existing job type, we get an error message as given below.

Now it is understood that there are jobs which currently exist in WFM for this particular job type. However there are a huge number of jobs for this particular job type. As we're using OS Authentication for WFM, the jobs and their job versions need to be deleted by the respective users. Before deleting the jobs, they should reconcile and post their edits in their job versions to SDE.Default to ensure their edits are not lost. Once all the jobs and versions are deleted, then it would be possible to delete the job type in WFM and import the updated job type JXL. This is a huge time-consuming activity. It would be great if there was a way to get around this and be able to simply overwrite the changes for this job type in WFM Administrator, without needing to delete any jobs. I know that there is an option to import the entire WFM configuration JXL file, and while doing that we do get an option whether to clear the existing jobs or to retain them. But we do not want to import the entire WFM configuration, for updating just one job type. Any suggestions are greatly appreciated.

We are using ArcGIS Desktop 10.4.1 with Workflow Manager. Database used is Oracle 12G, and the users connect via OS authentication to WFM.

Regards,

Apurv

0 Kudos
5 Replies
XiaomengZheng
Esri Contributor

Apurv,

I totally understand your pain. When migrating data for Workflow Manager repository, we have experienced the challenges that you described. There seems no perfect way to do the magic to migrate everything into the new database. Here I would like to share some of my thoughts.

1. Import jxl file is the best practice. This will handle almost everything including the configuration of extended properties, job maps, job queries, etc. However importing jxl file will not import data workspace connections into the new database. You would need to manually add them. As a result, if any job type is using data workspace, you will need to manually add them too after import. Here is what I usually do for the repository having many data workspaces. I would copy the Workflow Manager schema table JTX_DATABASES into the new database. This will copy the data workspace connection.

2. I am not 100% understanding your question regarding updating a specific job type by using jxl. Would you be able to update the job type in the old database then export the jxl from Workflow Manager Administrator in the database level then import? If you need to import jxl of job type which has been existing, unfortunately, we do not provide the option as we do for importing the entire database jxl to merge which will overwrite and update the existing job type. So I would suggest updating it in the old database then export jxl. 

Hope this help on the issues you are currently running into. Please let me know if you have more thoughts or questions.

Thanks,

Meggie 

0 Kudos
ApurvDanke
Occasional Contributor

Thanks a lot Meggie for your reply. 

Yes we were trying to "update" an existing job type by importing its updated JXL file. However when we tried that, nothing happened. As you suggest, we only have the option of importing the entire WFM JXL file in this scenario.

However now we're facing an even more serious issue. We have many existing job types, and our custom application creates jobs of those job types. Lets say for purpose of your understanding, these job types as A, B and C, and its custom application is "ABC". Now as part of enhancements, we created one more job type D. Now the jobs for job type D are created by a different custom application, compared to job type A, B and C. Lets say this custom application is called "XYZ". Now, we also had a defect in the custom application ABC. Both ABC and XYZ applications create jobs and versions using the IJtx ArcObjects interfaces, using OS authentication connection to WFM. So versions are created in the format "OS_Username"."Version_Name".

Now somehow for reasons which we're unable to understand, and we're at wits end on this; the jobs created using application ABC is showing "No Version" in WFM against the Version. Screenshot given below.

What is baffling is the version is created by the application ABC and the version name exists in the Version field of JTX_JOBS table in WFM database. Previously the versions used to appear. What is even more interesting is that the jobs which are created by application XYZ, does show the version name for job type D(screenshot given below). We're using exactly the same ArcObjects code to create the job and version in both applications. 

We checked all the configurations in WFM Administrator for job types A, B and C. Everything seems to be fine. We were not sure of this "No Version" observations started happening due to the the changes we did for application ABC or due to adding the new job type D and application XYZ. Both applications use the same user ID (OS authentication). So we created another simple application which uses the IJtx ArcObjects and only creates jobs and versions for job types A, B, C and D. Even with this simple application, for job types A, B and C, we see "No Version" in WFM, though the version has been created in database; whereas for job type D, we see the version in WFM.

What could be the reason for this, we're unable find find out the cause for this irritating issue.

Regards,

Apurv

0 Kudos
XiaomengZheng
Esri Contributor

Hi Apurv,

Could you check your WFM database and see if the data workspace is still configured? You can access Administrator - GeoDatabases - Data Workspaces. Like I mentioned in my earlier reply, importing JXL will not copy the Data Workspace connection. Most likely in your job type, it doesn't configure a data workspace either. You would need to manually add it. 

I am not 100% sure my understanding is correct based on your description. Let me know if it is something else.

Thanks,

Meggie

0 Kudos
ApurvDanke
Occasional Contributor

Hi Meggie,

Thanks for your reply. We tried many things such as -

1. Import entire JXL file which was exported from another environment, where everything was working fine, and set up the configurations again. This did not work.

2. Drop the WFM schema from database, and recreate it from scratch. We did this, and even after this it did not work.

3. Finally, we installed WFM on another machine, and deployed our application there and tested it, it worked! We could see the version appearing against the Job Information.

So there seems to be some problem to the WFM installation in the original machine. We're planning to repair or uninstall and reinstall the WFM from original machine and then check again there.

Regards,

Apurv

0 Kudos
by Anonymous User
Not applicable

Apurv,

The job types should 'upgrade' when a jxl is imported, however the update date of the imported one needs to be more recent than the update date of the existing one. Otherwise it'll stay the same.

As part of our longterm roadmap we are working on tools to make these transitions between environments much more streamlined and easier to use.

Thanks for your patience,

Michael

0 Kudos