Failure to apply a custom geographic transformation

[JoeLandersarup]

Hello,

I have create a custom geographic transformation for my project. I want to use this transform data on the fly in my map in ArcGIS Pro, so I can view multiple dataset in different co-ordinate systems together. Some of these cannot be reproejcted, such as the WMS layer from the British Geological Society (BGS) Borehole Database.

I have these layers:

• BGS Borehole WMS layer - British National Grid (In the form of a projected coord system OSGB_1936)
• historical borehole data for project - project specific projected coordinate system (Created by a company called SnakeGrid. The project specific projected coord system is called TPEN11-TM, and has an accompanying geographic coord system called TPEN11-IRF.

I have created a custom transformation using the ArcGIS tool (and using the TPEN11-TM.prj, TPEN11-IRF and NTv2 .gsb files provided by Snakegrid for this project). This transformation is for converting between geographic system GCS_ETRS89 to the geographic system TPEN11_IRF. The ArcGIS Pro built-in transformers can then convert the data from ETRS89 tto OSGB 1936 (using OTSN02 NTv2 .gsb). 

ArcGIS recognises the custom transformation I have created, see attached screenshot (select_transformation.png). The name of the custom transformation is 'TN15-ETRS89-to-TPEN11-IRF'. ArGIS Pro the adds '+OSGB 1936 to ETRS_1989_1' to convert it the rest of the way. However the problem is that once I click 'ok' to close the box it will not apply this transformation on my map. My data on the map doesn't transform and is not displaying together. When I re-open the map transformations list, the list is still at 'none'. See attached screenshot 'none_selected'. This is very frustrating.

To cut a long story short, Has anyone else come across this problem of ArcGIS Pro not applying a custom  transformation that is clearly being recognised in the list by ArcGIS Pro and is selectable? Any ideas why it won't apply?

On a semi-related note, the custom transformation works perfectly to project data. I have been successful in using the 'project' tool to use my custom transformation to convert data from British National Grid into my project specific co-ordinate system (see image 'project_transformer.png' So there is clearly nothing wrong with the accuracy of the custom transformation, it's working perfectly using the 'project' tool, it just won't be applied by ArcGIS Pro for on-the-fly transformation (which is what I badly need).

I'm using ArcGIS Pro 2.7

[DanPatterson]

@HeatherSmith posted a blog recently

Projection on the fly and geographic transformations (esri.com)

I am not sure whether that adding it to your "Favorites" will provide the capability to find it for coding or as defaults.

She will get a message about this post.  Perhaps she can comment further

[JoeLandersarup]

Thanks Dan, I came across the blog by @HeatherSmith before making the post (it's an excellent post btw), however that isn't the problem I'm experiencing.

The problem is to do with ArcGIS Pro not applying the transformation when I select it. It always reverts back to 'none', even when I select the custom transformation in the list.

I can provide my dataset, if people want to test it.

[DanPatterson · Accepted solution]

I just checked bug reports to see if anything was remotely connected

I think this is the one

BUG-000096165: ArcGIS Pro does not honor on-the-fly custom transfor.. (esri.com)

others for interest

BUG-000135135: In ArcGIS Pro, an applied custom transformation meth.. (esri.com)

BUG-000125948: Arcpy.ListTransformations() is inconsistent when wor.. (esri.com)

[JoeLandersarup]

Thanks Dan for the list of bug reports (I didn't know about them as a resource). They describe the problem exactly. 😞

[BojanŠavrič]

BUG-000096165 and  BUG-000135135 (based on their repro steps) appear to be resolved with the latest release, and both issues should be soon marked appropriately. Thanks!

[DanPatterson]

Perhaps you could  close... sad answer that it was

[Tim_McGinnes]

Maybe try manually putting the transformation in? Save the map to a .mapx file. Open in text editor and look for the datumTransforms section. Should look something like the below when you have one.  Modify as required (the ____ parts), save, and Import Map into a new project. Does the ETRS89 to OSGB 1936 transform get added from the properties? Maybe put it in by itself before you export as a good starting point.

    "datumTransforms" : [
      {
        "type" : "CIMDatumTransform",
        "forward" : true,
        "geoTransformation" : {
          "geoTransforms" : [
            {
              "wkid" : ____,
              "latestWkid" : ____,
              "transformForward" : false,
              "name" : "____"
            },
            {
              "wkid" : ____,
              "latestWkid" : ____,
              "transformForward" : true,
              "name" : "____"
            }
          ]
        }
      },
      {
        "type" : "CIMDatumTransform",
        "forward" : false,
        "geoTransformation" : {
          "geoTransforms" : [
            {
              "wkid" : ____,
              "latestWkid" : ____,
              "transformForward" : false,
              "name" : "____"
            },
            {
              "wkid" : ____,
              "latestWkid" : ____,
              "transformForward" : true,
              "name" : "____"
            }
          ]
        }
      }
    ],

[JoeLandersarup]

Thanks for the suggestion Tim.

I saved as mapx, and opened in text editor. See below block.

There is alot of transformations in the code. I'm not sure why. However when I look at the list of transformations within the map properties, there is a very long list under the 'Additional transformations' stating that 'These transformations are currently not in use on the map'. See my attached screenshot. The weird thing is, the first one on that list, is exactly the one I'm trying to use apply to the map!!!

As for the list of transformations in the mapx file, the 'wkt' for the custom transformation is a very long piece of text. But also there isn't any 'wkid' number like there is for the other built-in transformations. 

    "datumTransforms" : [
      {
        "type" : "CIMDatumTransform",
        "forward" : true,
        "geoTransformation" : {
          "geoTransforms" : [
            {
              "wkid" : 5339,
              "latestWkid" : 5339,
              "transformForward" : false,
              "name" : "OSGB_1936_To_WGS_1984_7"
            }
          ]
        }
      },
      {
        "type" : "CIMDatumTransform",
        "forward" : false,
        "geoTransformation" : {
          "geoTransforms" : [
            {
              "wkid" : 5339,
              "latestWkid" : 5339,
              "transformForward" : false,
              "name" : "OSGB_1936_To_WGS_1984_7"
            }
          ]
        }
      },
      {
        "type" : "CIMDatumTransform",
        "forward" : true,
        "geoTransformation" : {
          "geoTransforms" : [
            {
              "wkt" : "GEOGTRAN[\"TN15-ETRS89-to-TPEN11-IRF\",GEOGCS[\"GCS_ETRS_1989\",DATUM[\"D_ETRS_1989\",SPHEROID[\"GRS_1980\",6378137.0,298.257222101]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],GEOGCS[\"GCS_TPEN11-IRF\",DATUM[\"D_TPEN11-IRF\",SPHEROID[\"GRS_1980\",6378137.0,298.257222101]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],METHOD[\"NTv2\"],PARAMETER[\"Dataset_uk/TN15-ETRS89-to-TPEN11-IRF\",0.0]]",
              "transformForward" : false,
              "name" : "TN15-ETRS89-to-TPEN11-IRF"
            },
            {
              "wkid" : 5338,
              "latestWkid" : 5338,
              "transformForward" : false,
              "name" : "OSGB_1936_To_ETRS_1989_1"
            }
          ]
        }
      },
      {
        "type" : "CIMDatumTransform",
        "forward" : false,
        "geoTransformation" : {
          "geoTransforms" : [
            {
              "wkt" : "GEOGTRAN[\"TN15-ETRS89-to-TPEN11-IRF\",GEOGCS[\"GCS_ETRS_1989\",DATUM[\"D_ETRS_1989\",SPHEROID[\"GRS_1980\",6378137.0,298.257222101]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],GEOGCS[\"GCS_TPEN11-IRF\",DATUM[\"D_TPEN11-IRF\",SPHEROID[\"GRS_1980\",6378137.0,298.257222101]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],METHOD[\"NTv2\"],PARAMETER[\"Dataset_uk/TN15-ETRS89-to-TPEN11-IRF\",0.0]]",
              "transformForward" : false,
              "name" : "TN15-ETRS89-to-TPEN11-IRF"
            },
            {
              "wkid" : 5338,
              "latestWkid" : 5338,
              "transformForward" : false,
              "name" : "OSGB_1936_To_ETRS_1989_1"
            }
          ]
        }
      },
      {
        "type" : "CIMDatumTransform",
        "forward" : true,
        "geoTransformation" : {
          "geoTransforms" : [
            {
              "wkt" : "GEOGTRAN[\"TN15-ETRS89-to-TPEN11-IRF\",GEOGCS[\"GCS_ETRS_1989\",DATUM[\"D_ETRS_1989\",SPHEROID[\"GRS_1980\",6378137.0,298.257222101]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],GEOGCS[\"GCS_TPEN11-IRF\",DATUM[\"D_TPEN11-IRF\",SPHEROID[\"GRS_1980\",6378137.0,298.257222101]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],METHOD[\"NTv2\"],PARAMETER[\"Dataset_uk/TN15-ETRS89-to-TPEN11-IRF\",0.0]]",
              "transformForward" : false,
              "name" : "TN15-ETRS89-to-TPEN11-IRF"
            },
            {
              "wkid" : 5338,
              "latestWkid" : 5338,
              "transformForward" : false,
              "name" : "OSGB_1936_To_ETRS_1989_1"
            }
          ]
        }
      },
      {
        "type" : "CIMDatumTransform",
        "forward" : false,
        "geoTransformation" : {
          "geoTransforms" : [
            {
              "wkt" : "GEOGTRAN[\"TN15-ETRS89-to-TPEN11-IRF\",GEOGCS[\"GCS_ETRS_1989\",DATUM[\"D_ETRS_1989\",SPHEROID[\"GRS_1980\",6378137.0,298.257222101]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],GEOGCS[\"GCS_TPEN11-IRF\",DATUM[\"D_TPEN11-IRF\",SPHEROID[\"GRS_1980\",6378137.0,298.257222101]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],METHOD[\"NTv2\"],PARAMETER[\"Dataset_uk/TN15-ETRS89-to-TPEN11-IRF\",0.0]]",
              "transformForward" : false,
              "name" : "TN15-ETRS89-to-TPEN11-IRF"
            },
            {
              "wkid" : 5338,
              "latestWkid" : 5338,
              "transformForward" : false,
              "name" : "OSGB_1936_To_ETRS_1989_1"
            }
          ]
        }
      },
      {
        "type" : "CIMDatumTransform",
        "forward" : true,
        "geoTransformation" : {
          "geoTransforms" : [
            {
              "wkid" : 5338,
              "latestWkid" : 5338,
              "transformForward" : true,
              "name" : "OSGB_1936_To_ETRS_1989_1"
            },
            {
              "wkt" : "GEOGTRAN[\"TN15-ETRS89-to-TPEN11-IRF\",GEOGCS[\"GCS_ETRS_1989\",DATUM[\"D_ETRS_1989\",SPHEROID[\"GRS_1980\",6378137.0,298.257222101]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],GEOGCS[\"GCS_TPEN11-IRF\",DATUM[\"D_TPEN11-IRF\",SPHEROID[\"GRS_1980\",6378137.0,298.257222101]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],METHOD[\"NTv2\"],PARAMETER[\"Dataset_uk/TN15-ETRS89-to-TPEN11-IRF\",0.0]]",
              "transformForward" : true,
              "name" : "TN15-ETRS89-to-TPEN11-IRF"
            }
          ]
        }
      },
      {
        "type" : "CIMDatumTransform",
        "forward" : false,
        "geoTransformation" : {
          "geoTransforms" : [
            {
              "wkid" : 5338,
              "latestWkid" : 5338,
              "transformForward" : true,
              "name" : "OSGB_1936_To_ETRS_1989_1"
            },
            {
              "wkt" : "GEOGTRAN[\"TN15-ETRS89-to-TPEN11-IRF\",GEOGCS[\"GCS_ETRS_1989\",DATUM[\"D_ETRS_1989\",SPHEROID[\"GRS_1980\",6378137.0,298.257222101]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],GEOGCS[\"GCS_TPEN11-IRF\",DATUM[\"D_TPEN11-IRF\",SPHEROID[\"GRS_1980\",6378137.0,298.257222101]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],METHOD[\"NTv2\"],PARAMETER[\"Dataset_uk/TN15-ETRS89-to-TPEN11-IRF\",0.0]]",
              "transformForward" : true,
              "name" : "TN15-ETRS89-to-TPEN11-IRF"
            }
          ]
        }
      },
      {
        "type" : "CIMDatumTransform",
        "forward" : true,
        "geoTransformation" : {
          "geoTransforms" : [
            {
              "wkid" : 1149,
              "latestWkid" : 1149,
              "transformForward" : false,
              "name" : "ETRS_1989_To_WGS_1984"
            },
            {
              "wkt" : "GEOGTRAN[\"TN15-ETRS89-to-TPEN11-IRF\",GEOGCS[\"GCS_ETRS_1989\",DATUM[\"D_ETRS_1989\",SPHEROID[\"GRS_1980\",6378137.0,298.257222101]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],GEOGCS[\"GCS_TPEN11-IRF\",DATUM[\"D_TPEN11-IRF\",SPHEROID[\"GRS_1980\",6378137.0,298.257222101]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],METHOD[\"NTv2\"],PARAMETER[\"Dataset_uk/TN15-ETRS89-to-TPEN11-IRF\",0.0]]",
              "transformForward" : true,
              "name" : "TN15-ETRS89-to-TPEN11-IRF"
            }
          ]
        }
      },
      {
        "type" : "CIMDatumTransform",
        "forward" : false,
        "geoTransformation" : {
          "geoTransforms" : [
            {
              "wkid" : 1149,
              "latestWkid" : 1149,
              "transformForward" : false,
              "name" : "ETRS_1989_To_WGS_1984"
            },
            {
              "wkt" : "GEOGTRAN[\"TN15-ETRS89-to-TPEN11-IRF\",GEOGCS[\"GCS_ETRS_1989\",DATUM[\"D_ETRS_1989\",SPHEROID[\"GRS_1980\",6378137.0,298.257222101]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],GEOGCS[\"GCS_TPEN11-IRF\",DATUM[\"D_TPEN11-IRF\",SPHEROID[\"GRS_1980\",6378137.0,298.257222101]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],METHOD[\"NTv2\"],PARAMETER[\"Dataset_uk/TN15-ETRS89-to-TPEN11-IRF\",0.0]]",
              "transformForward" : true,
              "name" : "TN15-ETRS89-to-TPEN11-IRF"
            }
          ]
        }
      },
      {
        "type" : "CIMDatumTransform",
        "forward" : true,
        "geoTransformation" : {
          "geoTransforms" : [
            {
              "wkt" : "GEOGTRAN[\"TN15-ETRS89-to-TPEN11-IRF\",GEOGCS[\"GCS_ETRS_1989\",DATUM[\"D_ETRS_1989\",SPHEROID[\"GRS_1980\",6378137.0,298.257222101]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],GEOGCS[\"GCS_TPEN11-IRF\",DATUM[\"D_TPEN11-IRF\",SPHEROID[\"GRS_1980\",6378137.0,298.257222101]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],METHOD[\"NTv2\"],PARAMETER[\"Dataset_uk/TN15-ETRS89-to-TPEN11-IRF\",0.0]]",
              "transformForward" : true,
              "name" : "TN15-ETRS89-to-TPEN11-IRF"
            }
          ]
        }
      },
      {
        "type" : "CIMDatumTransform",
        "forward" : false,
        "geoTransformation" : {
          "geoTransforms" : [
            {
              "wkt" : "GEOGTRAN[\"TN15-ETRS89-to-TPEN11-IRF\",GEOGCS[\"GCS_ETRS_1989\",DATUM[\"D_ETRS_1989\",SPHEROID[\"GRS_1980\",6378137.0,298.257222101]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],GEOGCS[\"GCS_TPEN11-IRF\",DATUM[\"D_TPEN11-IRF\",SPHEROID[\"GRS_1980\",6378137.0,298.257222101]],PRIMEM[\"Greenwich\",0.0],UNIT[\"Degree\",0.0174532925199433]],METHOD[\"NTv2\"],PARAMETER[\"Dataset_uk/TN15-ETRS89-to-TPEN11-IRF\",0.0]]",
              "transformForward" : true,
              "name" : "TN15-ETRS89-to-TPEN11-IRF"
            }
          ]
        }
      },
      {
        "type" : "CIMDatumTransform",
        "forward" : true,
        "geoTransformation" : {
          "geoTransforms" : [
            {
              "wkid" : 1149,
              "latestWkid" : 1149,
              "transformForward" : false,
              "name" : "ETRS_1989_To_WGS_1984"
            }
          ]
        }
      },
      {
        "type" : "CIMDatumTransform",
        "forward" : false,
        "geoTransformation" : {
          "geoTransforms" : [
            {
              "wkid" : 1149,
              "latestWkid" : 1149,
              "transformForward" : false,
              "name" : "ETRS_1989_To_WGS_1984"
            }
          ]
        }
      },
      {
        "type" : "CIMDatumTransform",
        "forward" : true,
        "geoTransformation" : {
          "geoTransforms" : [
            {
              "wkid" : 5339,
              "latestWkid" : 5339,
              "transformForward" : true,
              "name" : "OSGB_1936_To_WGS_1984_7"
            },
            {
              "wkid" : 1149,
              "latestWkid" : 1149,
              "transformForward" : false,
              "name" : "ETRS_1989_To_WGS_1984"
            }
          ]
        }
      },
      {
        "type" : "CIMDatumTransform",
        "forward" : false,
        "geoTransformation" : {
          "geoTransforms" : [
            {
              "wkid" : 5339,
              "latestWkid" : 5339,
              "transformForward" : true,
              "name" : "OSGB_1936_To_WGS_1984_7"
            },
            {
              "wkid" : 1149,
              "latestWkid" : 1149,
              "transformForward" : false,
              "name" : "ETRS_1989_To_WGS_1984"
            }
          ]
        }
      }
    ],

[JoeLandersarup]

Thanks @Tim_McGinnes 

I tried your suggestion and the .mapx had really long list of 'datumTransforms'. I didn't really know what I was looking at so decided to leave it. As Bojan has kindly offered to look at the dataset, I'll see if ESRI can solve my issue.

Joe