topic Re: SOAP Server Object Extension (SOE) - how to deserialize custom types? in ArcObjects SDK Questions

SOAP Server Object Extension (SOE) - how to deserialize custom types?

ErinBrimhall — Tue, 04 Jan 2011 18:23:23 GMT

So far I've had success writing a handful of SOE methods that take parameters that are simple types (string and string array) and return my own custom, multi-level types (e.g. Customer, with an array of Address objects, etc.).

Now, what I would like to do is write an SOE method that accepts my own complex type(s) as its parameters, e.g. something like CustomerSearch, that accepts a custom Query object that defines the criteria to filter results by.

Everything seems to be fine up until the point where I attempt to deserialize the incoming request parameter as an instance of the Query class.

For example, inside the HandleSoapMessage method of my SOE class implementation, I try the following:

public void HandleSoapMessage(IMessage request, IMessage response)
{
    int parameterIndex = Utility.FindFieldIndex("inputQuery", request.Parameters, true);

    parameterIndex = Utility.FindFieldIndex("inputQuery", request.Parameters, true);
    Query query = (Query)request.Parameters.GetObject(parameterIndex, "http://www.mynamespace.com", "Query");

    .
    .
    .
}

parameterIndex correctly points to the first index in the set of request parameters where the "inputQuery" definition resides, but the following exception is thrown during the call to GetObject:

"Exception from HRESULT: 0x80043068"

A little searching reveals that the 0x80043068 code corresponds to the more usable message, "XML_SERIALIZE_E_CANT_MAP_XMLTYPE_TO_CLASS"

So, there was a problem mapping between the "Query" type defined in my WSDL and my concrete "Query" class. I decided to start small by defining only a single string attribute off the Query object, but I still receive the same error.

I replaced my usage of "Query" throughout with an ESRI type, "PointN", and I was able to deserialize the incoming data as IPoint using the same GetObject method. I also inspected the actual SOAP requests, comparing between the request that used "Query" and the request where I experimented with "PointN", and did not notice any obvious problem. See request comparisons below (note that the sole string property on "Query" is called "Test":

<?xml version="1.0" encoding="utf-8" ?><soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:tns="http://www.mynamespace.com" xmlns:esri="http://www.esri.com/schemas/ArcGIS/10.0"><soap:Body><tns:CustomerSearch><ns1:inputQuery xmlns:ns1=""><ns1:Test>Test Text</ns1:Test></ns1:inputQuery></tns:CustomerSearch></soap:Body></soap:Envelope>


<?xml version="1.0" encoding="utf-8" ?><soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:tns="http://www.mynamespace.com" xmlns:esri="http://www.esri.com/schemas/ArcGIS/10.0"><soap:Body><tns:CustomerSearch><ns1:inputPoint xmlns:ns1=""><ns1:X>0</ns1:X><ns1:Y>0</ns1:Y></ns1:inputPoint></tns:CustomerSearch></soap:Body></soap:Envelope>

This post is getting long, so I will finish by saying that the "Query" definition in my SOE WSDL correctly matches the COM-visible "Query" class that implements the IXMLSerialize interface, with both the Deserialize and Serialize methods accounting for the dummy "Test" string attribute.

I've also made doubly sure that my XmlSupport mapping file in the ArcGIS Server "XmlClassRegistry" folder (and sub-folder) correctly maps the "Query" XML type with the CSLID of the Query class.

I hope there is an SOE guru out there that can see where I am going wrong, or let me at least know if custom types as SOAP SOE input parameters is even supported.

Any help would be hugely appreciated!

Re: SOAP Server Object Extension (SOE) - how to deserialize custom types?

ErinBrimhall — Tue, 04 Jan 2011 21:41:17 GMT

**UPDATE**

After much experimenting, the problem seems to stem from the call to GetObject being limited to whatever XML / COM-visible type mappings are defined in the ESRI-specific XmlSupport.dat file located in the "bin" folder of the ArcGIS Server install folder.

In other words, my custom XML / COM-visible type mapping file in the XmlClassRegistry folder never seems to be used.

I was able to verify this by copying my type-mapping snippet into XmlSupport.dat, after which the call to GetObject on the SOAP request worked successfully for my custom "Query" type.

Per the ESRI SOAP SOE walkthrough I created the appropriate folders and dropped the specially named mapping file in them. The full path of the custom XML support file is: C:\Program Files (x86)\ArcGIS\Server10.0\XmlClassRegistry\MyCompany\XmlSupportMyProject.dat

So the next question is, why isn't ArcGIS Server looking to my custom type mapping file during calls to GetObject?

Re: SOAP Server Object Extension (SOE) - how to deserialize custom types?

Kevin_FernandoEscalera_Robles — Mon, 26 Mar 2012 01:36:04 GMT

So, no answer about it??, I have the same problem, I can't use a custom type for a parameter, did any one find a solution for this?

**UPDATE**

After much experimenting, the problem seems to stem from the call to GetObject being limited to whatever XML / COM-visible type mappings are defined in the ESRI-specific XmlSupport.dat file located in the "bin" folder of the ArcGIS Server install folder.

In other words, my custom XML / COM-visible type mapping file in the XmlClassRegistry folder never seems to be used.

I was able to verify this by copying my type-mapping snippet into XmlSupport.dat, after which the call to GetObject on the SOAP request worked successfully for my custom "Query" type.

Per the ESRI SOAP SOE walkthrough I created the appropriate folders and dropped the specially named mapping file in them. The full path of the custom XML support file is: C:\Program Files (x86)\ArcGIS\Server10.0\XmlClassRegistry\MyCompany\XmlSupportMyProject.dat

So the next question is, why isn't ArcGIS Server looking to my custom type mapping file during calls to GetObject?

Re: SOAP Server Object Extension (SOE) - how to deserialize custom types?

ErinBrimhall — Wed, 18 Apr 2012 14:36:20 GMT

Hi Kevin,

It's been a while, but I believe the problem turned out to be that my custom XML-type file was not in the correct folder. It should be in:

C:\Program Files (x86)\Common Files\ArcGIS\Server 10.0\XmlClassRegistery\SomeCompanyName

I somehow missed the "Common Files" folder and was putting the file in the wrong location.

Also, be sure to recycle the ArcSOM/ArcSOC windows services after copying your XML-types file or making changes to it.

Re: SOAP Server Object Extension (SOE) - how to deserialize custom types?

MiroslavVladár — Sun, 12 Aug 2012 08:58:17 GMT

I have same problem, but with ArcGis Server 10.1, and i cant find XmlClassRegistry folder. Is somethink changed in Server 10.1 ? Thanks

Re: SOAP Server Object Extension (SOE) - how to deserialize custom types?

ErinBrimhall — Thu, 27 Sep 2012 00:03:14 GMT

I have same problem, but with ArcGis Server 10.1, and i cant find XmlClassRegistry folder. Is somethink changed in Server 10.1 ? Thanks

The XmlClassRegistry folder is not created automatically when ArcGIS Server is installed. You will need to manually create any missing folders in the full path:

C:\Program Files\Common Files\ArcGIS\Server10.1\XmlClassRegistry\YourOrganizationName\

Also, be aware that there is a known bug with custom type deserialization in 10.1. See NIM083300.

Re: SOAP Server Object Extension (SOE) - how to deserialize custom types?

TomTyndall — Wed, 21 Nov 2012 18:06:09 GMT

Erin, the originator of this thread, writes that a SOAP SOE was written that successfully sends an array of strings as input. I have a working SOAP SOE that takes a custom type but have had no luck with sending arrays (either with my custom type or even with simple types). Does any one know of samples out there that show the correct syntax for this?

Re: SOAP Server Object Extension (SOE) - how to deserialize custom types?

ErinBrimhall — Sat, 11 Dec 2021 17:09:06 GMT

Erin, the originator of this thread, writes that a SOAP SOE was written that successfully sends an array of strings as input. I have a working SOAP SOE that takes a custom type but have had no luck with sending arrays (either with my custom type or even with simple types). Does any one know of samples out there that show the correct syntax for this?

Tom,

While I do not know of any online samples, I can at least provide you a few code snippets to illustrate how to use arrays as SOE method inputs.

The first snippet is from the hand-rolled WSDL for the SOE. It demonstrates how to define a SOAP method with a string array input parameter.

<xs:element name="GetStuffByStringArray">
  <xs:complexType>
    <xs:sequence>
      <xs:element name="aStringArray" type="ArrayOfString" />
    </xs:sequence>
  </xs:complexType>
</xs:element>
<xs:element name="GetStuffByStringArrayResponse">
  <xs:complexType>
    <xs:sequence>
      <xs:element name="resultString" type="xs:string" />
    </xs:sequence>
  </xs:complexType>
</xs:element>

Note that ArrayOfString is an XML type defined in the Esri namespace. On the ArcObjects side, this XML type can be translated as an IStringArray object as demonstrated in the code block below:

private void GetStuffByStringArray(IMessage request, IMessage response)
{
    // Begin by parsing the string array from the request parameter.
    IStringArray stringCollection;

    int fieldIndex = Utility.FindFieldIndex("aStringArray", request.Parameters, true);
    stringCollection = request.Parameters.GetObject(fieldIndex, Utility.EsriNamespace, "ArrayOfString") as IStringArray;

    // Do something with the string values...
}

Creating an SOE method that takes an array of custom type objects involves a bit more work. I'm happy to provide more detail but here are the essential concepts:

1) Create a new class (e.g. MyTypeCollection) that is derived from SerializableList<T> where T is your custom type that you want to pass a collection of. This collection class will be the type that the SOAP parameter can be deserialized into.

2) Define a corresponding XML type for MyTypeCollection. It will look something like the following:

<xs:complexType name="MyTypeCollection">
  <xs:sequence>
    <xs:element minOccurs="0"
          maxOccurs="unbounded"
          name="MyType"
          type="tns:MyType"/>
  </xs:sequence>
</xs:complexType>

3) Then, in your SOE code itself, deserialize the input parameter using code similar to the following:

MyTypeCollection collection = new MyTypeCollection();
int fieldIndex = Utility.FindFieldIndex("myTypeCollectionInput", request.Parameters, true);
collection = request.Parameters.GetObject(fieldIndex, Utility.CustomTypesNamespace, "MyTypeCollection ") as MyTypeCollection;

Note that your custom types/classes will need to implement the Deserialize and Serialize methods of the IXMLSerialize interface, and each custom type will need an entry in your XML type mapping file. Those topics are covered in detail in Esri's SOE samples.

Hope this helps!

Re: SOAP Server Object Extension (SOE) - how to deserialize custom types?

TomTyndall — Sat, 11 Dec 2021 17:09:09 GMT

Erin,

Thank you for your response and your carefully written questions and responses on this thread - they've helped me digest the many layers of abstraction that go along with SOAP SOE's (difficult, even with having experience writing REST SOE's).

I now have an SOE working that has methods for passing in string, string[] and a custom type and outputting string, string[] and arrays of custom types but still have been unsuccessful writing a method that takes an array of custom types as input. The critical parts of my code are listed below:

wsdl

<xs:element name="TestArrayArguments">
  <xs:complexType>
    <xs:sequence>
      <xs:element name="aCustomTypeCollection" type="tns:MyCustomTypeCollection" />
    </xs:sequence>
  </xs:complexType>
</xs:element>
  
<xs:complexType name="MyCustomTypeCollection">
  <xs:sequence>
    <xs:element minOccurs="0" maxOccurs="unbounded" name="MyCustomType" type="tns:MyCustomType"/>
  </xs:sequence>
</xs:complexType>

<xs:complexType name="MyCustomType">
  <xs:sequence>
    <xs:element name="ID" type="xs:string"/>
  </xs:sequence>
</xs:complexType>

<xs:element name="TestArrayArgumentsResponse">
  <xs:complexType>
    <xs:sequence>
      <xs:element name="Result" type="xs:string" />
    </xs:sequence>
  </xs:complexType>
</xs:element>

<message name="TestArrayArgumentsIn">
  <part name="parameters" element="e:TestArrayArguments" />
</message>
<message name="TestArrayArgumentsOut">
  <part name="parameters" element="e:TestArrayArgumentsResponse" />
</message>
  
<operation name="TestArrayArguments">
    <input message="e:TestArrayArgumentsIn" />
      <output message="e:TestArrayArgumentsOut" />
  </operation>

server side SOE code:

private void TestArrayArguments(IMessage reqMsg, IMessage respMsg)
{
    IXMLSerializeData reqParams = reqMsg.Parameters;
    string retString = "";

    int idx = reqParams.Find("aCustomTypeCollection");
    if (idx == -1)
         throw new ArgumentNullException("PROBLEM MAPPING FIELD TO INDEX");


     retString += "field index is " + idx;
     MyCustomTypeCollection myTypeColl = (MyCustomTypeCollection)reqParams.GetObject(idx, c_ns_soe, "MyCustomTypeCollection");


    respMsg.Name = "TestArrayArgumentsResponse";
    respMsg.NamespaceURI = c_ns_soe;
    respMsg.Parameters.AddString("Result", retString);
}

My custom types:

using System;
using System.Runtime.InteropServices;
using ESRI.ArcGIS.esriSystem;
using ESRI.ArcGIS.SOESupport;


namespace SoapSOE1
{
    [ComVisible(true)]
    [Guid("FCA9A8B6-4B78-45CB-B240-AEA7A6E2F489")]
    [ClassInterface(ClassInterfaceType.None)]
    public class MyCustomType : IXMLSerialize
    {
        public string ID { get; set; }


        public void Serialize(IXMLSerializeData data)
        {
            data.TypeName = this.GetType().Name;
            data.TypeNamespaceURI = SoapSOE1.c_ns_soe;


            data.AddString("ID", ID);
        }


        public void Deserialize(IXMLSerializeData data)
        {
            int idx = FindMandatoryParam("ID", data);
            this.ID = data.GetString(idx);
        }


        private int FindMandatoryParam(string fieldName, IXMLSerializeData data)
        {
            int idx = data.Find(fieldName);
            if (idx == -1)
                throw new MissingMandatoryFieldException(fieldName);
            return idx;
        }


        internal class MissingMandatoryFieldException : Exception
        {
            internal MissingMandatoryFieldException(string fieldName) : base("Missing mandatory field: " + fieldName) { }
        }
    }




    [ComVisible(true)]
    [Guid("D2BC82CA-F8C5-46AD-BFD6-0B80760A351E")]
    [ClassInterface(ClassInterfaceType.None)]
    public class MyCustomTypeCollection : SerializableList<MyCustomType>
    {
        public MyCustomTypeCollection(string namespaceURI) : base(namespaceURI) { }
    }
}

I've carefully constructed my XmlSupportAdot.dat file (everything works if I just send an instance of my custom type and not an array so I know this file is getting seen).

I've registered my dll on the server as described in NIM083300.

I'm getting the same "Exception from HRESULT: 0x80043068" you describe when executing the GetObject call. If I comment out that line I get no error and can see that the field is correctly mapped to index zero. At this point I'm not trying to access the contents of the input array, I'm just trying to instantiate MyCustomTypeCollection using the GetObject call.

Attached is a snapshot of the HTTP calls taking place when I try to invoke the SOE method from a client.

[ATTACH=CONFIG]19719[/ATTACH]

I'd greatly appreciate any suggestions you may have. Have you been successful in doing this before?

Thank you for your time,
Tom
AZ Dept of Transportation

Re: SOAP Server Object Extension (SOE) - how to deserialize custom types?

ErinBrimhall — Wed, 05 Dec 2012 19:10:02 GMT

I've carefully constructed my XmlSupportAdot.dat file (everything works if I just send an instance of my custom type and not an array so I know this file is getting seen).

Is there an entry in your XmlSupportAdot.dat file for the MyCustomTypeCollection class? Sounds like there most likely is but I wanted to confirm.

Beyond that, all the code examples you shared look correct to me. There are a few minor differences between your prototype and the code I've developed:

I only have a parameterless constructor for my Collection class (and pass a static namespace value to the base constructor).

The Collection object that is passed to my SOE methods is actually an attribute of a different custom class; I don't pass the Collection object by itself.

With that said, I don't believe either of these differences would cause the classic deserialization error you're seeing.

I'll take another look at your post and let you know if I can come up with any other ideas.

Re: SOAP Server Object Extension (SOE) - how to deserialize custom types?

TomTyndall — Wed, 05 Dec 2012 21:01:55 GMT

I do have an entry in XmlSupportAdot.dat for MyCustomTypeCollection.

I modified the constructor on my collection class to also use a static string and I do have a change of behaviour - I'm now getting invalid cast exception. I'll have to play around with this more to see what is happening.

[ATTACH=CONFIG]19734[/ATTACH]

Since I have been able to get a working method that I can pass a custom object to, I also considered passing the collection as an attribute on a custom object but got bogged down in the wsdl syntax and the serialize/deserialize code for that scenario. Do you have any examples of what that looks like?

Thanks,
Tom

Re: SOAP Server Object Extension (SOE) - how to deserialize custom types?

ErinBrimhall — Wed, 05 Dec 2012 21:59:42 GMT

Are you able to actively debug your SOE as it runs? My guess is that the cast-exception is stemming from this line:

MyCustomTypeCollection myTypeColl = (MyCustomTypeCollection)reqParams.GetObject(idx, c_ns_soe, "MyCustomTypeCollection");

It should be equivalent to what you already have, but you can also try using type attributes instead of hard-coding the "MyCustomTypeCollection" string. E.g.:

typeof(MyCustomTypeCollection).Name

I bring this up since it was another difference I noticed after examining my code. I'll post tomorrow an example of a custom type w/ a custom type collection attribute.

Re: SOAP Server Object Extension (SOE) - how to deserialize custom types?

TomTyndall — Sat, 11 Dec 2021 17:09:12 GMT

No, I don't have a working debugger; ArcGIS Server is on a different machine than my development workstation. I have been logging for debugging.

Using the runtime type name without any other casting still doesn't work.

logger.LogMessage(ServerLogger.msgType.warning, "segment extraction", -1, typeof(MyCustomTypeCollection).Name);
object myTypeColl = reqParams.GetObject(idx, c_ns_soe, typeof(MyCustomTypeCollection).Name);

Logger output: [ATTACH=CONFIG]19783[/ATTACH]

Comment out that GetObject call and it runs without problem.

My xml/soap comfort level is not that high and I get tangled up in namespace issues but I was thinking more about the change in behavior (throwing a invalid cast instead of the HRESULT: 0x80043068 error)when going from a passed in namespace

public MyCustomTypeCollection(string namespaceURI) : base(namespaceURI) { }

to a hardcoded one

public MyCustomTypeCollection() : base("http://www.myOrg.com/schemas/1.0") { }

In the "passed in" case ArcGIS Server is supplying the namespace when instantiating MyCustomTypeCollection. How is the namespace being set when it instantiates the MyCustomType's contained in that collection?

I order to get something up and running now I rewrote the service to take a string and I have the client build a delimited string of the MyCustomType payload data.

Thanks again for the help.

Re: SOAP Server Object Extension (SOE) - how to deserialize custom types?

TomTyndall — Sat, 11 Dec 2021 17:09:15 GMT

I wanted to see what ArcGIS Server was actually passing into the MyCustomTypeCollection constructor and I believe the answer is nothing.

public MyCustomTypeCollection(string namespaceURI) : base(namespaceURI)
{
      SoapSOE1.logger.LogMessage(ServerLogger.msgType.warning, "collection constructor", -1, namespaceURI);
}

The construction fails (see image of log) and I'm assuming that is because Server is trying to invoke a no parameter constructor and not finding one; hence the behavior change outlined in the previous post when changing to a hard coded namespace.

[ATTACH=CONFIG]19787[/ATTACH]

I got that pattern from the CustomLayerInfos constructor in ESRI's FindNearFeaturesSoapSOE sample. That CustomLayerInfos object is a Server output though, not an input so apparently that idiom only applies to outputs of a soap service.

TT

Re: SOAP Server Object Extension (SOE) - how to deserialize custom types?

ErinBrimhall — Sat, 11 Dec 2021 17:09:18 GMT

Hi Tom,

public MyCustomTypeCollection(string namespaceURI) : base(namespaceURI) { }
...

In the "passed in" case ArcGIS Server is supplying the namespace when instantiating MyCustomTypeCollection. How is the namespace being set when it instantiates the MyCustomType's contained in that collection?

I haven't verified this but I always assumed the typeNamespace parameter of the GetObject method was ultimately passed all the way through to the collection type's constructor. Regardless, I can definitely confirm that the parameterless-constructor w/ a static/hard-coded namespace value passed to the base constructor works fine in both AGS 10.0 and 10.1.

No, I don't have a working debugger; ArcGIS Server is on a different machine than my development workstation. I have been logging for debugging.

This makes troubleshooting more challenging. Even with the ability to develop and debug locally on the server, it still took me several days to work through all the deserialization issues in 10.1 and correctly implement the workaround (as described in NIM-083300).

Are you performing the 64-bit regasm.exe assembly registration each time you rebuild and redeploy your SOE code? I would also highly recommend developing on the server if that option is available to you. The reasons for some of the tougher issues I ran into were not apparent until I was able to actively debug the SOE.

One last quick set of thoughts and things to try:

The fact that you can pass a single custom type object as an input parameter to an SOE method tells me that your namespaces, XML type-mapping file, and SOE assembly registration (i.e. w/ the 64-bit version of regasm.exe) are correct.

You are inspecting the SOAP message you send, but it might also be a good idea to inspect that message once it reaches the SOE. The code snippet below will help support that:

public static string Read(this IMessage message)
{
  string xml;
  XMLStreamClass xmlStream = new XMLStreamClass();
  message.WriteXML(xmlStream);
  xml = xmlStream.SaveToString();

  return xml;
}

You may want to try just working with your collection type directly in the SOE, e.g. explicitly instantiate a collection object, add items to it, cast it, etc., just to make sure it is a valid class at runtime. I ran into a situation where multiple versions of my assemblies were somehow being registered, which was causing type conflict errors that were being swallowed unless I was actively debugging the code.

Add some logging inside the Deserialize method of the "MyCustomType" class. This method will be called for each object that was passed in the collection, so writing to the log from here will give you an idea if the code is even making it this far or if the "invalid cast exception" is being thrown earlier.

Re: SOAP Server Object Extension (SOE) - how to deserialize custom types?

DanaFernandez — Mon, 30 Mar 2015 16:28:08 GMT

I am having same issue in 10.3, I tried the steps above and they do not fix the problem, does anyone have a fix for this in 10.3?