Select to view content in your preferred language

How to Change the Tokenizer Used in ArcGIS Knowledge Search Interface

484
2
Jump to solution
09-21-2023 08:28 PM
Labels (1)
xingchenc
Occasional Contributor

@AdamMartin @MeganBanaski1 

When using the search interface in the ArcGIS Pro to search the knowlegde graph, at the back end it will convert what we entered into an AQL query. like the one shown below:

WITH `esri__search_idx` let esri__query_client_uuid = @esri__query_client_uuid FOR property IN ['TOWN_ID','metadata','keywords','globalid','PROVINCE_NAME','title','YARD_NAME','BUILDING_NAME','TOWN_NAME','REGION_NAME','CITY_NAME','name','originGlobalID','YARD_ID','text','destinationGlobalID','GEOENTITYID'] FOR doc IN esri__search_idx SEARCH (ANALYZER ((TOKENS("香", "text_en") ANY IN doc[property]), "text_en")) OPTIONS { collections: ["test","Document"]} COLLECT globalid = doc.globalid INTO group = {"property": property,"propertyScore": TFIDF(doc, true)} SORT MAX(group[*].propertyScore) DESC RETURN {globalid: globalid,propertyNames: group[*].property,propertyScores: group[*].propertyScore} Bind Variables: esri__query_client_uuid : 4ee97b79-7c9c-44e7-848d-1700a4c00c6a

By default, it will use a "text_en" tokenizer to split the words we entered. But if the knowledge graph is stored in other language, the search result accuracy will be pretty low. I was wondering if I can change the tokenizer to better match the language used in the knowledge graph?

Thanks!  

0 Kudos
1 Solution

Accepted Solutions
dvarshnay
New Contributor

Currently, our search operation does not offer support for user-defined tokenization. However, we have intentions to include this feature in a future release.

 

 

 

View solution in original post

0 Kudos
2 Replies
dvarshnay
New Contributor

Currently, our search operation does not offer support for user-defined tokenization. However, we have intentions to include this feature in a future release.

 

 

 

0 Kudos
xingchenc
Occasional Contributor

Hi @dvarshnay 

Thanks for the clarification.

0 Kudos