elastic/elasticsearch-net

A dynamic template must be defined with a name

Closed this issue · 3 comments

Elastic.Clients.Elasticsearch version: 8.18.1

Elasticsearch version: 8.6.1

.NET runtime version: .net9

Operating system version: not sure. Elastic cloud

Description of the problem including expected versus actual behavior:
I upgrade my library from 8.17.4 to 8.18.1. One of the breaking changes around the upgrade was to change TypeMapping.DynamicTemplates from ICollection<IDictionary<string, DynamicTemplate>> to a ICollection<KeyValuePair<string, DynamicTemplate>> collection.

Now when trying to create a new index, I get this error:

{"Request failed to execute. Call: Status code 400 from: PUT /ptest_blog1_articles?pretty=true&error_trace=true. ServerError: Type: mapper_parsing_exception Reason: "Failed to parse mapping: A dynamic template must be defined with a name" CausedBy: "Type: mapper_parsing_exception Reason: "A dynamic template must be defined with a name"""}

Steps to reproduce:

  1. Create an index with at least one Dynamic Template and you should run into the error.

Expected behavior
I should be able to create the index. I am guessing there is an issue with the request serializer which is not converting the key in the KeyValuePair<> to template name.

Hi @MikeAlhayek ,

could you please send me the JSON request? You can log this by enabling debug mode:

var settings = new ElasticsearchClientSettings(new Uri("https://..."))
    .Authentication(...)
    // ...
    .EnableDebugMode(cd => Console.WriteLine(cd.DebugInformation));

var client = new ElasticsearchClient(settings);

@flobernd Yes, as expected, there's no built-in serializer that can directly convert ICollection<KeyValuePart<string, DynamicTemplate>> into a dictionary-style JSON object (key-value pairs).

I hope the fix will be quick and we'll see 8.18.2 release very soon to avoid having to downgrade since it's a breaking change.

# Audit trail of this API call:
 - [1] BadResponse: Node: https://012e18f605c341ec9d5f861e8391e161.eastus.azure.elastic-cloud.com/ Took: 00:00:00.1809823
# OriginalException: Elastic.Transport.TransportException: Request failed to execute. Call: Status code 400 from: PUT /ptest_blog2_articlestest?pretty=true&error_trace=true. ServerError: Type: mapper_parsing_exception Reason: "Failed to parse mapping: A dynamic template must be defined with a name" CausedBy: "Type: mapper_parsing_exception Reason: "A dynamic template must be defined with a name""
# Request:
{
  "mappings": {
    "dynamic_templates": [
      {
        "key": "*.Inherited",
        "value": {
          "match_mapping_type": "string",
          "path_match": "*.Inherited",
          "mapping": {
            "type": "keyword"
          }
        }
      },
      {
        "key": "*.Ids",
        "value": {
          "match_mapping_type": "string",
          "path_match": "*.Ids",
          "mapping": {
            "type": "keyword"
          }
        }
      },
      {
        "key": "*.Location",
        "value": {
          "match_mapping_type": "object",
          "path_match": "*.Location",
          "mapping": {
            "type": "geo_point"
          }
        }
      }
    ],
    "_meta": {
      "last_task_id": 0
    },
    "properties": {
      "ContentItemId": {
        "type": "keyword"
      },
      "ContentItemVersionId": {
        "type": "keyword"
      },
      "Content.ContentItem.Owner": {
        "type": "keyword"
      },
      "Content.ContentItem.FullText": {
        "type": "text"
      },
      "Content.ContentItem.ContainedPart": {
        "properties": {
          "Ids": {
            "type": "keyword"
          },
          "Order": {
            "type": "float"
          }
        },
        "type": "object"
      },
      "Content.ContentItem.DisplayText": {
        "properties": {
          "Analyzed": {
            "type": "text"
          },
          "Normalized": {
            "type": "keyword"
          },
          "Keyword": {
            "type": "keyword"
          }
        },
        "type": "object"
      },
      "Content.ContentItem.ContentType": {
        "type": "keyword"
      }
    },
    "_source": {
      "enabled": true,
      "excludes": [
        "Content.ContentItem.DisplayText.Analyzed"
      ]
    }
  },
  "settings": {
    "analysis": {
      "analyzer": {
        "standard": {
          "type": "standard"
        }
      },
      "filter": {}
    }
  }
}
# Response:
{
  "error" : {
    "root_cause" : [
      {
        "type" : "mapper_parsing_exception",
        "reason" : "A dynamic template must be defined with a name",
        "stack_trace" : "org.elasticsearch.index.mapper.MapperParsingException: A dynamic template must be defined with a name\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.index.mapper.RootObjectMapper.processField(RootObjectMapper.java:492)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.index.mapper.RootObjectMapper.parse(RootObjectMapper.java:442)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.index.mapper.MappingParser.parse(MappingParser.java:102)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.index.mapper.MappingParser.parse(MappingParser.java:96)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.index.mapper.MapperService.parseMapping(MapperService.java:370)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:347)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.metadata.MetadataCreateIndexService.updateIndexMappingsAndBuildSortOrder(MetadataCreateIndexService.java:1331)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.metadata.MetadataCreateIndexService.lambda$applyCreateIndexWithTemporaryService$2(MetadataCreateIndexService.java:471)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.indices.IndicesService.withTempIndexService(IndicesService.java:686)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexWithTemporaryService(MetadataCreateIndexService.java:469)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequestWithV1Templates(MetadataCreateIndexService.java:586)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequest(MetadataCreateIndexService.java:419)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.metadata.MetadataCreateIndexService$1.execute(MetadataCreateIndexService.java:298)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.service.MasterService$UnbatchedExecutor.execute(MasterService.java:551)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.service.MasterService.innerExecuteTasks(MasterService.java:1052)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:1017)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:278)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:170)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:110)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:148)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:850)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:257)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:223)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1589)\n"
      }
    ],
    "type" : "mapper_parsing_exception",
    "reason" : "Failed to parse mapping: A dynamic template must be defined with a name",
    "caused_by" : {
      "type" : "mapper_parsing_exception",
      "reason" : "A dynamic template must be defined with a name",
      "stack_trace" : "org.elasticsearch.index.mapper.MapperParsingException: A dynamic template must be defined with a name\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.index.mapper.RootObjectMapper.processField(RootObjectMapper.java:492)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.index.mapper.RootObjectMapper.parse(RootObjectMapper.java:442)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.index.mapper.MappingParser.parse(MappingParser.java:102)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.index.mapper.MappingParser.parse(MappingParser.java:96)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.index.mapper.MapperService.parseMapping(MapperService.java:370)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:347)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.metadata.MetadataCreateIndexService.updateIndexMappingsAndBuildSortOrder(MetadataCreateIndexService.java:1331)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.metadata.MetadataCreateIndexService.lambda$applyCreateIndexWithTemporaryService$2(MetadataCreateIndexService.java:471)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.indices.IndicesService.withTempIndexService(IndicesService.java:686)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexWithTemporaryService(MetadataCreateIndexService.java:469)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequestWithV1Templates(MetadataCreateIndexService.java:586)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequest(MetadataCreateIndexService.java:419)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.metadata.MetadataCreateIndexService$1.execute(MetadataCreateIndexService.java:298)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.service.MasterService$UnbatchedExecutor.execute(MasterService.java:551)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.service.MasterService.innerExecuteTasks(MasterService.java:1052)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:1017)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:278)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:170)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:110)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:148)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:850)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:257)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:223)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1589)\n"
    },
    "stack_trace" : "org.elasticsearch.index.mapper.MapperParsingException: Failed to parse mapping: A dynamic template must be defined with a name\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.index.mapper.MapperService.parseMapping(MapperService.java:372)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:347)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.metadata.MetadataCreateIndexService.updateIndexMappingsAndBuildSortOrder(MetadataCreateIndexService.java:1331)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.metadata.MetadataCreateIndexService.lambda$applyCreateIndexWithTemporaryService$2(MetadataCreateIndexService.java:471)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.indices.IndicesService.withTempIndexService(IndicesService.java:686)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexWithTemporaryService(MetadataCreateIndexService.java:469)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequestWithV1Templates(MetadataCreateIndexService.java:586)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequest(MetadataCreateIndexService.java:419)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.metadata.MetadataCreateIndexService$1.execute(MetadataCreateIndexService.java:298)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.service.MasterService$UnbatchedExecutor.execute(MasterService.java:551)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.service.MasterService.innerExecuteTasks(MasterService.java:1052)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:1017)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:278)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:170)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:110)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:148)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:850)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:257)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:223)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1589)\nCaused by: org.elasticsearch.index.mapper.MapperParsingException: A dynamic template must be defined with a name\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.index.mapper.RootObjectMapper.processField(RootObjectMapper.java:492)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.index.mapper.RootObjectMapper.parse(RootObjectMapper.java:442)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.index.mapper.MappingParser.parse(MappingParser.java:102)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.index.mapper.MappingParser.parse(MappingParser.java:96)\n\tat org.elasticsearch.server@8.6.1/org.elasticsearch.index.mapper.MapperService.parseMapping(MapperService.java:370)\n\t... 21 more\n"
  },
  "status" : 400
}

# TCP states:
  Established: 72
  TimeWait: 8
  CloseWait: 1

# ThreadPool statistics:
  Worker: 
    Busy: 1
    Free: 32766
    Min: 24
    Max: 32767
  IOCP: 
    Busy: 0
    Free: 1000
    Min: 1
    Max: 1000

Hi @MikeAlhayek , I'll have a look. The surprising part is that we actually register a KeyValuePair converter to the internal JsonSerializerOptions. I'm going to double check.