Azure/azure-search-vector-samples

Bug : Unable to create merge skill from the skillset

Opened this issue · 2 comments

surendransuri commented

I am trying to load the split the documents into chunks and create embeddings from that. To save some contextual information I want to concatenate the file name of the each document to its chunk at the starting of the chunk.

I want to have like 'chunk: title: filename, chunk_data' in the field of chunk. how to achieve this. Normal text itself it is getting failed. Please help me resolve this

merge_skill = MergeSkill(
description="Skill to Merge text data via Skillset",
context= "/document/pages/",
insertPreTag= " ",
insertPostTag= " ",
inputs=[
InputFieldMappingEntry(name="text", source="/document/pages/"),
InputFieldMappingEntry(name="itemsToInsert", source="/document/metadata_storage_name"),
InputFieldMappingEntry(name="span", source=["0"]),
],
outputs=[
OutputFieldMappingEntry(name="mergedText", target_name="pageContent")
],
)

mattgotteiner commented

You can't do this easily using merge skill because /document/metadata_storage_name is not an array. Please consider using a custom skill to do this concatenation

  • 👀1
surendransuri commented

Can you please provide sample code for this. For using the custom skill is it required to have model/endpoint in Azure?

  • 👍1