enhance chunking mechanism for improved answer generation

[SaraElBrak]

User story

1. As a software developer
2. I want / need to include neighboring chunks in the input sent to the LLM responsible for answer generation
3. So that the chatbot can generate more accurate and contextually appropriate responses.

Acceptance criteria

• Update the existing chunking mechanism to retrieve not only the best matching chunk but also the neighboring chunks from the database based on the user's query.
• Test the enhanced feature using a diverse set of user queries and evaluate the chatbot's generated responses for accuracy and relevance.
• Document the enhancement.

Definition of done (DoD)

• Acceptance criteria are met.
• Work products are uploaded to the Github repository.
• A pull request is created for each related branch.
• The work products in the pull requests are reviewed.
• The corresponding branches are merged and closed.
• The bill of materials section of the planning documents is updated.
• All defined conventions are complied with.

DoD general criteria

• Feature has been fully implemented
• Feature has been merged into the mainline
• All acceptance criteria were met
• Product owner approved features
• All tests are passing
• Developers agreed to release