Feature: Plans for adding memories to the dev team skills.

[rysweet]

Some initial thoughts on adding memories.

Types of memories

I think we want the skills to be able to take advantage of multiple types of memories. This will allow for specialization, and for adding new types in the future, while keeping the collections topic focused and clean.

Memory collection for the Repo

Scope: per repository
This collection would include

• all the files in the repo, ideally chunked according to their type/structure
• Code Explanations for all the files in the repo, produced by the model
• Issues from the repo
• a "repo map" of project structure in natural language or maybe we find/invent a format

Shared Memory across the Skills

Scope: per repository (possibly per v-team - what if teams work on multiple repos?)
This collection would be used to help coordinate actions across the v-team of AI skills:

• including all instructions received from the human collaborators
• conversation history
• identity and specializations of each skill in the V-team
• role-assignments/what each member of the team is working on
• for shared systems, (if the plan specifies a shared component) then key details of the shared system (eg config var names, directory paths, class names, etc)
• think of this as shared memory for all the skills, we could actually make a record that stores important shared state in the vectored and is retrieved as needed by each skill

Working memory within a single skill

Scope: per repository (possibly per v-team - what if teams work on multiple repos?)
This collection would implement the working memory for an individual skill (or skill group) and include the original prompt and conversation history for that skill, as well as any data on the files that the skill may have modified or generated.

Specialization Memories

Scope: Any
These would be special read-only curated collections designed to give an AI agent a set of memories around some specialized knowledge. Think of in the Matrix when Neo says "Now I know KungFu!". They would be wired up through configuration or code depending on the needs of the skill.
eg:

• WAF framework, 12 Factor, etc for Azure architecture
• .NET learning content, .NET best practices, r9 SDK etc for .NET specialists
• ^^^^ for each programming language...
• CVEs and MITRE, security coding standards, examples of security fix before and after etc for security specialist
• patterns and practices for performance improvement for specific languages.
  Each of these types of memories should end up being a long running shared service with its own nuget etc that can just be included with a "using". They can improve over time as new documentation or code becomes available.

Implementation

Some thoughts on how to do the implementation:

• let's make each type of memory a separate project but stick to an API that is compatible with the SK MemoryStore
• while prompt templating {{recalll}} can be useful, more likely we will need a pattern that is specific to each skill for bringing in the memories at the right spot using the vector search.
• Each skill would likely apply an hierarchy to the memories, with an implied flow:
• 1. Prompt assembly
• 2. Shared Memory for details relevant to the prompt
• 3. Working Memory details relevant to the prompt
• 4. Specializations collections that are relevant to the prompt

We would need to allocate a certain amount of the token budget to each memory type, but for instance if vector search of working memory doesn't yield relevant results we might give some of its budget to the Specializations.

We should probably build the memory initialization as a background service that starts and does a warm-up when the app starts.