Support new backend implementation for rag-ai plugin

Question

Support new backend implementation for rag-ai plugin

Closed this issue 3 months ago · 2 comments

tranngoclam commented 6 months ago

Feature Suggestion

Support new backend of backstage for rag-ai plugin

Possible Implementation

N/A

Context

I have added rag-ai plugin and used with new backend of backstage, but got an error during startup

2024-05-08T04:17:34.370765253Z /app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1806
2024-05-08T04:17:34.370773628Z           throw new errors.ForwardedError(
2024-05-08T04:17:34.370776545Z                 ^
2024-05-08T04:17:34.370789462Z
2024-05-08T04:17:34.370791337Z ForwardedError: Plugin 'rag-ai' startup failed; caused by Error: augmentationIndexer must be registered
2024-05-08T04:17:34.370792753Z     at /app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1806:17
2024-05-08T04:17:34.370794170Z     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
2024-05-08T04:17:34.370795545Z     at async /app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1805:9
2024-05-08T04:17:34.370797045Z     at async Promise.all (index 9)
2024-05-08T04:17:34.370798378Z     ... 2 lines matching cause stack trace ...
2024-05-08T04:17:34.370799712Z     at async BackstageBackend.start (/app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1894:5) {
2024-05-08T04:17:34.370809295Z   cause: Error: augmentationIndexer must be registered
2024-05-08T04:17:34.370811170Z       at Object.init [as func] (/app/node_modules/@roadiehq/rag-ai-backend/dist/index.cjs.js:255:17)
2024-05-08T04:17:34.370813086Z       at /app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1805:31
2024-05-08T04:17:34.370814795Z       at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
2024-05-08T04:17:34.370816420Z       at async Promise.all (index 9)
2024-05-08T04:17:34.370818170Z       at async BackendInitializer.doStart_fn (/app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1766:3)
2024-05-08T04:17:34.370819920Z       at async BackendInitializer.start (/app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1626:5)
2024-05-08T04:17:34.370821378Z       at async BackstageBackend.start (/app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1894:5)
2024-05-08T04:17:34.370822836Z }
2024-05-08T04:17:34.370824086Z

My implementation

packages/backend/src/index.ts

import {legacyPlugin} from '@backstage/backend-common';
import {createBackend} from '@backstage/backend-defaults';

const backend = createBackend();

...
backend.add(legacyPlugin('ai', import('./plugins/ai')));
backend.add(import('@roadiehq/rag-ai-backend'));

packages/backend/src/plugins/ai.ts

import { createApiRoutes as initializeRagAiBackend } from '@roadiehq/rag-ai-backend';
import { initializeOpenAiEmbeddings } from '@roadiehq/rag-ai-backend-embeddings-openai';
import { createRoadiePgVectorStore } from '@roadiehq/rag-ai-storage-pgvector';
import { createDefaultRetrievalPipeline } from '@roadiehq/rag-ai-backend-retrieval-augmenter';
import { OpenAI } from '@langchain/openai';
import { CatalogClient } from '@backstage/catalog-client';
import { PluginEnvironment } from '../types';

export default async function createPlugin({
    logger,
    tokenManager,
    database,
    discovery,
    config,
}: PluginEnvironment) {
    const catalogApi = new CatalogClient({
        discoveryApi: discovery,
    });

    const vectorStore = await createRoadiePgVectorStore({ logger, database, config });

    const augmentationIndexer = await initializeOpenAiEmbeddings({
        logger,
        tokenManager,
        vectorStore,
        catalogApi,
        discovery,
        config,
    });

    const model = new OpenAI();

    const ragAi = await initializeRagAiBackend({
        logger,
        tokenManager,
        augmentationIndexer,
        retrievalPipeline: createDefaultRetrievalPipeline({
            discovery,
            logger,
            vectorStore: augmentationIndexer.vectorStore,
            tokenManager,
        }),
        model,
        config,
    });

    return ragAi.router;
}

package.json

  "dependencies": {
    "@backstage/backend-common": "^0.21.7",
    "@backstage/backend-defaults": "^0.2.17",
    "@backstage/backend-plugin-api": "^0.6.17",
    "@backstage/backend-tasks": "^0.5.22",
    "@backstage/catalog-client": "^1.6.4",
    "@backstage/catalog-model": "^1.4.5",
    "@backstage/config": "^1.2.0",
    "@backstage/plugin-app-backend": "^0.3.65",
    "@backstage/plugin-auth-backend": "^0.22.4",
    "@backstage/plugin-auth-backend-module-guest-provider": "^0.1.3",
    "@backstage/plugin-auth-node": "^0.4.12",
    "@backstage/plugin-catalog-backend": "^1.21.1",
    "@backstage/plugin-catalog-backend-module-gitlab": "^0.3.15",
    "@backstage/plugin-catalog-backend-module-gitlab-org": "^0.0.0-nightly-20240423021331",
    "@backstage/plugin-catalog-backend-module-scaffolder-entity-model": "^0.1.15",
    "@backstage/plugin-kubernetes-backend": "^0.17.0",
    "@backstage/plugin-permission-backend": "^0.5.41",
    "@backstage/plugin-permission-backend-module-allow-all-policy": "^0.1.14",
    "@backstage/plugin-permission-common": "^0.7.13",
    "@backstage/plugin-permission-node": "^0.7.28",
    "@backstage/plugin-proxy-backend": "^0.4.15",
    "@backstage/plugin-scaffolder-backend": "^1.22.5",
    "@backstage/plugin-search-backend": "^1.5.7",
    "@backstage/plugin-search-backend-module-catalog": "^0.1.23",
    "@backstage/plugin-search-backend-module-pg": "^0.5.26",
    "@backstage/plugin-search-backend-module-techdocs": "^0.1.22",
    "@backstage/plugin-search-backend-node": "^1.2.21",
    "@backstage/plugin-techdocs-backend": "^1.10.4",
    "@roadiehq/rag-ai-backend": "^0.3.0",
    "@roadiehq/rag-ai-backend-embeddings-openai": "^0.2.2",
    "@roadiehq/rag-ai-backend-retrieval-augmenter": "^0.3.0",
    "@roadiehq/rag-ai-storage-pgvector": "^0.1.1",
    "app": "link:../app",
    "better-sqlite3": "^8.0.0",
    "dockerode": "^3.3.1",
    "express": "^4.17.1",
    "express-promise-router": "^4.1.0",
    "pg": "^8.3.0",
    "winston": "^3.2.1"
  },

Answer 1 · 2024-06-06T12:01:59.000Z

This configuration was able to get me to startup, and queries OpenAI, but there is an auth error when querying the search endpoint for RAG entries.

// rag ai plugin
backend.add(import('@roadiehq/rag-ai-backend'));
backend.add(
  createBackendModule({
    pluginId: 'rag-ai',
    moduleId: 'config',
    register(env) {
      env.registerInit({
        deps: {
          tokenManager: coreServices.tokenManager,
          logger: coreServices.logger,
          database: coreServices.database,
          discovery: coreServices.discovery,
          config: coreServices.rootConfig,
          indexer: augmentationIndexerExtensionPoint,
          pipeline: retrievalPipelineExtensionPoint,
          model: modelExtensionPoint,
        },
        async init({ tokenManager, logger, database, discovery, config, indexer, pipeline, model }) {
          const catalogApi = new CatalogClient({ discoveryApi: discovery });
          const vectorStore = await createRoadiePgVectorStore({
            logger: loggerToWinstonLogger(logger),
            database,
            config,
          });

          indexer.setAugmentationIndexer(await initializeOpenAiEmbeddings({
            tokenManager,
            logger: loggerToWinstonLogger(logger),
            discovery,
            catalogApi,
            vectorStore,
            config,
          }));

          pipeline.setRetrievalPipeline(createDefaultRetrievalPipeline({
            tokenManager,
            logger: loggerToWinstonLogger(logger),
            discovery,
            vectorStore,
          }));

          model.setBaseLLM(new OpenAI({
            model: 'gpt-4o',
            temperature: 0.9,
            apiKey: config.getString('ai.embeddings.openai.openAIApiKey'),
          }));
        },
      });
    },
  })
);

Logged request error:

2024-06-06T12:01:00.847Z rag-ai info Received 0 embeddings from Vector store 
2024-06-06T12:01:00.851Z rootHttpRouter info ::ffff:127.0.0.1 - - [06/Jun/2024:12:01:00 +0000] "GET /api/search/query?term=Hello%20world&types[0]=software-catalog HTTP/1.1" 401 812 "-" "node" type=incomingRequest
2024-06-06T12:01:00.851Z rag-ai warn Unable to query Backstage search API for embeddable results. label=rag-ai-searchclient
2024-06-06T12:01:00.852Z rag-ai info Received 0 results when querying augmentations from search. 
2024-06-06T12:01:00.852Z rag-ai info Starting to prompt LLM. 
2024-06-06T12:01:01.667Z rootHttpRouter info ::ffff:127.0.0.1 - - [06/Jun/2024:12:01:01 +0000] "POST /backstage/api/rag-ai/query/catalog HTTP/1.1" 200 89 "[http://localhost:3000/"](http://localhost:3000/%22) "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36" type=incomingRequest

Answer 2 · 2024-08-05T12:07:24.000Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.