Pinecone Starter plan does not work

[ekusiadadus]

why

Pinecone has decided not to support the namespace feature for their free plan.

In my code, I was using the namespace feature like this:

const namespace = pineconeIndex.namespace(convertToAscii(fileKey));

Due to this limitation, I can't use the namespace feature in two places in my code. Although it's possible to proceed without using namespaces, this would result in answers being pulled from multiple files, which is not the desired behavior.

[MMcBreezy]

I came to the same conclusion when getting:

"PineconeBadRequestError: The requested feature 'Namespaces' is not supported by the current index type 'Starter'."

[battlepass210]

why

Pinecone has decided not to support the namespace feature for their free plan.

In my code, I was using the namespace feature like this:

const namespace = pineconeIndex.namespace(convertToAscii(fileKey));

Due to this limitation, I can't use the namespace feature in two places in my code. Although it's possible to proceed without using namespaces, this would result in answers being pulled from multiple files, which is not the desired behavior.

For me what I did was I upgrade my plan (wont charge you by upgrading, but will charge as you use), and delete the previous project and indexes, then create another new project with one of the free provided environments (not gcp-starter but something like Elliott's asia-southeast1-gcp-free), and yea you should be able to use namespaces.

[choopk]

it does not support namespace anymore. So if you still want to finish the project without namespace, you can use the following code

export async function loadS3IntoPinecone(fileKey: string) {
  console.log("downloading s3 into file system");
  const file_name = await downloadFromS3(fileKey);
  if (!file_name) throw new Error("could not download file from s3");
  const loader = new PDFLoader(file_name);
  const pages = (await loader.load()) as PDFPage[];

  const documents = await Promise.all(pages.map(prepareDocument));

  const vectors = await Promise.all(documents.flat().map(embedDocument));

  const client = await getPineconeClient();
  const pineconIndex = client.Index("chatpdf-youtube");

  console.log("Inserting vectors into pinecone");
  const request = {
    vectors,
  };
  await pineconIndex.upsert({ upsertRequest: request });
  console.log("Inserted vectors into pinecone");

  return documents[0];
}

[bagedaddy]

@choopk, the pineconeIndex that you are using I think is in the older version.

While trying to follow the updated pinecone 1.0.1 I am getting this: Argument of type '{ upsertRequest: { vectors: PineconeRecord[]; }; }' is not assignable to parameter of type 'PineconeRecord[]'.
Object literal may only specify known properties, and 'upsertRequest' does not exist in type 'PineconeRecord[]'.ts(2345)

Anyone know the right way to add into pinecone 1.0.1 without using namespace? I'm also looking into it

[choopk]

@choopk, the pineconeIndex that you are using I think is in the older version.

While trying to follow the updated pinecone 1.0.1 I am getting this: Argument of type '{ upsertRequest: { vectors: PineconeRecord[]; }; }' is not assignable to parameter of type 'PineconeRecord[]'. Object literal may only specify known properties, and 'upsertRequest' does not exist in type 'PineconeRecord[]'.ts(2345)

Anyone know the right way to add into pinecone 1.0.1 without using namespace? I'm also looking into it

the error message already points out the format you should pass. Check the type definition and pass in the correct format and it should work

[mezamateoj]

Hey, this fix worked for me:
if your environment for the project is gcp-starter it does not support namespaces. You can still use the free-tier by creating a new project:

1. Delete the project because the free tier only supports 1 project. (If you get a bad response deleting the project, delete any indexes you created first).
2. Create a new project, and index.
3. The new project will have a different environment that you choose when you create the project.
4. Create a new index and get the new API Key

source: https://stackoverflow.com/questions/76563752/pinecone-ingest-requested-feature-namespaces-is-not-supported

[ekusiadadus]

For me, using a chroma or indexed DB is a good alternative...

[CodeOfMugiwara]

@mezamateoj
Deleted the project and indexes, but while created a new project it only showing the gcp-starter environment and all the other environments are paid as it shows.
How's your going, have you solved the error.

@choopk the code you've provided returning errors how to fix them

@ekusiadadus How to use chroma and what could be the process and code for it

[kalilfagundes]

it does not support namespace anymore. So if you still want to finish the project without namespace, you can use the following code

export async function loadS3IntoPinecone(fileKey: string) {
  console.log("downloading s3 into file system");
  const file_name = await downloadFromS3(fileKey);
  if (!file_name) throw new Error("could not download file from s3");
  const loader = new PDFLoader(file_name);
  const pages = (await loader.load()) as PDFPage[];

  const documents = await Promise.all(pages.map(prepareDocument));

  const vectors = await Promise.all(documents.flat().map(embedDocument));

  const client = await getPineconeClient();
  const pineconIndex = client.Index("chatpdf-youtube");

  console.log("Inserting vectors into pinecone");
  const request = {
    vectors,
  };
  await pineconIndex.upsert({ upsertRequest: request });
  console.log("Inserted vectors into pinecone");

  return documents[0];
}

Is there any loss in not using the namespace? Like upload speed, processing or such...

Also, I'm getting:
Inserting vectors into pinecone PineconeArgumentError: The argument to upsert had type errors: argument must be array. at UpsertCommand.eval [as validator] (webpack-internal:///(rsc)/./node_modules/@pinecone-database/pinecone/dist/validator.js:196:19) at UpsertCommand.eval (webpack-internal:///(rsc)/./node_modules/@pinecone-database/pinecone/dist/data/upsert.js:144:30) at step (webpack-internal:///(rsc)/./node_modules/@pinecone-database/pinecone/dist/data/upsert.js:107:23) at Object.eval [as next] (webpack-internal:///(rsc)/./node_modules/@pinecone-database/pinecone/dist/data/upsert.js:48:20) at eval (webpack-internal:///(rsc)/./node_modules/@pinecone-database/pinecone/dist/data/upsert.js:26:71) at new Promise (<anonymous>) at __awaiter (webpack-internal:///(rsc)/./node_modules/@pinecone-database/pinecone/dist/data/upsert.js:8:12) at UpsertCommand.run (webpack-internal:///(rsc)/./node_modules/@pinecone-database/pinecone/dist/data/upsert.js:139:16) at Index.eval (webpack-internal:///(rsc)/./node_modules/@pinecone-database/pinecone/dist/data/index.js:386:49) at step (webpack-internal:///(rsc)/./node_modules/@pinecone-database/pinecone/dist/data/index.js:107:23) at Object.eval [as next] (webpack-internal:///(rsc)/./node_modules/@pinecone-database/pinecone/dist/data/index.js:48:20) at eval (webpack-internal:///(rsc)/./node_modules/@pinecone-database/pinecone/dist/data/index.js:26:71) at new Promise (<anonymous>) at __awaiter (webpack-internal:///(rsc)/./node_modules/@pinecone-database/pinecone/dist/data/index.js:8:12) at Index.upsert (webpack-internal:///(rsc)/./node_modules/@pinecone-database/pinecone/dist/data/index.js:380:16) at loadS3IntoPinecone (webpack-internal:///(rsc)/./src/lib/pinecone.ts:40:24) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async POST (webpack-internal:///(rsc)/./src/app/api/create-chat/route.ts:31:9) at async eval (webpack-internal:///(rsc)/./node_modules/next/dist/server/future/route-modules/app-route/module.js:254:37) { cause: undefined }

Any ideas?

[CodeOfMugiwara]

@kalilfagundes the solution to that error is in another issue #32

[rohitkf]

Just fix your regex using below code and you will not face anu issue

export function convertToAscii(inputString: string) {
  // remove non ascii characters
  return inputString.replace(/[^\x00-\x7F]/g, '').replace(/\s+/g, '_')
}