ScotterC/pinecone

Support more than one pinecone index

Sergezdv opened this issue · 5 comments

Sergezdv commented

it would be nice to be able to use multiple indexes like

index_1 = Pinecone::Client.new.index('index-1')
index_2 = Pinecone::Client.new.index('index-2')
ScotterC commented

Agreed. I'm pretty sure the problem is with setting HTTParty's base_uri in Vector.rb.

For testing, it should be easy to modify the index start/stop script to setup two example indexes.

I'd be curious to see anyone's approach to this.

pyrabbit commented

@ScotterC I can help with this... I dug into the catacombs of HTTParty issues and found this jnunemaker/httparty#167 ... Should be a relatively quick fix

pyrabbit commented

@ScotterC I have added a draft MR but need a bit of guidance on where you want to take the testing. I cannot run the test-suite locally because I am using up my one and only free tier index with a project I am working on... should we think about mocking client calls?

ScotterC commented

@pyrabbit thanks for taking this on!
Relying on live indexes isn't a great contributor experience right now. My issue with stubbing the API is it really reduces the surface area to be tested and the client is fairly thin already. A compromise is needed. I'm going to ping the Pinecone team and see if I can get a dedicated testing account of some kind. In the meantime, have your PR mock the API and I'll test it against live indexes before merging.

I'm also open to other ideas here if you find something more effective.

  • 👍1
pyrabbit commented

@ScotterC Ok I made my first pass at it and #24 is ready for your review. Here are some key changes:

  • Pinecone::Client#index was @index ||= Pinecone::Vector.new(index_name) but changed to Pinecone::Vector.new(index_name). I don't see a reason to keep the instance variable at this time -- guessing it is just a lingering piece of code from early development. Removing that piece was critical because having multiple indices on one client requires the index method to return a new instance of Pinecone::Vector each time you invoke it.
  • Removed self.class.base_uri from vector.rb in lieu of an instance variable appropriately called base_uri. This is the preferred solution suggested by the HTTParty maintainer for scenarios where you need a more dynamic base URI.
  • 👍1