ankane/ruby-polars

Read Database separate from the ActiveRecord default option

scart88 opened this issue · 3 comments

Hey @ankane,

Is there a way to connect and use a database different from an ActiveRecord?

I want to connect to the QuestDB directly from Polars. I can connect with the PG gem since they are Postgres compatible.
However, I must add a .to_a method to convert the response from the pg gem into an array so I can create the DataFrame.

Example:

  @req = @quest_db.exec(query)
  data = @req.to_a
  data = Polars::DataFrame.new(data)

Can you give me some hints on what I could do here?

Thank you!

Hey @scart88, that approach should be fine. What issue are you seeing?

The example I posted works fine; I'm using it in production on a small application.

Is there a way to do this without converting the response into an array?

Converting the pg gem response into an array makes the app slower since the array is created in memory.

When there is a lot of data, it hangs for several seconds.
When there are only several rows (1000 rows), it's almost instant.

I'm using Polars and Chartkick to build this histogram chart with data from QuestDB.
histogram

Thank you!

I just found another way, and it works just fine. :)

    @qdb = ActiveRecord::Base.establish_connection("postgres://#{qdb_url}")
    @req = @qdb.connection.select_all("select count from events;")
      (90.6ms)  select count from events;
    => 
    #<ActiveRecord::Result:0x000000010ce76c38
    
    polars = Polars.read_database(@req)
    => 
    shape: (1, 1)

Thank you again!