ricardochimal/taps

Incredible slow to import a 5mb database from heroku?

Opened this issue · 2 comments

I published this question in Stackoverflow, but maybe is more appropriate here

http://stackoverflow.com/questions/10853085/taps-gem-is-slow

For example, this table took over 3 hours to complete (from heroku to local pg db)

responses:     100% |==========================================| Time: 03:23:16

This is the current model

1.9.3p125 :001 > Response
 => Response(id: integer, generated_id: integer, response: string, parameters: string, created_at: datetime, updated_at: datetime) 

And the current count of objects.

1.9.3p125 :002 > Response.all.size
  Response Load (2908.1ms)  SELECT "responses".* FROM "responses" 
  EXPLAIN (34.5ms)  EXPLAIN SELECT "responses".* FROM "responses" 
EXPLAIN for: SELECT "responses".* FROM "responses" 
                            QUERY PLAN
------------------------------------------------------------------
 Seq Scan on responses  (cost=0.00..5622.13 rows=284213 width=47)
(1 row)

 => 284213

There's nothing fancy here yet it took 3 hours to process 250k objects saved.

Is this what it usually takes? If this is the case this gem looks totally inviable for production with not-so-much data stored. In the case of a real problem that would require i.e. restoring a production db based on a local copy it can take literally days over - let's say - a 1GB database.

I'm not sure if I'm missing something obvious here.

will commented

heroku db:push and pull will soon be replaced with pg:push and pull, which won't use taps, so it wouldn't have this problem. It'll have all of its own, new problems! If you want a preview check out heroku/heroku-pg-extras#42