CartoDB/CartoDB-SQL-API

Timeouts not working for exports

Opened this issue · 2 comments

At lest the tests do not pass when I run against pgbouncer.

Steps to reproduce

  1. Run pgbouncer listening on port 6432.
  2. Modify the config/environments/test.js to set module.exports.db_port = '6432';
  3. Apply the patch in #520 because otherwise the tests die in the setup phase (explained there).
  4. Execute the tests like this:
$ make test 2>&1 | tee /tmp/test.out

cause at some point the diffs display the bodies containing SHP and other binary formats, which screw my terminal output, so that way I could inspect the /tmp/test.out file.

Expected result

All the tests pass.

Actual result

Here's an excerpt of the results I got:

  415 passing (38s)
  13 pending
  9 failing

  1) timeout export database Geopackage export exceeding statement timeout responds 429 Over Limits:

      Uncaught AssertionError: Invalid response status code.
     Expected: 429
     Got: 200
     Body: SQLite format [...]

  2) timeout export database KML export exceeding statement timeout responds 429 Over Limits:

      Uncaught AssertionError: Invalid response status code.
     Expected: 429
     Got: 200
     Body: <?xml version="1.0" encoding="utf-8" ?> [...]

  3) timeout export database Shapefile export exceeding statement timeout responds 429 Over Limits:

      Uncaught AssertionError: Invalid response status code.
     Expected: 429
     Got: 200
     Body: [...]

(all the failures are related to exports)

(assigning to @dgaubert , for you to prioritize and dispatch at your leisure)

Hey!

That's because PGBoucer has its internal pool of connection against Postgres, before executing those tests we need to set a proper timeout for that user/database. it uses one connection of PGBouncer pool but rest of them keep the connection with the previous settings and the timeout will not be applied.

In fact, we had to clean up connections from the internal cartodb-psql pool.