GSA/touchpoints

Support CSV download for large response sets

Closed this issue · 1 comments

Support download through the web UI for at least several 100K responses.

Rough justification for how to choose a max size to support: If the exported file is small enough to import directly into Google sheets (or Excel, maybe), then we should be able to export that file. Once the file is bigger than that, the user is going to need more sophisticated tools to deal with their data, so we can probably get away with asking them to use the API to retrieve their response data.

Google sheets limits:

Up to 10 million cells or 18,278 columns for spreadsheets imported from Microsoft Excel. The limits are the same for Excel and CSV imports.

Since our CSV export has 13 non-question columns, that means ~700K responses for a single-question survey and ~300K responses for a survey with the maximum number of questions (20).

Given current response volume, supporting 200-300k downloads would cover MOST clients.
If a form has more responses than that, the user can use the API to access their data.

I've been instructing users with more than 10,000 responses to use the API.