Support running incrementally.

[erikjohnston]

For large databases we don't want to do all the work at once, so we should somehow look at doing work in incremental batches