18F/site-scanning

Document the previous domain gathering process

Closed this issue · 2 comments

Make sure to write how pulse assembled the target URL list it would use. Write down here.

Notes to self:

  • Domain-scan uses this to operate the overall gathering process.

  • Pulse would specific which gatherers to run here.

  • And the location of the data sources to be gathered was here.

  • (In an earlier era, when censys and rdns data was pulled live instead of from a static snapshot, the code to do that was here.)

First pass written here.