Problems with -logs option

[oskar456]

When used with -logs option to update the list of known and trusted CT logs using the log_list.json downloaded from Known Logs list, there are few issues:

• even disqualified logs (those with disqualified_at property) are scanned
• some logs like mammoth.ct.comodo.com return 404 errors due to double slash in the URL: https://mammoth.ct.comodo.com//ct/v1/get-sth
• when a new log is added, it's scanned for entries for all entries which takes a very long time

To workaround first two issues, I've created a small Python script. To workaround the third issue, it's necessary to delete the state files so "first run" is forced.

[AGWA]

even disqualified logs (those with disqualified_at property) are scanned

Generally you still want to monitor disqualified (now called retired) logs, because they can still be used to satisfy browser CT policy. If this is not what you want, you can remove them from the log list yourself.

some logs like mammoth.ct.comodo.com return 404 errors due to double slash in the URL: https://mammoth.ct.comodo.com//ct/v1/get-sth

Fixed in 185445e.

when a new log is added, it's scanned for entries for all entries which takes a very long time

If you don't want this, you can now specify -start_at_end.