edgi-govdata-archiving/web-monitoring-processing

Issues

• Tests should use VCR or a fixture server and avoid connecting to third-party services

  #236 opened 3 years ago by Mr0grog
  5

• Extract title from PDF content when importing

  #156 opened 4 years ago by Mr0grog
  4

• Radical suggestion: Deprecate this repo and break up pieces

  #206 opened 5 years ago by danielballan
  9

• Add resource limits for diffing server

  #154 opened 5 years ago by Mr0grog
  15

• Infer local file type from extension

  #231 opened 5 years ago by danielballan
  3

• Would we want a xml differ?

  #183 opened 5 years ago by lightandluck
  6

• Content is getting cut off in the HTML diff

  #259 opened 5 years ago by Mr0grog
  5

• html_token: Processing instruction nodes are converted to text

  #149 opened 5 years ago by Mr0grog
  3

• Add support for Sentry releases

  #194 opened 5 years ago by Mr0grog
  3

• Refactor on top of html5-parser

  #138 opened 5 years ago by danielballan
  4

• Choose better colors for highlighting changes

  #158 opened 6 years ago by Mr0grog
  6

• Create an analyzer that checks for simple, ignorable non-text changes

  #175 opened 7 years ago by Mr0grog
  4

• Package all the dependencies for conda.

  #220 opened 6 years ago by danielballan
  5

• Capture seed generation script

  #204 opened 6 years ago by danielballan
  4

• upload web-monitoring-processing into PyPI

  #168 opened 6 years ago by weatherpattern
  4

• Improve performance of other rendered differs (or precompute and cache)

  #139 opened 6 years ago by danielballan
  4

• Use third-party tools for backoff/retry of HTTP requests

  #261 opened 6 years ago by Mr0grog
  1

• Incorporate Suggestions from IA folks on CDX API methods/tests

  #202 opened 6 years ago by Mr0grog
  6

• Add health check endpoint to diff server

  #262 opened 6 years ago by Mr0grog
  2

• Add version endpoint to diff server

  #256 opened 6 years ago by Mr0grog
  2

• Create health check for Internet Archive data

  #125 opened 6 years ago by Mr0grog
  6

• HTML Differ breaks when there no body content can be found in the HTML

  #247 opened 6 years ago by Mr0grog
  1

• Fix issues with zombie processes on diffing server

  #207 opened 6 years ago by Mr0grog
  6

• Add contributors section to readme

  #235 opened 6 years ago by Mr0grog
  0

• NASA page appears to be causing great diffing server pain

  #169 opened 6 years ago by Mr0grog
  6

• Can/should we be more lenient about decoding errors?

  #203 opened 6 years ago by Mr0grog
  10

• Investigate bug with beautifulsoup4 v4.6.2

  #213 opened 6 years ago by danielballan
  3

• Scanner showing double content, rendering it as new

  #230 opened 6 years ago by gretchengehrke
  2

• Update to Circle 2.0.

  #221 opened 6 years ago by danielballan
  2

• Make tests runnable without credentials.

  #222 opened 6 years ago by danielballan
  2

• Do we need version information for dependencies?

  #195 opened 6 years ago by Mr0grog
  1

• Text of link diff can include non-visible content (e.g. the source of a <script>)

  #198 opened 6 years ago by Mr0grog
  0

• New links diff allows columns to get too wide/narrow

  #199 opened 6 years ago by Mr0grog
  0

• Discerning order of additions and removals of links next to each other is hard.

  #151 opened 6 years ago by weatherpattern
  1

• IA import scripts need option to include all snapshots

  #176 opened 7 years ago by Mr0grog
  4

• Upgrade CircleCI to v2

  #165 opened 7 years ago by Mr0grog
  2

• IA import scripts need rate limiting and retry/backoff semantics

  #177 opened 7 years ago by Mr0grog
  5

• Add issue template to processing repo

  #163 opened 7 years ago by weatherpattern
  1

• Change `site`/`agency` to `tags`/`maintainers`

  #166 opened 7 years ago by Mr0grog
  0

• html diffs: Should return early with a clear error if HTML does not parse

  #147 opened 7 years ago by Mr0grog
  2

• Diffing a version with no content results in an error

  #157 opened 7 years ago by Mr0grog
  2

• html_token: Stop using `lxml.html.diff` for tokenization

  #145 opened 7 years ago by Mr0grog
  1

• html_token: SVGs that have not changed look like they have and it’s a real problem

  #146 opened 7 years ago by Mr0grog
  1

• `html_token` differ returns different `change_count`s for same version

  #148 opened 7 years ago by lightandluck
  2

• “FERC Calendar of Events” page fails html_token diff (side-by-side only)

  #143 opened 7 years ago by Mr0grog
  2

• Implement 'change_count' for html_diff_render and links_diff

  #137 opened 7 years ago by danielballan
  2

• Reduce list-of-links issue (from CDC) to a clear test case

  #124 opened 7 years ago by Mr0grog
  6

• Add a `change_count` field in diff output indicating how many changes there were

  #130 opened 7 years ago by Mr0grog
  8

• Clean up differ names and responses

  #132 opened 7 years ago by danielballan
  0

• Ensure diffing server errors are JSON

  #121 opened 7 years ago by Mr0grog
  0