commoncrawl/ia-web-commons

Web archiving utility library

JavaApache-2.0

Issues

WAT: Duplicated payload metadata values for "Actual-Content-Length" and "Trailing-Slop-Length"
#43 opened 14 days ago by sebastian-nagel
0
WAT extractor: do not add <meta itemprop="..." > from body as metadata
#40 opened 17 days ago by sebastian-nagel
0
WAT extractor: add attributes of the <html> element as metadata
#35 opened 17 days ago by sebastian-nagel
0
Percent-encoded ampersands (&) in URL query string canonicalized incorrectly
#31 opened a year ago by tfmorris
3
Removal of leading WWWnnn. in URL canonicalization is too aggressive
#29 opened a year ago by tfmorris
1
WAT extraction: handle duplicate HTTP response headers
#18 opened a month ago by cldellow
1
Clear up all dependabot alerts
#39 opened a month ago by wumpus
1
WAT extractor: Document title bug
#36 opened 2 months ago by robertwaksmunski
1
WET extractor: add identified natural language of text content
#22 opened 5 years ago by sebastian-nagel
6
WAT extractor: Overlong truncated HTTP request header line throws exception and loss of request record
#32 opened a year ago by sebastian-nagel
3
Non-ASCII/UTF-8 characters lost in WARC-Target-URI during WAT/WET extraction
#27 opened 2 years ago by sebastian-nagel
0
WET files may include binary content if HTTP Content-Type header erroneously indicates HTML
#26 opened 2 years ago by sebastian-nagel
1
Failed tests: testInterruptibility (org.archive.util.InterruptibleCharSequenceTest): exception not throw
#25 opened 4 years ago by cronopioelectronico
4
WAT generator: do not fail on missing WARC-Filename in warcinfo record
#23 opened 5 years ago by sebastian-nagel
0
WAT extractor: WARC-Date to indicate capture time
#21 opened 5 years ago by sebastian-nagel
1
WAT/WET generator performance improvements
#15 opened 5 years ago by sebastian-nagel
13
WAT: only unescape complete XML/HTML character entities
#19 opened 5 years ago by sebastian-nagel
2
WAT: unescape XML/HTML character entities
#14 opened 5 years ago by sebastian-nagel
1
Links in onClick property not captured in WAT 'Links' metadata
#8 opened 7 years ago by e271828-
6
[WAT] Add rel attribute to A@/href links
#10 opened 7 years ago by sebastian-nagel
1
[WET] Missing spaces in parsed content
#13 opened 7 years ago by pipldev
1
Java stack overflow while matching cssUrlPattern
#12 opened 8 years ago by sebastian-nagel
1
[WAT extraction] Empty HTTP header fields are filled with value from preceding field
#11 opened 8 years ago by sebastian-nagel
1
data-href not captured in WAT 'Links' metadata
#7 opened 8 years ago by e271828-
4
Complete HTML link extraction to cover all element attributes of type URI
#9 opened 8 years ago by sebastian-nagel
0
WaybackURLKeyMaker to keep non-utf8 percent encodings
#6 opened 8 years ago by sebastian-nagel
1
WEATGenerator hanging while matching cssUrlPattern
#2 opened 8 years ago by sebastian-nagel
2
URLParser fails if URL contains empty port
#5 opened 8 years ago by sebastian-nagel
0
Add encoding detection to WET text extraction
#4 opened 8 years ago by sebastian-nagel
1
Add attribute `property` of HTML meta elements
#3 opened 8 years ago by sebastian-nagel
0
StringIndexOutOfBoundsException during WAT/WET generation
#1 opened 8 years ago by sebastian-nagel
1