ArchiveTeam/NewsGrabber

Issues

• youtube-dl issue

  #96 opened 6 years ago by Tsuser1
  0

• Warrior UI restarts in a loop

  #95 opened 6 years ago by s4y
  0

• Add some polish websites

  #93 opened 6 years ago by raspher
  0

• Handling of paywalled sites

  #91 opened 7 years ago by klslbmmytw
  0

• Is this python27 or python3.4+?

  #80 opened 8 years ago by luckcolors
  2

• last_upload_* files are sometimes not saved well during a bad event like a crash

  #42 opened 8 years ago by Arkiver2
  2

• New News-Sites suggestions

  #75 opened 8 years ago by Medowar
  0

• Skip site if its having technical issues

  #51 opened 9 years ago by HarryC145
  0

• Adding newspapers

  #52 opened 9 years ago by HarryC145
  0

• Use Thread/Process pools

  #61 opened 9 years ago by PromyLOPh
  1

• Ubuntu 16.04 gs-venv

  #81 opened 8 years ago by HarryC145
  0

• harrycross.me doesn't work

  #88 opened 8 years ago by rudolphos
  0

• rsync concurrency limit

  #74 opened 8 years ago by HarryC145
  1

• Assign a name to a worker

  #66 opened 8 years ago by HarryC145
  1

• German-language newssites

  #6 opened 9 years ago by PromyLOPh
  2

• Australian news websites

  #72 opened 8 years ago by rwoodpecker
  2

• Use sitemap.xml

  #68 opened 8 years ago by BnA-Robin
  0

• Some URLs are regrabbed.

  #21 opened 9 years ago by Arkiver2
  1

• Start listening to new socket instead of old socket if new socket for IRC is created

  #60 opened 9 years ago by Arkiver2
  0

• Check if file exists before starting uploadprocess

  #59 opened 9 years ago by Arkiver2
  1

• Add standard list of regexes for videoURLs and liveURLs

  #45 opened 9 years ago by Arkiver2
  0

• The IRC bot should reply to a command in the channel the command is given

  #48 opened 9 years ago by Arkiver2
  0

• Rewrite bad http:/ and https:/ URLs to http:// and https:// URLs if found

  #53 opened 9 years ago by Arkiver2
  0

• Decode parts of found percent encoded URLs that shouldn't be encoded

  #54 opened 9 years ago by Arkiver2
  0

• Rejoin IRC channel if disconnected

  #57 opened 9 years ago by Arkiver2
  1

• Only allow operators to make some IRC bot commands

  #56 opened 9 years ago by Arkiver2
  0

• Support videos and liveblogs not indicated in the URL

  #13 opened 9 years ago by PressStartandSelect
  1

• Youtube-dl is sometimes not starting.

  #19 opened 9 years ago by Arkiver2
  2

• Support selenium for better video and application extraction

  #47 opened 9 years ago by Arkiver2
  1

• Do not crash on bad Python service files

  #40 opened 9 years ago by Arkiver2
  2

• The IRC bot should reply to both private chat and public channel if a user has given commands though private chat to the IRC bot

  #50 opened 9 years ago by Arkiver2
  0

• Prevent static page requisites from being regrabbed

  #44 opened 9 years ago by Arkiver2
  1

• Print a warning in the IRC bot channel if no `regex` or `urls` is given for a service file

  #49 opened 9 years ago by Arkiver2
  0

• Automatically percent encode seed URLs

  #38 opened 9 years ago by Arkiver2
  1

• Custom URL extraction for found URLs that are being grabbed

  #46 opened 9 years ago by Arkiver2
  0

• Completely remove services dir when starting the scripts

  #41 opened 9 years ago by Arkiver2
  1

• Add an option for special URL extraction rules for seed URLs of a service

  #43 opened 9 years ago by Arkiver2
  0

• Grab seed URL in which new URL is found and the domain URL

  #39 opened 9 years ago by Arkiver2
  0

• Write new URLs to 'list' file and memory after !stop command

  #23 opened 9 years ago by Arkiver2
  1

• Restart upload item count for WARCs if date of latest upload is not date of uploading items.

  #20 opened 9 years ago by Arkiver2
  1

• Implement !start to undo !stop.

  #22 opened 9 years ago by Arkiver2
  1

• URL from RSS sometimes end with one or multiple spaces

  #24 opened 9 years ago by Arkiver2
  1

• The Intercept

  #2 opened 9 years ago by joepie91
  1

• Remove service from old refresh list if refresh is changed.

  #7 opened 9 years ago by Arkiver2
  1

• Implement IRC bot for the terminal

  #5 opened 9 years ago by Arkiver2
  1

• Are we logging timestamps into the WARC file name?

  #3 opened 9 years ago by HarryC145
  1

• Split items on Internet Archive in 10 GB items

  #17 opened 9 years ago by Arkiver2
  1

• YouTube-DL has stopped working again

  #4 opened 9 years ago by HarryC145
  1

• Bot shouldn't crash if a badly named service file is committed

  #9 opened 9 years ago by phuzion
  1

• Support ignoreregex for each site

  #12 opened 9 years ago by PressStartandSelect
  0