internetarchive/dweb-mirror

UI to manage downloading crawling and depth/breadth - {multiple subtasks}

Closed this issue · 6 comments

  • See TODO-UXLOCAL in code
  • On item, should indicate "downloaded all|details" rather than crawlinh
  • On item, drop down should say Downloaded rather than Download if done and "Dont download"
  • Note - movie can display Details without Downloading it.
  • Handle Search - more complex
  • Straw person UI - for Item
  • Text says "downloaded xx/yy
  • download xxMb
  • download all files yy Mb

This does once add check button that copies to crawl

  • Collection & Crawl
  • downloaded xx members yyMb
  • download tiles ~yyMb
  • download n items ~zzMb
    If <1000 then 3 10 30 100 300 all
    If > 1000 then 10 100 1000 10000 100000 all

Uses admin/crawl/add - see below

  • Write path is ....
  • mirrorHttp admin/crawl/setconfig/IDENTIFIER/LEVEL?q=
    • add rows OR replace with a new command
  • MirrorConfig.writeUserTaskLevel - extend to support search number

[ ] Crawl specifier

  • Add checkbox to UI
  • Calls admin/crawl/setconfig.... (see above)

] Adding one time to crawl -

  • mirrorHttp admin/crawl/add ? identifier=xxx query=xxx search=xxx .....
  • CrawlManager.add needs:
    • level: default to 'details'
    • rows: - then adds search: {rows, level}
  • edit CrawlConfig - refactor with NavButtons, and make configOptions depend on item
    • Item
      • Add files_details to downloaded info (on DM)
      • Select mark (maybe) depending on what done so far, or just skipping display
      • Add Mb on details and all
      • See bug meaning that "all" never completes as meta.xml failing #319
    • Search
      • ???

Basically done - any oddities can be posted as new issues