elixir-crawly/crawly

Issues

• mix project crawly prevent `iex -S mix` to give me the interactive console

  #298 opened 6 months ago by xward
  8

• Updating ex_doc to the latest version

  #302 opened 6 months ago by malinowskip
  1

• failed to run quickstart: Could not classify module <xxxxxxx> as spider

  #301 opened 7 months ago by shuiRong
  1

• Set base_url in init options instead of callback

  #296 opened 9 months ago by adonig
  1

• Running many instances of one spider

  #255 opened 2 years ago by serpent213
  3

• Make the management tool opt-in by default

  #262 opened 10 months ago by aej
  5

• Error: Could not load spiders.

  #271 opened 10 months ago by 3even
  5

• `Crawly.Fetchers.Fetcher` implementation for Playwright

  #246 opened 10 months ago by Nezteb
  4

• Protocol error

  #238 opened 10 months ago by assertnotnull
  7

• robots.txt matching is pretty buggy

  #254 opened 10 months ago by serpent213
  10

• [error] Pipeline crash by call: Crawly.Middlewares.UniqueRequest.run

  #274 opened a year ago by PotentSoftware
  0

• Encountering Complications with Forwarding to Crawly.API.Router

  #275 opened 10 months ago by oallanmendes
  1

• Upgrade to `httpoison` 2.x?

  #276 opened 10 months ago by samrat
  3

• management Web UI on localhost:4001 is not working

  #278 opened 10 months ago by cmnstmntmn
  4

• Does Crawly support requests using the POST method?

  #281 opened 10 months ago by oallanmendes
  1

• CI issue: Failed to upload the report to 'https://coveralls.io', Couldn't find a repository matching this job.

  #288 opened 10 months ago by OldhamMade
  1

• Advice wanted, usage with oban

  #179 opened 3 years ago by bapti
  6

• Crawly compilation warnings, undefined Floki functions

  #283 opened a year ago by dogweather
  1

• Q: Can the spider "fan out" on a website? (multiple next items)

  #263 opened 2 years ago by larshei
  1

• Improve documentation by linking from source docs to guides

  #174 opened 4 years ago by Ziinc
  1

• `Could not start application crawly` when trying to enter IEx

  #187 opened 2 years ago by thiagomajesk
  2

• Filter out requests when popping from request storage

  #146 opened 2 years ago by tanguilp
  2

• Provide more examples / detailed docs on custom request header override per spider

  #176 opened 4 years ago by DaniruKun
  3

• Setting up a parametric spider (dynamic base_url and start_urls)

  #207 opened 2 years ago by matteoredaelli
  1

• My Spider's code is never invoked, weird behavior with `Crawly.RequestsStorage.pop` in library code

  #224 opened 2 years ago by azrosen92
  5

• This is actually a question, Nested scraping

  #229 opened 2 years ago by shellking4
  2

• Genserver time out crash in long-running pipeline

  #230 opened 2 years ago by starcraft66
  1

• Stop and resume the spider where it stopped

  #232 opened 2 years ago by shellking4
  2

• jl files not found probably not writing

  #212 opened 2 years ago by shellking4
  1

• Any working examples?

  #210 opened 2 years ago by dogweather
  4

• Typespecs for crawly splash fetcher options are wrong

  #182 opened 2 years ago by Ziinc
  0

• Generic `Pipeline crash` error

  #185 opened 2 years ago by thiagomajesk
  4

• Suggestion for nameing of closespider_timeout in config

  #183 opened 2 years ago by EdmondFrank
  2

• custom parsar callback sample

  #189 opened 2 years ago by ziyouchutuwenwu
  7

• Move TestSpider definition from manager_test module

  #166 opened 2 years ago by oltarasenko
  0

• Remove URL fragment before storing in `Crawly.Middlewares.UniqueRequest`

  #144 opened 2 years ago by tanguilp
  3

• Configurable SendToUI Pipeline

  #152 opened 2 years ago by FrancescoZ
  4

• [question] scrapping dynamic urls

  #186 opened 2 years ago by mario-mazo
  1

• Demo page not loading

  #198 opened 2 years ago by braynm
  3

• Use a more reliable website to crawl in tutorial

  #209 opened 2 years ago by james-s-w-clark
  1

• Crawly.fetch giving 301 response instead of 200

  #214 opened 2 years ago by shellking4
  1

• Could not compile dependency :epipe

  #190 opened 3 years ago by avillen
  5

• curl fetcher

  #181 opened 3 years ago by dkuku
  1

• Observed crash of Genserver

  #184 opened 4 years ago by bapti
  1

• js render problem

  #188 opened 4 years ago by ziyouchutuwenwu
  6

• Improvements to file logging

  #163 opened 4 years ago by Ziinc
  1

• any way to skip certificate validation?

  #160 opened 4 years ago by dkuku
  1

• Question: Chat for crawly users

  #147 opened 4 years ago by oltarasenko
  1

• How to wait for running spiders to finish?

  #153 opened 4 years ago by phortx
  10

• Feature request: make request storage configurable and pluggable

  #145 opened 4 years ago by tanguilp
  1