Qianlitp/crawlergo

Issues

• [BUG] 使用 simple 模式过滤时出现不正常任务

  #111 opened 2 years ago by PIGfaces
  4

• Duplicate Requests

  #144 opened 2 years ago by redkit75
  2

• 输出的json不合法

  #124 opened 2 years ago by ox1234
  6

• 使用--push-to-proxy无法获取数据包

  #156 opened 9 months ago by dkive
  0

• 设置了post请求的post-data，实际请求并没有发以post-data为body的post请求

  #155 opened 9 months ago by dimsoul
  0

• Any suggestions on authenticated crawling?

  #126 opened 2 years ago by rohitcoder
  1

• 代理存在账号密码验证的时候会请求超时

  #148 opened 10 months ago by 5Laos
  1

• 是否可以添加参数类似`--url-keywords`，只爬取包含特定关键字的url

  #153 opened a year ago by tdragon6
  1

• Request: Change default to not submit forms to prevent accidental spam

  #154 opened a year ago by MarkRosemaker
  2

• crawlergo陷入假死状态发现大量Chromium僵尸进程

  #149 opened 10 months ago by evilc0deooo
  3

• --ignore-url-keywords参数不管用和需求fuzz payload

  #152 opened a year ago by Tian866
  1

• 导航超时 navigate timeout

  #132 opened a year ago by byposeidon
  2

• --robots-path 参数问题

  #146 opened a year ago by Cl0udG0d
  3

• Crawlergo not working

  #134 opened a year ago by byposeidon
  3

• Navigate timeout

  #135 opened a year ago by redkit75
  1

• OpenAPI Support

  #138 opened 2 years ago by redkit75
  0

• 增加sitemap.xml(网站地图)解析功能，其读取主要和robots.txt差不多

  #143 opened 2 years ago by sairson
  1

• 只有get请求

  #142 opened a year ago by c0d1007
  0

• may i know what tool is this?

  #140 opened a year ago by seczoid
  0

• 功能建议

  #141 opened 2 years ago by pwnhxl
  0

• may i know about this?

  #139 opened 2 years ago by seczoid
  0

• [Bug] 不断请求阻塞请求导致导航页超时

  #105 opened 3 years ago by PIGfaces
  2

• 请删除

  #127 opened 2 years ago by byposeidon
  0

• 表格自动填写

  #131 opened 2 years ago by byposeidon
  0

• Form auto-fill not working

  #128 opened 2 years ago by byposeidon
  1

• crawlergo not working

  #129 opened 2 years ago by byposeidon
  1

• Show status code for each page

  #118 opened 2 years ago by undergroundwires
  0

• Do not write to stderr

  #117 opened 2 years ago by undergroundwires
  0

• Mac系统报错

  #125 opened 2 years ago by li-baige
  1

• 多级目录爬取不到，只能爬取到二级目录，希望可以增加一下功能

  #122 opened 2 years ago by Kepler587
  3

• [Code Review] js 回调 golang 的方法

  #119 opened 2 years ago by PIGfaces
  0

• Output Request Response to ".txt" File

  #97 opened 3 years ago by gromhacks
  2

• readme需更新

  #114 opened 2 years ago by gjm-anban
  1

• [code review] 阻断静态资源代码块

  #106 opened 2 years ago by PIGfaces
  1

• 【新需求】--fuzz-path-dict 参数增加多级目录支持

  #108 opened 2 years ago by soapffz
  2

• [Code Review] 函数中的无效参数

  #113 opened 2 years ago by PIGfaces
  0

• 无法爬取动态渲染的表单

  #99 opened 2 years ago by tyraniter
  1

• 无法爬取vue项目的地址

  #109 opened 2 years ago by gjm-anban
  2

• vue页面无法爬取

  #110 opened 2 years ago by gjm-anban
  0

• [code review] 计算 url path 的唯一值的策略不解的代码段

  #100 opened 2 years ago by PIGfaces
  1

• http://dvwa.bihuo.cn/login.php没有爬到POST login.php

  #104 opened 3 years ago by chushuai
  2

• push to proxy not working

  #103 opened 3 years ago by melmel27
  8

• Add option to not crawl outside of root domain

  #102 opened 3 years ago by arnoldasr
  2

• Cookie

  #90 opened 3 years ago by Mount4in
  2

• 请问为什么crawlergo在shell下能爬出来的内容在python脚本调用里就爬不出来了

  #88 opened 3 years ago by zongdeiqianxing
  3

• 非无头模式下，能不能添加人工操作的功能？

  #95 opened 3 years ago by chushuai
  1

• Cookie

  #96 opened 3 years ago by gosky666
  1

• upload

  #92 opened 3 years ago by Mount4in
  0

• using one protocol

  #91 opened 3 years ago by MindPatch
  1

• crawlergo don't work with some sites like twitter

  #89 opened 3 years ago by am0o0
  6