Blackweb es un proyecto que pretende recopilar la mayor cantidad de listas negras públicas de dominios (para bloquear porno, descargas, drogas, malware, spyware, trackers, bots, redes sociales, warez, venta de armas, etc), con el objeto de unificarlas y hacerlas compatibles con Squid-Cache (Tested in v3.5.x). Para lograrlo, realizamos una depuración de urls, para evitar duplicados, dominios inválidos (validación de ccTLD, ccSLD, sTLD, uTLD, gSLD, gTLD, etc), y un filtrado con listas blancas de dominios (falsos positivos, como google, hotmail, yahoo, etc), para obtener una mega ACL, optimizada para Squid-Cache, libre de "overlapping domains" (e.g: "ERROR: '.sub.example.com' is a subdomain of '.example.com'").
Blackweb is a project that aims to collect as many public domain blacklists (to block porn, downloads, drugs, malware, spyware, trackers, Bots, social networks, warez, arms sales, etc.), in order to unify them and make them compatible with Squid-Cache (Tested in v3.5.x ). To do this, we perform a debugging of urls, to avoid duplicates, invalid domains (validation, ccTLD, ccSLD, sTLD, uTLD, gSLD, gTLD, etc), and filter with white lists of domains (false positives such as google , hotmail, yahoo, etc.), to get a mega ACL, optimized for Squid-Cache, free of overlapping domains (eg: "ERROR: '.sub.example.com' is a subdomain of '.example.com'").
| ACL | Black Domains | txt size | tar.gz size |
|---|---|---|---|
| blackweb.txt | 2.405.211 | 56,6 MB | 12,1 MB |
git squid bash tar zip wget subversion python
git clone --depth=1 https://github.com/maravento/blackweb.git
La ACL blackweb.txt ya viene optimizada para Squid-Cache. Descárguela y descomprimala en la ruta de su preferencia:
The ACL blackweb.txt is already optimized for Squid-Cache. Download it and unzip it in the path of your preference:
wget -q -N https://github.com/maravento/blackweb/raw/master/blackweb.tar.gz && cat blackweb.tar.gz* | tar xzf -
wget -q -N https://github.com/maravento/blackweb/raw/master/blackweb.md5
md5sum blackweb.txt | awk '{print $1}' && cat blackweb.md5 | awk '{print $1}'
El script bwupdate.sh actualiza la ACL blackweb.txt, realizando la captura, depuración y limpieza de dominios, sin embargo puede generar conflíctos por errores en las FUENTES, por tanto deberá depurarlos manualmente. Tenga en cuenta que este script consume gran cantidad de recursos de hardware durante el procesamiento y puede tomar mucho tiempo.
The bwupdate.sh script updates blackweb.txt ACL, doing the capture, debugging and cleaning of domains, however it can generate conflicts for errors in the SOURCES, therefore you must manually debug conflicts. Keep in mind that this script consumes a lot of hardware resources during processing and it can take a long time.
wget -q -N https://github.com/maravento/blackweb/raw/master/bwupdate/bwupdate.sh && sudo chmod +x bwupdate.sh && sudo ./bwupdate.sh
path: /var/log/syslog
Blackweb for Squid: Done 06/05/2017 15:47:14
Y verifique el contenido del archivo SquidError.txt en su escritorio para corregirlos / And check the contents of the SquidError.txt file on your desktop to fix them
- Antes de utilizar bwupdate.sh debe activar la regla en Squid-Cache / You must activate the rule in Squid-Cache before using bwupdate.sh
- bwupdate.sh debe ejecutarse en equipos de pruebas. Nunca en servidores en producción. / bwupdate.sh must run on test equipment. Never on servers in production.
- bwupdate.sh no incluye sitios cloud (Mega, Dropbox, Pcloud, iCloud, etc) o de soporte remoto (Teamviewer, Anydesk, logmein, etc), excepto si ya vienen bloqueados desde las FUENTES. Para bloquearlos o excluirlos debe activar la línea según su elección: / bwupdate.sh does not include cloud sites (Mega, Dropbox, Pcloud, iCloud, etc) or remote support (Teamviewer, Anydesk, logmein, etc), except if they are already blocked from the SOURCES. To block or exclude them you must activate the line according to your choice:
# unblock
#sed '/^$/d; /#/d' {cloudsync,remoteurls}.txt | sort -u >> urls.txt
# block
#sed '/^$/d; /#/d' {cloudsync,remoteurls}.txt | sort -u >> bwtmp/bw.txt
REGLA Squid-Cache / Squid-Cache RULE
Edite / Edit:
/etc/squid/squid.conf
Y agregue las siguientes líneas: / And add the following lines:
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
acl blackweb dstdomain -i "/path_to_acl/blackweb.txt"
http_access deny blackweb
Blackweb contiene millones de dominios bloqueados, por tanto: / Blackweb contains millions of blocked domains, so:
- Utilice la ACL whitedomains para excluir dominios falsos positivos (y repórtelo) u otros dominios que quiera excluir (ejemplo: accounts.youtube.com desde Feb 2014, Google utiliza el subdominio accounts.youtube.com para autenticar sus servicios) / Use the ACL whitedomains to exclude false-positive (and report it) domains or other domains that you want to exclude (e.g.: accounts.youtube.com since Feb 2014, Google uses the subdomain accounts.youtube.com to authenticate its services)
- Utilice la ACL blackdomains para agregar dominios no incluidos en Blackweb (ejemplo: .youtube.com .googlevideo.com, .ytimg.com) / Use the ACL blackdomains to add domains not included in Blackweb (e.g.: .youtube.com .googlevideo.com, .ytimg.com)
acl whitedomains dstdomain -i "/path_to_acl/whitedomains.txt"
acl blackdomains dstdomain -i "/path_to_acl/blackdomains.txt"
acl blackweb dstdomain -i "/path_to_acl/blackweb.txt"
http_access allow whitedomains
http_access deny blackdomains
http_access deny blackweb
adblockplus malwaredomains_full
ABPindo indonesianadblockrules
Capitole - Direction du Système d'Information (DSI)
chadmayfield (included: porn_all, porn top)
CHEF-KOCH BarbBlock-filter-list
Disconnect.me (included: simple_ad, simple_malvertising, simple_tracking)
dshield.org (included: Low, Medium, High)
firebog.net (included: AdguardDNS, Airelle-hrsk, Airelle-trc, BillStearns, Easylist, Easyprivacy, Kowabit, Prigent-Ads, Prigent-Malware, Prigent-Phishing, Shalla-mal, WaLLy3K)
hosts-file.net (included: ad_servers, emd, grm, hosts, psh)
Matomo-org referrer-spam-blacklist
mitchellkrogza (included: Badd-Boyz-Hosts, Hacked Malware Web Sites, Nginx Ultimate Bad Bot Blocker, The Big List of Hacked Malware Web Sites, Ultimate Hosts Blacklist)
Passwall SpamAssassin (Server Down since Dec 2016). Last Update
Perflyst (included: android-tracking, SmartTV)
quidsup (included: notrack-blocklists, notrack-malware, trackers, qmalware)
Ransomware Abuse (included: CryptoWall, Locky, Domain Blocklist, Ransomware Abuse ,URL Blocklist ,TorrentLocker)
securemecca.net and hostsfile.org
squidblacklist.org (included: dg-ads, dg-malicious.acl)
StevenBlack (included: add.2o7Net, add.Risk, fakenews-gambling-porn-social, hosts, spam, uncheckyAds)
zerodot1 CoinBlockerLists (included: Host, host_browser, host_optional, list, list_browser, list_browser_UBO)
UrlBlacklist (Server Down since July 24, 2017)
ipv6-hosts (Partial)
O365IPAddresses (Partial)
University Domains and Names Data List
Agradecemos a todos aquellos que han contribuido a este proyecto. Los interesados pueden contribuir, enviándonos enlaces de nuevas listas, para ser incluidas en este proyecto / We thank all those who have contributed to this project. Those interested can contribute, sending us links of new lists, to be included in this project
BTC: 3M84UKpz8AwwPADiYGQjT9spPKCvbqm4Bc
© 2018 Maravento Studio
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.