Projects/Adblocker

From x22889 Wiki
Revision as of 18:29, 13 January 2017 by Root.s (talk | contribs)
Jump to navigation Jump to search

Hardware

  • Virtual Machine on HP Proliant Gen8 with eSXI Hypervisor
  • 40GB HDD
  • 1x 2,3 GhZ CPU
  • 1GB RAM
  • 2x 1 GBit Network connection (only one being used currently)

Software

Operating system

  • Ubuntu 16.04.1 LTS

Services

  • Squid3 (Ad filtering)
  • Apache2 (only for WPAD file distribution)
  • DHCP-Server (isc-dhcp-server) to automatically distribute Proxy settings via DHCP attribute

System Configuration

  • Assign system a fixed IP address, DNS and Gateway

IP address and DNS server

sudo nano /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto ens160
iface ens160 inet static
address 192.168.178.13
netmask 255.255.255.0
network 192.168.178.0
gateway 192.168.178.1
dns-nameservers 192.168.178.1

Default gateway route

sudo ip route change default via 192.168.178.1 dev ens160

Restart networking to apply changes: 

sudo service networking restart

Update software

Run the following commands to install the latest software updates available for your system: 

sudo apt update
sudo apt -y upgrade
<source>
==Services configuration==
Install required packages:
<source>sudo apt install apache2 isc-dhcp-server squid squidguard

Apache

Create a new site for WPAD distribution 

sudo -u www-data nano /etc/apache2/sites-availabe/wpad.conf

And paste the following: 

# Auto Proxy Configuration
<VirtualHost *:80>
    ServerAdmin webmaster@example.com
    DocumentRoot /var/www/wpad
    ServerName wpad.webfilter.local
    ServerAlias wpad

    AddType application/x-ns-proxy-autoconfig  .dat
    AddType application/x-ns-proxy-autoconfig  .pac
</VirtualHost>

Create 3 files for the automatic proxy server distribution via DHCP: 

sudo -u www-data nano /var/www/wpad/XXXXXXXXX.XXX

Replace "XXXXXXXXX.XXX" with the following filenames:

proxy.dat 

function FindProxyForURL(url,host) {
    if(isInNet(host,"127.0.0.1","255.0.0.0"))
        return "DIRECT";

    if(isPlainHostName(host))
        return "DIRECT";

    if(isInNet(host,"192.168.178.0","255.255.255.0"))
        return "PROXY 192.168.178.13:8080; DIRECT";

    else
        return "PROXY 192.168.178.13:8080; DIRECT";
}

proxy.pac 

function FindProxyForURL(url,host) {
    if(isInNet(host,"127.0.0.1","255.0.0.0"))
        return "DIRECT";

    if(isPlainHostName(host))
        return "DIRECT";

    if(isInNet(host,"192.168.178.0","255.255.255.0"))
        return "PROXY 192.168.178.13:8080; DIRECT";

    else
        return "PROXY 192.168.178.13:8080; DIRECT";
}

wpad.dat 

function FindProxyForURL(url,host) {
    if(isInNet(host,"127.0.0.1","255.0.0.0"))
        return "DIRECT";

    if(isPlainHostName(host))
        return "DIRECT";

    if(isInNet(host,"192.168.178.0","255.255.255.0"))
        return "PROXY 192.168.178.13:8080; DIRECT";

    else
        return "PROXY 192.168.178.13:8080; DIRECT";
}

Enable the new Apache2 site by running 

sudo a2ensite wpad

Reload Apache to apply the changes 

sudo service apache2 reload

DHCPd

Edit /etc/dhcp/dhcpd.conf: 

 sudo nano /etc/dhcp/dhcpd.conf

Adjust as follows: 

#
# Sample configuration file for ISC dhcpd for Debian
#
# Attention: If /etc/ltsp/dhcpd.conf exists, that will be used as
# configuration file instead of this file.
#
#

# The ddns-updates-style parameter controls whether or not the server will
# attempt to do a DNS update when a lease is confirmed. We default to the
# behavior of the version 2 packages ('none', since DHCP v2 didn't
# have support for DDNS.)
ddns-update-style none;

# option definitions common to all supported networks...
option domain-name "homenet.local";
option domain-name-servers 192.168.178.1;

default-lease-time 600;
max-lease-time 7200;

# If this DHCP server is the official DHCP server for the local
# network, the authoritative directive should be uncommented.
#authoritative;

# Use this to send dhcp log messages to a different log file (you also
# have to hack syslog.conf to complete the redirection).
log-facility local7;

# No service will be given on this subnet, but declaring it helps the
# DHCP server to understand the network topology.

#subnet 10.152.187.0 netmask 255.255.255.0 {
#}

# This is a very basic subnet declaration.

  option local-wpad code 252 = text;

subnet 192.168.178.0 netmask 255.255.255.0 {
  range 192.168.178.20 192.168.178.40;
  option routers 192.168.178.1;
  option local-wpad "http://192.168.178.13/proxy.pac";
}

Restart DHCPd to apply the changes: 

sudo service isc-dhcp-server restart

You can use the tool DHCP Test by Vladimir Panteleev to verify your configuration was applied and works. The output should look something like this: 

dhcptest v0.5 - Written by Vladimir Panteleev
https://github.com/CyberShadow/dhcptest
Run with --help for a list of command-line options.

Listening for DHCP replies on port 68.
Type "d" to broadcast a DHCP discover packet, or "help" for details.
d
Sending packet:
  op=BOOTREQUEST chaddr=20:6B:AD:EB:E2:85 hops=0 xid=26195EB3 secs=0 flags=8000
  ciaddr=0.0.0.0 yiaddr=0.0.0.0 siaddr=0.0.0.0 giaddr=0.0.0.0 sname= file=
  1 options:
     53 (DHCP Message Type): discover
Received packet from 192.168.178.13:67:
  op=BOOTREPLY chaddr=20:6B:AD:EB:E2:85 hops=0 xid=26195EB3 secs=0 flags=8000 ciaddr=0.0.0.0 yiaddr=192.168.178.40 siaddr=192.168.178.13 giaddr=0.0.0.0 sname= file=
  8 options:
     53 (DHCP Message Type): offer
     54 (Server Identifier): 192.168.178.13
     51 (IP Address Lease Time): 600 (10 minutes)
      1 (Subnet Mask): 255.255.255.0
      3 (Router Option): 192.168.178.1
      6 (Domain Name Server Option): 192.168.178.1
    252 (Unknown): "http://192.168.178.13/proxy.pac" (68 74 74 70 3A 2F 2F 31 39 32 2E 31 36 38 2E 31 37 38 2E 31 33 2F 70 72 6F 78 79 2E 70 61 63)
     15 (Domain Name): homenet.local

If the DHCP is working as intended it should return an IP address from the pool we specified earlier and also should include the proxy.pac configuration (option 252).

Squid

Edit squid.conf 

sudo nano /etc/squid/squid.conf
#acl lists
acl manager proto cache_object
acl localhost src 127.0.0.1/32 ::1
#acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
acl localnet src 192.168.178.0/24 # RFC1918 possible internal network
acl ads dstdom_regex "/etc/squid/ad_block.txt"
http_access deny ads

#port connections
acl SSL_ports port 443
acl SSL method CONNECT
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT

#allow/deny
http_access allow manager localhost
http_access deny manager
http_access allow localnet

# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports

# Example rule allowing access from your local networks.
http_access allow localnet
http_access allow localhost

# And finally deny all other access to this proxy
http_access deny all

#bind address default port is 3128
http_port 192.168.178.13:8080

#cache directory
cache_dir ufs /squid-cache/ 512 16 128
#cache_mem 2048MB
#coredump_dir /home/serveruser/squidcache/

#log
cache_store_log /var/log/squid/store.log

#rewrite program squidGuard
#url_rewrite_program /usr/bin/squidGuard -c /etc/squidguard/squidGuard.conf
#url_rewrite_children 5 #threads
#url_rewrite_concurrency 0 #jobs per threads

SquidGuard

Edit /etc/squidguard/squidguard.conf 

sudo nano /etc/squidguard/squidguard.conf
#
# CONFIG FILE FOR SQUIDGUARD
#
# Caution: do NOT use comments inside { }
#

dbhome /var/lib/squidguard/db
logdir /var/log/squidguard

#
# TIME RULES:
# abbrev for weekdays:
# s = sun, m = mon, t =tue, w = wed, h = thu, f = fri, a = sat

time workhours {
        weekly mtwhf 08:00 - 16:30
        date *-*-01  08:00 - 16:30
}

#
# SOURCE ADDRESSES:
#

src admin {
        ip              1.2.3.4  1.2.3.5
        user            root foo bar
        within          workhours
}

src foo-clients {
        ip              172.16.2.32-172.16.2.100 172.16.2.100 172.16.2.200
}

src bar-clients {
        ip              172.16.4.0/26
}

#
# DESTINATION CLASSES:
#
# [see also in file dest-snippet.txt]

dest good {
}

dest local {
}

dest porn {
}

#dest adult {
#       domainlist      BL/adult/domains
#       urllist         BL/adult/urls
#       expressionlist  BL/adult/expressions
#       redirect http://admin.foo.bar.de/cgi-bin/blocked.cgi?clientaddr=%a&clientname=%n&clientuser=%i&clientgroup=%s&targetgroup=%t&url=%u
#}

#
# ACL RULES:
#

acl {
        admin {
                pass     any
        }

        foo-clients within workhours {
                pass     good !in-addr !porn any
        } else {
                pass any
        }

        bar-clients {
                pass    local none
        }

        default {
                pass     local none
                redirect http://admin.foo.bar.de/cgi-bin/blocked.cgi?clientaddr=%a&clientname=%n&clientuser=%i&clientgroup=%s&targetgroup=%t&url=%u
        }
}

#create ads category
dest ads {
        #location of blacklists, domains, urls, expressions.
        domainlist blacklists/ads/domains
        urllist blacklists/ads/urls
        #expressionlist blacklists/ads/expressions
}
acl {
        default {
                #allow except 'ads'
                pass !ads all
                #redirect to transparent gif
                redirect http://localhost/blank.gif

        }
}

Keeping the lists up to date

sudo curl -sS -L --compressed "http://pgl.yoyo.org/adservers/serverlist.php?hostformat=nohtml&showintro=0&mimetype=plaintext" > ad_block.txt

Useful links

http://www.itbert.de/raspberry-pi-squid-block-ads/ https://wiki.gentoo.org/wiki/ProxyAutoConfig https://calomel.org/squid_adservers.html http://askubuntu.com/questions/140126/how-do-i-install-and-configure-a-dhcp-server http://wiki.squid-cache.org/SquidFaq/ConfiguringBrowsers#Automatic_WPAD_with_DNS http://serverfault.com/questions/633108/automatic-proxy-discovery-wpad-example-com-doesnt-work https://en.wikipedia.org/wiki/Web_Proxy_Autodiscovery_Protocol https://codepoets.co.uk/2014/squid-3-4-x-with-ssl-for-debian-wheezy/ http://ubuntuserverguide.com/2012/06/how-to-setup-squid3-as-transparent-proxy-on-ubuntu-server-12-04.html http://pgl.yoyo.org/adservers/

Retrieved from "https://wiki.x22889.de/wiki/index.php?title=Projects/Adblocker&oldid=10"