Developers Club geek daily blog

2 years, 8 months ago
It is no secret that in big offices subject of filtering of the Internet the quite actual. Software and hardware solutions cope with this task many. But now all those sites which we cut earlier, work under the HTTPS protocol, i.e. port 443. It is known that this protocol to trace, listen , etc., it is impossible. And any caching filtering proxy server, redirector , etc. filters only HTTP, i.e. port 80. How to cut VKontakte, Schoolmates, iphide.info and many other similar sites? How to block access to personal mail to the organizations if use of it is forbidden by orders in the organization? Yes, it is possible to filter to IP addresses, but they quite often change and on many resources some IP addresses. To block them at the level of firewall somehow not so orthodox solution, and not the absolutely convenient.

And here, quite recently, one companion has told me that he lifts at himself in office the caching proxy with filtering of HTTPS, it has interested me. And it lifted Squid 3.5.8. As it has become clear, in it is mute the organization of interception of the encoded HTTPS sessions (ssl_bump) is processed, instead of the modes of interception ("modes") are entered into use of action ("actions") which including can be used in ACL. The server-first mode at which connection with the target server is carried out in the beginning, and is created then the protected connection between the client and proxy, is available as action of "bump" now. The none mode at which the TCP tunnel without decoding of traffic is created, is available as action of "splice" now.

For ensuring backward compatibility action of "peek-and-splice" at which the solution on type of the connection created in the beginning (the client proxy or the proxy server) is accepted on the basis of SSL of hello-messages is added. Actions of "peek" and "stare" for obtaining the client or server certificate with saving of possibility of further application of the "splice" and "bump" modes for connections are added. Action of "terminate" for closing of connections is added to the client or the server. Here is how time of SSL BUMP, PEEK-and-SPLICE and Terminate are also necessary to us. In general, scheme of work the actually quite simple. Squid is connected to HTTPS to resource, receives its certificate, and can "look" at some data on resource, in particular server name which just and is necessary for us for blocking! All manuals which are on the Internet, continually describe Man in the middle (MITM) attack with substitution of certificates at which in principle some sites and bank clients do not work and users obviously see that them monitor. We with the companion through joint efforts have achieved sbosob of filtering and tracking of HTTPS without substitution of certificates, without MITM and other, and all this in the transparent mode without setup of browsers!

Afterwards I have faced some difficulties, in particular Squid started segfoltitsya on heavy load. But the problem has been solved. The matter is that in Openssl there are some bugs which are corrected in Libressl library. Therefore it is necessary to integrate at first into Libressl system, then after entering of patch into the bio.cc file in source codes of Squid to start compilation of the last. Have gone! I will make a reservation that I use distribution kit of Debian Jessie x86, and Squid I as a result have collected version 3.5.9 (the version last at the moment), and article is expected more or less experienced user of Linux as some moments fall, and only the most important is told as to me everything to chew laziness. So, if it is interesting to you, we go under kat.

For a start, we will be prepared for packet assembly:

apt-get install git fakeroot build-essential devscripts
apt-cache policy squid3
apt-get build-dep squid3
apt-get build-dep libecap2
apt-get install libssl-dev libgnutls28-dev

Do not forget to pass into that folder where you will collect source codes not to zasrat to yourself Home. Further we will download, we will compile and we will set Libressl:

wget http://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-2.1.6.tar.gz
tar -xzvf libressl-2.1.6.tar.gz
cd libressl-2.1.6

We collect and set then we will re-read hashes of libraries:

./configure
make
checkinstall --pkgname libressl --pkgversion 2.1.6
dpkg -i libressl_2.1.6-1_i386.deb
ldconfig

Well it is also necessary to configure use of LibreSSL by default:

mv /usr/bin/openssl /usr/bin/openssl-1
update-alternatives --install /usr/bin/openssl openssl /usr/bin/openssl-1 10
update-alternatives --install /usr/bin/openssl openssl /usr/local/bin/openssl 50
update-alternatives --config openssl

Let's check, whether it has turned out to put Libressl:

openssl version
    LibreSSL 2.1.6

It has turned out!

After execution of these actions, it is necessary to edit sources.list, having included source codes from testing branch there (it is necessary as we need to compile new libecap which is in turn necessary for assembly of Squid):

deb-src http://ftp.de.debian.org/debian/ testing main contrib non-free

Let's update cache of packets:

apt-get update

And now we will download the necessary source codes from Testing:


apt-get source squid3/testing
apt-get source libecap3/testing

Further we will collect libecap:

cd libecap-1.0.1/
dpkg-buildpackage -us -uc -nc -d

Let's delete second-hand articles, and we will set to novya:

apt-get purge libecap2
dpkg -i libecap3_1.0.1-2_i386.deb
dpkg -i libecap3-dev_1.0.1-2_i386.deb

We swing the last the freshest and Squid'a working snepshot:

wget http://www.squid-cache.org/Versions/v3/3.5/squid-3.5.8.tar.gz

Let's update Squid'a source codes received earlier to new, and we will work further already in directory with newly made source codes:

cd squid3-3.5.7/
uupdate -v 3.5.8 ../squid-3.5.8.tar.gz
cd ../squid3-3.5.8/

That all buns necessary to us were earned, Squid should compile with the necessary options therefore we will enter the following options of compilation to debian/rules:

--enable-ssl
--enable-ssl-crtd
--with-openssl

Let's download patch for bio.cc which is necessary for correct work with Libressl, bugs.squid-cache.org/attachment.cgi from here? id=3216 also we will apply it

patch -p0 -i bug-4330-put_cipher_by_char-t1.patch

Now it is possible to start compilation and assembly of Squid'a. But not so quickly! Everything will be compiled without problems, at least on h86 to architecture, but right at the end, on assembly step of packets of deb, to you will kindly tell in the console: "ouch ouch ouch, I cannot understand, what dependences are necessary for libssl.so.32" (it is the version of library from Libressl). It also is clear, from where Debian'U to know about it. We will deceive system, having specified the option "not check dependence", here so:

export DEB_DH_SHLIBDEPS_ARGS_ALL=--dpkg-shlibdeps-params=--ignore-missing-info

And now we will start compilation and assembly:

dpkg-buildpackage -us -uc -nc

After assembly we will set packet with languages for Squid'a:

apt-get install squid-langpack

Further we will set newly made packets:

dpkg -i squid-common_3.5.8-1_all.deb
dpkg -i squid_3.5.8-1_i386.deb
dpkg -i squid3_3.5.8-1_all.deb
dpkg -i squidclient_3.5.8-1_i386.deb

If the system zamateritsya on unsatisfied dependences, we will make:

apt-get -f install

It is almost ready. Let's pass into directory/etc/squid, there we will change something. Let's create pem the file necessary for SSL-Bump'inga:

openssl req -new -newkey rsa:1024 -days 365 -nodes -x509 -keyout squidCA.pem -out squidCA.pem

Also we will lead squid.conf to the following look:

acl localnet src 192.168.1.0/24	# RFC1918 possible internal network

acl SSL_ports port 443
acl Safe_ports port 80		# http
acl Safe_ports port 21		# ftp
acl Safe_ports port 443		# https
acl Safe_ports port 70		# gopher
acl Safe_ports port 210		# wais
acl Safe_ports port 1025-65535	# unregistered ports
acl Safe_ports port 280		# http-mgmt
acl Safe_ports port 488		# gss-http
acl Safe_ports port 591		# filemaker
acl Safe_ports port 777		# multiling http
acl CONNECT method CONNECT

dns_nameservers 8.8.8.8
http_access deny !Safe_ports

http_access deny CONNECT !SSL_ports

http_access allow localhost manager
http_access deny manager

http_access allow localnet
http_access allow localhost
http_access deny all

#прозрачный порт указывается опцией intercept
http_port 192.168.1.254:3128 intercept options=NO_SSLv3:NO_SSLv2

#также нужно указать непрозрачный порт, ибо если захотите вручную указать адрес
#прокси в браузере, указав прозрачный порт, вы получите ошибку доступа, поэтому нужно
#указывать непрозрачный порт в браузере, если конечно такое желание будет, к тому же в логах #сыпятся ошибки о том, что непрохрачный порт не указан=) 
http_port 192.168.1.254:3130 options=NO_SSLv3:NO_SSLv2 

#и наконец, указываем HTTPS порт с нужными опциями
https_port 192.168.1.254:3129 intercept ssl-bump options=ALL:NO_SSLv3:NO_SSLv2 connection-auth=off cert=/etc/squid/squidCA.pem

always_direct allow all
sslproxy_cert_error allow all
sslproxy_flags DONT_VERIFY_PEER

#укажем правило со списком блокируемых ресурсов (в файле домены вида .domain.com)
acl blocked ssl::server_name  "/etc/squid/blocked_https.txt"
acl step1 at_step SslBump1
ssl_bump peek step1

#терминируем соединение, если клиент заходит на запрещенный ресурс
ssl_bump terminate blocked 
ssl_bump splice all

sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/ssl_db -M 4MB

coredump_dir /var/spool/squid
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
refresh_pattern .               0       20%     4320
cache_dir aufs /var/spool/squid 20000 49 256
maximum_object_size 61440 KB
minimum_object_size 3 KB

cache_swap_low 90
cache_swap_high 95
maximum_object_size_in_memory 512 KB
memory_replacement_policy lru
logfile_rotate 4



I will a little tell about config. Documentation on ssl_bump, peek-n-splice quite extensive, but for our task it is necessary to know the following. There are some stages "handshaking" (i.e. handshake, interaction with the server). They are described in of.dokumentation. We are interested in example of Peek at SNI and Bump. That is, as appears from the name, we watch SNI information and bampy connection. Before it, we specify by the option DONT_VERIFY_PEER that it is necessary to accept certificates even if they have not undergone testing and we specify by the option sslproxy_cert_error that it is necessary to disconnect verification of certificates on the server. The specified rule "acl step1 at_step SslBump1" — it is the first of three possible steps of ssl_bump'a. At execution of this step we receive only SNI, and anything more. We have enough of it. Further we use this ACL in the line ssl_bump peek step1, that is directly we watch SNI, and after it we block connection if server_name from the blocked list is found in SNI.


Further we will wrap firewall the necessary ports on Squid:

iptables -t nat -A PREROUTING -p tcp -m tcp -s 192.168.1.0/24 --dport 443 -j REDIRECT --to-ports 3129
iptables -t nat -A PREROUTING -p tcp -m tcp -s 192.168.1.0/24 --dport 80 -j REDIRECT --to-ports 3128

Everything is ready! Squid can start solemnly:

systemctl start squid

Let's look, whether everything from Squid'om is normal:

systemctl status squid
● squid.service - LSB: Squid HTTP Proxy version 3.x
   Loaded: loaded (/etc/init.d/squid)
   Active: active (running) since Сб 2015-09-26 21:09:44 YEKT; 2h 6min ago
  Process: 1798 ExecStop=/etc/init.d/squid stop (code=exited, status=0/SUCCESS)
  Process: 1818 ExecStart=/etc/init.d/squid start (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/squid.service
           ├─1850 /usr/sbin/squid -YC -f /etc/squid/squid.conf
           ├─1852 (squid-1) -YC -f /etc/squid/squid.conf
           ├─1853 (logfile-daemon) /var/log/squid/access.log
           └─1854 (pinger)

сен 26 21:09:44 squid squid[1850]: Squid Parent: will start 1 kids
сен 26 21:09:44 squid squid[1850]: Squid Parent: (squid-1) process 1852 started
сен 26 21:09:44 squid squid[1818]: Starting Squid HTTP Proxy: squid.

If errors are not present, it is possible to work. Unfortunately, when blocking HTTPS of resources, there is no message of Squid'a "Access Is Prohibited", and instead the browser gives out error about impossibility of creation of connection. If someone prompts as to make it, I will be very glad.

UPD: in the version of the Squid which was compiled by me initially, i.e. 3.5.9, the annoying bug (or feature) is found because of which later time some HTTPS the sites cease to open. Solution: to compile version 3.5.8.

UPD2: has created next bagreport on problem in 3.5.9, I will update subject if something clears up.
UPD3: there was version 3.5.10 with the corrected bugs, at least, the patch on the bio.cc file is already applied there. Did not test meanwhile
UPD4: has edited article a little.
UPD5: direct reference for downloading of archive with all necessary packets (SQUID 3.5.8), and also direct reference on downloading of archive with all necessary packets (SQUID 3.5.10 — stable release last at the moment in which the set of bugs is corrected and is made set of optimization, judging by cheynzhlog, including the bug because of which it was necessary to apply in article patch to bio.cc is corrected. ATTENTION! It was not TESTED on the fighting server!)

I want to tell thanks to companion Dmitry Rakhmatullin, without it it would not turn out to make that is written above. Also, separate thanks to the Squid'a developers who have quickly answered my bug contango about error in libssl. And thanks to children of Nadz Goldman and gmax007 from the Toaster which have sent to the necessary bed my ideas on transfer of Squid'a on the server, physically separate of the main gateway.

This article is a translation of the original post at habrahabr.ru/post/267851/
If you have any questions regarding the material covered in the article above, please, contact the original author of the post.
If you have any complaints about this article or you want this article to be deleted, please, drop an email here: sysmagazine.com@gmail.com.

We believe that the knowledge, which is available at the most popular Russian IT blog habrahabr.ru, should be accessed by everyone, even though it is poorly translated.
Shared knowledge makes the world better.
Best wishes.

comments powered by Disqus