HTTP Anti-Virus Proxy

Official HAVP Support Forum
Registration disabled, I'm tired of spambots. E-mail havp@hege.li if you have questions.
HAVP project is pretty much frozen/abandoned at this time anyway.
It is currently 22 Jun 2014 09:52

All times are UTC + 2 hours [ DST ]




Post new topic Reply to topic  [ 10 posts ] 
Author Message
 Post subject: HTTPS scanning
PostPosted: 27 Nov 2006 18:00 
Offline

Joined: 27 Nov 2006 17:58
Posts: 1
Hi to all.
Now with simple havp is not possible to scan https pages.

Is there someone that can halp me and other people to work around this problem?

Future implementation, conversione, any ideas?

Thanks Nicola

:)


Top
 Profile  
 
 Post subject:
PostPosted: 26 Mar 2007 22:43 
Offline

Joined: 06 Apr 2006 21:33
Posts: 21
Its possible with apache 2.2.4
You will need to create a CA-root, then sign a wildcard certificate with the CA-root, then tell apache to use these files.

Then you need to import the root in your webbrowser. (IE 7.0)
IE 6.0 dosent like wildcard certs, so you have to uncheck "Warn for invalid site certificates" in advanced settings. (I use swedish IE but I dont know which is the english equalivent)

You can set up a virtualhost on 8443, transparently forward all outgoing SSL traffic to this port, then tag all SSL-traffic with a extra header, then let it trough HAVP, and then read off this header at the other end to decide if the connection should be SSL or not, and then rewrite the urls so they go through HAVP proxy.

Then you need a virtualhost on 8448, accepting standard HTTP, that tags traffic for HTTP and send it to HAVP.


Then a virtualhost on 8449 , that has ProxyRequests on , that accept a upstream request from HAVP, and then rewrite the urls back to their original urls, based on the content in the header you set before....

Like this:
X-SSL-Originating-Traffic: YES
X-SSL-Originating-Traffic: NO

(And invalid headers you can rewrite to [F] so nobody can manipulate the headers)

------------------------------------------------------------------------------------

A little while after next Copfilter comes out, I will post a configuration example here for apache 2.2.4 , so you can copy+paste


Top
 Profile  
 
 Post subject:
PostPosted: 06 Apr 2007 20:13 
Offline

Joined: 06 Apr 2006 21:33
Posts: 21
Here you have it....
Make sure you have apache 2.2.4 added, and compile in almost every module.... (I dont exactly know which ones, but you can compile in mod_so, and the load in each module, as apache requires it...)

Make sure server.crt is a wildcard certificate which has its CN set to * with a matching server.key private key.

8080 must be replaced with the port HAVP proxy is listening on, and HAVP must have its PARENTPROXY set to 127.0.0.1:8445

Also add the following firewall rules:
iptables -t nat -A PREROUTING -p tcp -i <INTERFACE_ON_INSIDE> --dport 80 -j REDIRECT --to-port 8444
iptables -t nat -A PREROUTING -p tcp -i <INTERFACE_ON_INSIDE> --dport 443 -j REDIRECT --to-port 8443
(Of course you replace <INTERFACE_ON_INSIDE> with the name of the interface on the LAN side)


Then add this in the apache config:

listen 8443
listen 8444
listen 8445
<VirtualHost _default_:8443>
RewriteEngine on
RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK|OPTIONS)
RewriteRule .* - [F]
DocumentRoot /home/httpd/html
ServerAdmin root@localhost
ErrorLog /var/log/httpd/error_log
TransferLog /var/log/httpd/access_log
SSLEngine on
SSLProtocol all -SSLv2
SSLCipherSuite ALL:!ADH:!EXPORT56:!eNULL:!SSLv2:RC4+RSA:+HIGH:+MEDIUM:+LOW:+$
SSLCertificateFile /etc/apache2/server.crt
SSLCertificateKeyFile /etc/apache2/server.key
SetEnv HOME /home/nobody
SetEnvIf User-Agent ".*MSIE.*" \
nokeepalive ssl-unclean-shutdown \
downgrade-1.0 force-response-1.0

ProxyRemote * http://127.0.0.1:8080
ProxyPreserveHost On
SetOutputfilter DEFLATE
DeflateCompressionLevel 9
SetInputFilter INFLATE
RequestHeader unset xsslcatch
RequestHeader set xsslcatch ison
RewriteRule ^(.*)$ proxy:http://%{HTTP_HOST}$1
</VirtualHost>

<VirtualHost _default_:8444>
RewriteEngine on
RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK|OPTIONS)
RewriteRule .* - [F]
DocumentRoot /home/httpd/html
ServerAdmin root@localhost
ErrorLog /var/log/httpd/error_log
TransferLog /var/log/httpd/access_log
ProxyRemote * http://127.0.0.1:8080
ProxyPreserveHost On
SetOutputFilter DEFLATE
DeflateCompressionLevel 9
SetInputFilter INFLATE

RequestHeader unset xsslcatch
RequestHeader set xsslcatch isoff
RewriteRule ^(.*)$ proxy:http://%{HTTP_HOST}$1
</VirtualHost>


<VirtualHost _default_:8445>
ProxyRequests on
SSLProxyEngine on
ProxyVia block
ProxyPreserveHost On
DocumentRoot /home/httpd/html
ServerAdmin root@localhost
ErrorLog /var/log/httpd/error_log
TransferLog /var/log/httpd/access_log
SetOutputFilter INFLATE
<Proxy *>
RequestHeader unset Via
RequestHeader unset X_FORWARDED_FOR
RewriteEngine on
RewriteCond %{HTTP:xsslcatch} ^ison$
RewriteRule ^proxy:http://(.*)$ proxy:https://$1
RewriteCond %{HTTP:xsslcatch} ^isoff$
RewriteRule ^proxy:http://(.*)$ proxy:http://$1
RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK|OPTIONS)
RewriteRule .* - [F]
</Proxy>

</VirtualHost>


Top
 Profile  
 
 Post subject: Re: HTTPS scanning
PostPosted: 30 Nov 2009 16:17 
Offline

Joined: 20 Nov 2009 18:38
Posts: 7
Hello !

This topic is old but it corresponds exactly to the problem I had : filter the HTTPS content. Thank you sebastian for posting your conf (if you ever come back here...)

This method works well, but it has a little inconvenient : it is not possible to get the certificate from the remote site ; instead I get the wildcard I set. I don't want to allow access to every site, but I neither want sites signed with a known CA to generate a warning from my browser.

Is there something that can be done to make the proxy forward the certificate (I guess not ; or it would be too easy doing man in the middle attacks), or if I accept the wildcart cert in my browser, that the proxy blocks sites signed with unknown CA (like browsers do) ?

I hope I have been clear ; tell me if I wasn't !
Thanks for helping me


Top
 Profile  
 
 Post subject: Re: HTTPS scanning
PostPosted: 02 Dec 2009 14:53 
Offline

Joined: 06 Apr 2006 21:33
Posts: 21
About the wildcard cert, you can import this in browser and then turn off "Check hostnames in certificates" or the equvalient in your browser and you will never get HTTPS warnings.

If you want to block sites with a unknown CA, set the directive:

SSLProxyCACertificateFile

to a path to a PEM file where all the cerficates of intemediary and root CAs you want to accept, is stored. Just store them one after one in one single PEM file. You can do this by exporting all intemediary and root CAs in your browser, and then CAT:ing them together in one single file. Be sure to export in CRT format (its almost equvalient to the PEM format)

You can also enable and disable remote hostname check with:
SSLProxyCheckPeerCN (on or off)

And you can also set if the remote server certificate expiration date is checked:
SSLProxyCheckPeerExpire (on or off)

If your users are accessing sites which require client certificates, request their certificates and keys, probably in .p12 format along with their passwords, export them with "openssl -pkcs12 -in certificatefile.p12 -out certfile.pem -nodes"

Then you need to go in with nano or another text editior, and search for the first private key and put it after the first certificate... The openssl exports in this format:
Certificate A
Certificate B
Certificate C
Privatekey A
Privatekey B
Privatekey C

You need to get it as this:

Certificate A
Privatekey A
Certificate B
Privatekey B
Certificate C
Privatekey C

Then save the unencrypted PEM file in the apache folder and set:

SSLProxyClientCertificateFile to point to that file.
Then users will log on automatically to sites requiring client certificate, like banking sites.

All these directives SHOULD be set AFTER:
SSLProxyEngine on

since its that server that acts as a proxy to the whole world.

***PLEASE NOTE***

There will NOT be possible for the end user to dismiss the certificate error, its a all or nothing setting, so if you select to block bad certificates in the proxy, user will not be able to access these sites, and thats pretty bad since theres much sites out there that use Self-signed and certificates with invalid dates and such.

The proxy will err out with a 502 HTTP error code if the certificate is invalid, the remote server requires a clientcertificate or the date/CN is incorrect and you have set the equvalient settings noted above.


Top
 Profile  
 
 Post subject: Re: HTTPS scanning
PostPosted: 03 Dec 2009 11:09 
Offline

Joined: 20 Nov 2009 18:38
Posts: 7
Thank you very much for your answer. Since I posted the message, I tried different settings : I added

SSLProxyVerify require
SSLProxyVerifyDepth 10
SSLProxyCACertificatePath /usr/lib/ssl/certs

where the path contains the certificates trusted by openssl -but it's a good idea to export the ones of the browser. The problem is that it works even if the directory is empty. I will try the directives you gave to me.

About the client certificates, thanks, it could be very useful to me.

But still, it remains a big problem ; is there no way to redirect the 502 error to a web page stored in the server, where the certificate could be displayed ? Then with a click the content of the certificate would be added to the accepted certs...


Top
 Profile  
 
 Post subject: Re: HTTPS scanning
PostPosted: 03 Dec 2009 11:40 
Offline

Joined: 06 Apr 2006 21:33
Posts: 21
Rankin:

Try:
ErrorDocument 502 /502error.cgi

and then place a 502error.html in your webroot (configure a webroot for the proxy if you don't have one).

But one problem is that I think the server is only loading certs at startup, so maybe you need to have a script that places the server certificate in the correct directory/file and does a graceful restart.

You can find the url or host of the page in one of the enviroment variables Apache spits out when doing a ErrorDocument. Then you can fetch the certificate for yourself and then use it in some way.


Last edited by sebastian on 03 Dec 2009 12:05, edited 2 times in total.

Top
 Profile  
 
 Post subject: Re: HTTPS scanning
PostPosted: 03 Dec 2009 12:02 
Offline

Joined: 20 Nov 2009 18:38
Posts: 7
Thanks for the piece of advice about the 502 error, I'll try it !

I 've just made some tests and there are the results : I post the configuration of my proxy virtualhost :

<VirtualHost _default_:8445>
ProxyRequests on
SSLProxyEngine on
SSLProxyVerify require
SSLProxyVerifyDepth 10
SSLProxyCACertificatePath /usr/lib/ssl/certs
SSLProxyCheckPeerCN on
SSLProxyCheckPeerExpire on
ProxyVia block
ProxyPreserveHost On
DocumentRoot /var/www
ServerAdmin root@localhost
ErrorLog /var/log/apache2/havp-ssl-error.log
TransferLog /var/log/apache2/havp-ssl-access.log
SetOutputFilter INFLATE
<Directory /var/www>
Order deny,allow
Deny from all
Allow from 127.0.0.1
</Directory>
<Proxy *>
Order deny,allow
Deny from all
Allow from 127.0.0.1
RequestHeader unset Via
RequestHeader unset X_FORWARDED_FOR
RewriteEngine on
RewriteCond %{HTTP:xsslcatch} ^ison$
RewriteRule ^proxy:http://(.*)$ proxy:https://$1
RewriteCond %{HTTP:xsslcatch} ^isoff$
RewriteRule ^proxy:http://(.*)$ proxy:http://$1
RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK|OPTIONS)
RewriteRule .* - [F]
</Proxy>

</VirtualHost>

With this conf, every site is allowed even ones with self-signed certificates. If I replace
SSLProxyCACertificatePath /usr/lib/ssl/certs
by
SSLProxyCACertificateFile /usr/lib/ssl/certs/Wells_Fargo_Root_CA.pem (it's an example), I got a 502 error.
It seems that the checkpeers directive don't work as well as they should ; an error in my conf ?

Little hint, when I try to remove the SSLProxyCACertificate* directive, I got this warning at startup :
Oops, you want to request client authentication, but no CAs are known for verification!?

It's not about client authentication but server authentication, right ?


--
For 502 : no problem if I have to restart apache, if I can do what I want it won't be too annoying. The main problem I think, will be to retrieve the certificate.


Top
 Profile  
 
 Post subject: Re: HTTPS scanning
PostPosted: 03 Dec 2009 12:06 
Offline

Joined: 06 Apr 2006 21:33
Posts: 21
But really, just dont do certificate validation, its not THAT high risk for a MITM attack with invalid certificates. As long as you dont go over untrusted networks, it should not be a problem to just accept all certificates.

And your network is probably trusted. I think your network is like my network:
My computer --- (My LAN, trusted) -- Firewall -- (The Apartment LAN, of course only the managers has the key to the switch room, I trust the managers) -- (My ISP network, I trust my ISP so no problem here. Also all network rooms are alarmed and locked with AccessControl, so nobody can break in and even sniff HTTP traffic) -- (Backbone in my country. Highly restricted network. Breaking in here would probably cause the military to search for the problem, so here is high security too, I trust the backbone) -- (The backbone in the country of the target server, I trust too.) -- (The server's ISP, of course I trust the server's ISP) -- firewall of target host -- (Local server LAN, I trust) -- Server.

I trust the full path of the connection, HTTPS/SSL is really not needed over secure ISP networks. Its just over untrusted networks HTTPS/SSL is required.


Top
 Profile  
 
 Post subject: Re: HTTPS scanning
PostPosted: 01 Jun 2011 11:12 
Offline

Joined: 01 Jun 2011 11:11
Posts: 1
HTTPS is protected how it can be scanned?
astigmatism


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 10 posts ] 

All times are UTC + 2 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group