| HTTP Anti-Virus Proxy http://havp.hege.li/forum/ |
|
| Havp & Debian FTP servers http://havp.hege.li/forum/viewtopic.php?f=3&t=69 |
Page 1 of 1 |
| Author: | Osiris X [ 05 Apr 2006 11:56 ] |
| Post subject: | Havp & Debian FTP servers |
HAVP 0.78 seems to work great for ftp servers accessed through a web browser. But when I use the command "apt-get" (Debian Linux platform) to access servers such as ftp.uk.debian.org and ftp.fi.debian.org the downloads timeout at between 60% and 80% complete. I get the following messages Error reading from server - read (104 Connection reset by peer) & 403 Server down from HAVP the havp.log says 05/04/2006 09:03:07 (127.0.0.1) Could not read server body (ftp.fi.debian.org:21) 05/04/2006 09:04:41 (127.0.0.1) Could not read server header (ftp.uk.debian.org:21) Any ideas as to what might be causing this behaviour? and more importantly any ideas on how I can fix it? Regards Os |
|
| Author: | hege [ 05 Apr 2006 13:05 ] |
| Post subject: | |
Seems to be a problem with apt-get sending HTTP/1.1 pipelined requests. HAVP has some problems with them. I'll fix this is next version.. You can fix apt-get yourself by adding this line to /etc/apt/apt.conf: Acquire::http::Pipeline-Depth 0; Cheers, Henrik |
|
| Author: | Osiris X [ 05 Apr 2006 16:17 ] |
| Post subject: | |
Thanks for the quick response Alas this doesnot seem to solve the problem for me, possibly because it is using the ftp method and not the http method. Strangely if I change the ftp_proxy environment variable from http://localhost:8080 to ftp://localhost:8080 apt-get cannot even connect to the ftp servers. Os |
|
| Author: | hege [ 05 Apr 2006 16:22 ] |
| Post subject: | |
It did fix it for me, there should be no reason it not to work after the pipeline setting. Though you seem to have more problems.. if you get 403 Server down, then it's possible something is down  I didn't try 0.78, but the newly released 0.79.. try if it fixes something.
And naturally ftp://localhost:8080 doesn't work. Proxies are always defined with http:// because they talk HTTP. Oh and just to make sure.. you are using PARENTPROXY with HAVP, right? 
Cheers, Henrik |
|
| Author: | Osiris X [ 05 Apr 2006 22:46 ] |
| Post subject: | |
I am using squid as the PARENTPROXY (I would happily use something different if I could find one that proxied ftp). I have now upgraded to HAVP 0.79 ftp_proxy & FTP_PROXY are both set to http://localhost:8080/ in /etc/environment I have added the line Acquire::http::Pipeline-Depth 0; to /etc/apt/apt.conf Unfortunately the problem still remains (i.e. ftp transfer aborts halfway through), however the error message is a little different "Error reading from server. Remote end closed connection" I have tried with several different debian ftp servers. Any other suggestions? By the way, I tried usine SmartFTP from a windows host through the proxy using CONNECT. It generates an error in HAVP - "Invalid request from browser" Thanks for your help Os |
|
| Author: | Osiris X [ 06 Apr 2006 06:36 ] |
| Post subject: | |
Something interesting from the havp logs 06/04/2006 04:24:05 127.0.0.1 ftp://ftp.de.debian.org/debian/dists/te ... ckages.bz2 Virus: BZip.ExceededFileSize 06/04/2006 04:24:20 127.0.0.1 ftp://ftp.de.debian.org/debian/dists/st ... ackages.gz Virus: GZip.ExceededFileSize Does this indicate a virus has been detected? and could this potentially be the cause of the incomplete ftp download? havp settings: RANGE false KEEPBACKBUFFER 200000 (The package files are all larger than this) KEEPBACKTIME 5 MAXSCANSIZE 0 TRICKLING 30 Os |
|
| Author: | hege [ 06 Apr 2006 09:26 ] |
| Post subject: | |
Hehe yes that is the reason.. ClamAV is blocking files that exceed limits by default in 0.79. You should change CLAMBLOCKMAX to false. # Should we block files that go over maximum archive values? # CLAMBLOCKMAX true # Scanning limits for archives (filesize = MB): # CLAMMAXFILES 1000 # CLAMMAXFILESIZE 10 # CLAMMAXRECURSION 8 # CLAMMAXRATIO 250 Cheers, Henrik |
|
| Author: | Osiris X [ 06 Apr 2006 13:53 ] |
| Post subject: | |
It works! During an apt-get update the % complete counter for each package source is misleading - maybe it is calculated on time and estimated data rate. It appears to not reach anywhere near 100% before just disappearing. Maybe that this is due to the certain amount of data that HAVP holds back before sending in a burst. If I whitelist the server the % complete counter works correctly, but the same number of bytes are transferred in total as before. One thing I did notice is that I can't have ftp.*.debian.org/debian/dists/* in the whitelist as it generates an error - 06/04/2006 11:30:58 URLList - Too many wildcards in ftp.*.debian.org 06/04/2006 11:30:58 URLList invalid Domain: ftp.*.debian.org I also tried usine SmartFTP from a windows host through an "HTTP proxy using a CONNECT tunnel". It generates an error in HAVP - "Invalid request from browser". Any ideas as to why? Thanks for all of your help Os |
|
| Author: | hege [ 06 Apr 2006 13:58 ] |
| Post subject: | |
Osiris X wrote: One thing I did notice is that I can't have ftp.*.debian.org/debian/dists/* in the whitelist as it generates an error - 06/04/2006 11:30:58 URLList - Too many wildcards in ftp.*.debian.org 06/04/2006 11:30:58 URLList invalid Domain: ftp.*.debian.org I guess thats some limitation in the whitelist, maybe Christian will have a look as he did it. Quote: I also tried usine SmartFTP from a windows host through an
"HTTP proxy using a CONNECT tunnel". It generates an error in HAVP - "Invalid request from browser". Any ideas as to why? CONNECT is meant for SSL requests, and your HAVP isn't compiled with the support. And anyways, all proxies allow only ports 443 and 563 by default with CONNECT. You are trying to use port 21 for FTP. You cannot scan the files like this anyway. In any case, use HTTP rather than FTP for the mirrors. It is much more efficient. 
Cheers, Henrik |
|
| Author: | Christian [ 06 Apr 2006 17:01 ] |
| Post subject: | |
The domain of the white/blacklist is limited to one wildcard at the beginning. *.debian.org would be ok Also only complete domain names are supported. e.g. *ebian.org/ will not blacklist debian.org but ebian.org, xyz.ebian.org ... I will try to allow "middle" wildcard in the next version. |
|
| Author: | eakinasila [ 25 Nov 2009 12:41 ] |
| Post subject: | Re: Havp & Debian FTP servers |
I need to set up a page where a client can login with a username password and access their FTP folder.? They also need an option to upload to their ftp folder. What's the best/easiest way to configure something like this? _________________ external keyword tool ~ keyworddiscovery.com ~ keycompete.com ~ compete.com ~ webmasterworld.com |
|
| Author: | karesmakro [ 26 Nov 2009 11:16 ] |
| Post subject: | Re: Havp & Debian FTP servers |
Hi eakinasila! What are the credentials? With FTP-Client or explorer to handle the FTP file transfers? Should it be possible to download files with explorer (perhaps with virus filter)? In this case I would prefer apache2 and proftpd (frox for virus filter), but this is a matter of taste. regards |
|
| Page 1 of 1 | All times are UTC + 2 hours [ DST ] |
| Powered by phpBB® Forum Software © phpBB Group https://www.phpbb.com/ |
|