HTTP Anti-Virus Proxy

Official HAVP Support Forum
Registration disabled, I'm tired of spambots. E-mail havp@hege.li if you have questions.
HAVP project is pretty much frozen/abandoned at this time anyway.


All times are UTC + 2 hours [ DST ]




Post new topic Reply to topic  [ 10 posts ] 
Author Message
PostPosted: 12 Aug 2009 04:43 
Offline

Joined: 12 Aug 2009 00:19
Posts: 10
HAVP version: 0.91
ClamAV version: 0.95.2
Slackware Linux kernel: 2.6.24.5-smp

havp.config:
USER havp
GROUP havp
LOG_OKS false
TEMPDIR /mnt/ramdisk0
PARENTPROXY 192.168.1.101
PARENTPORT 3128
MAXSCANSIZE 15000000
STREAMUSERAGENT Player Winamp iTunes QuickTime Audio RMA/ MAD/ Foobar2000 XMMS
STREAMSCANSIZE 20000
ENABLECLAMLIB true
CLAMDBDIR /usr/share/clamav
ENABLEFPROT false
ENABLEAVG false
ENABLEAVESERVER false
ENABLESOPHIE false
ENABLETROPHIE false
ENABLENOD32 false
ENABLEAVAST false
ENABLEARCAVIR false
ENABLEDRWEB false


HAVP runs fine when using an ext3 partition with {mand}.
But it fails with the following errors when I try to run it using an ext2 ram disk.

Code:
# mount | grep ram0
/dev/ram0 on /mnt/ramdisk0 type ext2 (rw,mand,noatime)

# df -h | grep ram0
/dev/ram0              16M  140K   16M   1% /mnt/ramdisk0

# /usr/local/sbin/havp -c /usr/local/etc/havp/havp.config
Starting HAVP Version: 0.91
LibClamAV Error: cli_untgz: Wrote 0 instead of 512 (/mnt/ramdisk0/clamav-4021213a28028a03f4f7188968193037/main.mdb)
LibClamAV Error: cli_cvdload(): Can't unpack CVD file.
LibClamAV Error: Can't load /usr/share/clamav/main.cvd: Broken or not a CVD file
One or more scanners failed to initialize!
Check errorlog for errors.
Exiting..

# tail /var/log/havp/error.log
06:29:12 === Starting HAVP Version: 0.91
06:29:12 Running as user: havp, group: havp
06:29:12 Use parent proxy: 192.168.1.101:3128
06:29:12 --- Initializing ClamAV Library Scanner
06:29:12 ClamAV: Using database directory: /usr/share/clamav
06:29:13 ClamAV: Could not load database: Broken or not a CVD file
06:29:13 Error initializing ClamAV Library Scanner!


What's causing this and how do I try to debug it?


Top
 Profile  
 
PostPosted: 12 Aug 2009 13:49 
Offline
HAVP Developer

Joined: 27 Feb 2006 18:12
Posts: 687
Location: Finland
What does clamscan --tempdir=/mnt/ramdisk0 say?

Maybe try ext3, or maybe such "old style"(?) ramdisk just doesn't work with certain system functions.


Top
 Profile  
 
PostPosted: 12 Aug 2009 21:22 
Offline

Joined: 23 Apr 2008 09:36
Posts: 101
Hy Strykar,
I see you are using ram with 16MB disk size and actual this is far too little.

Database sizes today:
45M 14. Mai 18:00 main.cld
3,8M 12. Aug 20:00 daily.cld

If you take a look at output errror message, you can see: Can't unpack CVD file
and this is not a ext2, or havp related problem!

Update: :arrow: Sorry for my mistaken, I have to correct myself, I took a look on a system, where I'm running havp with ramdisk size 8MB! Used size is 1,5M and using ext3. But I remember, it was a little bit trickly to get it running. Perhaps a user right problem?

This is a part of my init script, to get it running:
Code:
                        echo -n "Creating File System and starting HAVP ..."
                        /sbin/mkfs.ext3 /dev/ram0
                        sleep 2
                        `mount /dev/ram0 /var/tmp/havp -o mand`
                        `chown -R clamav:clamav /var/tmp/havp/`
                        $HAVP_BIN -c $HAVP_CONFIG


Code:
/dev/ram0             7,8M  1,5M  6,0M  20% /var/tmp/havp


Top
 Profile  
 
PostPosted: 13 Aug 2009 02:39 
Offline

Joined: 12 Aug 2009 00:19
Posts: 10
hege wrote:
What does clamscan --tempdir=/mnt/ramdisk0 say?

Maybe try ext3, or maybe such "old style"(?) ramdisk just doesn't work with certain system functions.

I still get the same error.
"Old style" being ext2 fs?

Code:
# sudo -u clamav clamscan --tempdir=/mnt/memory/
LibClamAV Error: cli_untgz: Wrote 0 instead of 512 (/mnt/memory//clamav-c94d88b9                                                                                                    64f002aa2b8c788dc6ffef9c/main.mdb)
LibClamAV Error: cli_cvdload(): Can't unpack CVD file.
LibClamAV Error: Can't load /usr/share/clamav/main.cvd: Broken or not a CVD file
ERROR: Broken or not a CVD file

Can I turn up HAVP log levels to see what could be going wrong?
As root/clamav, I can cp/rm files to the ramdisk just fine :?


Last edited by Strykar on 13 Aug 2009 02:46, edited 1 time in total.

Top
 Profile  
 
PostPosted: 13 Aug 2009 02:43 
Offline

Joined: 12 Aug 2009 00:19
Posts: 10
karesmakro wrote:
Update: :arrow: Sorry for my mistaken, I have to correct myself, I took a look on a system, where I'm running havp with ramdisk size 8MB! Used size is 1,5M and using ext3. But I remember, it was a little bit trickly to get it running. Perhaps a user right problem?

This is a part of my init script, to get it running:
Code:
 snipped


karesmakro,
I did think it was a permission issue and double checked that before posting.
Even when I chmod 666 I still get the same error.

Code:
root@barge:~# /sbin/mkfs.ext3 /dev/ram1
mke2fs 1.40.8 (13-Mar-2008)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
4096 inodes, 16384 blocks
819 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=16777216
2 block groups
8192 blocks per group, 8192 fragments per group
2048 inodes per group
Superblock backups stored on blocks:
        8193

Writing inode tables: done
Creating journal (1024 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 26 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
root@barge:~# mount /dev/ram1 /mnt/memory -o mand
root@barge:~# chown -R clamav:clamav /mnt/memory/
root@barge:~# mount|grep ram
/dev/ram1 on /mnt/memory type ext3 (rw,mand)
root@barge:~# /usr/local/sbin/havp -c /usr/local/etc/havp/havp.config
Starting HAVP Version: 0.91
LibClamAV Error: cli_cvdload(): Can't create temporary directory /mnt/memory/clamav-5c5835284204d5668440a8af96268175
LibClamAV Error: Can't load /usr/share/clamav/main.cvd: Can't create temporary directory
One or more scanners failed to initialize!
Check errorlog for errors.
Exiting..


Top
 Profile  
 
PostPosted: 13 Aug 2009 07:57 
Offline
HAVP Developer

Joined: 27 Feb 2006 18:12
Posts: 687
Location: Finland
No kares, you are right, good catch. It's has to be the small ramdisk size. The error clearly shows it's being used as ClamAV temporary directory, thus there is no space to unpack stuff. You must have some other TEMPDIR directory set. SCANTEMPFILE is different thing.

So the "fix" here is TEMPDIR to another bigger disk. Using ramdisk is not strictly necessary anyway, it's very marginal performance gain. Unless you are very low on memory and OS doesn't cache files.


Top
 Profile  
 
PostPosted: 14 Aug 2009 03:21 
Offline

Joined: 12 Aug 2009 00:19
Posts: 10
hege wrote:
No kares, you are right, good catch. It's has to be the small ramdisk size. The error clearly shows it's being used as ClamAV temporary directory, thus there is no space to unpack stuff. You must have some other TEMPDIR directory set. SCANTEMPFILE is different thing.

So the "fix" here is TEMPDIR to another bigger disk. Using ramdisk is not strictly necessary anyway, it's very marginal performance gain. Unless you are very low on memory and OS doesn't cache files.


Hi hege,

I increased the RAM disk size to 64 MB, with some interesting results:
Code:
# mount | grep ram0
/dev/ram0 on /mnt/memory type ext3 (rw,mand)

# df -h | grep ram0
/dev/ram0              61M  5.3M   53M  10% /mnt/memory


# sudo -u clamav clamscan --tempdir=/mnt/memory
/root/.htoprc: OK
/root/dead.letter: OK
/root/.fonts.conf: OK
/root/.joe_state: Access denied
/root/memtest86-3.5.tar.gz: OK
/root/.mysql_history: Access denied
/root/.serverauth.1606: Access denied
/root/.bash_history: OK
/root/.qmake.vars: OK
/root/clamava.debug: OK
/root/.lesshst: Access denied
/root/.Xauthority: Access denied
/root/.mcoprc: Access denied
/root/.kderc: Access denied
/root/.ICEauthority: Empty file
/root/.xinitrc: OK
/root/du.sh~: OK
/root/du.sh: OK
/root/.serverauth.5416: Access denied
/root/.nvidia-settings-rc: OK
/root/findlargefiles: OK

----------- SCAN SUMMARY -----------
Known viruses: 608668
Engine version: 0.95.2
Scanned directories: 1
Scanned files: 12
Infected files: 0
Data scanned: 3.27 MB
Data read: 0.95 MB (ratio 3.46:1)
Time: 10.086 sec (0 m 10 s)


So clamscan now has enough space and "clamscan --tempdir=/mnt/memory" works.
But the error still persists when I run HAVP:

Code:
# /usr/local/sbin/havp -c /usr/local/etc/havp/havp.config
Starting HAVP Version: 0.91
LibClamAV Error: cli_cvdload(): Can't create temporary directory /mnt/memory/clamav-049b6f9156bc66262b0a9587851194e2
LibClamAV Error: Can't load /usr/share/clamav/main.cvd: Can't create temporary directory
One or more scanners failed to initialize!
Check errorlog for errors.
Exiting..

# tail -f /var/log/havp/error.log
14/08/2009 05:46:19 === Starting HAVP Version: 0.91
14/08/2009 05:46:19 Running as user: havp, group: havp
14/08/2009 05:46:19 Use parent proxy: 192.168.1.101:3128
14/08/2009 05:46:19 --- Initializing ClamAV Library Scanner
14/08/2009 05:46:19 ClamAV: Using database directory: /usr/share/clamav
14/08/2009 05:46:20 ClamAV: Could not load database: Can't create temporary directory
14/08/2009 05:46:20 Error initializing ClamAV Library Scanner!


Someone in #Clamav on Freenode suggested "64 MB might be cutting it close depending on what you're scanning too."
Which made sense, but since HAVP's MAXSCANSIZE was set to 15000000 (14 MB), I thought that HAVP (clamscan) would never need more than 14-20MB of TEMPDIR space.
Am I incorrect in assuming this?

karesmakro has said above that he uses an 8MB RAMdisk and HAVP works fine :?
Do you still think it is an issue of free space in TEMPDIR?


Top
 Profile  
 
PostPosted: 14 Aug 2009 03:57 
Offline

Joined: 12 Aug 2009 00:19
Posts: 10
I increased the RAMdisk size to 128 MB, and the same HAVP error as above persists.
And I ran strace to see what was going on:

Code:
# strace -o /tmp/havvvp.log /usr/local/sbin/havp -c /usr/local/etc/havp/havp.config
Starting HAVP Version: 0.91
LibClamAV Error: cli_cvdload(): Can't create temporary directory /mnt/memory/clamav-3e971867c53d0a37ad27aebdbd13d4c4
LibClamAV Error: Can't load /usr/share/clamav/main.cvd: Can't create temporary directory
One or more scanners failed to initialize!
Check errorlog for errors.
Exiting..

The owner of the TEMPDIR is clamav:
# ls -alh /mnt/memory/
total 17K
drw-rw-rw- 3 clamav clamav 1.0K 2009-08-14 06:06 ./
drwxr-xr-x 11 root root 4.0K 2009-08-12 05:44 ../
drwx------ 2 clamav clamav 12K 2009-08-14 06:04 lost+found/
The strace log is big and I can paste it here if someone wants, but here's the interesting bit from it:

Code:
# grep -w "mkdir" /tmp/havvvp.log
mkdir("/mnt/memory/clamav-3e971867c53d0a37ad27aebdbd13d4c4", 0700) = -1 EACCES (Permission denied)


So the errors:
Code:
LibClamAV Error: cli_cvdload(): Can't create temporary directory /mnt/memory/clamav-clamav-3e971867c53d0a37ad27aebdbd13d4c4
LibClamAV Error: Can't load /usr/share/clamav/main.cvd: Can't create temporary directory

are not disk space related but permission related.

Since the TEMPDIR is owned by clamav, I don't understand this behavior or is this a bug?


Top
 Profile  
 
PostPosted: 14 Aug 2009 09:26 
Offline

Joined: 23 Apr 2008 09:36
Posts: 101
As I see in your havp config on first post, that you are running havp under user and group havp, but permission on your tempdir belongs to clamav, which leads to this error!
My init script was an example, which depends on the settings related to clamav and havp. In my example, havp and clamav is running with user clamav, because if you are using the option in havp.config
Code:
ENABLECLAMLIB true
and running both services under an unprivileged user, you have to run both services with same user names (sorry for my english).

Read the comments in havp.config for tempdir and you can see
Code:
#
# Directory for ClamAV and other scanner created tempfiles.
# Needs to be writable by HAVP user. Use ramdisk for best performance.
#
# Default:
TEMPDIR /var/tmp/havp

which can't be successful in your example, as you created your tempdir with clamav user rights.

I hope this helps, to solve your problem!

Regards, karesmakro
p.s.: and keep smiling, there is always a logical explanation :wink:


Top
 Profile  
 
PostPosted: 14 Aug 2009 22:59 
Offline

Joined: 12 Aug 2009 00:19
Posts: 10
kares,

Thanks a lot for your pointers :D
It was a permissions issue, changing RAMdisk ownership to clamav instead of HAVP fixed it!

Maybe this should be included in the RAMdisk section of the README?

Thanks a lot for all your time hege and kares.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 10 posts ] 

All times are UTC + 2 hours [ DST ]


Who is online

Users browsing this forum: Google [Bot] and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group