HostingHacks.net | Tools to Build a Linux Webhosting Service

Spamassassin support for Sendmail

INSTALLATION: The installation commands can be run from a Putty window in a "cut-and-paste" style layout or copied a script. Notes on Putty best practices can be found here.

last updated: Apr. 2005

Spamassassin requires many perl modules to be installed on the system. Your system may already have some of these perl modules. You can do a quick check of the all perl CPAN modules installed by using:

cat > listpm.pl << "EOF"
#!/usr/bin/perl
use strict;
use ExtUtils::Installed; 

my $installed = ExtUtils::Installed->new(); 
foreach my $module ( grep( !/^Perl$/, $installed->modules() ) ) 
{ 
    my $version = $installed->version( $module ) || "---"; 
    print( "$module|" ); 
}
EOF

chmod 755 listpm.pl
./listpm.pl

Time-HiRes: (Needed by Net::DNS) The Time::HiRes module implements a Perl interface to the usleep, ualarm, gettimeofday, system calls ; high resolution time and timers. http://search.cpan.org/~jhi/Time-HiRes-1.65/

mkdir -p /usr/src/CPAN
cd /usr/src/CPAN
wget http://search.cpan.org/CPAN/authors/id/J/JH/JHI/Time-HiRes-1.65.tar.gz
# wget http://hostinghacks.net/dist/CPAN/Time-HiRes-1.65.tar.gz
tar -zxf Time-HiRes-1.65.tar.gz
cd /usr/src/CPAN/Time-HiRes-1.65
LC_ALL=C
export LC_ALL
perl Makefile.PL
make
#make test
make install

Digest::base - Digest base class (needed by Digest-SHA1) http://search.cpan.org/~gaas/Digest-1.10/

mkdir -p /usr/src/CPAN
cd /usr/src/CPAN
wget http://search.cpan.org/CPAN/authors/id/G/GA/GAAS/Digest-1.10.tar.gz
# wget http://hostinghacks.net/dist/CPAN/Digest-1.10.tar.gz
tar -zxf Digest-1.10.tar.gz
cd /usr/src/CPAN/Digest-1.10
perl Makefile.PL
make
#make test
make install

Digest-SHA1-2.10 (Digest-HMAC claims Digest-SHA1 as a dependancy ; used to speed up hash tests but not absolutely required) ; http://search.cpan.org/~gaas/Digest-SHA1-2.10/ ; The Digest::SHA1 module allows you to use the NIST SHA-1 message digest algorithm from within Perl programs.

mkdir -p /usr/src/CPAN
cd /usr/src/CPAN
# wget http://hostinghacks.net/dist/CPAN/Digest-SHA1-2.10.tar.gz
wget http://search.cpan.org/CPAN/authors/id/G/GA/GAAS/Digest-SHA1-2.10.tar.gz
tar -zxf Digest-SHA1-2.10.tar.gz
cd /usr/src/CPAN/Digest-SHA1-2.10
perl Makefile.PL
make
#make test
make install

Digest-HMAC-1.01 (needed by Net::DNS); http://search.cpan.org/~gaas/Digest-HMAC-1.01/

cd /usr/src/CPAN
# wget http://hostinghacks.net/dist/CPAN/Digest-HMAC-1.01.tar.gz
wget http://search.cpan.org/CPAN/authors/id/G/GA/GAAS/Digest-HMAC-1.01.tar.gz
tar -zxf Digest-HMAC-1.01.tar.gz
cd /usr/src/CPAN/Digest-HMAC-1.01
perl Makefile.PL
make
#make test
make install

Net-DNS-0.48 (needed for RBL lookups) ; http://search.cpan.org/~crein/Net-DNS-0.48/ ; Net::DNS is a collection of Perl modules that act as a Domain Name System (DNS) resolver. It allows the programmer to perform DNS queries that are beyond the capabilities of gethostbyname and gethostbyaddr.

mkdir -p /usr/src/CPAN
cd /usr/src/CPAN
wget http://search.cpan.org/CPAN/authors/id/C/CR/CREIN/Net-DNS-0.48.tar.gz
# wget http://hostinghacks.net/dist/CPAN/Net-DNS-0.48.tar.gz
tar -zxf Net-DNS-0.48.tar.gz
cd /usr/src/CPAN/Net-DNS-0.48

echo "no" > no.txt 
perl Makefile.PL < no.txt

# perl Makefile.PL
# Do you want to enable these tests? [y] n

make
#make test
make install

DB_File (SpamAssassin will work without DB_File, but Bayes support will be disabled) ; http://search.cpan.org/~pmqs/DB_File-1.810/ ; Although DB_File is intended to be used with Berkeley DB version 1, it can also be used with version 2, 3 or 4. In this case the interface is limited to the functionality provided by Berkeley DB 1.x.

mkdir -p /usr/src/CPAN
cd /usr/src/CPAN
wget http://search.cpan.org/CPAN/authors/id/P/PM/PMQS/DB_File-1.810.tar.gz
# wget http://hostinghacks.net/dist/CPAN/DB_File-1.810.tar.gz
tar -zxf DB_File-1.810.tar.gz
cd /usr/src/CPAN/DB_File-1.810  
perl Makefile.PL
make
#make test
make install

ExtUtils-MakeMaker - (Version 6.16 or later is recommended) ; http://search.cpan.org/~mschwern/ExtUtils-MakeMaker-6.24/ ; This utility is designed to write a Makefile for an extension module from a Makefile.PL.

mkdir -p /usr/src/CPAN
cd /usr/src/CPAN
wget http://search.cpan.org/CPAN/authors/id/M/MS/MSCHWERN/ExtUtils-MakeMaker-6.24.tar.gz
# wget http://hostinghacks.net/dist/CPAN/ExtUtils-MakeMaker-6.21.tar.gz
tar -zxf ExtUtils-MakeMaker-6.24.tar.gz
cd /usr/src/CPAN/ExtUtils-MakeMaker-6.24 
perl Makefile.PL
make
#make test
make install

HTML-Tagset-3.04 ; http://search.cpan.org/~sburke/HTML-Tagset-3.04/ ; This module contains several data tables useful in various kinds of HTML parsing operations.

mkdir -p /usr/src/CPAN
cd /usr/src/CPAN
wget http://search.cpan.org/CPAN/authors/id/S/SB/SBURKE/HTML-Tagset-3.04.tar.gz
# wget http://hostinghacks.net/dist/CPAN/HTML-Tagset-3.04.tar.gz
tar -zxf HTML-Tagset-3.04.tar.gz
cd /usr/src/CPAN/HTML-Tagset-3.04
perl Makefile.PL
make
#make test
make install

HTML-Parser-3.41 ; http://search.cpan.org/~gaas/HTML-Parser-3.41/ ; Objects of the HTML::Parser class will recognize markup and separate it from plain text (alias data content) in HTML documents. As different kinds of markup and text are recognized, the corresponding event handlers are invoked.

mkdir -p /usr/src/CPAN
cd /usr/src/CPAN
wget http://search.cpan.org/CPAN/authors/id/G/GA/GAAS/HTML-Parser-3.41.tar.gz
# wget http://hostinghacks.net/dist/CPAN/HTML-Parser-3.41.tar.gz
tar -zxf HTML-Parser-3.41.tar.gz
cd /usr/src/CPAN/HTML-Parser-3.41

echo "no" > no.txt 
perl Makefile.PL < no.txt

# perl Makefile.PL
# Do you want decoding on unicode entities? [no] (???)

make
#make test
make install

IO-stringy-2.109 ; http://search.cpan.org/~eryq/IO-stringy-2.109/ ; This toolkit primarily provides modules for performing both traditional and object-oriented i/o) on things other than normal filehandles; in particular, IO::Scalar, IO::ScalarArray, and IO::Lines.

mkdir -p /usr/src/CPAN
cd /usr/src/CPAN
wget http://search.cpan.org/CPAN/authors/id/E/ER/ERYQ/IO-stringy-2.109.tar.gz
# wget http://hostinghacks.net/dist/CPAN/IO-stringy-2.109.tar.gz
tar -zxf IO-stringy-2.109.tar.gz
cd /usr/src/CPAN/IO-stringy-2.109
perl Makefile.PL
make
#make test
make install

MailTools ; http://search.cpan.org/~markov/MailTools-1.65/ ; Mail::Address Parse mail addresses 1.65 Mail::Cap Parse mailcap files 1.65 Mail::Field Base class for manipulation of mail header fields 1.65 Mail::Field::AddrList object representation of e-mail address lists 1.65 Mail::Field::Date 1.65 Mail::Filter Filter mail through multiple subroutines 1.65 Mail::Header manipulate mail RFC822 compliant headers 1.65 Mail::Internet manipulate Internet format (RFC 822) mail messages 1.65 Mail::Mailer Simple interface to electronic mailing mechanisms

mkdir -p /usr/src/CPAN
cd /usr/src/CPAN
wget http://search.cpan.org/CPAN/authors/id/M/MA/MARKOV/MailTools-1.65.tar.gz
# wget http://hostinghacks.net/dist/CPAN/MailTools-1.65.tar.gz
tar -zxf MailTools-1.65.tar.gz
cd /usr/src/CPAN/MailTools-1.65
perl Makefile.PL
make
#make test
make install

Mail-Audit-2.1 ; http://search.cpan.org/~simon/Mail-Audit-2.1/ ; Mail::Audit allows a piece of email to be logged, examined, accepted into a mailbox, filtered, resent elsewhere, rejected, replied to, and so on. It's designed to easily create filter programs to stick in a .forward file or similar.

mkdir -p /usr/src/CPAN
cd /usr/src/CPAN
wget http://search.cpan.org/CPAN/authors/id/S/SI/SIMON/Mail-Audit-2.1.tar.gz
# wget http://hostinghacks.net/dist/CPAN/Mail-Audit-2.1.tar.gz
tar -zxf Mail-Audit-2.1.tar.gz
cd /usr/src/CPAN/Mail-Audit-2.1 &&
perl Makefile.PL
make
#make test
make install

MIME-tools ; http://search.cpan.org/~dskoll/MIME-tools-5.416/ ; MIME-tools is a collection of Perl5 MIME:: modules for parsing, decoding, and generating single- or multipart (even nested multipart) MIME messages.

mkdir -p /usr/src/CPAN
cd /usr/src/CPAN
wget http://search.cpan.org/CPAN/authors/id/D/DS/DSKOLL/MIME-tools-5.416.tar.gz
# wget http://hostinghacks.net/dist/CPAN/MIME-tools-5.416.tar.gz
tar -zxf MIME-tools-5.416.tar.gz
cd /usr/src/CPAN/MIME-tools-5.416 &&
perl Makefile.PL
make
#make test
make install

Term-ReadLine-Gnu ; http://search.cpan.org/~hayashi/Term-ReadLine-Gnu-1.15/ ; This is an implementation of Term::ReadLine using the GNU Readline/History Library.

mkdir -p /usr/src/CPAN
cd /usr/src/CPAN
wget http://search.cpan.org/CPAN/authors/id/H/HA/HAYASHI/Term-ReadLine-Gnu-1.15.tar.gz
# wget http://hostinghacks.net/dist/CPAN/Term-ReadLine-Gnu-1.15.tar.gz
tar -zxf Term-ReadLine-Gnu-1.15.tar.gz
cd /usr/src/CPAN/Term-ReadLine-Gnu-1.15
perl Makefile.PL
make
#make test
make install

Term-ReadLine-Perl http://search.cpan.org/dist/Term-ReadLine-Perl/ ??????????????????????/ Perl 5 module for Term::ReadLine editing without GNU readline Perl 5 ships with a module called Term::ReadLine which is an interface to command line editing and recall. The version that ships with Perl is only a stub, and offers little functionality. This module supplants the Term::ReadLine stubs with real command line editing and recall facilities, written entirely in Perl. Applications that use Term::ReadLine do not need to be modified to gain the benefits of this package; it will happen transparently upon installation.

mkdir -p /usr/src/CPAN
cd /usr/src/CPAN
# wget http://hostinghacks.net/dist/CPAN/Term-ReadLine-Perl-1.0203.tar.gz
wget http://search.cpan.org/CPAN/authors/id/I/IL/ILYAZ/modules/Term-ReadLine-Perl-1.0203.tar.gz
tar -zxf Term-ReadLine-Perl-1.0203.tar.gz
cd /usr/src/CPAN/Term-ReadLine-Perl-1.0203
perl Makefile.PL
make
#make test
make install

Term::ReadKey - A perl module for simple terminal control ; http://search.cpan.org/~jstowe/TermReadKey-2.30/

mkdir -p /usr/src/CPAN
cd /usr/src/CPAN
wget http://search.cpan.org/CPAN/authors/id/J/JS/JSTOWE/TermReadKey-2.30.tar.gz
# wget http://hostinghacks.net/dist/CPAN/TermReadKey-2.30.tar.gz
tar -zxf TermReadKey-2.30.tar.gz
cd /usr/src/CPAN/TermReadKey-2.30
perl Makefile.PL
make
#make test
make install

libnet-1.1 (may already be installed!!!) ; http://search.cpan.org/~gbarr/libnet-1.19/ ; libnet is a collection of perl5 modules which all related to network programming. The majority of the modules available provided the client side of popular server-client protocols that are used in the internet community.

mkdir -p /usr/src/CPAN
cd /usr/src/CPAN
# wget http://search.cpan.org/CPAN/authors/id/G/GB/GBARR/libnet-1.19.tar.gz
wget http://hostinghacks.net/dist/CPAN/libnet-1.19.tar.gz
tar -zxf libnet-1.19.tar.gz
cd /usr/src/CPAN/libnet-1.19

# perl Makefile.PL
# Ah, I see you already have installed libnet before.
# Do you want to modify/update your configuration (y|n) ? [no] y

echo "yes" > yes.txt
perl Makefile.PL < yes.txt
make
#make test
make install

Mail::SpamAssassin ; http://search.cpan.org/~felicity/Mail-SpamAssassin-3.0.2/

mkdir -p /usr/src/CPAN
cd /usr/src/CPAN
wget http://search.cpan.org/CPAN/authors/id/F/FE/FELICITY/Mail-SpamAssassin-3.0.2.tar.gz
# wget http://hostinghacks.net/dist/CPAN/Mail-SpamAssassin-3.0.2.tar.gz
tar -zxf Mail-SpamAssassin-3.0.2.tar.gz
cd /usr/src/CPAN/Mail-SpamAssassin-3.0.2
# perl Makefile.PL
LANG=C perl Makefile.PL

# insert e-mail here:

make
#make test
make install

spamassassin config file:

mkdir -p /etc/mail/spamassassin
mkdir -p /var/run/spamd 
mkdir -p /home/spamd

groupadd spamd
useradd -s /bin/nologin -d /home/spamd -g spamd -c "spamd user" spamd
chown spamd /etc/mail/spamassassin -R

chown -R spamd:spamd /var/run/spamd 

cat > /etc/mail/spamassassin/local.cf << "EOF"

# See 'perldoc Mail::SpamAssassin::Conf' for configuration details

required_hits           5.0
rewrite_header Subject  [SPAM?]
report_safe             1
use_bayes               1
blacklist_from          blacklist@localhost
skip_rbl_checks         1
#skip_rbl_checks        0
#use_razor2             1
#use_dcc                1
#use_pyzor              1
ok_languages            en
ok_locales              en
report_contact          postmaster@localhost

### Bayesian settings 
use_bayes 1 
bayes_auto_learn 1 
# bayes_path /var/lib/spamassassin/ 
bayes_file_mode 0666 
bayes_expiry_max_db_size 1500000 

### Tests to use 
skip_rbl_checks 0 
#use_razor2 1 
#use_pyzor 1 
#use_dcc 1 

dns_available yes 

# add_header all DCC _DCCB_: _DCCR_ 

### Don't flag local mail 
header LOCAL_RCVD Received =~ /\S+\.hostinghacks\.net\s+\(.*\[.*\]\)/ 
describe LOCAL_RCVD Received from local machine 
score LOCAL_RCVD -50 
trusted_networks 127/8 

EOF

# chmod /etc/mail/spamassassin

spamassassin init script:

cat > /etc/rc.d/rc.spamd << "EOF"
#!/bin/sh 
# Spamd init script for Slackware 10
# Martin Ostlund, nomicon  2003
# Modified by Troy Belding for Qmailrocks 2004
# Edited by boneman for Hostinghacks.net 2005

PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin 
DAEMON=/usr/bin/spamd 
NAME=spamd 
SNAME=rc.spamd 
DESC="SpamAssassin Mail Filter Daemon" 
PIDFILE="/var/run/spamd/$NAME.pid" 
PNAME="spamd" 

DOPTIONS="-x -u spamd -H /home/spamd -d --pidfile=$PIDFILE" 

KILL="/bin/killall spamd" 
KILLALL="/bin/killall" 
# Defaults - don't touch, edit /etc/mail/spamassassin/local.cf 
ENABLED=0 
OPTIONS="" 

set -e 

case "$1" in 
start) 
echo -n "Starting $DESC: " 
$PNAME $OPTIONS $DOPTIONS 

echo "$NAME." 
;; 
stop) 
echo -n "Stopping $DESC: " 
$KILL -9 `cat $PIDFILE` 
/bin/rm $PIDFILE 
echo "$NAME." 
;; 
restart|force-reload) 
echo -n "Restarting $DESC: " 
$0 stop 
$0 start 

echo "$NAME." 
;; 
*) 
ME=/etc/rc.d/$SNAME 
echo "Usage: $ME {start|stop|restart|force-reload}" >&2 
exit 1 
;; 
esac 

exit 0 

EOF

chmod 755 /etc/rc.d/rc.spamd 
/etc/rc.d/rc.spamd start

echo "/etc/rc.d/rc.spamd start" >> /etc/rc.d/rc.local

Set procmail to pass mail to Spamassassin:

cat > /etc/procmailrc << "EOF"

#DROPPRIVS=yes
#Uncomment the following lines to allow for logging
#
LOGFILE=/var/log/procmail
VERBOSE=ON

# :0 indicates a recipe. 'fw' indicates a filter.
:0fw
# send messages to Spamassassin if they are less than 256000 bytes. 
* < 256000
# original message is replaced with SpamAssassin's output
# '-f' = return the original message if it can't connect to spamd. 
| /usr/bin/spamc -f

#functional but less efficient >>> | /usr/bin/spamassassin -P

:0e
{
EXITCODE=$?
}  

EOF

touch /var/log/procmail

Distributed Checksum Clearinghouse : the DCC or Distributed Checksum Clearinghouse is a system of millions of users, tens of thousands of clients and more than 250 servers collecting and counting checksums related to more than 150 million mail messages on week days. The counts can be used by SMTP servers and mail user agents to detect and reject or filter spam or unsolicited bulk mail. A DCC server accumulates counts of cryptographically secure checksums of messages but not the messages themselves. DCC clients send reports of checksums related to incoming mail to a nearby DCC server. | http://www.rhyolite.com/anti-spam/dcc/source/

mkdir -p /usr/src
cd /usr/src
wget http://www.rhyolite.com/anti-spam/dcc/source/dcc-dccd-1.2.74.tar.Z
# wget http://hostinghacks.net/dist/dcc-dccd-1.2.74.tar.Z 

gunzip dcc-dccd-1.2.74.tar.Z
tar -xf dcc-dccd-1.2.74.tar
cd dcc-dccd-1.2.74

./configure \
--bindir=/usr/bin \
--mandir=/usr/man

make && make install

The way DCC is called by SpamAssassin requires /bin/sh in order to work properly. You should be aware that this requirement reduces the overall security of the system. Without /bin/sh, you'll receive this error: DCC -> check failed: no response . If you are willing to accept this risk then make the following edit to /etc/passwd:

bin:x:1:1:bin:/bin:/bin/sh

finally : add to spamassassin conf >>> use_dcc 1

DCC traffic is like DNS traffic. You should treat port 6277 like port 53. Allow outgoing packets to distant UDP port 6277 and incoming packets from distant UDP port 6277. | To keep a server's database of checksums from growing without bound, checksums are forgotten when they become old. Checksums with large totals are kept longer. See dbclean(8). | DCC clients pick the nearest working DCC server using a small shared or memory mapped file, /var/dcc/map. It contains server names, port num- bers, passwords, recent performance measures, and so forth.

once everything is set up correctly the following command should generate some useful information:

/usr/bin/cdcc 'info'

manually starting spamd:

spamd -x -u spamd -H /home/spamd -d --pidfile=/var/run/spamd/spamd.pid
/usr/bin/dccproc -H -R

check for problems in the config file:

 spamassassin --lint

list all perl CPAN modules installed:

perl -MCPAN -e 'install ExtUtils::Installed'
perl -MCPAN -e 'install Net::DNS'

cat > listpm.pl << "EOF"
#!/usr/bin/perl
use strict;
use ExtUtils::Installed; 

my $installed = ExtUtils::Installed->new(); 
foreach my $module ( grep( !/^Perl$/, $installed->modules() ) ) 
{ 
    my $version = $installed->version( $module ) || "---"; 
    print( "$module|" ); 
}
EOF

chmod 755 listpm.pl
./listpm.pl

http://wiki.apache.org/spamassassin/BayesFaq

http://surbl.org

spamassassin-users-subscribe at incubator.apache.org

SA needs a minimum of 200 ham and 200 spam messages to
enable bayesian filtering.

The lines 
bayes_min_ham_num       5
bayes_min_spam_num      20
are ignore in 2.55

After feeding an amount of ham and spam into it it works. Can be watched
by using the -D switch when starting SA

thanks
Alexander Topolanek

>    If I remember correctly (verify this with the SpamAssassin 
> docs), Bayesian filtering won't happen until you have a large 
> enough corpus, and I think that corpus was 300 messages.  So,

This is the GTUBE, the
	Generic
	Test for
	Unsolicited
	Bulk
	Email

If your spam filter supports it, the GTUBE provides a test by which you
can verify that the filter is installed correctly and is detecting incoming
spam. You can send yourself a test mail containing the following string of
characters (in upper case and with no white spaces and line breaks):

XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X

You should send this test mail from an account outside of your network.

To test from the command line:

spamassassin -D < /usr/share/doc/spamc/sample-spam.txt

...where /usr/share/doc/spamc/sample-spam.txt is the path to a sample
email that you want to test. Read the resulting debug text and you will
have a better idea of how SA is running checks.


# [option: add -DSPAMC_SSL to $CFLAGS to build an SSL-enabled spamc]
# Test
# spamassassin -t < sample-nonspam.txt > nonspam.out
# spamassassin -t < sample-spam.txt > spam.out

# In the header of spam.out find a spam indicator

# http://wiki.apache.org/spamassassin
# http://wiki.apache.org/spamassassin/UsedViaProcmail

SpamAssassin can directly report your spam to the three HashSharingSystem 
it supports: UsingRazor, UsingPyzor, UsingDcc  

To report a message, enter: 

spamassassin -r < message.txt

Any SpamAssassin markup will automatically be removed (i.e., the -d option is implicitly on). 

The message will also be submitted to SpamAssassin's learning systems; 
currently this is the internal Bayesian statistical-filtering system 
(BayesInSpamAssassin). Note that if you only want to perform statistical 
learning, and do not want to report mail to a third-party server, 
you should use the sa-learn command directly instead.

1. How can I tell what SpamAssassin is doing / if 
a SpamAssassin feature is enabled?
The first thing to do is to run with debugging on. 
For "spamassassin" and "spamd", add a "-D" parameter. 
spamassassin -tD < [message_file] This will usually 
tell you what you want to know. 

The best messages to use for this test are the samples 
included with the distribution, sample-spam.txt and sample-nonspam.txt.