SRS integration with sendmail

Scope

This document describes a procedure to integrate the SRS (Sender Rewriting Scheme) into sendmail. It is targeted at system administrators and/or mail operators, and assumes a workable knowledge of sendmail administration and access to the sendmail configuration files.

Purpose

The purpose of this project is to integrate SRS into sendmail in such a way, that ALL outgoing envelope-from addresses are SRS signed; and, conversely, that incoming envelope-from addresses are SRS validated. The two put together yield an air-tight basis for eliminating fake "bounces".

N.B. This project describes making changes to sendmail.cf manually. Everything you need, however, is also available in this m4. Copy perlsrs.m4 to your ./cf/hack directory, and add "HACK(perlsrs)dnl" to your local .mc file.

If you are using sendmail 8.13.x, you can use the socketmap approach. The socketmap route is the preferred method of implementation. The "program" map method is now DEPRECATED. It will remain part of this document to demonstrate proof of concept, and to fulfill the needs of those who have not upgraded their sendmail yet; but the program map approach will not be developed any further.

The Problem

Consider the following SMTP dialogue:


Friend or foe?

>>> MAIL From:<>
<<< 250 2.1.0 <>... Sender ok
>>> RCPT To:<admin@asarian-host.net>

Well, friend or foe? Ay, there's the rub: without SRS, there is no way of knowing.

To understand where SRS comes in, we need to understand that a legitimate DSN message takes its single (!) recipient from the envelope-from of the message it seeks to bounce: what goes around, comes around. Hence, if we send mail, using an SRS signed envelope-from, then a DSN will show this same, signed, address as recipient upon return:


Sending MTA received undeliverable message with this envelope-from:

<SRS0=+fZygoFE=5I=asarian-host.net=admin@asarian-host.net>

And now initiates a bounce dialogue:

>>> MAIL From:<>
<<< 250 2.1.0 <>... Sender ok
>>> RCPT To:<SRS0=+fZygoFE=5I=asarian-host.net=admin@asarian-host.net>

This provides the basis to make the determination about the validity of the bounce message. At this point in the communication we validate the SRS signature.

Valid SRS address? PASS
Invalid SRS signature? FAIL

A tempting, but nonetheless RISKY, next step, is to also, immediately, conclude the following:


>>> MAIL From:<>
<<< 250 2.1.0 <>... Sender ok
>>> RCPT To:<admin@asarian-host.net>
<<< 550 5.1.1 <admin@asarian-host.net>... Bounce address not SRS signed!

That is the general idea, of course. :) But, at this stage, it may be too soon too judge. Why? Because of what, in and by themselves, could be considered perfectly legit SMTP probes of this kind:


Heya, are you for real?

>>> MAIL From:<>
<<< 250 2.1.0 <>... Sender ok
>>> RCPT To:<admin@asarian-host.net>
<<< 250 2.1.5 <admin@asarian-host.net>... Recipient ok
>>> RSET
<<< 250 2.0.0 Reset state
>>> QUIT
<<< 221 2.0.0 asarian-host.net closing connection

A certified bounce, in contradistinction to a probe of this kind, would have for recipient not my regular email address, but the SRS signed envelope-from. Yet this is NOT an unsigned bounce which must be rejected, but simply a valid SMTP probe!

The Solution

How to tell the difference then? Fortunately, the practical solution is rather simple. Since a regular SMTP 'callback' probe does NOT enter the DATA phase (why would it?), all we do at this point, is mark the transaction as suspect, and wait for a possible DATA phase; only THEN we reject, should the recipient be unsigned (*). This solution does not require a tricky scrutiny of the message DATA, and is solely based on the "logic" of how regular SMTP probes operate.

(*) Rejecting at the DATA phase (mind you, not at the data COMMAND), for all practical purpose and intent, just means you will reject at the first available opportunity; in a Milter, this would be as early as the header_callback. The DATA phase would still be completed, but the message simply not be accepted for delivery.

To recapitulate:

Empty envelope-from (<>) + DATA phase = message for which we require a cryptographically signed SRS recipient;
Empty envelope-from (<>) - DATA phase = regular SMTP probe; no cryptographically signed SRS recipient required.

Like so:


>>> MAIL from: <>
<<< 250 2.1.0 <>... Sender ok
>>> RCPT To:<admin@asarian-host.net>
<<< 250 2.1.5 <admin@asarian-host.net>... Recipient ok
>>> DATA
<<< 554 5.7.1 Bounce address not SRS signed!

That is why this method does not require a cumbersome scrutiny of the DATA to determine whether we are dealing with a legitimate bounce: because the very tripartite condition of having,

An empty envelope-from,
AND a DATA phase,
AND a cryptographically unsigned recipient,

Is itself enough to REJECT the message as a forged bounce.

N.B. This document follows a more conservative approach than what I eventually implemented and went with. Empirical data over several months showed that hardly anyone, if at all, makes a legitimate CBV with an empty envelope-from to an unsigned recipient. Nor is there any good reason they should, really. If people per se must check the validity of my unsigned address, then they can do so with a non-zero envelope-from.

The cost of wasting all that extra bandwidth (always a scarce commodity) to wait out the whole DATA phase, just to accommodate the odd broken client, just did not add up. And I say "broken" with good reason, because a real DSN takes for recipient the address it received as envelope-from in the SMTP dialogue. That there are folks out there who think it a good idea to "fake" a DSN (for whatever purpose), and use for recipient something which did NOT come out of an original SMTP dialogue, is really, ultimately, their problem.

There is always the risk of blocking mail if you REJECT every unsigned recipient in a DSN, prior to the DATA phase. This document will err on the side of caution, therefore. But I would not hold it against you if you did the same thing I do now. :)

Implementation

Integrating SRS into sendmail consists of two steps;

The signing of outgoing envelope-from addresses,
Validating SRS signatures on incoming traffic.

Signing outgoing envelope-from addresses

What is the best way to have sendmail sign outgoing envelope-from addresses? That answer will vary, depending on who you ask. Some will use .forward files, containing a piped SRS script:


Taking the .forward route:

.forward before: final@destination.com
.forward after:  "| /usr/bin/srs --secret=/etc/srs.secret final@destination.com"

The most immediate drawback of this method is, of course, obvious: you can only use SRS on forwarding. :) Though SRS was actually originally designed for that precise purpose, the goal of this project, however, is to sign ALL outgoing envelope-from addresses.

If your server configuration looks anything like mine, you will have plenty of piped aliases and/or CGI scripts, autoresponders, and other assorted places, where mail is being sent out. It would be thoroughly undoable, impossible even in many cases, to modify all these scripts to start sending out SRS signed envelope-from addresses. So, instead, we look at sendmail itself, and make an "incision" at a carefully targeted location in its configuration file and insert our SRS signing tool.

The method I have chosen does not involve .forward files or aliases. Nor does it expose any SRS secrets. Instead, I use a single Ruleset, invoked from within sendmail.cf, at the precise moment sendmail is about to sign off on the envelope-from of an outgoing message; ANY outgoing message. Where does sendmail do that? At EnvFromSMTP, which will be defined as the S= entry for the (E)SMTP mailer:


Where I, sendmail, set the envelope-from:

Mesmtp, P=[IPC], F=mDFMuXa, S=EnvFromSMTP/HdrFromSMTP, R=EnvToSMTP, E=\r\n, L=990,
        T=DNS/RFC822/SMTP,
        A=TCP $h

Let us have a look:


#
#  envelope sender rewriting
#

SEnvFromSMTP
R$+                     $: $>PseudoToReal $1    sender/recipient common
R$* :; <@>              $@                      list:; special case
R$*                     $: $>MasqSMTP $1        qualify unqual'ed names
R$+                     $: $>MasqEnv $1         do masquerading

m We add a single line:


SEnvFromSMTP
R$+                     $: $>PseudoToReal $1    sender/recipient common
R$* :; <@>              $@                      list:; special case
R$*                     $: $>MasqSMTP $1        qualify unqual'ed names
R$+                     $: $>MasqEnv $1         do masquerading
R$*                     $: $(make_srs $1 $)

x Is sendmail bulking? Remember: the whitespace between R$* and $ is NOT filled with spaces, but with TAB characters!

m Now we define a map, of type "program", that will do the actual SRS translation:


##################
#   local info   #
##################

....

# SRS program map

Kmake_srs program /etc/scripts/env2srs.pl

In this example, I chose a Perl "program" map. Perl has a start-up performance issue, of course; which is why you should take the socketmap route.

As you can see, map "make_srs" is called with only one parameter: the original envelope-from address; and it returns only one parameter: the new SRS address.

m Now, the env2srs script itself. It is pretty basic; it performs standard SRS0 rewriting, and comes with some sanity-checks. You should examine and configure/modify this script, at least to set $secret (or have it retrieved from file, etc), and adjust the $srs object parameters according to your server's policy.


#!/usr/local/bin/perl
#
# Sendmail "program" map script to rewrite envelope-from
# address to SRS0 address. Called from macro EnvFromSMTP.
#
# Code by Mark Kramer <admin@asarian-host.net>
#
# Version 0.30
#
# Last revision: March 24, 2004
#
# Licensed under GPL
#
# For detailed installation notes, read:
#
#           http://srs-socketmap.info/sendmailsrs.htm
#
# See also: http://www.libsrs2.org/
#           http://spf.pobox.com/
#
# This version requires at least Sendmail 8.12.10 + Mail::SRS 0.30


use Mail::SRS;
use strict;

# No funny business in our output, please

close (STDERR);

my $old_address = $ARGV[0];
my $secret = 'whateverfloatsyourboat';
my ($new_address, $use_address);
my $fwdomain = 'mydomain.com';
my $srs = new Mail::SRS (Secret => $secret, HashLength => 8, AlwaysRewrite => 1);

# Our original envelope-from may look funny on entry
# of this Ruleset:
#
#     admin<@asarian-host.net.>
#
# We need to preprocess it some:

   ($use_address = $old_address) =~ s/[<>]//g;
    $use_address =~ s/\.$//g;

# Here, at EnvFromSMTP, we do not loop our address through an
# extra IsSrs macro: we want SRS1 forwarding functionality!
# (relaying reversed third-party SRS1 addresses is a
# different story, though; but here we just allow for SRS0
# addresses to be promoted to SRS1 ones).
#
# Ok, first check whether we already have a signed SRS address;
# if so, just return the old address: we do not want to double-sign
# by accident! (Non-locally generated SRS0 addresses, by nature
# of the protocol, will not 'eval'; so, they will simply become
# SRS1 addresses. Thus, only locally generated SRS0 addresses are
# exempted from double-signing.)
#
# Else, gimme a valid SRS signed address, munge it back the way
# sendmail wants it at this point; or just return the old address,
# in case nothing went.

if (eval {$new_address = $srs -> reverse ($use_address)}) {
    print "$old_address\n";
} elsif (eval {$new_address = $srs -> forward ($use_address, $fwdomain)}) {
    $new_address .= '.>';
    $new_address =~ s/\@/<@/;
    print "$new_address\n";
} else {
    print "$old_address\n";
}

exit 0;

m Restart sendmail, and TEST your configuration. I recommend using running "sendmail -bt" (address test mode. This mode reads addresses and shows the steps in parsing; it is used for debugging configuration tables):


asarian-host: {root} % sendmail -bt
ADDRESS TEST MODE (ruleset 3 NOT automatically invoked)
Enter <ruleset> <address>
> /tryflags SE
> /try esmtp admin@asarian-host.net
Trying envelope sender address admin@asarian-host.net for mailer esmtp
canonify       input: admin @ asarian-host . net
Canonify2      input: admin < @ asarian-host . net >
Canonify2    returns: admin < @ asarian-host . net . >
canonify     returns: admin < @ asarian-host . net . >
1              input: admin < @ asarian-host . net . >
1            returns: admin < @ asarian-host . net . >
EnvFromSMTP    input: admin < @ asarian-host . net . >
PseudoToReal   input: admin < @ asarian-host . net . >
PseudoToReal returns: admin < @ asarian-host . net . >
MasqSMTP       input: admin < @ asarian-host . net . >
MasqSMTP     returns: admin < @ asarian-host . net . >
MasqEnv        input: admin < @ asarian-host . net . >
MasqEnv      returns: admin < @ asarian-host . net . >
EnvFromSMTP  returns: SRS0 + Q1Ju93UT=FT=asarian-host . net=admin < @ asarian-host . net . >
final          input: SRS0 + Q1Ju93UT=FT=asarian-host . net=admin < @ asarian-host . net . >
final        returns: SRS0 + Q1Ju93UT=FT=asarian-host . net=admin @ asarian-host . net
Rcode = 0, addr = SRS0+Q1Ju93UT=FT=asarian-host.net=admin@asarian-host.net

x If you did NOT get the desired result, CHECK your logs, and make sure you have a safe path to the SRS program!


Hi, I am sendmail, complaining about your poor administrative skills:

Feb 20 05:35:21 asarian-host sendmail[4824]: i1K4ZLcV004821: Warning:
prog_open: program /etc/scripts/env2srs.pl unsafe: World writable directory

Validating/converting SRS addresses

Without a proper reversal track, sending mail to an SRS signed, local recipient, either yields an "Unknown user" error, assuming you have no "catchall" user defined, or leads to a single "controlling user", instead of the intended recipient. Therefore, the path to having sendmail validate/reverse SRS addresses is remarkably akin to the procedure we followed for signing them. We look for an entry-point in the configuration file, embed our SRS validation/reversal tool, convert the address if possible/valid (!), and return it to sendmail for further processing.

I have chosen Ruleset 98, ParseLocal, in sendmail.cf, as location to insert our SRS reversal tool:


###################################################################
###  Ruleset 98 -- local part of ruleset zero (can be null)     ###
###################################################################

SParseLocal=98

# addresses sent to foo@host.REDIRECT will give a 551 error code

R$* < @ $+ .REDIRECT. >         $: $1 < @ $2 . REDIRECT . > < ${opMode} >
R$* < @ $+ .REDIRECT. > <i>     $: $1 < @ $2 . REDIRECT. >
R$* < @ $+ .REDIRECT. > < $- >  $#error $@ 5.1.1 $: "551 User has moved; ...

m We add two local macros, and a final reversal line:


###################################################################
###  Local SRS Macros (IsSrs macro, courtesy of Alain Knaff)    ###
###################################################################

SIsSrs
R$*                            $: $(is_srs $1 $)
R$@                            $@ YES
R$*                            $@ NO

SReverseSrs
R$*                            $: $1 $>IsSrs $1
R$* NO                         $@ $1
R$* YES                        $@ $(reverse_srs $1 $)

###################################################################
###  Ruleset 98 -- local part of ruleset zero (can be null)     ###
###################################################################

SParseLocal=98

# Do we need to reverse SRS address?

R$*                             $: $>ReverseSrs $1

# addresses sent to foo@host.REDIRECT will give a 551 error code

R$* < @ $+ .REDIRECT. >         $: $1 < @ $2 . REDIRECT . > < ${opMode} >
R$* < @ $+ .REDIRECT. > <i>     $: $1 < @ $2 . REDIRECT. >
R$* < @ $+ .REDIRECT. > < $- >  $#error $@ 5.1.1 $: "551 User has moved; ...

x Is sendmail bulking? Remember: the whitespace between R$* and $ is NOT filled with spaces, but with TAB characters!

m Now we define two maps; one of type "program", that will do the actual SRS translation, and one of type "regex", that we use as a quick early-out to avoid having to call the srs2env.pl script.


##################
#   local info   #
##################

....

# SRS program map

Kreverse_srs program /etc/scripts/srs2env.pl

# SRS regex map

Kis_srs regex ^<?SRS[01][-+=].*

m The srs2env script is slightly simpler than its counterpart, as all it needs to do is reverse an SRS address. You should examine and configure/modify this script, at least to set $secret (or have it retrieved from file, etc), and adjust the $srs object parameters according to your server's policy.


#!/usr/local/bin/perl
#
# Sendmail "program" map script to revert SRS0 or SRS1 address
# back to regular recipient. Called from macro ParseLocal.
#
# Code by Mark Kramer <admin@asarian-host.net>
#
# Version 0.30
#
# Last revision: March 24, 2004
#
# Licensed under GPL
#
# For detailed installation notes, read:
#
#           http://srs-socketmap.info/sendmailsrs.htm
#
# See also: http://www.libsrs2.org/
#           http://spf.pobox.com/
#
# This version requires at least Sendmail 8.12.10 + Mail::SRS 0.30


use Mail::SRS;
use strict;

# No funny business in our output, please

close (STDERR);

my $old_address = $ARGV[0];
my $secret = 'whateverfloatsyourboat';
my $use_address;
my $srs = new Mail::SRS (Secret => $secret, HashLength => 8, AlwaysRewrite => 1);

# Munge ParseLocal recipient in the same manner as required
# in EnvFromSMTP.

   ($use_address = $old_address) =~ s/[<>]//g;
    $use_address =~ s/\.$//g;

# Just try and reverse the address. If we succeed, return this
# new address; else, return the old address (quoted if it was
# a piped alias).
#
# We do an exhaustive while loop, so that SRS1 address may
# become SRS0, which, in turn, may become reverted to
# a local recipient.
#
# Mail:SRS, as of 0.30, is now case-insensitive. Added the
# /i switch to accomodate for the change.

if ($use_address =~ /^SRS[01][-+=]/i) {
    $use_address = $_ while (eval {$_ = $srs -> reverse ($use_address)});
    $use_address .= '.>';
    $use_address =~ s/\@/<@/;
    print "$use_address\n";
} elsif ($use_address =~ /^\|/) {
    print "\"$old_address\"\n";
} else {
    print "$old_address\n";
}

exit 0;

m Restart sendmail, and TEST your configuration. I recommend using running "sendmail -bt" again. We use Ruleset 0, this time:


asarian-host: {root} % sendmail -bt
ADDRESS TEST MODE (ruleset 3 NOT automatically invoked)
Enter <ruleset> <address>
> 0 SRS0=s0jfOihW=FU=forward.com=contact@forward.com
parse        input: SRS0=s0jfOihW=FU=forward . com=contact @ forward . com
Parse0       input: SRS0=s0jfOihW=FU=forward . com=contact @ forward . com
Parse0     returns: SRS0=s0jfOihW=FU=forward . com=contact @ forward . com
ParseLocal   input: SRS0=s0jfOihW=FU=forward . com=contact @ forward . com
ParseLocal returns: contact < @ forward . com . >
Parse1       input: contact < @ forward . com . >
Recurse      input: johndoe @ asarian-host . net
canonify     input: johndoe @ asarian-host . net
Canonify2    input: johndoe < @ asarian-host . net >
Canonify2   returns: johndoe < @ asarian-host . net . >
canonify    returns: johndoe < @ asarian-host . net . >
parse         input: johndoe < @ asarian-host . net . >
Parse0        input: johndoe < @ asarian-host . net . >
Parse0      returns: johndoe < @ asarian-host . net . >
ParseLocal   input: johndoe < @ asarian-host . net . >
ParseLocal returns: johndoe < @ asarian-host . net . >
Parse1       input: johndoe < @ asarian-host . net . >
Parse1     returns: $# local $: johndoe
parse      returns: $# local $: johndoe
Recurse    returns: $# local $: johndoe
Parse1     returns: $# local $: johndoe
parse      returns: $# local $: johndoe

Behold, after a few more passes, we see the address changing to a different local user. That is because, in our example, "contact@forward.com" was an entry in a virtusertable. In this dialogue we witness the correct stage at which our SRS reversal tool intervenes. Sendmail is now able to make proper local delivery.

N.B. If you have a Milter running, doing front-end SRS validation for you, instead of the back-end SRS sendmail integration outlined in this document, then sendmail itself must still be able to accept SRS signed addresses. Otherwise, even if the Milter would "SMFIS_CONTINUE" the SRS address (at envrcpt_callback), upon returning control to sendmail, where the recipient "falls back" to sendmail for further processing, the recipient would then still bounce with an "Unknown user" error, despite your Milter status-code. This caveat can be elegantly solved, using sendmail's "plussed users" facility, where you create a dummy alias like this:

SRS0+*: dummy@yourdomain.com

Which, in principle, makes all SRS0+ addresses valid to sendmail (rejection would occur at the Milter). Then you set the SRS 'separator' to the "+" sign.

Mind you, this is NOT necessary if you follow this document's way of integrating SRS into sendmail.

m You're done! :)

Summary

The beauty of this approach is, that it requires NO Milter. Being integrated into sendmail.cf, you have now a stand-alone sendmail, that will SRS-sign outgoing envelope-from addresses, and that can, on its own, reverse/validate these signed addresses. You have often seen me use "reverse/validate" in combo. And this, of course, because they are leaves on the same tree: if sendmail is asked to make delivery to an SRS signed envelope recipient, and the reversal at ParseLocal fails, then sendmail will simply REJECT this recipient, because the unreversed user does not exist on the system! Thus, since only valid SRS addresses can be reversed, we have now a built-in validation mechanism.

N.B. Needless to say, if you have defined "catchall" users, you can, of course, not rely on "User unknown" bounces, as your "catchall" made them all go away. IMHO, having "catchall" users is a Bad Idea anyway.

You could certainly put a front-end Milter at the gate. Especially so, if you wish to implement the above outlined "empty envelope-from + no SRS signed recipient + DATA phase" anti-spammer policy. But that falls outside the scope of this document, which is to integrate SRS functionality into sendmail.

Acknowledgments

Shevek, the author of SRS, deserves full credit for the Perl implementation of Mail::SRS. He has been an instrumental force in forwarding the SRS cause, and inspired me to embark on this project.

Disclaimer

This document comes with no guarantees, express or implied.

Mark <admin@asarian-host.net>