Sender Rewriting Scheme
Encyclopedia
Sender Rewriting Scheme (SRS) is a technique to re-mail an email message so that eventual Delivery Status Notifications can reach the original message sender. In this context, re-mailing is an alternative to Email forwarding
, which is not allowed by the Sender Policy Framework
.
added their host name to the reverse path. In
the Simple Mail Transfer Protocol
(SMTP) this
reverse path is also known as MAIL FROM, but
paths were also used before and outside of SMTP,
e.g. as bang paths in UUCP
and Usenet
(NetNews). All news articles still contain a
Path header, example:
Path: news.server.example!other.example!not-for-mail
The same information in an RFC 5321 e-mail envelope -
that is the SMTP info like MAIL FROM - would be:
The 1st step reflects the sender, the 2nd step the
next MTA, etc. In this example let's assume that
the 2nd MTA forwards the mail to a 3rd MTA, where
it is finally delivered. The final MTA is also
known as Mail delivery agent
(MDA), putting
the mail into the mailbox of the recipient. The
MDA transforms the reverse path into the known
Return-Path header field:
Return-Path:<@news.server.example:not-for-mail@other.example>
SMTP uses MX record
s for its forward routing.
Explicit source routes as in…
RCPT TO:<@news.server.example:user@destination.example>
…to route mail from other.example
via MTA news.server.example
to MDA destination.example were
cumbersome. To make things worse sometimes the
new (1982) style of addresses was mixed with
old UUCP bang paths in constructs like...
destination.example!user@news.server.example
other.example!not-for-mail@news.server.example
...and various other kludges. SMTP and MX records
made all this essentially useless. Therefore
source routing was deprecated 1989 in RFC 1123.
One special case in RFC 1123 are gateways from or
to other networks like UUCP and NetNews, where the
first sending MTA cannot reach the final receiver
directly with
TCP
. It is
solved by MX records and if necessary rewriting
foreign addresses at the gateway. MX is an
acronym for Mail eXchanger.
Another special case are
mailing list
s, where
the list server rewrites all reverse paths to
its own error handling address for
bounces
(error messages) by
recipients. The list server could automatically
unsubscribe bouncing recipients. This type of
address rewriting is known since RFC 821 and still
used today ( RFC 5321, as well as RFC 2821, updated the SMTP chapter
in RFC 1123 ).
Last but not least forwarding to another address
always worked by rewriting the address in the
forward path also known as RCPT TO, if and
only if the forwarding MTA accepted the
responsibility for both forwarding the mail and
returning potential bounce messages
to the sender. RFC 821 and all later SMTP
specifications offer two result codes for this
situation:
251 user not local (attempted forward)
551 user not local (mail rejected)
For privacy reasons these result codes are today
rarely used; they include the forwarded to (251) or
not forwarded to (551) address. But the meaning
and the effect of forwarding to third parties is
identical for result code 250 and error code 550
respectively.
As noted RFC 1123 deprecated source routing, that
implicitly also deprecated the reverse routing of
bounces. It was a relatively
small Internet back in 1989, mail admins
(postmasters) often knew each other and fixed problems
on the fly. Routing bounce messages
back via any forwarders was a waste of time and
bandwidth if the MTA noting a problem (e.g. a
rejection with a 5xx error code) could send the
error message directly back to the MX of sender.
Since RFC 1123 forwarders to third parties still
rewrote the RCPT TO address, but kept the MAIL
FROM as is. As a side effect MTAs wishing to
accept mail from forwarders generally accept any
MAIL FROM address.
More than a decade later spammers
started to abuse this flaw in post-1123 SMTP,
today most mails are spam
and
most reverse paths are forged. Note that
spammers
typically forge
working reverse paths, many MTAs reject mail
if callback verification
or other
plausibility checks fail for the reverse path.
RFC 5321, as well as RFC 2821, states that non-delivery reports ( bounces
) must be sent to the originator as indicated in the reverse path after an MTA accepted the responsibility for delivery. However, the bounce message may be suppressed when the original content is hostile (cf. spam or virus mail) or the message is forged (RFC 5321, Section 6). Note that all current forgery detection methods require the mailbox owner to supply information for them to work. Failing to supply the criteria should not make any bounce message classifiable as backscatter
, although some people mistakenly think it should.
Open relays and forwarders are in an unlucky
position with regards to this issue, generally they can't
guarantee that the MAIL FROM address indicates
the originator, and they also can't guarantee
that final delivery will succeed.
This SMTP problem caused as side effect of
RFC 1123 is addressed by
SPF
, and the
executive summary is SPF breaks forwarding -
actually that's not the case, SPF FAIL only asks
receivers to check SPF at their border MTA, not
later.
Receivers can arrange their forwarding in a way
that works with SPF with in essence three
strategies:
Sender Rewriting Scheme (SRS) is one way for
the third strategy.
Email forwarding
Email forwarding generically refers to the operation of re-sending an email message delivered to one email address on to a possibly different email address...
, which is not allowed by the Sender Policy Framework
Sender Policy Framework
Sender Policy Framework is an email validation system designed to prevent email spam by detecting email spoofing, a common vulnerability, by verifying sender IP addresses. SPF allows administrators to specify which hosts are allowed to send mail from a given domain by creating a specific SPF...
.
Background
Historically all Mail transfer agents (MTAs)added their host name to the reverse path. In
the Simple Mail Transfer Protocol
Simple Mail Transfer Protocol
Simple Mail Transfer Protocol is an Internet standard for electronic mail transmission across Internet Protocol networks. SMTP was first defined by RFC 821 , and last updated by RFC 5321 which includes the extended SMTP additions, and is the protocol in widespread use today...
(SMTP) this
reverse path is also known as MAIL FROM, but
paths were also used before and outside of SMTP,
e.g. as bang paths in UUCP
UUCP
UUCP is an abbreviation for Unix-to-Unix Copy. The term generally refers to a suite of computer programs and protocols allowing remote execution of commands and transfer of files, email and netnews between computers. Specifically, a command named uucp is one of the programs in the suite; it...
and Usenet
Usenet
Usenet is a worldwide distributed Internet discussion system. It developed from the general purpose UUCP architecture of the same name.Duke University graduate students Tom Truscott and Jim Ellis conceived the idea in 1979 and it was established in 1980...
(NetNews). All news articles still contain a
Path header, example:
Path: news.server.example!other.example!not-for-mail
The same information in an RFC 5321 e-mail envelope -
that is the SMTP info like MAIL FROM - would be:
- MAIL FROM:<not-for-mail@other.example>
- MAIL FROM:<@news.server.example:not-for-mail@other.example>
The 1st step reflects the sender, the 2nd step the
next MTA, etc. In this example let's assume that
the 2nd MTA forwards the mail to a 3rd MTA, where
it is finally delivered. The final MTA is also
known as Mail delivery agent
Mail delivery agent
A mail delivery agent or message delivery agent is a computer software component that is responsible for the delivery of e-mail messages to a local recipient's mailbox...
(MDA), putting
the mail into the mailbox of the recipient. The
MDA transforms the reverse path into the known
Return-Path header field:
Return-Path:<@news.server.example:not-for-mail@other.example>
SMTP uses MX record
MX record
A mail exchanger record is a type of resource record in the Domain Name System that specifies a mail server responsible for accepting email messages on behalf of a recipient's domain, and a preference value used to prioritize mail delivery if multiple mail servers are available...
s for its forward routing.
Explicit source routes as in…
RCPT TO:<@news.server.example:user@destination.example>
…to route mail from other.example
via MTA news.server.example
to MDA destination.example were
cumbersome. To make things worse sometimes the
new (1982) style of addresses was mixed with
old UUCP bang paths in constructs like...
destination.example!user@news.server.example
other.example!not-for-mail@news.server.example
...and various other kludges. SMTP and MX records
made all this essentially useless. Therefore
source routing was deprecated 1989 in RFC 1123.
One special case in RFC 1123 are gateways from or
to other networks like UUCP and NetNews, where the
first sending MTA cannot reach the final receiver
directly with
TCP
Transmission Control Protocol
The Transmission Control Protocol is one of the core protocols of the Internet Protocol Suite. TCP is one of the two original components of the suite, complementing the Internet Protocol , and therefore the entire suite is commonly referred to as TCP/IP...
. It is
solved by MX records and if necessary rewriting
foreign addresses at the gateway. MX is an
acronym for Mail eXchanger.
Another special case are
mailing list
Mailing list
A mailing list is a collection of names and addresses used by an individual or an organization to send material to multiple recipients. The term is often extended to include the people subscribed to such a list, so the group of subscribers is referred to as "the mailing list", or simply "the...
s, where
the list server rewrites all reverse paths to
its own error handling address for
bounces
Bounce message
In the Internet's standard e-mail protocol SMTP, a bounce message, also called a Non-Delivery Report/Receipt , a Delivery Status Notification message, a Non-Delivery Notification or simply a bounce, is an automated electronic mail message from a mail system informing the sender of another...
(error messages) by
recipients. The list server could automatically
unsubscribe bouncing recipients. This type of
address rewriting is known since RFC 821 and still
used today ( RFC 5321, as well as RFC 2821, updated the SMTP chapter
in RFC 1123 ).
Last but not least forwarding to another address
always worked by rewriting the address in the
forward path also known as RCPT TO, if and
only if the forwarding MTA accepted the
responsibility for both forwarding the mail and
returning potential bounce messages
to the sender. RFC 821 and all later SMTP
specifications offer two result codes for this
situation:
251 user not local (attempted forward)
551 user not local (mail rejected)
For privacy reasons these result codes are today
rarely used; they include the forwarded to (251) or
not forwarded to (551) address. But the meaning
and the effect of forwarding to third parties is
identical for result code 250 and error code 550
respectively.
As noted RFC 1123 deprecated source routing, that
implicitly also deprecated the reverse routing of
bounces. It was a relatively
small Internet back in 1989, mail admins
(postmasters) often knew each other and fixed problems
on the fly. Routing bounce messages
back via any forwarders was a waste of time and
bandwidth if the MTA noting a problem (e.g. a
rejection with a 5xx error code) could send the
error message directly back to the MX of sender.
Since RFC 1123 forwarders to third parties still
rewrote the RCPT TO address, but kept the MAIL
FROM as is. As a side effect MTAs wishing to
accept mail from forwarders generally accept any
MAIL FROM address.
More than a decade later spammers
E-mail spam
Email spam, also known as junk email or unsolicited bulk email , is a subset of spam that involves nearly identical messages sent to numerous recipients by email. Definitions of spam usually include the aspects that email is unsolicited and sent in bulk. One subset of UBE is UCE...
started to abuse this flaw in post-1123 SMTP,
today most mails are spam
E-mail spam
Email spam, also known as junk email or unsolicited bulk email , is a subset of spam that involves nearly identical messages sent to numerous recipients by email. Definitions of spam usually include the aspects that email is unsolicited and sent in bulk. One subset of UBE is UCE...
and
most reverse paths are forged. Note that
spammers
E-mail spam
Email spam, also known as junk email or unsolicited bulk email , is a subset of spam that involves nearly identical messages sent to numerous recipients by email. Definitions of spam usually include the aspects that email is unsolicited and sent in bulk. One subset of UBE is UCE...
typically forge
working reverse paths, many MTAs reject mail
if callback verification
Callback verification
Callback verification, also known as callout verification or Sender Address Verification, is a technique used by SMTP software in order to validate e-mail addresses. The most common target of verification is the sender address from the message envelope...
or other
plausibility checks fail for the reverse path.
RFC 5321, as well as RFC 2821, states that non-delivery reports ( bounces
Bounce message
In the Internet's standard e-mail protocol SMTP, a bounce message, also called a Non-Delivery Report/Receipt , a Delivery Status Notification message, a Non-Delivery Notification or simply a bounce, is an automated electronic mail message from a mail system informing the sender of another...
) must be sent to the originator as indicated in the reverse path after an MTA accepted the responsibility for delivery. However, the bounce message may be suppressed when the original content is hostile (cf. spam or virus mail) or the message is forged (RFC 5321, Section 6). Note that all current forgery detection methods require the mailbox owner to supply information for them to work. Failing to supply the criteria should not make any bounce message classifiable as backscatter
Backscatter (e-mail)
Backscatter is incorrect automated bounce messages sent by mail servers, typically as a side effect of incoming spam....
, although some people mistakenly think it should.
Open relays and forwarders are in an unlucky
position with regards to this issue, generally they can't
guarantee that the MAIL FROM address indicates
the originator, and they also can't guarantee
that final delivery will succeed.
This SMTP problem caused as side effect of
RFC 1123 is addressed by
SPF
Sender Policy Framework
Sender Policy Framework is an email validation system designed to prevent email spam by detecting email spoofing, a common vulnerability, by verifying sender IP addresses. SPF allows administrators to specify which hosts are allowed to send mail from a given domain by creating a specific SPF...
, and the
executive summary is SPF breaks forwarding -
actually that's not the case, SPF FAIL only asks
receivers to check SPF at their border MTA, not
later.
Receivers can arrange their forwarding in a way
that works with SPF with in essence three
strategies:
- not checking SPF behind their border, e.g. white list forwarders
- just reject SPF FAIL, resulting in a bounce (SMTP error 550)
- rewrite the MAIL FROM at the forwarder (as done by mailing lists)
Sender Rewriting Scheme (SRS) is one way for
the third strategy.
See also
- Sender Policy FrameworkSender Policy FrameworkSender Policy Framework is an email validation system designed to prevent email spam by detecting email spoofing, a common vulnerability, by verifying sender IP addresses. SPF allows administrators to specify which hosts are allowed to send mail from a given domain by creating a specific SPF...
(SPF) - Bounce messageBounce messageIn the Internet's standard e-mail protocol SMTP, a bounce message, also called a Non-Delivery Report/Receipt , a Delivery Status Notification message, a Non-Delivery Notification or simply a bounce, is an automated electronic mail message from a mail system informing the sender of another...
(SMTP non-delivery report) - Bounce Address Tag ValidationBounce Address Tag ValidationIn computing, Bounce Address Tag Validation is a method, defined in an Internet Draft, for determining whether the bounce address specified in an E-mail message is valid...
(BATV) - Simple Mail Transfer ProtocolSimple Mail Transfer ProtocolSimple Mail Transfer Protocol is an Internet standard for electronic mail transmission across Internet Protocol networks. SMTP was first defined by RFC 821 , and last updated by RFC 5321 which includes the extended SMTP additions, and is the protocol in widespread use today...
(SMTP)
External links
- libsrs2 home page
- Paper on SRS (PDF)
- Historical SRS draft by Meng Weng WongMeng Weng WongMeng Weng Wong is a serial entrepreneur. In 1994 he founded pobox.com, an email services company. In 2003 he led the group that designed the Sender Policy Framework standard which was later embraced and extended by Microsoft...
(2003) - qmail SRS patch