Managing Internet Mail: Setting Up and Trouble-Shooting Sendmail and DNS


Class Outline:

Introduction To Managing Internet Mail:

Setting UP & Trouble Shooting Sendmail & DNS
Introduction And New Features Of Sendmail
What Are The Functions Of Sendmail?
Sendmail Is A Rule Based Mail Routing Agent
Programs And Support Files Used By Sendmail
Addressing, Mail Routing, Sendmail And DNS
RFC 1869, Extended SMTP (ESMTP)
Mail Routing With A Domain Style Address
Mail Routing With MX Records
RFC 822 Message Format
Simple Mail Transfer Protocol, SMTP

Overview Of Sendmail Operation

The Sendmail SMTP Server, sendmail -bd
Sendmail 8.12's Non-SUID Root Operation
The Queue Daemon, sendmail q15m
Persistent Queue Runner, sendmail -qptime
Sendmail's Queue Directory, mqueue
Common Command Line Arguments To Sendmail
Command Line Arguments To Sendmail Useful For Troubleshooting
Sendmail's -d Debug Flags
Tracing An Email Message In The Log Files

In Class Demonstration:

Forwarding An SMTP Message Through A Firewall SMTP Relay
Tracing The Message Through The Log Files

Overview Of The sendmail.cf File

Configuration Of The sendmail.cf File
Sendmail Rulesets And Rule Definitions
Sendmail Macros And Classes
Configuration Options To Sendmail
Common Configuration Options To Sendmail
Define A Mailer

Using M4 To Generate sendmail.cf Files

Organization Of The Cf Configuration Directory Used By M4
Basic M4 Syntax:
The define, VERSIONID, OSTYPE, MAILER, FEATURE, DOMAIN And HACK macros
Order Of The Host.mc File
An Overview Of Makefiles
Setting Up The sendmail/cf Sub-Directories

In Class Demonstration:

Setting Up The sendmail/cf M4 Template Directory
Generating A sendmail.cf File

Introduction To Sendmail's Rules

sendmail's Address Rewriting Rules
Sendmail's Address Workspace
Left Hand Side (LHS) Metasymbols For Pattern Matching
Right Hand Side (RHS) Metasymbols For Substitutions, Ruleset Control And Database Rewriting
Using The Comment Field To Write Or Understand A Rule

Introduction To Sendmail's Rulesets

The Important Rulesets Canonify (S3), Final (S4) And Parse (S0)
The Less Important Rulesets Sender (S1), Recipient (S2)and Localaddr (S5)
Ruleset Sequence, "The Diagram"
Example Of Rewriting Of A Message
Pre-Processing Ruleset, S3 Or canonify
Sendmail Address Focus, ">" "<"
Ruleset Canonify2 (S96), Localization And Canonicalization
Post-Processing Ruleset, S4 Or final
The Delivery Ruleset, Ruleset S0 Or parse
Mailer Specific Rulesets
Sendmail's -d Debug Flags For Address Test Mode
Sendmail's -bt Address Test Mode
Sendmail Debugging With checksendmail
Strategy For Using Sendmail's Address Test Mode

In Class Demonstration:

Trouble Shooting A Mail Delivery Problem:
Using checksendmail And Address Test Mode

Using Databases In Sendmail

Defining A Database In The sendmail.cf With The K Line
The Right Hand Side Database Look-up Metasymbols $( And $)
Changing The Database Default Rewrite With $:
Passing Additional Parameters To The Database Rewrite With $@
Debug Flags Useful For Sendmail Databases
Database Debugging Commands In Sendmail's Address Test Mode (-bt)
Building And Updating Databases With Sendmail makemap And editmap
Using The domaintable Database To Rewrite Host And Domain Names
Specifying Delivery For A Specific Addresses With mailertable
The access Database For Selective Accepting And Rejecting Of Mail
Restricting A Lookup Key To A Specific Check
Trouble Shooting Sendmail Database Problems

In Class Demonstration:

Verifying Database Operation In Address Test Mode
Using The domaintable And mailertable To Rewrite Addresses And Route Mail

Sendmail Version 8.14 Installation

Should You Use Open Source Or Vendor Supported Sendmail
Organization Of The sendmail-8.14 Release Directory
Building Sendmail Version 8.14
Checking The PGP Signature
Check The Results Of Make
Making A Tar Archive Of The Sendmail Binaries
Using -d0.1 To Print Version Of Binary And C Compile Time Definitions

Converting An Existing sendmail.cf File Into A M4 Template

M4 Macros That Set Sendmail Macros
M4 Macros That Set Sendmail Classes
Finding M4 Macros That Set Options
Merging Custom Rules Into Existing Rulesets

In Class Demonstration:

Converting An Existing sendmail.cf File Back Into A sendmail.mc file

MILTER, Sendmail's Mail FILTER API

How Milters Work
Milter API Operation
Milter Callback Types And Return Values
Compiling Milter Support Into The Sendmail Binaries
Defining A Milter In The sendmail.cf File

Introduction To DNS,

What Services Does Domain Name System Provide
Delegation Of A Zone Of Authority
Functions Of Name Servers
Reverse And Double Reverse PTR Record Lookups
Parts Of Bind, named, Named Support Programs, And The Resolver Libraries

Setting Up DNS Clients

The Client Process, The Resolver Libraries
The /etc/resolv.conf File
DNS Resource Records
dig - Dump DNS Information For A DNS Zone
host - Make A DNS Query
The nslookup Command

Named, The DNS Server

Syntax Of The Bind 8 named.conf File
Bind 8 Name Daemon Control, ndc And Bind 9 rndc
Caching-Only Named Server
Authoritative Name Server
Setting Up A Recursive Name Server
Sharing A Zone File With Multiple Domains
DNS Zone Files And Resource Records
Format Of The Resource Records In DNS ASCII Source Files
Common Zone File Syntax Errors
Using $INCLUDE And $ORIGIN
The SOA And NS Resource Records For The Domain
The A, CNAME, And PTR Resource Records For A Host
Responsible Person, RP, And Text, TXT, Resource Records
The SeRVice, SRV, Resource Records

Setting Up A Simple DNS Domain

Setting Up The named.conf For The Master Named Server
DNS Zone Data And Reverse PTR Records Zone Data Files For Gadget.com Domain
Generating A named.ca Cache File Using Dig
Setting Up The named.conf For The Slave Named Server

In Class Demonstration:

Setting Up Master And Slave Named Servers
Verify Named By Dumping Memory To /var/named/named_dump.db

Debugging DNS

Exhaustively Check The Zone And Delegation Of The Zone With doc
Checking Consistency Of DNS Data In DNS Database Files With nslint
Checking Consistency Of Data In Remote DNS Zone Files With dnswalk
h2n - Translate Host Table To Named Zone Files
Named Debugging Output

Setting Up A Simple Sendmail SMTP Relay

Mail Relays Need To Enable SMTP Relaying
How To Forward Mail Out Of The Domain
How To Forward Mail In The Domain To An Internal Relay
Custom Sendmail Rules Verses Sendmail Database Rewriting
MASQUERADE_AS And Masquerading FEATUREs
Masquerading Internal SMTP Hosts
Forwarding To A Smart Host Relay Using The SMART_HOST Macro
Accepting Mail On A Mailhub For Of Mail Clients Using Class $=w
Using FEATURE(use_cw_file) To Add Hosts To Class $=w
Forwarding SMTP Mail For Internal Hosts Directly Using Rules And LOCAL_NET_CONFIG
How To Forward Internal And External Mail Using The mailertable

In Class Demonstration:

Configuring Sendmail For A Firewall And Mail Hub
Using checksendmail To Verify The sendmail.cf Configuration
Enabling Masquerading FEATUREs

Setting Up The Mail Clients

Mail Servers And Clients
Functional Rolls Of Email Hosts With Sample Configurations.
nullclient Mail Clients
Masquerading And EXPOSED_USERs On A Nullclient
Turning Off The Sendmail SMTP Server On A Mail Client
Taking Away Sendmail's SUID Root Privileges
Using submit.cf On A Mail Client Running Sendmail 8.12
Running Sendmail Exclusively Using Submit Mode

In Class Demonstration:

Configuring A submit.cf Only Sendmail Client

Sendmail Address Rewriting Rules

OperatorChars Token Separator Option
Tokenizing An Address By Sendmail
The Left Hand Side (LHS) Metasymbols For Generic And Specific Pattern Matching
The Right Hand Side (RHS) Metasymbols For Substitutions And Recursion Control
Calling Another Ruleset With $>number And $>name
Sample Use Of Ruleset Control
Calling A Mailer In The Right Hand Side (RHS)
The Rule LHS -vs.- The Rule RHS
Examples Of Comment Fields
Writing A Custom Rule Using The Comment Field Revisited
Dollar Dot Conditional Tests
How Rulesets Are Called
Debug Commands Useful In Sendmail's Address Test Mode

In Class Demonstration:

Following The Application Of Mini Ruleset S3

Sendmail Rulesets

Rulesets Called By The Sendmail Binary
The Ruleset Sequences:
The Envelope Sender Address
The Envelope Recipient Address
Collect The Message
Rewrite And Deliver The Message
Pre-Processing Ruleset, S3 Or canonify
Why Is Ruleset S3 Important?
Pseudo Domains For Non-Domain Style Address
*LOCAL* Pseudo Domains For Unqualified User Addresses
Ruleset Canonify2 (S96), Localization And Canonicalization
The Generic Sender And Recipient Rulesets sender (S1) And recipient (S2)
Post-Processing Ruleset, final (S4)
The Delivery Ruleset, Ruleset parse (S0)
Ruleset MailerToTriple (S95) Resolves A Lookup Focus
Alternate Local Delivery Ruleset, Localaddr (S5)
Mailer Specific Rulesets

Sendmail check_* Rulesets

Use check_rcpt To Dis-Allow Relaying Through This Host
Rejecting Mail Before Calling The Mailer With check_compat
Debugging check_compat And check_relay Which Use $|
Adding check_compat Style Rules To The check_rcpt Ruleset
Adapting check_mail And check_rcpt To The check_compat Ruleset
Debugging Flags Useful For check_* Rulesets

Sendmail Mailer Definitions

The Mailer Definition
Sendmail Split Mailer Split Envelope/Header Rewriting Rules
The Standard Mailers: local, program, *file*, *include*, smtp And error
Sendmail Mailer Flag For:
Invoking The Mailer Program
Invoking The Local Mailers
Control Calling Of The Mailer
The User ID For Execution Of Programs
Affecting The Body Of The Message And MIME Conversions
SMTP And Extended SMTP, ESMTP
Generating Headers
Header Format Definitions
Creating Custom Header Format Definitions
Conditionally Including Custom Headers By Setting Macros
Modifying The Received: And Message-Id: Headers
Debugging Flag Useful For Mailers
Including An Error Header In Bounce Messages
Mailer Specific Ruleset Order
In Debug Mode printaddr() Prints Internal Address Structure

Delivery Agents, Local Delivery, And Aliasing

What Is A Message Delivery Agent?
Local Delivery And Aliases
Using LMTP (Local Mail Transport Protocol)
POP And IMAP
Using aliases And .forward Files To Mail To Files And Programs
Dealing With Former Users Using FEATURE(redirect)

In Class Demonstration:

Forwarding Mail With Aliases And Rejecting Mail With FEATURE(redirect)

Sendmail Queuing, The mqueue Directory

Printing The Queue With Mailq
Limiting Queue Processing With -qS, -qR, And -qI
Sendmail's Queue Files:
Queue Control (qf), Data File (df), And Transcript File (xf)
Persistent Queue Runners, -qp
Queue Control Process, QCP And Work Group Processors, WGP
Sendmail Options Used For The Queue:
Queue Time-outs
Managing The Queue Load
Controlling Number Of Connections To The SMTP Server
Connection Caching To Enhance SMTP Performance
Persistent Host Status Information In Sendmail
Debugging Flags For The Queue
Dealing With Spam Bounces By Deleting Bounce Mail
Delivering Mail To Nobody
Putting The "nobody" Rules Into An M4 HACK

In Class Demonstration:

Selectively Processing The Queue With -qS And -qR
Timing Messages Out Of The Queue
Deleting Mail With "nobody" Rules

Queue Groups & Split Queues

Queue Group Declaration: Q Line Type
Queue Runners
Running The Queue For A Specific Queue Group With -qG
Controlling Number Of Queue Runners
Splitting A Message Envelope
Queue Group Selection
Defining Queue Groups For Standard Mailers
Debugging Flags For Queue Groups
Some Ideas About Types Of Queue Groups

In Class Demonstration:

Using Split Queue Directories
Using Queue Groups For Inbound And Outbound Mail

Sendmail Monitoring And Logging

sendmail Logging Levels
Tools To Process Sendmail's Log File: ssl, smtpstats, And fromto
Monitoring A Mail Server
Reading Mail Headers

Virtual Domains

Virtual Domain Mail Delivery With The virtusertable Databases
Applying The virtusertable To All Hosts In A Domain
The genericstable Database
Apply The Generics Database To Entire Domain
Building genericstable And virtusertable From A Single File
Differences Between The Aliases File And The virtusertable

In Class Demonstration:

Accepting Mail For Multiple Domains With The virtusertable
Changing UserID Addresses To First.Last Address With The genericstable

Sub-domains

Creating A Sub-Domain In An Existing Zone Of Authority
Creating A New Zone Of Authority For A Sub-Domain
Delegating The Sub-Domain To A Different DNS Server
Issues With Sendmail And Sub-domains

In Class Demonstration:

Delegating A DNS Sub-Domain

Simple Mail Transfer Protocol, SMTP

SMTP Commands
Structure Of SMTP Numeric Reply Codes
RFC1893, Enhanced Mail System Status Codes
Extended SMTP (ESMTP) Support For:
Message Size Declaration
8bit-MIME Body Type (8BITMIME)
Delivery Status Notifications (DSN)
ETRN Command
Speeding Up Sendmail By Running Fast And Slow Daemons
Debugging Flags Useful For SMTP
Debugging Flags Useful For MIME

In Class Demonstration:

Using Fast And Slow Sendmail SMTP Daemons

Advanced BIND 8 And Bind 9 Features

BIND 8 Configuration Syntax
The master, slave, stub, And hint Zone Statements
The options Statement
Using Access Control To Limit Zone Transfers And DNS Client Queries
The server Statement
The acl Statement
The logging channel Statement And category Phrase
An Example Of Logging Different Category(ies) To Specific Channels
Overview Of Bind 9
rndc, Remote Name Daemon Control Application
/etc/rndc.conf, rndc Configuration File
Generating A rndc.conf File Using rndc-confgen

In Class Demonstration:

Using Advanced Bind Features

Lightweight Directory Access Protocol, LDAP

What Is A Directory Service?
How Is Data Stored In LDAP
Relative Distinguished Name (RDN)
Introduction To Abstract Syntax Notation One (ASN.1) And Backus-Naur Form
LDAP Object Classes And Attribute Types
LDAP Supports Access Control
Sendmail LDAP Support
Sendmail LDAP Map Specifications
Compiling In LDAP Support Into Sendmail
Sendmail LDAP Debugging Flags
Using LASER (LDAP Schema For E-Mail Routing) Mail Routing With FEATURE(ldap_routing)
Testing The LDAP Directory With ldapsearch
Testing The Sendmail LDAP Configuration
Using LDAP For Aliases
A Sample Host Specific LDAP LDIF Entries For An Alias
Using LDAP For Database FEATURE()s
A Sample Host Specific LDAP LDIF Entire For The mailertable
Using LDAP For Classes
A Sample Host Specific LDAP LDIF Entire For The $=R Class

In Class Demonstration:

Rouging Mail With FEATURE(ldap_routing)

Sendmail Security

Why Is Sendmail A Security Target?
What Are The Avenues Of Attack Against Sendmail
How Is Sendmail Compromised?
smrsh, Sendmail Restricted Shell
smap, Smapd, Secure SMTP Server
smtpd, An Alternative To Smap
Sendmail Support For Tcpwrapper
Using RunAsUser To Run A Sendmail SMTP Relay As Mailnull
Using trustedUser To Maintain Aliases And Database Files
Sendmail Control Socket
General UNIX Security
Options Relating To Security
Restrict SMTP Commands With PrivacyOptions
DontBlameSendmail Loosens Sendmail's Security Checks
Limit Header Sizes With MaxHeadersLength
Debugging Flags Useful For Security
Some Useful Security Resources

In Class Demonstration:

Limiting Root Use On A Firewall With RunAsUser, trustedUser, And The Control Socket

Sendmail And DNS On A Firewall SMTP Relay

WorkAroundBrokenAAAA, Avoiding Problems With Severs That Do Not Support Quad A Records
Setting DNS Resolver Time-out For Sendmail
The Service Switch Configures Sources Of Naming Service Information
$[ And $] DNS Lookup Meta-Symbols
Split DNS, Using Two Separate DNS Systems
Using Multiple MX Records Which Point To SMTP Relays
Using Multiple MX Records For Mailbox Hosts
Redundant Internet Connections
Forwarding Mail To Hosts Behind An Internal Firewall
Debugging Flags Useful For MX Records
Sendmail And The Firewall Relay Host
Full DNS Information Published To The Internet
Disabling Local And Prog Mailers
Delivery To An Internal SMTP Relay Using Sendmail Rules
Delivery To An Internal SMTP Relay Using Sendmail mailertable
Using Internal DNS Information With Split DNS Domains

Debugging Sendmail And SMTP

Initial Message Processing By Sendmail
Message Processing By Sendmail From An External Delivery Agent
Manually Feeding A Message To Sendmail
Using Debug Flags When Delivering Mail
Testing Incoming SMTP Delivery
Sendmail Address Test Mode
checksendmail Perl Debugging Script
Strategy For Using Sendmail's Address Test Mode For:
Message Routing And Delivery, Envelope And Header Addresses
Strategy For Debugging Problems At Remote Hosts
Determining Which Rules Sets Should Be Looked At

Selecting & Tuning The Hardware & OS For Sendmail

Heuristic Performance Tuning
Subtractive Performance Tuning
Selecting Hardware For Sendmail
Sendmail Likes Memory
Disk Drive Architectures
Multiple Disk Drives: Just A Bunch Of Disk (JBOD) And RAID
Using Sendmail Split qf, df And xf Directories With Multiple Disks
Installing The Operating System For Sendmail

Sendmail Performance Tuning

Overall Tuning Strategies
Pulling It All Together With Already Covered Sendmail Options For Performance Tuning
Dealing With Spam
Re-enabling Limited SMTP Relaying
Enabling Relaying For An Entire Domain
Allowing Relaying Based On MX Records
The access Database For Selective Accepting And Rejecting
New Sendmail Features:
access Database Features
FEATURE(compat_check)
FEATURE(greet_pause)
FEATURE(ratecontrol)
FEATURE(conncontrol)
Using Private Result Codes
Sendmail DNS Blackhole List
Sendmail Enhanced DNS Blackhole List
Regular Expression Map
Using The Regex Map To Reject Specific Address Prefixes
discard Mailer To Silently Delete A Message
Message Header Rulesets And check_eoh Ruleset
Checking For The Existence Of A Message-ID: Header
Conditionally Including A Spam Header By Setting A Macro
Using A DNS Blocking List To Add An X-may-be-spam: Header
Rejecting Subject: Lines Starting With ADV: Or AD:
Debug Flag -d31.9 Useful For Header Checking Rulesets