Security+ TechNotes - Email Security


Email Security
- Encryption
- Signing
- X.509

Email Security

Email has become a very popular way to communicate and is therefore also a popular target for malicious attackers and cyber vandalism. In addition to concerns regarding privacy, message integrity, and sender authenticity, a mail system’s users need to be protected from a daily flood of spam and viruses. These TechNotes cover some of the popular technologies for secure messaging, focusing on the Security+ exam objectives listed at the bottom.


Email was originally developed to exchange plain text messages, but after some time people wanted to be able to include files such as images, reports, and even executables in the message. To allow these attachments with different formats and to allow interoperability between email systems, the Multipurpose Internet Mail Exchange (MIME) standard was developed. To secure the message as well as the attachments, an extended version of MIME was developed, which is called Secure MIME (S/MIME). S/MIME is a comprehensive technology that encapsulates many different security functions to satisfy some of the concerns mentioned earlier. S/MIME allows users to encrypt and sign messages as described in the following paragraphs.

Message Encryption

A user that sends a message with confidential information can keep the contents private while it travels to its destination by using message encryption. For message encryption, a symmetric algorithm (DES, 3DES, or in older implementations RC2) is used to encrypt the message data. The key used for this process is a one-time bulk key generated at the email client. The recipient of the encrypted message needs the same symmetric key to decrypt the data, so the key needs to be communicated to the recipient in a secure manner. To accomplish that, an asymmetric key algorithm (RSA or Diffie-Hellman) is used to encrypt and securely exchange the symmetric key. The key used for this part of the message encryption process is the recipient’s public key. When the recipient receives the encrypted message, he will use his private key to decrypt the symmetric key, which in turn is used to decrypt the message data.

As you can see, this type of message encryption uses a hybrid system, which means it uses both symmetric and asymmetric algorithms. The reason for not using the public key system to encrypt the data directly is that it requires a lot of CPU resources, symmetric encryption is much faster than asymmetric encryption. Only the contents of a message is encrypted; the header of the message is not encrypted so mail gateways can read addressing information and forward the message accordingly.

Digital Signatures

In addition to keeping message contents confidential, S/MIME offers sender authenticity, non-repudiation, and message integrity by adding a digital signature to a message. When a user signs a message, a hashing algorithm (MD5 or SHA-1) is used to create a message digest. This message digest is in turn encrypted with the sender’s private key by using an asymmetric algorithm (RSA or DSA).

When the recipient receives the message, he will use the sender’s public key to decrypt the message digest value. If the recipient is able to do this, the sender proved he is who he claims to be (authenticity), and because that person should be the only one with the private key, the recipient can prove that he received a message from that person, so the sender cannot deny sending it (non-repudiation). When the recipient decrypted the message digest value (or hash value), he will run the message through the same hashing algorithm and compare the outcome with the value in the digital signature. If the message digest values match, the message has not been tampered with during transmission (integrity), hence it proves the message is the exactly the same message the sender originally sent.

There are two different types of signing, opaque, and clear signing. With clear signing, the digital signature is added to the message as a separate attachment while the message itself remains in clear text. This may be required when sending a message to an older email client. With opaque signing, the message and the digital signature are combined into a single binary file. Signing a message does not mean it is encrypted, and encrypting a message does not require a digital signature.

• When a user sends an encrypted message, the recipient’s public key is used to encrypt the message. The recipient’s private key is used to decrypt the message.
• When a user signs a message, the sender’s private key is used to encrypt the message digest. The recipient uses the sender’s public key to verify the signature.

X.509 Certificates

S/MIME relies on an X.509 certificate Public Key Infrastructure (PKI) for private/pubic key management. This means users use X.509 certificates to exchange the public keys used to encrypt a message or verify a digital signature. All users involved must trust the Certification Authority (CA) that issued the users’ certificates, either directly or indirectly, in a hierarchical trust model.

The first time an email client is enabled for signing and encryption, the client contacts a certificate authority (CA), which generates the private/public key pairs (modern systems use a different pair for signing and encryption) and uses one-time session keys to securely communicate the key pairs to the client. The private key is usually stored on the local computer and can be decrypted and used for signing or encrypting by using a password. As the name implies, private keys should be kept private at all time. In modern systems a copy of the key is stored in a central database to allow key recovery in case the user lost his private key (or the password to access it). PKI and certificates will be covered in more detail in other TechNotes.


PGP was originally a freeware message encryption system and offers users to choose from several encryption algorithms. It was released by Philip R. Zimmermann in 1991, and was the first encryption technology widely available to the general public (something the US government wasn’t too happy about). It does not need a hierarchical trust model like S/MIME with X.509v3 certificates, but instead users themselves are responsible for exchanging the public keys and building a ring of trust with other users.

The public key is a simple block of text, making it easy to exchange the key with others by just including it in an email message directly or exporting it to a text file. Users can also make their public keys available through a keyserver on the Internet, i.e. The private key is stored on the local computer and is protected with a passphrase.

PGP is also a hybrid encryption system. The latest versions of PGP support AES, CAST, IDEA, Twofish, and 3DES algorithm for the symmetric portion; to encrypt the actual data. That symmetric key is in turn encrypted by using an asymmetric algorithm (RSA, DSS/Diffie-Hellman) to allow users to securely exchange the symmetric key with each other. It is now also possible to use X.509 certificates from a CA to prove identity and ownership of PGP public keys. A sender’s private key is used for signing and the recipient’s public key is used to for encryption, just like in S/MIME.

Because the source code of PGP has always been available to the public, several free, commercial, legal, pirated, and international versions of PGP are available, supporting several different services including encryption and digital signatures and support a wide range of algorithms including ElGamal, DSA, AES, 3DES. Blowfish, Twofish, CAST, MD5, SHA-1, and more. PGP Inc. still offers a free version for personal non-commercial use, which you can download here.

S/MIME and PGP can keep message contents confidential on their way from email client to email client. However, other vulnerable parts in an email system, i.e. client-server or server-server connections, need to be secured as well by using SSL, TLS, IPsec, VPNs, or some other means of secure communication.


Spam is unsolicited, typically commercial, junk mail. Nobody wants it but virtually anyone with a mailbox gets it. Apart from spam filters at clients and servers, you can reduce the effect spam has on your SMTP mail servers, and hence the legitimate users, by disabling relaying and enabling reverse lookups.

Especially older systems relay every message regardless of the source domain in the address. This means you could send a message with the source address from a mail server hosting users for the email domain So basically, a server that allows relaying allows users to send messages with a fake or ‘spoofed’ source address. The major spammers continuously scan the Internet for ‘open’ mail relays, so they can abuse those for sending thousands of messages while using someone else’s resources and hiding behind them. Although most modern email systems relay only for the domain(s) local to the server and those explicitly specified by an administrator, there are still a lot of mail servers that allow relaying for any domain.

You can check a server manually by opening a Telnet session to port 25 on the mail server and use the following command sequence when the welcome message appears that says it is ready to accept commands:

Your salary has been doubled because of your excellent work for the company.

(Note the dot on the last line). If the server accepts the message for delivery, the server is vulnerable. In that case you will have to disable relaying and specify the domains the server should relay manually. If the server is configured correctly, it will reply with something like “Relaying not allowed”, which it usually does after you issue the MAIL FROM: command with a fake source address.

Unfortunately, there will always be careless administrators that leave their servers open for relaying. To reduce the chance of receiving spam from those servers, you should enable reverse lookups. This allows your mail server to verify if the IP address of a remote server is actually mapped to the domain for which that server wants to send messages. So when a spammer abuses an open server to send messages to your server, the incoming connection will be rejected because your server will consult DNS and see that the source domain of the spam messages doesn’t map to the IP address of the remote server.


Current related exam topics for the Security+ exam:

DOMAIN 2.0: Communication Security

2.2 Recognize and understand the administration of the following email security concepts
- S/MIME (Secure Multipurpose Internet Mail Extensions)

- PGP (Pretty Good Privacy) like technologies
- Vulnerabilities

2.3 Recognize and understand the administration of the following Internet security concepts
- Vulnerabilities
-- SMTP (Simple Mail Transfer Protocol) Relay

DOMAIN 3.0: Infrastructure Security

3.5 Understand the following concepts of Security Baselines, be able to explain what a Security Baseline is, and understand the implementation and configuration of each kind of intrusion detection system
- Application hardening
-- E-mail Servers

Date: Thursday, January 20, 2004
Author: Johan Hiemstra
MCSA 2000/2003 Security+ CWNA