How does an email server work?

You may use email every day. How does email securely send mail around the world?

Email client to server diagram with SMTP protocol.

Email has existed for over four decades with billions of emails sent and received every single day. It's arguably one of the oldest means of digital communication and mature enough such that one could configure and run their own email server to serve emails using their own domain name. Governments, multinational corporations, and families all rely on email as a cornerstone of asynchronous communication and management.On a high level, running one's own email server a.k.a SMTP server involves running an incoming email server to receive email messages, an outgoing email server to send email messages, and a email client or web interface to perform the compose, send and read operations.Let's understand the components involved in running a email server.Email Client or Mail User Agent (MUA)It is a web-based mail client used to compose, send, receive & read messages (eg. Gmail, Yahoo, Outlook).The two popular ways of accessing email messages are Internet Message Access Protocol (IMAP) defined in the RFC 9051 standard and Post Office Protocol version 3 (POP3) defined in the RFC 1939 standard.In case of Skiff Mail, the email client is a private and end-to-end encrypted where email messages are first encrypted before sending them. For more details on how Skiff provides privacy & encryption follow this link.Outgoing Mail Server or Mail Submission Agent (MSA)Responsible for sending messages to Message Transfer Agent (MTA). MSA must only accept messages from authenticated users.Incoming Mail Server or Mail Transfer Agent (MTA)Responsible for delivering the messages sent by other email servers. It queues the messages and/or forward to a Message Delivery Agent (MDA) for further processing & storage.Message Store (MS)A message store can be any database or file system used for storing the incoming & outgoing email messages. It is accessed by the MUA to view the received & sent messages.Simple Mail Transfer Protocol (SMTP)A spec that defines how an email client (MUA) submits email messages and how the MTA's should transfer those messages.

Architecture

This section walks through how a typical email send and receive workflow works, as well as how an email server itself operates.Email messages between senders and recipients flow through multiple steps. In many cases, the sender and recipient are using different email service providers and different email servers around the world, requiring multiple steps before an email gets to its final destination.Sender users the email client to compose the message and send it via the outgoing mail server and store the message in the message store so it can be accessed later.The outgoing mail server then communicates with recipient’s incoming mail server over SMTP protocol and performs a handshake to deliver the email message to recipient’s email address or email account.The recipient’s server then stores the email message on the recipient’s email provider message store.Ultimately, the recipient can access the email message using the email client and compose a response which will be sent via the recipients email provider outgoing mail server.Alternatively, a Mail Delivery Agent (MDA) may be used to process the email messages queued by the incoming mail server. The MDA is responsible for actually storing the email messages and perform any business logic - custom spam filtering, user lookups and so on. This is to keep the functionality of the incoming mail server to a minimum and achieve high availability & low latency for email delivery.

Security

Running an email server comes with security challenges. There is huge potential for spam & abuse if the email server is not configured properly. A large part of running an email server successfully involves effectively filtering spam email messages and maintaining high IP reputation of the IPs associated to the outgoing mail server.Here are a few basic security configurations that are critical to maintaining high deliverability of email messages and filtering spam messages. These apply to all email or smtp servers.TLSJust like web servers, communication between the client and email servers can be secured using Transport Layer Security (TLS). STARTTLS is the extension for SMTP protocol specified in the RFC 3207 as a way to explicitly request TLS connections by clients.AuthenticationOnly authenticated users/clients must be able to send messages via the outgoing mail server. This is important because bad actors could leverage an open outgoing email server to send spam and damage the reputation of the email server.In addition, it is recommended to run the outgoing email server over a private network behind NAT (Network Address Translation) gateways to restrict public access.

Email Server DNS (Domain Name System) Configuration

DKIM RecordsDKIM is Domain-Keys Identified Mail is a RFC 6376 standard that specifies signing emails sent by an outgoing email server. The domain owner can publish asymmetric key pair. The private key is used by the outgoing email server to sign the content and/or headers of the email message. The public key is used by incoming mail server to verify the email message.This establishes the ownership of the email message being sent and the incoming mail server can verify authenticity of the email. The verification of signature allows incoming server to filter out spam email messages before delivering them to the users’ mailbox.We recommend that domain owners rotate their DKIM keys periodically. Since the public key is published in the domain configuration, it opens the DKIM keys for phishing attacks. Rotating the keys helps reduce this attack vector significantly.SPF RecordSPF stands for Sender Policy Framework. It is specified in RFC 7208 standard. The domain owner can create a TXT record in their domain configuration and list the IP addresses of the outgoing email server. This record is used by incoming mail server to verify if the email message is indeed sent from one of those IP addresses and reject them if the IP is not part of the SPF record.This maintain the IP reputation of the domain’s outgoing email servers.MX RecordMX is Mail Exchange record which specifies the email server that is responsible for handling incoming mail messages for a particular domain.It is recommended to run multiple groups of email servers with varying levels of priority. This is to provide high availability and uptime.DMARC RecordDMARC is Domain-based Message Authentication, Reporting, and Conformance standard specified in RFC 7489.While SPF & DKIM records allow the domain owners to protect the authenticity of the incoming email messages, it does not influence the action that should be taken by the incoming mail server if SPF or DKIM checks fail.DMARC allows specifying a policy by the owner of the domain which dictates how the incoming mail server should handle non-authentic emails. This is done by publishing a TXT record in the domain configuration which specifies the policy.

Skiff Mail

Skiff is a provider of end-to-end encrypted email service. All the emails sent & received between users of Skiff Mail are 100% private and only they have access to those email messages.To deliver on our mission of providing privacy, at Skiff we build and run our own email/smtp servers. This allows us to further extend our email server capabilities to enhance security and scalability.Skiff’s encrypted email servers offer a high level of security for businesses and individuals who need to communicate sensitive information. By encrypting emails, these servers prevent third-party interception and protect the privacy of communications.Skiff Mail also allows consumers and businesses to onboard their own custom domains for sending and receiving emails. Users of Skiff Mail can import all their emails from other providers by importing .eml files as well as take advantage of direct import feature for importing emails from Microsoft Outlook and Gmail.

Conclusion

Email protocols are the set of rules and standards that are used to exchange messages over the internet. The most commonly used email protocol is the Simple Mail Transfer Protocol (SMTP), which is used to send and receive messages between servers.When a user composes and sends an email, their email client sends it to their email server using SMTP. The email server then looks up the recipient's email address to determine their email server, and sends the message to that server using SMTP. The recipient's email server then receives the message and delivers it to the recipient's email client.Other email protocols include the Post Office Protocol (POP) and the Internet Message Access Protocol (IMAP), which are used to retrieve messages from the email server and access them on the user's email client. These protocols also use a system of commands and responses to transfer messages between the email client and server.A complex system of authentication and DNS records (DKIM, SPF, DMARC, and MX records) govern email deliverability and email security.Overall, email protocols are an essential part of the email system, allowing users to send and receive messages reliably and securely over the internet.