How to make email-lists and do not mow: practical advice
The developer, who first encountered the generation of e-mails, has almost no chance to write an application that will do it correctly. About 40% of emails generated by enterprise applications have some kind of violation of standards, and as a result, problems with delivery and display. There are reasons for this: e-mail is technically much more complicated than the web, mail work is regulated by several hundred standards and an uncountable number of common (and not so) practices, and email clients are diverse and unpredictable. Testing can significantly improve the situation, but there are practically no materials devoted to testing the mail.
Mail.Ru regularly interacts with its users via e-mail. In our project, all the components responsible for the generation of letters, and even single mailings are mandatory testing. In this article we will share our experience (and stuffed cones).
What are the electronic letters
Who participates in the testing and control process
The postal message and the postal transport
The interface of the mail infrastructure and the boundaries of the tested application
Determination of the test parameters
Typical structure of the generating application
What and when to test
Infrastructure of delivery
Structural and vestochnye templates of the letter
Basic requirements for infrastructure testing
Requirements for authorization
Checking the generating application
Requirements for postal addresses
Requirements for the headers of letters
Requirements for the structure of the letter
Requirements for URI
Requirements for the layout of the letter
Conducting of split tests
in this article .
Check DNS record DKIM, validity and composition of DKIM-signature (DKIM-Signature). Verify that you are using a DKIM key with a size of at least 1024 bits. The recommended mode for hashing the DKIM signature is: relaxed /relaxed. Make sure that all the important headers (From, To, Subject, Date, Message-ID, MIME-Version, Content-Type) are signed, Received, Delivered-To, Return-Path are not signed, and DKIM passes validation the main postal services. Configure one of the mail services to forward to the other, DKIM should not "beat" on forwarded emails. Verify that the DKIM signature domain matches the sender domain from the From header.
Check the DMARC walkthrough on the main mail services. Check the receipt of DMARC-reports, identify and eliminate the problems with the passage of SPF and DKIM for all IP-addresses of your infrastructure.
Verify that the messages are delivered to external servers using encryption (TLS). You can sometimes check the presence of TLS by the Received header on the recipient's server: for example, specifying the ESMTPS protocol or having parameters of the form (version = TLS1_2 cipher = ECDHE-RSA-AES128-GCM-SHA256 bits = 128/128); indicates the presence of TLS.
.https://example.com/somepath ). Typical errors are the use of relative links (/somepath) and the absence of a protocol (//example.com/somepath), which is not allowed for emails, because in them the default protocol can be file: //.
Any service and non-ASIC symbols (in particular, Cyrillic) in the URI must necessarily be encoded with percent encoding.
A link inserted as text (ie visible to the user as a URL, rather than as a piece of text) should still be marked through the <а> tag. , otherwise the user will not be able to click on it. Some web mails mark such links themselves, but this is not standard behavior. The href address inside A must then match the link text, otherwise the content filter can react to such a link as an attempt to deceive the user. This should be especially noted when there are "clickers" that track the user's transitions from the letter.
It is better to use the httr: //, httrs: //and malito: protocols.
With high security requirements, you should completely stop using http: //in favor of httrs: //.
Do not use non-standard ports (for example, Example.com : 8080 /somepath), because they may not be available to the user.
Going through the link inside the HTML part should not lead to any changes in the status of the application (subscription, unsubscription, cancellation, etc.) without additional confirmation by the user on the page, because Some content filtering systems can independently check the security of such a transition by requesting a page by reference; mail application can show a preview of the page by reference when hovering the mouse, and modern browsers can load the page before the user clicked on the link to shorten the download time (in the web application, it is generally not recommended to do any modifying actions on the GET request, requests must go through POST or PUT).
Go to the link in the List-Unsubscribe header, on the contrary, should not require any additional actions from the user, because for the user an unsubscription from this header is usually made by the mail program.
Do not expect from the user that he is reading the message and clicking on the link in the same browser in which he initiates the action that leads to the sending of the message (for example, registers an account). The link should work in any other browser or mobile device. In particular, the user can open the link, being unauthorized, or authorized in an account other than the one to which the letter was sent.
Because The length of the URI can be limited, it is not worthwhile for large objects to use the URI of the dаta: type. For the same reason, you should not use too long URIs in your links.
You can not use external shortcuts, it negatively affects the delivery of letters. It's better if all links point to your domain, this will reduce the potential negative impact of someone else's reputation on the delivery of letters.
Do not place external images on any public services or free hosting.
otherwise they can be used for spamming (in the field displayed in the letter, for example, in the name, the spam text is inserted and the address of the victim is indicated as the address). For example, if you can send obscene text to the developer's address on behalf of your service, then there is a problem.
Check the absence of external images on third-party services.
Check the availability of counters for sending, delivering, reading letters, transitions. Some of them are in the letter itself (for example, the counter-pixel reading of the letter), some are tracked by the mailer, but, as a rule, all are available in the dispatcher's admin panel.
Check the correctness of the subscription category and the work of the user's unsubscription for this category through the link in the letter.
Check the display at:
popular web versions of mail: to the "big three" Mail.Ru, Yandex, Gmail, you can add Rambler and Outlook.com;
mobile applications listed above postal providers;
standard mobile applications using IMAP protocol, taking into account popular mobile platforms, at least for iPhone, Pixel (reference platform Android), Samsung (the most common for Android), MIUI (second place for Android-platforms);
various desktop browsers: Chrome, Firefox, Edge, Internet Explorer, Opera, etc .;
desktop applications (e-mail programs), necessarily Thunderbird, Outlook and Apple Mail, optional The Bat! and Opera Mail;
popular corporate solutions with a web interface (Exchange, optional Roundcube, Communigate, Zimbra, SquirrelMail) - for B2B solutions;
do not forget to check the layout on both Retina-monitors, and on monitors with a lower resolution.
During the check in each case, you need to pay attention to:
Passing authorization headers, SPF /DKIM /DMARC.
Speed of letter loading: it should be loaded quickly, do not hang.
The display of the letter in the list of letters: avatar, sender's name and subject that falls into the snippet letters, whether its category was correctly defined (for example, whether the order has fallen into the category of "social network").
The layout of the letter as a whole: nothing is broken up, there are no incorrect hyphenations, etc., including when scaling and resizing the window.
Fonts should not be small or poorly readable.
Background images and background colors.
Matching brand book.
Convenience of carrying out actions implied by the letter. For example, if a letter contains a confirmation code or other information that may need to be stored somewhere, then it should not only be read well, it should also be conveniently selected and copied even in the mobile interface.
Keep track of the overall size of the letter (including external images) and so that it does not exceed reasonable values. The heavier the letter on traffic and download time, the more likely a negative reaction to it.
Even letters to which no changes are made should be checked periodically, because changes may occur on the side of the postal service, and may, for example, "fire" an previously unseen problem.
Some parameters need to be monitored in all tests. For example, problems with DKIM authentication can be due to infrastructure problems (DNS problems or DKIM signature generation, time synchronization errors), due to the errors of the forming program (incorrect sender address, incorrect characters in headers, missing or duplicate the mandatory From, Date or Message-ID headers) and because of content errors (incorrect line terminators, too long lines, incorrectly specified addresses). At the same time, the letter can not be "beaten" anywhere, and the problem can not be manifested on any service.
It is necessary to conduct a split-testing of various mailing options for obtaining optimal indicators. Just changing the name of the sender and the subject of the letter can increase the CTR by several times and repeatedly reduce the number of complaints. The number of letters should be statistically significant for the evaluation of results (for large projects it is usually several thousand). The final version of the letter is sent (in several stages for additional measurement of indicators and "warming up" - starting from about 1?000 recipients, with an increase of about an order of magnitude per day).
The main idea: emails are part of your application, perhaps one of the most complex and problematic. At the same time, this is often a "blind spot" in terms of testing. I hope that I managed to draw your attention to this problem.
I express great gratitude for the help in the preparation of the article to Vladimir Dubrovin z3apa3a and Alena Likhacheva s4ever . The article also used the materials of Eduard Tiantov EdT and Alexander Purtov 4Alexander .