FileMaker 18: How to Send HTML Emails Using Insert from URL

HTML Emails

FileMaker 18 added SMB, LDAP(S), and SMTP(S) to the list of protocols supported by the Insert From URL script step. The full list of supported protocols now is HTTP(S), FTP(S), FILE, SMB, LDAP(S), and SMTP(S). This blog post will focus on making use of cURL and the SMTP protocol to send HTML-formatted emails.

Prior to FileMaker 18, plain text emails could be sent using the Send Mail script step. FileMaker 17 introduced the ability to include multiple attachments. Now, fully stylized rich-text emails with attachments can be sent, but the process is not as simple as it was with the Send Mail step. This blog post breaks down much of what you need to know and supplements that with a demo file for some additional hands-on learning.

Insert From URL

To send an HTML email, the Insert From URL script step needs to have the following items specified:

• Target – stores the result of the cURL command
• URL – specifies the SMTP server URL where the cURL command will be directed to
• Verify SSL Certificates – determines whether the SMTP server SSL certificate should be verified
• cURL options – specifies the options used to fine tune the cURL command

URL

The URL specifies the protocol (SMTP or SMTPS), the fully qualified domain name of the SMTP server, and the port.

Here are some examples of SMTP URLs:

• smtp://smtp.mydomain.com:25/
• smtps://internal-smtp.mydomain.com:587/

cURL Options

The cURL options specify additional instructions for delivery of the email. Here are some of the options that you may use, along with some comments for each one:

--mail-rcpt cturner@email.com
--mail-rcpt awright@email.com

Message Format

The SMTP URL and cURL options tell cURL where to send the email, to whom, and how, but they do not directly specify what to send. Instead, the cURL options reference a mail file which includes the content of what is to be sent. This mail file, referred to as the message, has to be constructed following particular formatting rules.

The rules can get quite complex, and a big driver of the complexity is the fact that each new email standard aims to be backward-compatible with the older standard that it supersedes. Some examples of this are the rules that require content to use 7-bit US-ASCII characters with each line of text limited to 78 characters or less (more on that later).

In my testing, I’ve found variations in the extent to which some of these rules are enforced. My guess is this is due to differences in how the various SMTP servers and email clients have implemented the standards that define how internet email works, similar to how support for HTML and JavaScript is implemented differently across different web browsers. The lesson from this is that you may get something to work for a particular SMTP server or email client, but if one of those factors later changes, it may end up breaking the functionality.

MIME Formatting

In the simplest case, where the email message consists of only ASCII characters, none of the lines in the message is longer than 998 characters, and the email does not have any attachments, the message can be constructed using just one “part.” In more complex cases, the email message will use Multipurpose Internet Mail Extensions (MIME) formatting to handle different character encodings, to allow for attachments, or to include alternate (e.g. plain text) versions of the email message.

The message will always consist of a header and a body. In the case of MIME multipart messages, the body itself will be split up into multiple parts, each of which will have its own header and body.

Each part in a multipart message will specify a MIME type to indicate the type of content included in that part. Some examples of MIME types include:

• Plain text: text/plain
• HTML text: text/html
• PNG image: image/png
• PDF: application/pdf
• Zip: application/zip

The full list of defined MIME types can be found here.

So that email clients can tell where one part ends and another begins, each part in a multipart message is separated by a boundary which has to be a string of printable 7-bit ASCII characters (codes 32 through 126) with a length of 70 characters or less, and which must not appear in any of the encapsulated parts. Here is an example of a valid boundary: gc0p4Jq0M2Yt08j34c0p.

Each time a boundary is used, it is prefixed with two hyphens. The last boundary is also suffixed with an additional two hyphens.

MIME Parts

There are three kinds of MIME parts, and therefore three kinds of part boundaries.

Alternative: The “alternative” boundary separates alternate versions of the message. For example, there could be a plain text version of the email to accompany the HTML version, so that old-school plain-text-only email clients can display the message correctly.

The other two types of MIME parts have to do with attachments. Attachments can be embedded within the email message, or they can be attached to the email message in such a way so that they have to be saved out of the email before they can be accessed.

Related: The “related” boundary separates embedded attachments from each other and from the email message.

Mixed: The “mixed” boundary separates non-embedded attachments from each other and from the rest of the message.

The various different parts are nested with the mixed parts forming the outer-most portion of the structure and the alternative parts forming the inner-most portion.

At this point, your head is probably spinning with all of the details, so hopefully, this visual explanation will help clarify your understanding:

The main header section can contain fields such as From, To, Cc, Subject, and Date, but note that the To and Cc fields are not used to determine where the message should be delivered. That information is determined from the cURL options (covered above). The analogy is a paper letter that contains the destination address both on the envelope and on the letter itself. The address on the envelope is the one that is used for delivery. The address shown at the top of the letter is included for informational purposes only.

MIME Format Example

Here is an example of a simple, non-MIME formatted message:

From: Liam Lopez <llopez@email.com>
To: Cameron Turner <cturner@email.com>, April Wright <awright@email.com>,
 Theodore Hall <thall@email.com>
Cc: Zara Allen <zallen@email.com>
Subject: Test
Date: Wed, 14 May 2019 19:34:21
Content-Type: text/html; charset="utf8"

<html><body>Hello, here is my <b>bold</b> email message.</body></html>

A couple of things to point out:

• The message body is separated from the header section with an empty line.
• Each line should be separated by CRLF (carriage return and line feed) characters.
• Each line should be kept to a length of no more than 78 characters. The technical limit is actually 998 characters, but the recommended limit is 78. Longer lines should be broken out into multiple lines with a whitespace character beginning each new line. This process is called “folding.”
• The message (both the header and the body) must be comprised of 7-bit ASCII characters.
• The Bcc addresses, if there are any, are typically not included in the message header.

For MIME-formatted messages, non-ASCII data can be converted to 7-bit ASCII using an encoding scheme such as “quoted-printable” or base64 (RFC 2045). When encoding content that consists of mostly Latin-based text, the “quoted-printable” encoding is preferable, since it is human-readable. This makes troubleshooting easier. Both of these encoding schemes automatically fold lines at lengths of 76 characters.

Quoted-printable Encoded Text

Here’s an example of what quoted-printable encoded text looks like:

Original:

Congratulations to Luka Modrić 🇭🇷⚽ for winning the Ballon d'Or!

Quoted-printable:

Congratulations to Luka Modri=C4=87 =F0=9F=87=AD=F0=9F=87=B7=E2=9A=BD for w=
inning the Ballon d'Or!

Unfortunately, FileMaker doesn’t provide a native quoted-printable encoding function, so I used base64 encoding in the demo that accompanies this blog post.

Additional MIME Examples

Here are some additional visual guides that show how email messages (both MIME and non-MIME) are put together.

The pilcrow character (¶) is displayed to denote places where an empty line should be inserted.

To simplify things, some of the text is represented using placeholders. This is denoted with angled brackets; e.g. <<EMAIL_MESSAGE_HTML_TEXT>>.

HTML Email Only

The first example is of a simple non-MIME HTML email.

When I tested sending out this kind of email, I was able to include non-ASCII characters, but the standard specifies that only printable 7-bit ASCII characters should be used. This is another example of the point made earlier, which is that different SMTP servers and email clients implement the standards in different ways, including building in support that allows for stretching the rules. However, if you want to ensure that your email implementation will continue to work regardless of what server or client is used, better adhere to the rules defined in the standards.

HTML Email + Attached Files

This second example shows two attachments included along with the HTML email. In this case, the attachments are not embedded within the email message.

The preamble message is there so that non-MIME compliant email readers can have some intelligible message to show to the user. This message can be anything you want. For example:

This is a message in MIME format.  If you see this, your mail reader does not support this format.

HTML Email + Embedded Files + Attached Files

The following example adds two additional attachments, but unlike the previous two, these new ones are embedded within the email. This is done by setting the content disposition to “inline”.

Embedding attachments is commonly done with images. When the HTML for the email is constructed, these attachments can be referred to by their content ID, which is defined in the part header. For example, suppose that the content ID for one of the attachments is defined as follows:

Content-ID: <image001.png@01D4FC56.1590DFB0>

Then the image can be referenced inside the HTML email like this:

<img src="cid:image001.png@01D4FC56.1590DFB0">

Plain Text Email + HTML Email + Embedded Files + Attached Files

The last example uses “alternative” boundaries to add a plain text version of the email message.

Using Encoded-Words in Message Headers

The preceding examples describe how a message body can be encoded to account for non-ASCII characters, but what if the message header also contains these unsupported characters?

For instance, the name of a recipient or the subject of the email could contain characters that fall outside of the 7-bit ASCII range. In that case, you can use encoded-words to convert sequences of unsupported text using the following syntax:

=?charset?encoding?encoded-text?=

For example, suppose the email subject consists simply of the following smiley face emoji: 😋

As a base64 encoded-word, the subject message header would appear as:

Subject: =?UTF-8?B?8J+Yiw==?=

In the example above, the character set is specified as “UTF-8”, the encoding is specified as “B” (for base64), and the encoded text is specified as “8J+Yiw==”, which is the base64 equivalent of 😋

The two encoding options are “B” for base64 and “Q” for quoted-printable.

Similarly, the following recipient message header…

To: 老子 <laotzu@email.com>

…could be encoded as follows:

To: =?UTF-8?B?6ICB5a2Q?= <laotzu@email.com>

You can read more about encoded-word syntax in RFC 2047.

Conclusions and Demo File

We now have the ability to send HTML emails with attachments, but the step up in complexity is significant compared to the Send Mail script step.

The accompanying demo file includes a script which does a lot of the heavy lifting for you. The script was written to be portable so that you can copy it into your solution without having to set up any tables or fields. (The script does rely on the “GetMIMEType” and “ErrorMessage” custom functions though, so you will need to copy those before copying the script.)

The demo file supports fully stylized emails which can be achieved through HTML or through FileMaker field rich text formatting (using the GetAsCSS function). Attachments can be designated as embedded (“Inline”) or as regular, non-embedded (“Attachment”).

The demo file does not, however, support alternative MIME parts, which are used to provide alternate (e.g. plain text) versions of the email message. And it does not do any folding or encoding of message headers. So if your message headers (e.g. subject or recipient email addresses) use non-ASCII characters or are longer than 998 characters in length, the email might not get sent out.

If you do not have an SMTP server readily available for testing, you can create a Gmail account and use the Gmail SMTP server. If you have two-factor authentication enabled (and you should), you will need to create an application-specific password. You can follow these instructions to do so.

As a final word, I’d like to thank my colleagues Brian Engert and Marcelo Piñeyro for their assistance with preparing this blog post and the demo file. If you are looking to implement something similar for your FileMaker solution, please contact our team.

References

• Message format for non-MIME emails: https://en.wikipedia.org/wiki/Email#Message_format
• RFC 5322 standard for non-MIME emails: https://tools.ietf.org/html/rfc5322
• Wikipedia article on MIME messages (includes links to the relevant RFC standards): https://en.wikipedia.org/wiki/MIME
• Message header fields: https://www.iana.org/assignments/message-headers/message-headers.xhtml
• Encoded-word syntax (RFP 2047): https://tools.ietf.org/html/rfc2047
• Complete list of MIME types: https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Complete_list_of_MIME_types
• An excellent guide to multi-part MIME messages written by Daniel Clark: http://qcode.co.uk/post/70