Tom Wemyss

Writing emails in markdown with neomutt and pandoc




Mutt is a fast, user-friendly and easily-extensible terminal-based email client. I’ve been using mutt for my emails for a couple of years, but I’ve found myself falling back to using Google’s Gmail for emails that I cannot easily compose in plain text - such as those that require text formatting. I still really enjoy using mutt (there are a lot of reasons to prefer it over other email clients), so I wanted to find a way to bring it back into my daily email workflow.

Neomutt is a fork of mutt which retains compatibility with mutt configuration files. One of the advantages of using neomutt over mutt is that it has support for sending emails with a multipart/alternative content type. This means that you can send both HTML content (with fancier formatting for email clients which support it) and plain-text content (great for viewing in email clients like mutt) in the same email, and simply let email clients decide which version they prefer to display.

However, manually writing out every email in both plain-text and HTML would be tiresome. It’s much easier to just write emails once, in a user-friendly markup language such as markdown and let software such as pandoc convert it into HTML and plain text.

With help from neomutt macros, this conversion can be integrated with neomutt, so that there’s no need to exit neomutt or significantly alter your workflow for sending emails.

Configuration

This writeup assumes that you have neomutt installed and working. If not, the following example configuration files might be useful:

  • A sample set of configuration files for neomutt with two accounts, using the SMTP client built into neomutt. This is my favoured approach at the moment, because the inbuilt SMTP support within (neo)mutt is good enough for my purposes, and I don’t want an MTA installed locally.
  • A more complex configuration using offlineimap to sync IMAP folders and mSMTP MTA. This could be helpful for later integration with powerful search programs, such as notmuch.
  • The Arch Linux wiki page for mutt - most configuration options are directly compatible between mutt and neomutt.
  • A solarized colourscheme for mutt/neomutt.

Once you’ve got neomutt installed and working, then you can proceed with the following instructions.

1. Installing pandoc

On my system, I chose to run pandoc inside a docker container. The reason for this is that the Arch Linux pandoc package comes with over 100 dependencies, totally nearly a gigabyte, which would increase the number of packages installed on my system by more than 15%.

To start using pandoc in docker, first install docker and then download the official pandoc docker image by running docker pull pandoc/core.

For those who don’t like docker, it’s also entirely possible to run this workflow using pandoc installed through your favourite package manager (or compiled directly from source). In those cases, the docker commands within the neomutt macro will need to be adjusted to call pandoc directly.

2. Set up a pandoc template for HTML emails

Add the following HTML template to ~/.mutt/templates/email.html.

~/.mutt/templates/email.html

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="$lang$" xml:lang="$lang$"$if(dir)$ dir="$dir$"$endif$>
<head>
  <meta charset="utf-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" />
  <style>
    $styles.html()$
  </style>
$for(css)$
  <link rel="stylesheet" href="$css$" />
$endfor$
  <!--[if lt IE 9]>
    <script src="//cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv-printshiv.min.js"></script>
  <![endif]-->
$for(header-includes)$
  $header-includes$
$endfor$
</head>
<body>
  $body$
  $for(include-after)$
  $include-after$
  $endfor$
</body>
</html>

3. Configure the mutt macro

Adding the code below to your ~/.muttrc sets up a macro which compiles markdown to HTML and plain text when you press “m” after composing an email.

~/.muttrc (partial excerpt)

1
2
3
4
5
6
7
8
9
macro compose m \
"<enter-command>set pipe_decode<enter>\
<pipe-message>docker run -i -v /tmp:/tmp --rm pandoc/core -f gfm -t plain -o /tmp/msg.txt<enter>\
<pipe-message>docker run -i -v /tmp:/tmp -v ~/.mutt/templates/email.html:/mutt/templates/email.html --rm pandoc/core -s -f gfm --self-contained -o /tmp/msg.html --resource-path /mutt/templates/ --template email<enter>\
<enter-command>unset pipe_decode<enter>\
<attach-file>/tmp/msg.txt<enter>\
<attach-file>/tmp/msg.html<enter>\
<tag-entry><previous-entry><tag-entry><group-alternatives>" \
"Convert markdown to HTML5 and plaintext alternative content types"

Mutt macros can be a little intimidating at first, so there’s a line by line description of what this macro does later in this article.

4. Sending your first email

Once you’ve typed your markdown email in your favourite text editor, you’ll arrive at the neomutt compose screen, where you’d usually rename attachments, edit the subject, or press ‘y’ to send the message, with the default key bindings.

Before pressing ‘y’ to send the message:

  1. Press ‘m’ to run the macro. All being well, this will automatically create another inline attachment containing both the HTML and the plain text.
  2. If you’re happy with the new attachment that got generated, then delete your markdown file by using the arrow keys to navigate the list of attachments and detaching the original text (pressing Shift and D with the default key bindings).
  3. Press send!

It’s worth experimenting a little bit with emails to yourself, to make sure things are actually appearing how you want.

Once you’re familiar with the process, you can even adjust your email signature to use markdown for formatting.

How does it work?

The configuration above works well for me - but an explanation of the setup is given below for those who may wish to customise it.

What does the neomutt macro do?

A line-by-line description of the neomutt macro is given below. The neomutt documentation provides further detail on configuring macros.

  1. macro compose m
    • This line tells neomutt that a new macro is being bound to the “m” key for the email “compose” screen.
  2. <enter-command>set pipe_decode<enter>
    • This sets pipe_decode which tells neomutt to decode the email it sends to the next stages of the macro.
    • Decoding the email piped to later stages of the macro means that the next stages of the macro only receive the text content of the email, rather than the entire email including headers.
  3. <pipe-message>docker run -i -v /tmp:/tmp --rm pandoc/core -f gfm -t plain -o /tmp/msg.txt<enter>
    • This line begins with <pipe-message>, which pipes the message you composed to the following command.
    • The following command runs the pandoc docker image to produce a plaintext email (from the markdown message you wrote), which it saves to /tmp/msg.txt.
  4. <pipe-message>docker run -i -v /tmp:/tmp -v ~/.mutt/templates/email.html:/mutt/templates/email.html --rm pandoc/core -s -f gfm --self-contained -o /tmp/msg.html --resource-path /mutt/templates/ --template email<enter>
    • This line again pipes the message to docker/pandoc.
    • This docker/pandoc command converts the markdown message to a HTML message, using the template at ~/.mutt/templates/email.html.
  5. <enter-command>unset pipe_decode<enter>
    • This line unsets the change made on line 2, so other macros you may use afterwards are not affected.
  6. <attach-file>/tmp/msg.txt<enter>
    • Attaches the plain-text message to the email.
  7. <attach-file>/tmp/msg.html<enter>
    • Attaches the HTML message to the email.
  8. <tag-entry><previous-entry><tag-entry><group-alternatives>
    • First this line tags (selects) the current attachment (the HTML message).
    • This line then moves to the previous attachment, and tags that too.
    • Finally, it groups the two selected attachments together as alternatives, which is necessary to achieve the correct content-type on the email.
    • Having the correct content-type means that most clients will automatically select the plain-text or the HTML, depending upon what they are capable of displaying.
  9. "Convert markdown to HTML5 and plaintext alternative content types"
    • This line is the macro description, which is shown in the help screen in neomutt. The help screen is displayed by pressing “?”.

Can I put inline images in these messages?

Kind of - technically images can be embedded, but the images won’t display in most email clients. It’s better to attach images to the email instead, and reference them from the text.

If you really want to put inline images in (bearing in mind they won’t display for most people), then you can copy your images to /tmp, where the docker container will be able to access them. You can then include them in the email using the standard markdown syntax for images (![alt text](/path/to/image.png)), making sure to reference images using an absolute path (/tmp/example.png).

It’s worth paying extra attention to the alt-text, because that will be very prominent for the majority of users to whom the image will not appear: users viewing the plain-text version of your email, and the majority of users with clients that won’t display the image.

Putting images inline (sort of) works because the pandoc command is run with the option --self-contained, which means that any images in the email text will be encoded into base64 and stored within the HTML email itself. In recent versions of pandoc, you can even adjust the size of the images by writing markdown like ![alt text](/tmp/image.png){width=300px height=200px}. In order to enable this feature, replace the two occurrences of -f gfm in the macro definition with -f markdown+link_attributes. That slightly changes the markdown syntax for the document (from GitHub Flavoured Markdown to Pandoc Markdown), but the differences will be negligible for most.

Why use a custom pandoc template?

Using the default HTML5 template provided with pandoc generates a title element within the email, which has the content “-“. This title element doesn’t appear on most clients, but it does show prominently on some, such as Thunderbird. It’s not ideal to have emails which display differently depending what software is used to view them. Thankfully, there are a few possible approaches to solving this:

  • setting the title of the HTML email to be the email subject. This can be achieved by using a bash script to extract the subject from the email, and then passing the subject into the pandoc command (e.g. --metadata title=$SUBJECT). However, the default HTML5 pandoc template will also prepend this title (in a H1 element) to the main email body, which would lead to it displaying before your email content. This would look quite unprofessional;
  • or setting the HTML title to whitespace. This can be done by adding --metadata title=' ' to the pandoc command that creates the HTML file. It’s a hacky solution, and it leads to an H1 element containing a space being created within the document when using the default pandoc HTML5 template, which could cause problems with screen readers or other programs which parse the HTML mail;
  • or using a custom HTML template for pandoc which does not have a title attribute.

The latter solution - to leave out the title element from the generated HTML - is the solution that seems most likely to avoid differences in formatting/display between different email clients viewing the same email. Furthermore, leaving out the title element is actually compatible with the HTML5 specification, which states:

The title element is a required child in most situations, but when a higher-level protocol provides title information, e.g. in the Subject line of an e-mail when HTML is used as an e-mail authoring format, the title element can be omitted.

The template used in this setup was created from the default pandoc HTML5 template, which can be extracted using pandoc -D html5 and then deleting the unwanted elements - mostly those related to the title and header attributes.

Can I write emails in LaTeX/Haddock/[language]?

Any formats supported by pandoc should work. For example, to write emails in LaTeX and have them compiled to HTML and plaintext, just change -f gfm to -f latex in the docker commands within the macro you added to .muttrc. The pandoc manual has a list of supported input formats.