I’ve been working on a small side project that involves processing incoming email. In particular, it’s an app that needs to do something for each email it receives from (hopefully paying!) users.
I am not interested in storing user mail, so SES is just too costly, at least according to a quick worst-case calculation.
That leaves me with two options:
1. Self-hosted Postfix
2. Mail service like Mailgun
With (1), there is no need to worry about overages, but scaling the mail server might be challenging.
The advantage of (2) over SES is that you are only charged a flat fee for each email, regardless of size. Emails are then automatically deleted after some period of time. Scaling up and down is easy.
For now, I am using Mailgun, but I am writing the mail processing daemon in a way that will make it easy to transition to Postfix, if needed.
Also, I decided to write the mail processing backend in Rust, so I’ve been learning the language as I go!
> I’ve been working on a small side project that involves processing incoming email. In particular, it’s an app that needs to do something for each email it receives from (hopefully paying!) users.
I wish you all the best! Mind if I ask for the link?
> With (1), there is no need to worry about overages, but scaling the mail server might be challenging.
Honestly, quite the opposite.
1. Duplicate your MX box.
2. Duplicate your MX record.
That is it :)
> I am writing the mail processing daemon ... in Rust...
You might like to take a look in https://github.com/mailman/mailman for ideas/inspiration. It's a great tool for processing emails too, but cannot deny I'm now curious to see how one in rust will look like.
Yes, I only learned about MX record priorities last night haha. With Postfix, the most straightforward way to run code on receiving an email seems to be through a pipe filter. Running multiple filter processes probably requires a beefy server.
Thanks for that link! I might just use a similar approach to allow users to configure how to receive emails (HTTP or stdin, etc.).
Then limit the number of filters... you can have postfix run a fixed number of smtpd processes, and each process handles only one message at a time.
When they're all full, your server will just stop handling messages, but SMTP will retry anyway, giving you plenty of time to scale up if the load is consistently too high
I’m actually doing something similar, I opted for SES, S3 and Lamda, I am a bit worried about costs, especially when i need to scale up, and handle larger emails.
I am not interested in storing user mail, so SES is just too costly, at least according to a quick worst-case calculation.
That leaves me with two options:
1. Self-hosted Postfix
2. Mail service like Mailgun
With (1), there is no need to worry about overages, but scaling the mail server might be challenging.
The advantage of (2) over SES is that you are only charged a flat fee for each email, regardless of size. Emails are then automatically deleted after some period of time. Scaling up and down is easy.
For now, I am using Mailgun, but I am writing the mail processing daemon in a way that will make it easy to transition to Postfix, if needed.
Also, I decided to write the mail processing backend in Rust, so I’ve been learning the language as I go!