I'm attempting to set up account confirmation and password resets passing through an intermediate email validation step first, but I'm still working through some of the details before implementation as I'm fairly new to back end development. My tech stack is Node, Express, mongoDB, Mongoose, Nodemailer, and potentially Mailgun (as it seems to be free for the purposes of my app).
As I have learned, sending emails from the web server where my app would be hosted on is generally a bad idea given that it has no reputation as an email server, and therefore likely to end up in a spam folder or rejected altogether.
Does Mailgun offer free email servers, or must I look elsewhere?
You can send up to 300 emails/day from your sandbox server to authorized recipients only.
On signup, Mailgun provides a sandbox server to test email delivery, however the sandbox server only allows emails to be sent to authorized recipients (i.e., recipients explicitly added to an authorized list in Mailgun). Of course, this will not do for anything production related.
In order to send emails to recipients without having them on an authorized list, one must provide their own domain.
Related
I don't have any technical problems, but I have a question that I would like to be answered out of curiosity.
Here is my current understanding of how email works:
One of the privileges of having your own domain is that you can hook it up to IMAP/POP3/SMTP servers and use them to send and receive messages to and from "anyone#yourdomain.com". With spam being such a problem, however, the SMTP server that you use to send messages must add a bunch of headers (DKIM, SPF, etc) to each message that you send in order to prove that the SMTP server has the authority to send emails from that domain. The receiving SMTP server can cross-check those headers with DNS records that it finds to verify the legitimacy of the email message.
So if you want to send emails with your domain cheaply, you can use Gmail's "Send Mail As" feature. I followed this help article to get mine working: https://support.google.com/domains/answer/9437157
Note: I unchecked the "Treat as an alias" option during the setup.
But wait... no additional DNS configuration required? I have my domain registered with Cloudflare, and there are no entries related to Google in there.
There is this step in the setup process:
But it seems that this only for Google to prevent you from using their servers to send spam. What is stopping Google from impersonating any email address they want? Why do receiving SMTP servers trust an email from "anyone#yourdomain.com" if Google's SMTP servers have no way of adding legitimate SPF/DKIM headers?
The short answer is that nothing prevents Google from doing this, and that DMARC was created for exactly this case.
There is nothing that stops Google from impersonating any domain. However, there are things receivers can (and should) do when they receive an email which isn't send from the server indicated in the From: field.
Try sending an email from the alias you just added to a different #gmail.com inbox. You will see that it says via gmail.com behind the sender email address. But other email receivers might do more: flag this message with red exclamation marks and scam warnings, throw it into spam or even deny receiving it completely. Gmail probably has some hardcoded trust, but try doing this from your own SMTP server and the above will very likely happen.
As you say in your question, you can authorize your emails by marking gmail.com as an authorized sender with SPF (which protects against forging from other domains, but doesn't stop Google), or even sign your emails with DKIM (not possible from Gmail UI, but you can do this in some email clients or send email with a custom Python script like me; Google can't do this without knowing the key).
However, that only solves one side of the problem – authorizing legitimate email messages. But what if an SMTP server still receives an unverified email? What if they have previously received an email from the same sender which was DKIM signed? What if DKIM passes, but SPF fails?
Because the behavior in that case is largely unspecified, and also the sender wants to check if their DKIM/SPF authorizations are actually working, and if anyone is attempting to spoof them, another standard was created: DMARC. It introduces another DNS TXT record where you can say what checks are required to pass, what to do if they fail, and also what basic analytics should the receivers report to the owner of the domain.
Of all webmail client providers, Google's Send mail as is actually the most well-implemented for a variety of reasons.
First of all, how it works is not different from when you set up POP3 or IMAP using a mail client like Outlook or Thunderbird. You have to specify the domain and port where you receive emails from, and the domain and port where you send emails from. For example, Google's incoming and outgoing servers for IMAP are as follows:
imap.gmail.com:993
smtp.gmail.com:465
The Send mail as feature is a partial implementation of that. It only implements the outgoing part.
How mail clients like Outlook and Thunderbird send emails, is basically that it sends the email to the outgoing mail server, and the outgoing mail server then sends the email. Usually, outgoing mail servers will require some sort of authentication, and will allow authenticated users to only send from specific email addresses.
Gmail works the same way. The outgoing mail server is the one that has to pass the SPF and DKIM tests, not Google's servers.
No other webmail clients do this. Hotmail used to do this, but they recently removed the feature. Now, the option is very difficult to find, and they just rewrite your FROM address and sends your email from Hotmail's SMTP server, which creates delivery problems.
They don't provide you with the option to send emails from another SMTP server, because this allows people to very easily set up virtual mail servers that can send emails under a domain of your choice, but use say a typical free Hotmail account to store incoming mails. This takes away business from their paid services, because both Hotmail and Gmail sell the option to host your company emails. I'm sure Google also knows about this, but it is really awesome of them to still keep the option available to free Gmail users.
If you want to learn more about virtual email servers, you can check out this article here: https://blog.terresquall.com/2022/01/setting-up-a-virtual-postfix-mail-server-part-1/
I am creating a node js application, where I required to send and receive the email.
I wanted to create a custom email like myname#mycompany.com for every user of this web app.
How to create this programmatically using the SMTP Server?
also wanted to send and receive email through the SMTP server.
any suggestion for SMTP server provider to achieve this programmatically?
tldr
If you want your app to be self-contained it needs to function as an SMTP client to send email directly to recipients, and as an SMTP server to receive email. If you want to use a provider Amazon SES is a solid offering.
You might be misunderstanding SMTP
In SMTP, the client (the one who initiates the connection) is always the email sender and the server is always the recipient. SMTP gets used in 2 contexts, but fundamentally it's the same thing going on, just with and without requiring authentication.
The first is when users send email to their email provider. For example: I, Alice, use Gmail. I want to send an email to bob#yahoo.com. I start by sending the email to Gmail via SMTP. Gmail's SMTP server is smtp.gmail.com. smtp.gmail.com requires authentication. I authenticate using my Gmail username and password. Once authenticated I am allowed to send email from alice#gmail.com to anyone.
The second is when a server wants to send email to another server. For example: I'm Gmail. One of my users, Alice, just submitted an email to bob#yahoo.com. I need to send that message to the mail servers responsible for the yahoo.com domain, so I look at the MX records for yahoo.com. I see that yahoo.com has 3 mail servers listed: mta5.am0.yahoodns.net, mta6.am0.yahoodns.net, and mta7.am0.yahoodns.net. I am supposed to try the one with the lowest priority value first, but all 3 have a priority of 1, so I just pick one at random. I open an SMTP connection to mta7.am0.yahoodns.net. This server does not require authentication. I am allowed to send email from anyone to anyone#yahoo.com.
SMTP does not have commands to create an account. In fact, SMTP can operate without accounts at all, just like HTTP can operate without a directory of files to serve.
As far as SMTP is concerned the only way to receive mail is to be the SMTP server listening on port 25 for the address listed in the MX record for mycompany.com. There is no way to connect to another SMTP server and ask "give me all the email for myname#mycompany.com". That is what POP and IMAP are for.
How to do what you want
You have 2 separate problems. You need to be able to send mail and you need to be able to receive mail.
Sending
Whenever you need to email foo#bar.com, you could look up the MX records for bar.com, open an SMTP connection to that server, and deliver the message directly. The main problem with this is trying not to wind up in people's spam folder. There are several tricks to this, but that is a whole subject of it's own.
You could use an intermediate SMTP server that you pay someone else to operate. SendGrid and MailGun are 2 examples here. Just like when Alice connected to smtp.gmail.com, you could connect to SendGrid's SMTP server. You would authenticate with them, but unlike with Alice who could only send from alice#gmail.com to anyone, you would be allowed to send email from anyone#mycompany.com to anyone. SendGrid would take care of making sure your emails don't wind up in people's spam.
You could use some other non-SMTP API to send email. Amazon SES for example.
Receiving
You could operate an SMTP server. It could even be built in to your node app if you wanted. You would just need to make sure your server was listening on port 25, and that 1st MX record (the one with the lowest priority) for mycompany.com pointed to that server. This lets you react the second you get an email. You might get a lot of spam doing this, so be ready for that.
You could pay someone to operate an SMTP server for you, and have them dump the email they get into a mailbox accessible via POP or IMAP. You would add an MX record for mycompany.com pointing to the company's SMTP server. You would add accounts using their API. You would poll for new email via POP or IMAP (IMAP supports notifications, but it is a more complex protocol).
Amazon SES also supports receiving email.
I have a website that requires people to confirm their emails upon registration. My backend server is a node server. I am using nodemailer to deliver these confirmation emails.
For testing, I am currently using a gmail account to send these emails. And this is working perfectly.
Now however, I would like to upgrade and use my own domain email. And I would like to send out these emails under an email address like "no-reply#mydomain.com"
One of the requirements for my site is that it can handle sending up to 2000 confirmation emails in a day.
I have looked at email hosting like godaddy and it says that one email account with them allows only 250 emails a day, which is way less than 2000.
So, how can I achieve 2000 emails a day in the cheapest way possible? Any recommendations will be helpful.
Thank you in advance for helping me with my question.
I highly recommend GSuite by Google.
Once registered simply create a no-reply email (If I recall correctly, nodemailer needs the email to exist on the smtp server to work) and link it up. Good luck
I want to use node-mailer module to send emails to registering users on app for verification and other purposes also. I am not sure which transport mechanism to use so that emails I send are not a spam and also get delivered.
I also want to use the same domain to receive emails regarding support and bugs(i.e lets say on email on support#example.com). I am planning to set-up google apps account for this. Basically I will set-up the google apps account with me being an user and having an alias support#example.com. So can I also set-up an alias with say admin#example.com and send the emails using nodemailer from my app using this email address? But obviously I don't want to store my email id and password for google apps in the code for security reasons. So how Should I do it?
Other option is to Amazon SES, I believe its a good service and would be very useful. But can I use the same domain name for serving email from AMAZON SES and google apps also being able to host my app? So how to send emails from AMAZON SES by authenticating SPF and DKIM so that these emails reach the inbox of the user.
Saransh,
You absolutely can use the same domain and email address with both Google Apps and AmazonSES without any issues. When it comes to receiving emails, all messages have to be directed to a single server (your MX records). But sending email is very different in that you can send email from many different sources without any issues.
To ensure your messages don't go to the SPAM folder just be sure to properly authenticate all your messages with SPF and DKIM. I'm not sure how this works with AmazonSES, but it was really easy with the provider that I chose for my email http://socketlabs.com
I have signed up with Google Apps, I am using a third party SMTP Server to send the emails from the web app, Emails are reaching inbox for all the email clients (yahoo, gmail, hotmail...)
But Emails sent to my domain (mydomain.com) are reaching SPAM, This is happening only for my domain Which is google apps account domain. The solution to the problem is the sender must be in the contacts of the receiver. So its a overhead involved in adding the sender in the contacts before receiving the mail. This is not automated.
NOTE: Mails sent from <xyz#mydomain.com> to <abc#mydomain.com> are reaching SPAM. It means, Its happening only when the sender and receiver belongs to same domain.
ASSUMPTION: ALL the emails will be sent from mydomain.com .
So, I want to write a script using google apps script So that when a mail arrives from mydomain.com (having FROM address belongs to mydomain.com), The script must execute and add the senders emails address in the contacts. So it will avoid the spam rate.
I am very new to google apps script, Please help me doing this, Or if any better solution is there please feel free to post.
Vinay,
It sounds like you may have an incorrectly configured SPF record. Please see this article:
https://support.google.com/a/bin/answer.py?hl=en&answer=33786&topic=2759192&ctx=topic
Email that has a mydomain.com sender address, needs to come from a mail server listed in your SPF record or risk being sent to spam.
By chance, has Contact Sharing been enabled for your apps domain? I'm just offering this as an hunch that might help rather than a specific solution to your problem.
If this doesn't help, its a straightforward but non-trivial problem to add all your domain accounts to each others contacts list (and keeping it synced over time), but you'll be leaning heavily on a version of the Contacts API rather than plain Apps Script because the Contacts service only operates on the contacts of the user executing your script.