I am working on an application that requires communication via email with the user and the client for whom the application is being developed.
Basically the flow is thus :
Program Does Something
Program sends an email detailing if it worked or not to the user and my client
We have established a Gmail account to make this happen. Unfortunately my clients users are multinational and Gmail freaked out and started blocking users, which started throwing exceptions when they were trying to use the program.
In the emails I saw when I checked the account, I saw I see at least two of these emails that say "Suspicious sign in prevented", and there was a red bar that said multiple attempts were tried to sign in and asked if it was me, and there were points all over the world.
How would I go about making it so that Gmail doesn't freak out when it sees users trying to sign in all over the planet?
Isn't sending through a some kind of a server application not an option? I.e. instead of making SMTP connection from your end user app directly to Gmail server your app sends a request to your web server which does actual SMTP connection. This way you'll also eliminate issues when SMTP ports are blocked on end users' machines.
Related
About two years ago, I created a Node.js application that would send emails periodically through Gmail's Less Secure Apps option. This worked quite well, but they recently got rid of it. I have G-Suite email addresses under my domain that I would like to send emails from, but the new Gmail API takes quite a bit of effort to set up and maintain. Is there a way that I can use Node.js to send emails from those addresses, but in a simpler way? Could I possibly use another SMTP server, like Yahoo or iCloud, with Nodemailer?
if you want to continue as you are just create an apps password.
otherwise you can configure a service account and set up domain wide deligation through your workspace domain account.
The service account will be able to send email on behalf of a user on your domain without any authorization needed. This is probably the most robust solution
I don't have any technical problems, but I have a question that I would like to be answered out of curiosity.
Here is my current understanding of how email works:
One of the privileges of having your own domain is that you can hook it up to IMAP/POP3/SMTP servers and use them to send and receive messages to and from "anyone#yourdomain.com". With spam being such a problem, however, the SMTP server that you use to send messages must add a bunch of headers (DKIM, SPF, etc) to each message that you send in order to prove that the SMTP server has the authority to send emails from that domain. The receiving SMTP server can cross-check those headers with DNS records that it finds to verify the legitimacy of the email message.
So if you want to send emails with your domain cheaply, you can use Gmail's "Send Mail As" feature. I followed this help article to get mine working: https://support.google.com/domains/answer/9437157
Note: I unchecked the "Treat as an alias" option during the setup.
But wait... no additional DNS configuration required? I have my domain registered with Cloudflare, and there are no entries related to Google in there.
There is this step in the setup process:
But it seems that this only for Google to prevent you from using their servers to send spam. What is stopping Google from impersonating any email address they want? Why do receiving SMTP servers trust an email from "anyone#yourdomain.com" if Google's SMTP servers have no way of adding legitimate SPF/DKIM headers?
The short answer is that nothing prevents Google from doing this, and that DMARC was created for exactly this case.
There is nothing that stops Google from impersonating any domain. However, there are things receivers can (and should) do when they receive an email which isn't send from the server indicated in the From: field.
Try sending an email from the alias you just added to a different #gmail.com inbox. You will see that it says via gmail.com behind the sender email address. But other email receivers might do more: flag this message with red exclamation marks and scam warnings, throw it into spam or even deny receiving it completely. Gmail probably has some hardcoded trust, but try doing this from your own SMTP server and the above will very likely happen.
As you say in your question, you can authorize your emails by marking gmail.com as an authorized sender with SPF (which protects against forging from other domains, but doesn't stop Google), or even sign your emails with DKIM (not possible from Gmail UI, but you can do this in some email clients or send email with a custom Python script like me; Google can't do this without knowing the key).
However, that only solves one side of the problem – authorizing legitimate email messages. But what if an SMTP server still receives an unverified email? What if they have previously received an email from the same sender which was DKIM signed? What if DKIM passes, but SPF fails?
Because the behavior in that case is largely unspecified, and also the sender wants to check if their DKIM/SPF authorizations are actually working, and if anyone is attempting to spoof them, another standard was created: DMARC. It introduces another DNS TXT record where you can say what checks are required to pass, what to do if they fail, and also what basic analytics should the receivers report to the owner of the domain.
Of all webmail client providers, Google's Send mail as is actually the most well-implemented for a variety of reasons.
First of all, how it works is not different from when you set up POP3 or IMAP using a mail client like Outlook or Thunderbird. You have to specify the domain and port where you receive emails from, and the domain and port where you send emails from. For example, Google's incoming and outgoing servers for IMAP are as follows:
imap.gmail.com:993
smtp.gmail.com:465
The Send mail as feature is a partial implementation of that. It only implements the outgoing part.
How mail clients like Outlook and Thunderbird send emails, is basically that it sends the email to the outgoing mail server, and the outgoing mail server then sends the email. Usually, outgoing mail servers will require some sort of authentication, and will allow authenticated users to only send from specific email addresses.
Gmail works the same way. The outgoing mail server is the one that has to pass the SPF and DKIM tests, not Google's servers.
No other webmail clients do this. Hotmail used to do this, but they recently removed the feature. Now, the option is very difficult to find, and they just rewrite your FROM address and sends your email from Hotmail's SMTP server, which creates delivery problems.
They don't provide you with the option to send emails from another SMTP server, because this allows people to very easily set up virtual mail servers that can send emails under a domain of your choice, but use say a typical free Hotmail account to store incoming mails. This takes away business from their paid services, because both Hotmail and Gmail sell the option to host your company emails. I'm sure Google also knows about this, but it is really awesome of them to still keep the option available to free Gmail users.
If you want to learn more about virtual email servers, you can check out this article here: https://blog.terresquall.com/2022/01/setting-up-a-virtual-postfix-mail-server-part-1/
I am creating a node js application, where I required to send and receive the email.
I wanted to create a custom email like myname#mycompany.com for every user of this web app.
How to create this programmatically using the SMTP Server?
also wanted to send and receive email through the SMTP server.
any suggestion for SMTP server provider to achieve this programmatically?
tldr
If you want your app to be self-contained it needs to function as an SMTP client to send email directly to recipients, and as an SMTP server to receive email. If you want to use a provider Amazon SES is a solid offering.
You might be misunderstanding SMTP
In SMTP, the client (the one who initiates the connection) is always the email sender and the server is always the recipient. SMTP gets used in 2 contexts, but fundamentally it's the same thing going on, just with and without requiring authentication.
The first is when users send email to their email provider. For example: I, Alice, use Gmail. I want to send an email to bob#yahoo.com. I start by sending the email to Gmail via SMTP. Gmail's SMTP server is smtp.gmail.com. smtp.gmail.com requires authentication. I authenticate using my Gmail username and password. Once authenticated I am allowed to send email from alice#gmail.com to anyone.
The second is when a server wants to send email to another server. For example: I'm Gmail. One of my users, Alice, just submitted an email to bob#yahoo.com. I need to send that message to the mail servers responsible for the yahoo.com domain, so I look at the MX records for yahoo.com. I see that yahoo.com has 3 mail servers listed: mta5.am0.yahoodns.net, mta6.am0.yahoodns.net, and mta7.am0.yahoodns.net. I am supposed to try the one with the lowest priority value first, but all 3 have a priority of 1, so I just pick one at random. I open an SMTP connection to mta7.am0.yahoodns.net. This server does not require authentication. I am allowed to send email from anyone to anyone#yahoo.com.
SMTP does not have commands to create an account. In fact, SMTP can operate without accounts at all, just like HTTP can operate without a directory of files to serve.
As far as SMTP is concerned the only way to receive mail is to be the SMTP server listening on port 25 for the address listed in the MX record for mycompany.com. There is no way to connect to another SMTP server and ask "give me all the email for myname#mycompany.com". That is what POP and IMAP are for.
How to do what you want
You have 2 separate problems. You need to be able to send mail and you need to be able to receive mail.
Sending
Whenever you need to email foo#bar.com, you could look up the MX records for bar.com, open an SMTP connection to that server, and deliver the message directly. The main problem with this is trying not to wind up in people's spam folder. There are several tricks to this, but that is a whole subject of it's own.
You could use an intermediate SMTP server that you pay someone else to operate. SendGrid and MailGun are 2 examples here. Just like when Alice connected to smtp.gmail.com, you could connect to SendGrid's SMTP server. You would authenticate with them, but unlike with Alice who could only send from alice#gmail.com to anyone, you would be allowed to send email from anyone#mycompany.com to anyone. SendGrid would take care of making sure your emails don't wind up in people's spam.
You could use some other non-SMTP API to send email. Amazon SES for example.
Receiving
You could operate an SMTP server. It could even be built in to your node app if you wanted. You would just need to make sure your server was listening on port 25, and that 1st MX record (the one with the lowest priority) for mycompany.com pointed to that server. This lets you react the second you get an email. You might get a lot of spam doing this, so be ready for that.
You could pay someone to operate an SMTP server for you, and have them dump the email they get into a mailbox accessible via POP or IMAP. You would add an MX record for mycompany.com pointing to the company's SMTP server. You would add accounts using their API. You would poll for new email via POP or IMAP (IMAP supports notifications, but it is a more complex protocol).
Amazon SES also supports receiving email.
I know there's an API for a Gmail plugin in your browser. Is there one for the server? For example, if I want to write an anti-virus app that reads emails coming in to an account on the server, then let's them continue or quarantines them - all before the user ever sees them?
Obviously it would have to be authorized for the users it did this for.
EDIT: As of 26/06/2014, this answer has become incorrect, as Google just released a dedicated API to GMail. See the link in this post for reference.
Original answer below this line :
From a technological perspective, it's a mail server just like any other.
Thus, your choice should fall on a mail protocol. It's also already answered in the question you linked: https://stackoverflow.com/a/2820280/1680196
Your app would open an IMAP connection to your mail server of choice (in this case gmail), scan for incoming mail, read the content and trigger a delete / move to trash if necessary.
Personally, I wouldn't trust any app like this running outside my own mailserver / mail client, since this use case requires full read and write access to a users mails.
There might be some API for Google Apps for Domains mail accounts, but I doubt this goes beyond the RSS mail notifications.
Edit: Official Google api documentation also refers to IMAP for accessing user inboxes. Link to IMAP OAuth overview. With application specific passwords users could enable your app to access their account without exposing their actual credentials, but beyond that I fear you are out of luck.
There is an HTTP REST API now!
http://developers.google.com/gmail/api/
(And has been for years if you were willing to use IMAP. :-D)
I have a SMTP server set up for my domain in IIS6. The mail sends just fine from the site, there are no errors. But the email is never delivered to the recipient. I checked my mail folder on the server and the emails are stuck in \inetpub\mailroot\drop
Any idea why they wouldn't be getting sent out? This is new territory for me and I'm not having much luck finding a solution.
Since the mail is making its way into the mail folder, I assume the problem lies somewhere in my SMTP server settings
There could be a couple of reasons for this and it sounds to me that your provider has blocked port 25 which means your server can't send the emails, hence, they are still in the queue.
Your best bet is to use a 3rd party SMTP service such as Amazon SES or Mandrill (free), you can either configure your application to send mail directly through these providers, or, you can continue to use IIS SMTP and configure smart host, this will tell IIS SMTP to not bother trying to send email directly, but instead relay it through the 3rd party SMTP service.
More about SMTP and smart hosts here: http://support.microsoft.com/kb/303734/en-us