Desired Behaviour
Use Gmail, OAuth2 and Nodemailer to send an email from a server side node.js file.
What I've Tried
Relevant Documentation
https://nodemailer.com/smtp/oauth2
https://nodemailer.com/usage/using-gmail
https://developers.google.com/gmail/api/auth/web-server
Relevant Questions
send emails from MY gmail account with OAuth2 and nodemailer
How do I authorise an app (web or installed) without user intervention?
https://stackoverflow.com/a/47936349
https://stackoverflow.com/a/22572776
There were gaps in the instructions of the above sources and some information was outdated, so the answer below was my final implementation which appears to be working.
I'm posting this solution for confirmation it is best practice and, if it is, to save others time.
The following worked for me, there are two parts:
01) app.js
02) Google and OAuth2 setup
app.js
var nodemailer = require("nodemailer");
var transporter = nodemailer.createTransport({
host: 'smtp.gmail.com',
port: 465,
secure: true,
auth: {
type: 'OAuth2',
user: local_settings.my_gmail_username,
clientId: local_settings.my_oauth_client_id,
clientSecret: local_settings.my_oauth_client_secret,
refreshToken: local_settings.my_oauth_refresh_token,
accessToken: local_settings.my_oauth_access_token
}
});
var mail = {
from: "John Smith <me#mydomain.com>",
to: "user#userdomain.com",
subject: "Registration successful",
text: "You successfully registered an account at www.mydomain.com",
html: "<p>You successfully registered an account at www.mydomain.com</p>"
}
transporter.sendMail(mail, function(err, info) {
if (err) {
console.log(err);
} else {
// see https://nodemailer.com/usage
console.log("info.messageId: " + info.messageId);
console.log("info.envelope: " + info.envelope);
console.log("info.accepted: " + info.accepted);
console.log("info.rejected: " + info.rejected);
console.log("info.pending: " + info.pending);
console.log("info.response: " + info.response);
}
transporter.close();
});
Google and OAuth Setup
The code above requires the following setup:
01) Go to https://console.developers.google.com
02) If you don't have a project, you will be prompted to create one
03) Click on Create Project
04) Click on Create
05) Enter a Project Name and click Create
06) Select the Gmail API
07) Click on Enable
08) Click on Create Credentials
09) Enter the required settings
10) Give the OAuth client a name and ensure you add https://developers.google.com/oauthplayground as a redirect URI in order to generate the refresh and access tokens later
11) Define the consent screen settings
12) Click I'll do this later and Done
13) Click on the Edit icon, to view your Client ID and Client Secret
14) To generate access and refresh tokens, go to https://developers.google.com/oauthplayground
15) Click on the cog icon in the top right, check Use your own OAuth credentials and enter Client ID and Client Secret
16) In the left column, select Gmail API v1 and click Authorise APIs
17) If you are signed into multiple accounts, when prompted select the relevant account
18) Click Allow
19) Click Exchange authorisation code for tokens
I'm not sure why there is a count down on the access token, but hopefully the message at the bottom of the screen means that the token won't expire.
OAuth Consent Screen
You are definetely right about the gaps and outdated information, and you did a really great job on documenting the steps needed to use Gmail with OAuth and nodemailer!
Nevertheless, I think it worths mentioning that in the Credentials page there is another step: the OAuth Consent Screen tab.
It contains a form like a Google Play app submission that requires validation from Google, if you choose your app to not being validated, you have a limitation of 100 calls of what they call Sensitive scopes before being asked for submission.
What about quota?
It's still not clear to me if this 100 calls quota will be consumed even if you don't select any additional permission to use sensitive scopes (the default ones are email, profile, openid). I hope not, since the OAuth Consent Screen asks for things like the Application Homepage Link and Authorised domains that is something you might not have if you are working on a backend application.
I think that this whole procedure is really slow and uselessly complex since most people do all these steps to just send an email from their app using nodemailer...
Related
I am working on deploying my Node.js app. However I am having issues with having the registration email getting sent out.
const transporter = nodemailer.createTransport({
service: 'Gmail',
auth: {
user: GMAIL_USER,
pass: GMAIL_PASS,
},
});
......
......
transporter.sendMail({
to: newUser.email,
subject: 'Confirm Email',
html: `Please click this email to confirm your email: ${url}`
});
This works perfectly when I try running it on local host, but as soon as I upload the files to my server and try it, google blocks the sign in attempt, and I get an email saying
Someone just used your password to try to sign in to your account. Google blocked them, but you should check what happened.
Every time, I click the button "this was me", but any future attempts still get blocked.
I have "less secure apps" enabled. Is there a way to whitelist an IP to send from my gmail? or a way to get this working in general?
You have two options.Either you set the access to less secure apps setting to Enabled or you obtain an accessToken and a refreshToken from Google OAuth2.0 and use them in your nodemailer config
For option one go to https://www.google.com/settings/security/lesssecureapps
For option two go to https://console.developers.google.com/apis/credentials
if you choose option two your config for the transport will look something like this:
auth: {
type: 'OAuth2',
user: 'user#example.com',
accessToken: 'ya29.Xx_XX0xxxxx-xX0X0XxXXxXxXXXxX0x'
}
Sometimes people who don't assign recovery email, phone number .In that scenario also google hinders login. In my case google first allowed me on turning less secure app but very next day it denied. So I added recovery email for that particular email and it worked. So that's how I think google security algo works.
Had to fill out some hidden captcha for gmail, then everything worked fine. Sadly dont have the link, or id post it.
I'm creating a Twitter bot and I'm implementing a method that sends me a email if there is an error. As I'm already using the google API to access Google Drive (have no problem here), I decided to use the service account to send the email (Google console says it could be used that way)
The method I've come up to send the email so far is:
var config = require('./config/mail');
var google = require('./config/google');
var nodemailer = require('nodemailer');
var send = function (args) {
let transporter = nodemailer.createTransport({
'service': 'gmail',
'auth': {
'type': 'OAuth2',
'user': google.client_email,
'serviceClient': google.client_id,
'privateKey': google.private_key
}
});
transporter.on('token', token => console.log(token));
let message = {
'from': `"${config.serverFromName}" <${config.serverFromMail}>`,
'to': args.to,
'subject': args.subject,
'text': args.text,
'html': `<p>${args.text}</p>`
};
transporter.sendMail(message, (err, info) => {
if (err) {
console.log('Mail couldn\'t be sent because: ' + err);
} else {
console.log('Mail sent');
}
});
};
The config/google file contains the data that Google generates for you when you create a service account. config.serverFromName and config.serverFromMail are the name and email of the sender (not the same as the service account id). args contains the recipent email and the content
When I test the send method, I got the following message in my console:
Mail couldn't be sent because: Error: Invalid login: 535-5.7.8 Username and Password not accepted. Learn more at
535 5.7.8 https://support.google.com/mail/?p=BadCredentials z123sm543690vkd.10 - gsmtp
I know the token is being created correctly because the listener I created is printing it:
{ user: 'name#project.iam.gserviceaccount.com',
accessToken: 'ya29.ElmIBLxzfU_kkuZeyISeuRBeljmAe7HNTlwuG4K12ysUNo46s-eJ8NkMYHQqD_JrqTlH3yheNc2Aopu9B5vw-ivEqvPR4sTDpWBOg3xUU_4XiJEBLno8FHsg',
expires: 1500151434603 }
Searching on the Internet I found that it may be a problem with the OAuth scope. However, all the info that talks about it refers to using Client IDs, not service accounts. I don't find that option in the Google developer console, either.
Any ideas of what I'm doing wrong?
Bottom Line: The specific way Google describes a service account is INCOMPATIBLE with nodemailer. BUT there is a way!
I have just spent countless hours myself up over this same issue! I have come to the conclusion, Google's Admin Console has removed half this capability indirectly. The console does not provide a way to authorize (a user accepting the consent screen) the desired scope the very first time with a service account.
First up, follow the Node.JS Quickstart instructions for Google Drive API to authorize a scope and receive a refresh token.
Go to console.developers.google.com, build a OAuth2.0 Client Id, and download the client_secret.json file.
Create a separate temporary module folder and use NPM to download google api modules
npm install googleapis
npm install google-auth-library
Create a quickstart.js file
Place your client_secret.json file next to quickstart.js
Line 7 in the quickstart.js is the array to define the scopes you intend to allow the application to access. Modify it as you see necessary. It is highly recommended to only provision access for what is intended. See Gmail API Scopes.
RUN node quickstart.js
Open the URL in a browser, authenticate, and copy the code from the browser back into the terminal window. This will download a nodejs-gmail-quickstart.json file which the location will be provided in stdout.
This is the part you are unable to accomplish for a Service Account. This action authorizes the scopes provided in the SCOPES array to the downloaded access_token & refresh token.
NOTE: access_token's have a lifespan of 1 hour. refresh_token's are immortal.
Now you have an authorized refresh_token!
Next is setting up your auth object with 3LO in Nodemailer. I would look more at the bottom examples because not all values are required. My auth looks like this:
const mailbot = nodemailer.createTransport({
host: 'smtp.gmail.com',
port: 587, // TLS (google requires this port for TLS)
secure: false, // Not SSL
requireTLS: true, // Uses STARTTLS command (nodemailer-ism)
auth: {
// **HIGHLY RECOMMEND** ALL values be
// read in from a file not placed directly in code.
// Make sure that file is locked down to only the server daemon
type : 'OAuth2',
user : config.client_email,
scope : "https://www.googleapis.com/auth/gmail.send",
clientId : config.client_id,
clientSecret: secret,
refreshToken: activeToken.refresh_token
// AT RUNTIME, it looks like this:
//type : 'OAuth2',
//user : 'user#gmail.com', // actual user being impersonated
//scope : "", //Optional, but recommend to define for the action intended
//clientId : '888888888998-9xx9x99xx9x99xx9xxxx9xx9xx9x88x8xxx.apps.googleusercontent.com',
//clientSecret: 'XxxxxXXxX0xxxxxxxx0XXxX0',
//refreshToken: '1/XXxXxsss-xxxXXXXXxXxx0XXXxxXXx0x00xxx'
}
});
TIP: Gmail will rewrite the FROM field from any email sent with the authorized user account (user impersonated). If you want to customize this slightly, use the syntax { FROM: '"Display NAME" <user email>' } and it will not overwrite your display name choice since the email matches.
NOTE: nodemailer will make a token request out to https://accounts.google.com/o/oauth2/token with the refresh token to automatically obtain an access_token.
Unfortunately, nodemailer lacks the functionality to save a received token out to a file directly but instead just uses this.emit(). If the server stays active it will not be an issue but as mine is only bursting, it will always incur a delay as a new access_token will be requested every time.
[SECURITY] Hopefully this works for you! It is disappointing to loose the private key encryption a service account with 2LO would bring but at least this Client ID way is very hard to spoof. I was concerned about security but reading more I am okay with this implementation. See Google Identity Platform (Nodemailer uses the HTTP/REST details) and given
[1] Google's OAuth 2.0 endpoint is at
https://accounts.google.com/o/oauth2/v2/auth. This endpoint is
accessible only over HTTPS. Plain HTTP connections are refused.
[5] After the web server receives the authorization code, it can exchange
the authorization code for an access token.
you are using TLS to connect initially for an authorization code, then matching it with your client ID data, and a refresh_token (you must go through the hassle we did above) then you can receive an access_token to actually interact with Google APIs.
As long as you increase your security posture with keeping the OAuth2.0 Client ID (highly random username), secret, and refresh token as separate, secure, and hidden as much as possible, you should be able to sleep soundly. GOOD LUCK!
After visiting the OAuth 2.0 Playground and experimenting with all possible variations of gmail-related sub-scopes, even selecting them altogether...
https://www.googleapis.com/auth/gmail.labels
https://www.googleapis.com/auth/gmail.send
https://www.googleapis.com/auth/gmail.readonly
https://www.googleapis.com/auth/gmail.compose
https://www.googleapis.com/auth/gmail.insert
https://www.googleapis.com/auth/gmail.modify
https://www.googleapis.com/auth/gmail.metadata
https://www.googleapis.com/auth/gmail.settings.basic
https://www.googleapis.com/auth/gmail.settings.sharing
...the error message described in the OP title still persist:
Error: Invalid login: 535-5.7.8 Username and Password not accepted
It seems that NodeMailer is not capable of connecting via the scopes mentioned above. In fact, it explicitly mentions in the "Troubleshooting" section of its OAuth2 SMTP transport docs
The correct OAuth2 scope for Gmail SMTP is https://mail.google.com/, make sure your client has this scope set when requesting permissions for an user
Although this gives access to more than just sending emails, it works!
The only alternative to reach a more fine grained scope solution seems to be to resort to google's own Gmail API, where you can pass scopes when generating the OAuth2 client (which should of course at least include the scopes granted at the time the OAuth consent screen was shown):
oAuth2Client.generateAuthUrl({
access_type: 'offline',
scope: SCOPES,
})
I was able to get service accounts working with Google & nodemailer:
these were the steps:
Log in to console.- https://console.cloud.google.com/
Create a service account under the project.
Click on the new service account, go to permissions and add a member. You will use this member's email address when sending the request.
Create keys for the service account. - keys -> add key. https://console.cloud.google.com/iam-admin/serviceaccounts
Download your key file. You will get something like service-account-name-accountid.json. It will have all the information you need to get the code below running.
Delegate authority to your service account https://developers.google.com/identity/protocols/oauth2/service-account#delegatingauthority. Addhttps://mail.google.com/ as the scope.
Write some code like below:
const nodemailer = require('nodemailer');
const json = require('./service-account-name-accountid.json');
const sendEmail = async (email, subject, text) => {
try {
const transporter = nodemailer.createTransport({
host: 'smtp.gmail.com',
port: 465,
secure: true,
auth: {
type: 'OAuth2',
user: email, //your permissioned service account member e-mail address
serviceClient: json.client_id,
privateKey: json.private_key
}
});
await transporter.verify();
await transporter.sendMail({
from: json.service_email,
to: email, //you can change this to any other e-mail address and it should work!
subject,
text
});
console.log('success!');
return {
status : 200
}
} catch (error) {
console.log(error);
return {
status : 500,
error
}
}
}
sendEmail('your_permissioned_service_account_email_address#some_place.com, 'testing 123', 'woohoo!');
I'm trying to build a web app on Meteor with Outlook mail API. The first step is to get user authentication through the Outlook mail API, which requires a custom URL with local host, client id, secret, etc attached.
I've looked at a few node.js oauth2 tutorials and downloaded simple-oauth2. All the tutorials use something like:
const oauth2 = require('oauth2').create({
client: {
id: '<client-ID>',
secret: '<app-password>',
},
auth: {
tokenHost: 'https://login.microsoftonline.com',
authorizePath: 'common/oauth2/v2.0/authorize',
tokenPath: 'common/oauth2/v2.0/token'
}
});
When I run it, the console logs "Uncaught TypeError: require(...).create is not a function." Is create() something that comes with some sort of download that I don't have? The require works fine if I split it up into two parts, so I know it's an issue with create().
First of all wenn you list const then don´t extend them at the end only you know what you do.
Second I don´t use Outlook API but I use OneDrive API and it should be similar. You need to follow the docs about how to get the auth code. Or here my guide for creating it for OneDrive should be same to your problem
Add application at https://apps.dev.microsoft.com/ you need microsoft account for this.
Input your Application Name. In this case, don't use Guided Setup
Create Application secret and copy it to save loaction
Add Platform and choose web. In this case, redirect URL is http://localhost/dashboard
Go to Graph access below and choose for both options Files.ReadWrite.All
Make sure you press save.
Enter your client ID in this url where xxxxxx is. Notice that client ID is the Application ID you can find at the top of your application settings.
https://login.microsoftonline.com/common/oauth2/v2.0/authorize?client_id=xxxxxxxxxx&scope=offline_access%20files.readwrite.all&response_type=code&redirect_uri=http://localhost/dashboard
Please inport above URL to your browser. Log in with your microsoft account and then give your application access by clicking yes. Then you will get redirected to http://localhost/dashboard/?code=xxxxxxxxx-xxxxxx-xxxxx-xxx-xxxxx
This auth code can be used for the access_token that will be generated with this step
request.post({url:'https://login.microsoftonline.com/common/oauth2/v2.0/token', form: {
redirect_uri: 'https://login.live.com/oauth20_desktop.srf',
client_id: 'xxxxxxxxxxxxxxxxx',
client_secret: 'xxxxxxxxxxxxxxxx',
code: 'xxxxxxxxxxxxxxxxxx',
grant_type: 'authorization_code'
}
}, function(err,httpResponse,body){ /* ... */
console.log('err: ' + err)
console.log('body: ' + body)
});
This is basicly the same process for every oAuth2 API. Notice that I use for other API´s in the access_token request the redirect_uri http://localhost/dashboard
If you make a request on and console.log as example the body and you get object object as result then please use stringify. Hope it could help you
I am working on some client side web app like a chrome extension that needs access to outlook mail and calendar. I followed the instruction on https://dev.outlook.com/RestGettingStarted and successfully got access and refresh tokens to retrieve data.
However, I cannot find any way of implementing "logout". The basic idea is to let user sign out and login with a different outlook account. In order to do that, I removed cached tokens, requested access tokens in interactive mode. The login window did pop out, but it took any valid email address, didn't let me input password and finally returned tokens for previous account. So I was not able to really use a different account until the old token expired.
Can anyone please tell me if it is possible to send a request to revoke the tokens so people can use a different account? Thanks!
=========================================================
Update:
Actually it is the fault of chrome.identity api. I used chrome.identity.LaunchWebAuthFlow to init the auth flow. It caches user's identity but no way to remove it. So we cannot really "logout" if using this api.
I used two logouts via launchWebAuthFlow - first I called the logout link to my app, then secondly, I called the logout link to Google.
var options = {
'interactive': false,
'url': 'https://localhost:44344/Account/Logout'
}
chrome.identity.launchWebAuthFlow(options, function(redirectUri) {});
options = {
'interactive': false,
'url': 'https://accounts.google.com/logout'
}
chrome.identity.launchWebAuthFlow(options, function(redirectUri) {});
I'm trying to do a simple contact form in a node app, using nodemailer. I want all the msg to be sent from a gmail account I made for this purpose, to my personnal mail.
on the client side, all I do is to get the name/mail/message of the customer and send it to the server. It works fine locally but fails to work when deployed (on heroku btw).
After a quick search, it seems I have to generate a ClientId and ClientSecret from Google Developers Console - which I did - but when it comes to generating a "refresh token" iI'm completely lost.
var smtpTransport = nodemailer.createTransport("SMTP",{
service:"Gmail",
auth:{
XOAuth2: {
user:"myaccount#gmail.com",
clientId:"",
clientSecret:"",
refreshToken:""
}
}
});
I am confused : What exactly is a refresh token and how do I get one ?
Notes by this answer original's author:
So, I finally managed to figure it out. I'm surprised I couldn't find more ressources about that so for those who need to use Gmail with Nodemailer
I found the answer here: http://masashi-k.blogspot.fr/2013/06/sending-mail-with-gmail-using-xoauth2.html
Try creating a new User if you already had one and things ain't working fine. It was the case for me.
I hope this will be useful to someone,
Cheers
Question 1: What exactly is a refresh token?
From documentation found here:
A refresh token provides your app continuous access to Google APIs while the user is not logged into your application.
(...)
Considerations:
Be sure to store the refresh token safely and permanently, because you can only obtain a refresh token the first time that you perform the code exchange flow.
There are limits on the number of refresh token that are issued—one limit per client/user combination, and another per user across all clients. If your application requests too many refresh tokens, it may run into these limits, in which case older refresh tokens stop working.
See also Offline Access and Using a refresh token.
Question 2: How do I get one?
Step 1: Obtain OAuth 2.0 credentials at Google Developers Console
As stated here, you should:
Go to the Google Developers Console.
Select a project, or create a new one.
In the sidebar on the left, expand APIs & auth. Next, click APIs. Select the Enabled APIs link in the API section to see a list of all your enabled APIs. Make sure that the "Gmail API" is on the list of enabled APIs. If you have not enabled it, select the Gmail API from the list of APIs (under Google Apps APIs), then select the Enable API button for the API.
In the sidebar on the left, select Credentials.
If you haven't done so already, create your project's OAuth 2.0 credentials by clicking Create new Client ID, and providing the information needed to create the credentials.
Look for the Client ID and Client secret in the table associated with each of your credentials.
PAY SPECIAL ATTENTION TO specifying https://developers.google.com/oauthplayground
as a Redirect URI when you create a new User in the console.
Otherwise, you will have an error.
Step 2: Obtain the refresh token at Google OAuth2.0 Playground
Go to the Google Oauth2.0 Playground.
Click the Gear Button on the right-top. Set your Client ID and Client Secret obtained from the Google Developers Console, and select Access token location as Authorization header w/ Bearer prefix. Close this configuration overlay.
Set up the scopes. Use https://mail.google.com/ as it's the one need by nodemailer. Then click the Authorize APIs button.
After OAuth2.0 authorization, exchange authorization code for tokens and voilá! your refresh token is ready-to-use
For those who have been looking around for a working example/code snippet, follow Radioreve's Answer until you are able to get the access token and refresh token. (Basically, go to the playground, make sure it asks for access for sending mail and mail.google.com, give permission, exchange authorization code for tokens)
Note that the expires time I entered was new Date().getTime() + 2000 which was close to the expiration seconds seen on the playground. I am not sure if I had to enter access token and expiration time accurately as it seems to be refreshing the token automatically.
Use this sample code written in ECMAScript 6:
const user_name = 'something#gmail.com';
const refresh_token = '';
const access_token = '';
const client_id = '';
const client_secret = '';
const email_to = 'receiver#gmail.com';
const nodemailer = require('nodemailer');
let transporter = nodemailer
.createTransport({
service: 'Gmail',
auth: {
type: 'OAuth2',
clientId: client_id,
clientSecret: client_secret
}
});
transporter.on('token', token => {
console.log('A new access token was generated');
console.log('User: %s', token.user);
console.log('Access Token: %s', token.accessToken);
console.log('Expires: %s', new Date(token.expires));
});
// setup e-mail data with unicode symbols
let mailOptions = {
from : user_name, // sender address
to : email_to, // list of receivers
subject : 'Hello ✔', // Subject line
text : 'Hello world ?', // plaintext body
html : '<b>Hello world ?</b>', // html body
auth : {
user : user_name,
refreshToken : refresh_token,
accessToken : access_token,
expires : 1494388182480
}
};
// send mail with defined transport object
transporter.sendMail(mailOptions, function (error, info) {
if (error) {
return console.log(error);
}
console.log('Message sent: ' + info.response);
});
You can Simple use Google SMTP to send email. Use nodemailer with smtp.google.com and email and App password (not gmail password).
How to Get App Password.
Now you have to enable 2 Step Verification in Google (How to Enable 2 Step Auth)
You need to generate App Specific Password. Goto Google My Account > Security
Click on App Password > Select Other and you will get App Password
You can use normal smtp with email and App password.