Nodemailer/Gmail - What exactly is a refresh token and how do I get one? - node.js

I'm trying to do a simple contact form in a node app, using nodemailer. I want all the msg to be sent from a gmail account I made for this purpose, to my personnal mail.
on the client side, all I do is to get the name/mail/message of the customer and send it to the server. It works fine locally but fails to work when deployed (on heroku btw).
After a quick search, it seems I have to generate a ClientId and ClientSecret from Google Developers Console - which I did - but when it comes to generating a "refresh token" iI'm completely lost.
var smtpTransport = nodemailer.createTransport("SMTP",{
service:"Gmail",
auth:{
XOAuth2: {
user:"myaccount#gmail.com",
clientId:"",
clientSecret:"",
refreshToken:""
}
}
});
I am confused : What exactly is a refresh token and how do I get one ?

Notes by this answer original's author:
So, I finally managed to figure it out. I'm surprised I couldn't find more ressources about that so for those who need to use Gmail with Nodemailer
I found the answer here: http://masashi-k.blogspot.fr/2013/06/sending-mail-with-gmail-using-xoauth2.html
Try creating a new User if you already had one and things ain't working fine. It was the case for me.
I hope this will be useful to someone,
Cheers
Question 1: What exactly is a refresh token?
From documentation found here:
A refresh token provides your app continuous access to Google APIs while the user is not logged into your application.
(...)
Considerations:
Be sure to store the refresh token safely and permanently, because you can only obtain a refresh token the first time that you perform the code exchange flow.
There are limits on the number of refresh token that are issued—one limit per client/user combination, and another per user across all clients. If your application requests too many refresh tokens, it may run into these limits, in which case older refresh tokens stop working.
See also Offline Access and Using a refresh token.
Question 2: How do I get one?
Step 1: Obtain OAuth 2.0 credentials at Google Developers Console
As stated here, you should:
Go to the Google Developers Console.
Select a project, or create a new one.
In the sidebar on the left, expand APIs & auth. Next, click APIs. Select the Enabled APIs link in the API section to see a list of all your enabled APIs. Make sure that the "Gmail API" is on the list of enabled APIs. If you have not enabled it, select the Gmail API from the list of APIs (under Google Apps APIs), then select the Enable API button for the API.
In the sidebar on the left, select Credentials.
If you haven't done so already, create your project's OAuth 2.0 credentials by clicking Create new Client ID, and providing the information needed to create the credentials.
Look for the Client ID and Client secret in the table associated with each of your credentials.
PAY SPECIAL ATTENTION TO specifying https://developers.google.com/oauthplayground
as a Redirect URI when you create a new User in the console.
Otherwise, you will have an error.
Step 2: Obtain the refresh token at Google OAuth2.0 Playground
Go to the Google Oauth2.0 Playground.
Click the Gear Button on the right-top. Set your Client ID and Client Secret obtained from the Google Developers Console, and select Access token location as Authorization header w/ Bearer prefix. Close this configuration overlay.
Set up the scopes. Use https://mail.google.com/ as it's the one need by nodemailer. Then click the Authorize APIs button.
After OAuth2.0 authorization, exchange authorization code for tokens and voilá! your refresh token is ready-to-use

For those who have been looking around for a working example/code snippet, follow Radioreve's Answer until you are able to get the access token and refresh token. (Basically, go to the playground, make sure it asks for access for sending mail and mail.google.com, give permission, exchange authorization code for tokens)
Note that the expires time I entered was new Date().getTime() + 2000 which was close to the expiration seconds seen on the playground. I am not sure if I had to enter access token and expiration time accurately as it seems to be refreshing the token automatically.
Use this sample code written in ECMAScript 6:
const user_name = 'something#gmail.com';
const refresh_token = '';
const access_token = '';
const client_id = '';
const client_secret = '';
const email_to = 'receiver#gmail.com';
const nodemailer = require('nodemailer');
let transporter = nodemailer
.createTransport({
service: 'Gmail',
auth: {
type: 'OAuth2',
clientId: client_id,
clientSecret: client_secret
}
});
transporter.on('token', token => {
console.log('A new access token was generated');
console.log('User: %s', token.user);
console.log('Access Token: %s', token.accessToken);
console.log('Expires: %s', new Date(token.expires));
});
// setup e-mail data with unicode symbols
let mailOptions = {
from : user_name, // sender address
to : email_to, // list of receivers
subject : 'Hello ✔', // Subject line
text : 'Hello world ?', // plaintext body
html : '<b>Hello world ?</b>', // html body
auth : {
user : user_name,
refreshToken : refresh_token,
accessToken : access_token,
expires : 1494388182480
}
};
// send mail with defined transport object
transporter.sendMail(mailOptions, function (error, info) {
if (error) {
return console.log(error);
}
console.log('Message sent: ' + info.response);
});

You can Simple use Google SMTP to send email. Use nodemailer with smtp.google.com and email and App password (not gmail password).
How to Get App Password.
Now you have to enable 2 Step Verification in Google (How to Enable 2 Step Auth)
You need to generate App Specific Password. Goto Google My Account > Security
Click on App Password > Select Other and you will get App Password
You can use normal smtp with email and App password.

Related

How to get a refresh token using the Microsoft Graph Javascript client library?

I am using the Microsoft graph Javascript client library to get a refresh token for a user. I created an app that connects doctors and patients. I want to create and delete events on the doctors' calendars. I first need their authorization to access their outlook account. Unfortunately, when I make the api call to get the refresh token, I get back an access token and an id token but no refresh token. Can someone please help?
Here's my code
const msalConfig = {
auth: {
clientId: process.env.OUTLOOK_OAUTH_CLIENT_ID,
authority: process.env.OUTLOOK_OAUTH_AUTHORITY,
clientSecret: process.env.OUTLOOK_OAUTH_CLIENT_SECRET
},
system: {
loggerOptions: {
loggerCallback(loglevel, message, containsPii) {
console.log(message);
},
piiLoggingEnabled: false,
logLevel: msal.LogLevel.Verbose,
}
}
};
// Create msal application object
const ouathClient = new msal.ConfidentialClientApplication(msalConfig);
const response = await ouathClient.acquireTokenByCode(tokenRequest);
I am using node js.
This method can also be used to get the refresh token, Please refer this Microsoft documentation.To get refresh token
Client Credentials flow does not support user context thus no refresh token is supported in this case.
If you are using MSAL depending on whether you are using Public client (Mobile, Desktop or Single Page apps) where users sign-in to your app then you may need a refresh token and you should be using flows listed here
If you are using a private client like a serveside daemon then you dont need a refresh token.

NodeMailer - send mail with Google service account fails because "Username and Password not accepted"

I'm creating a Twitter bot and I'm implementing a method that sends me a email if there is an error. As I'm already using the google API to access Google Drive (have no problem here), I decided to use the service account to send the email (Google console says it could be used that way)
The method I've come up to send the email so far is:
var config = require('./config/mail');
var google = require('./config/google');
var nodemailer = require('nodemailer');
var send = function (args) {
let transporter = nodemailer.createTransport({
'service': 'gmail',
'auth': {
'type': 'OAuth2',
'user': google.client_email,
'serviceClient': google.client_id,
'privateKey': google.private_key
}
});
transporter.on('token', token => console.log(token));
let message = {
'from': `"${config.serverFromName}" <${config.serverFromMail}>`,
'to': args.to,
'subject': args.subject,
'text': args.text,
'html': `<p>${args.text}</p>`
};
transporter.sendMail(message, (err, info) => {
if (err) {
console.log('Mail couldn\'t be sent because: ' + err);
} else {
console.log('Mail sent');
}
});
};
The config/google file contains the data that Google generates for you when you create a service account. config.serverFromName and config.serverFromMail are the name and email of the sender (not the same as the service account id). args contains the recipent email and the content
When I test the send method, I got the following message in my console:
Mail couldn't be sent because: Error: Invalid login: 535-5.7.8 Username and Password not accepted. Learn more at
535 5.7.8 https://support.google.com/mail/?p=BadCredentials z123sm543690vkd.10 - gsmtp
I know the token is being created correctly because the listener I created is printing it:
{ user: 'name#project.iam.gserviceaccount.com',
accessToken: 'ya29.ElmIBLxzfU_kkuZeyISeuRBeljmAe7HNTlwuG4K12ysUNo46s-eJ8NkMYHQqD_JrqTlH3yheNc2Aopu9B5vw-ivEqvPR4sTDpWBOg3xUU_4XiJEBLno8FHsg',
expires: 1500151434603 }
Searching on the Internet I found that it may be a problem with the OAuth scope. However, all the info that talks about it refers to using Client IDs, not service accounts. I don't find that option in the Google developer console, either.
Any ideas of what I'm doing wrong?
Bottom Line: The specific way Google describes a service account is INCOMPATIBLE with nodemailer. BUT there is a way!
I have just spent countless hours myself up over this same issue! I have come to the conclusion, Google's Admin Console has removed half this capability indirectly. The console does not provide a way to authorize (a user accepting the consent screen) the desired scope the very first time with a service account.
First up, follow the Node.JS Quickstart instructions for Google Drive API to authorize a scope and receive a refresh token.
Go to console.developers.google.com, build a OAuth2.0 Client Id, and download the client_secret.json file.
Create a separate temporary module folder and use NPM to download google api modules
npm install googleapis
npm install google-auth-library
Create a quickstart.js file
Place your client_secret.json file next to quickstart.js
Line 7 in the quickstart.js is the array to define the scopes you intend to allow the application to access. Modify it as you see necessary. It is highly recommended to only provision access for what is intended. See Gmail API Scopes.
RUN node quickstart.js
Open the URL in a browser, authenticate, and copy the code from the browser back into the terminal window. This will download a nodejs-gmail-quickstart.json file which the location will be provided in stdout.
This is the part you are unable to accomplish for a Service Account. This action authorizes the scopes provided in the SCOPES array to the downloaded access_token & refresh token.
NOTE: access_token's have a lifespan of 1 hour. refresh_token's are immortal.
Now you have an authorized refresh_token!
Next is setting up your auth object with 3LO in Nodemailer. I would look more at the bottom examples because not all values are required. My auth looks like this:
const mailbot = nodemailer.createTransport({
host: 'smtp.gmail.com',
port: 587, // TLS (google requires this port for TLS)
secure: false, // Not SSL
requireTLS: true, // Uses STARTTLS command (nodemailer-ism)
auth: {
// **HIGHLY RECOMMEND** ALL values be
// read in from a file not placed directly in code.
// Make sure that file is locked down to only the server daemon
type : 'OAuth2',
user : config.client_email,
scope : "https://www.googleapis.com/auth/gmail.send",
clientId : config.client_id,
clientSecret: secret,
refreshToken: activeToken.refresh_token
// AT RUNTIME, it looks like this:
//type : 'OAuth2',
//user : 'user#gmail.com', // actual user being impersonated
//scope : "", //Optional, but recommend to define for the action intended
//clientId : '888888888998-9xx9x99xx9x99xx9xxxx9xx9xx9x88x8xxx.apps.googleusercontent.com',
//clientSecret: 'XxxxxXXxX0xxxxxxxx0XXxX0',
//refreshToken: '1/XXxXxsss-xxxXXXXXxXxx0XXXxxXXx0x00xxx'
}
});
TIP: Gmail will rewrite the FROM field from any email sent with the authorized user account (user impersonated). If you want to customize this slightly, use the syntax { FROM: '"Display NAME" <user email>' } and it will not overwrite your display name choice since the email matches.
NOTE: nodemailer will make a token request out to https://accounts.google.com/o/oauth2/token with the refresh token to automatically obtain an access_token.
Unfortunately, nodemailer lacks the functionality to save a received token out to a file directly but instead just uses this.emit(). If the server stays active it will not be an issue but as mine is only bursting, it will always incur a delay as a new access_token will be requested every time.
[SECURITY] Hopefully this works for you! It is disappointing to loose the private key encryption a service account with 2LO would bring but at least this Client ID way is very hard to spoof. I was concerned about security but reading more I am okay with this implementation. See Google Identity Platform (Nodemailer uses the HTTP/REST details) and given
[1] Google's OAuth 2.0 endpoint is at
https://accounts.google.com/o/oauth2/v2/auth. This endpoint is
accessible only over HTTPS. Plain HTTP connections are refused.
[5] After the web server receives the authorization code, it can exchange
the authorization code for an access token.
you are using TLS to connect initially for an authorization code, then matching it with your client ID data, and a refresh_token (you must go through the hassle we did above) then you can receive an access_token to actually interact with Google APIs.
As long as you increase your security posture with keeping the OAuth2.0 Client ID (highly random username), secret, and refresh token as separate, secure, and hidden as much as possible, you should be able to sleep soundly. GOOD LUCK!
After visiting the OAuth 2.0 Playground and experimenting with all possible variations of gmail-related sub-scopes, even selecting them altogether...
https://www.googleapis.com/auth/gmail.labels
https://www.googleapis.com/auth/gmail.send
https://www.googleapis.com/auth/gmail.readonly
https://www.googleapis.com/auth/gmail.compose
https://www.googleapis.com/auth/gmail.insert
https://www.googleapis.com/auth/gmail.modify
https://www.googleapis.com/auth/gmail.metadata
https://www.googleapis.com/auth/gmail.settings.basic
https://www.googleapis.com/auth/gmail.settings.sharing
...the error message described in the OP title still persist:
Error: Invalid login: 535-5.7.8 Username and Password not accepted
It seems that NodeMailer is not capable of connecting via the scopes mentioned above. In fact, it explicitly mentions in the "Troubleshooting" section of its OAuth2 SMTP transport docs
The correct OAuth2 scope for Gmail SMTP is https://mail.google.com/, make sure your client has this scope set when requesting permissions for an user
Although this gives access to more than just sending emails, it works!
The only alternative to reach a more fine grained scope solution seems to be to resort to google's own Gmail API, where you can pass scopes when generating the OAuth2 client (which should of course at least include the scopes granted at the time the OAuth consent screen was shown):
oAuth2Client.generateAuthUrl({
access_type: 'offline',
scope: SCOPES,
})
I was able to get service accounts working with Google & nodemailer:
these were the steps:
Log in to console.- https://console.cloud.google.com/
Create a service account under the project.
Click on the new service account, go to permissions and add a member. You will use this member's email address when sending the request.
Create keys for the service account. - keys -> add key. https://console.cloud.google.com/iam-admin/serviceaccounts
Download your key file. You will get something like service-account-name-accountid.json. It will have all the information you need to get the code below running.
Delegate authority to your service account https://developers.google.com/identity/protocols/oauth2/service-account#delegatingauthority. Addhttps://mail.google.com/ as the scope.
Write some code like below:
const nodemailer = require('nodemailer');
const json = require('./service-account-name-accountid.json');
const sendEmail = async (email, subject, text) => {
try {
const transporter = nodemailer.createTransport({
host: 'smtp.gmail.com',
port: 465,
secure: true,
auth: {
type: 'OAuth2',
user: email, //your permissioned service account member e-mail address
serviceClient: json.client_id,
privateKey: json.private_key
}
});
await transporter.verify();
await transporter.sendMail({
from: json.service_email,
to: email, //you can change this to any other e-mail address and it should work!
subject,
text
});
console.log('success!');
return {
status : 200
}
} catch (error) {
console.log(error);
return {
status : 500,
error
}
}
}
sendEmail('your_permissioned_service_account_email_address#some_place.com, 'testing 123', 'woohoo!');

create oauth2 isn't working

I'm trying to build a web app on Meteor with Outlook mail API. The first step is to get user authentication through the Outlook mail API, which requires a custom URL with local host, client id, secret, etc attached.
I've looked at a few node.js oauth2 tutorials and downloaded simple-oauth2. All the tutorials use something like:
const oauth2 = require('oauth2').create({
client: {
id: '<client-ID>',
secret: '<app-password>',
},
auth: {
tokenHost: 'https://login.microsoftonline.com',
authorizePath: 'common/oauth2/v2.0/authorize',
tokenPath: 'common/oauth2/v2.0/token'
}
});
When I run it, the console logs "Uncaught TypeError: require(...).create is not a function." Is create() something that comes with some sort of download that I don't have? The require works fine if I split it up into two parts, so I know it's an issue with create().
First of all wenn you list const then don´t extend them at the end only you know what you do.
Second I don´t use Outlook API but I use OneDrive API and it should be similar. You need to follow the docs about how to get the auth code. Or here my guide for creating it for OneDrive should be same to your problem
Add application at https://apps.dev.microsoft.com/ you need microsoft account for this.
Input your Application Name. In this case, don't use Guided Setup
Create Application secret and copy it to save loaction
Add Platform and choose web. In this case, redirect URL is http://localhost/dashboard
Go to Graph access below and choose for both options Files.ReadWrite.All
Make sure you press save.
Enter your client ID in this url where xxxxxx is. Notice that client ID is the Application ID you can find at the top of your application settings.
https://login.microsoftonline.com/common/oauth2/v2.0/authorize?client_id=xxxxxxxxxx&scope=offline_access%20files.readwrite.all&response_type=code&redirect_uri=http://localhost/dashboard
Please inport above URL to your browser. Log in with your microsoft account and then give your application access by clicking yes. Then you will get redirected to http://localhost/dashboard/?code=xxxxxxxxx-xxxxxx-xxxxx-xxx-xxxxx
This auth code can be used for the access_token that will be generated with this step
request.post({url:'https://login.microsoftonline.com/common/oauth2/v2.0/token', form: {
redirect_uri: 'https://login.live.com/oauth20_desktop.srf',
client_id: 'xxxxxxxxxxxxxxxxx',
client_secret: 'xxxxxxxxxxxxxxxx',
code: 'xxxxxxxxxxxxxxxxxx',
grant_type: 'authorization_code'
}
}, function(err,httpResponse,body){ /* ... */
console.log('err: ' + err)
console.log('body: ' + body)
});
This is basicly the same process for every oAuth2 API. Notice that I use for other API´s in the access_token request the redirect_uri http://localhost/dashboard
If you make a request on and console.log as example the body and you get object object as result then please use stringify. Hope it could help you

CRON Node.js Gmail API script

How do I properly setup Gmail API script that sends emails?
I am about to use this method and I started building my script from this quickstart guide.
Is there alternative way to do this without using OAuth 2 validation? Or a way to validate once for all?
Well, in using Gmail APi with your app, you need to use OAuth 2.0 because all request to the Gmail API must be authorized by an authenticated user. And if you notice the quickstart, there is a step here that you need to create a credentials/Outh client ID to make this API work.
For more information, there is another way to authorize your app with Gmail. You can do it with the help of Google+ Sign-in that provide a "sign-in with Google" authentication method for your app.
While asking for authorization from GMail, OAuth 2.0 gives one access token and one refresh token. To avoid validation every time, store the access token. Use the refresh token to get the new access token after it is expired (access token expires every one hour).
Read about this process here: https://developers.google.com/identity/protocols/OAuth2
I found solution using JWT to authorize OAuth2.
You need to have admin account to create Domain wide delegation service account. Then in Developer console you need to download service key JSON file which you load as credentials.
First fetch all users like this: (here you need to use account with admin directory rights)
const google = require('googleapis');
const gmail = google.gmail('v1');
const directory = google.admin('directory_v1');
const scopes = [
'https://www.googleapis.com/auth/gmail.readonly',
'https://www.googleapis.com/auth/admin.directory.user.readonly'
];
const key = require('./service_key.json');
var authClient = new google.auth.JWT(
key.client_email,
key,
key.private_key,
scopes,
"authorized#mail.com"
);
authClient.authorize(function(err, tokens){
if (err) {
console.log(err);
return;
}
directory.users.list(
{
auth: authClient,
customer: 'my_customer',
maxResults: 250,
orderBy: 'email'
}, (err, resp) => {
if (err) {
console.log(err);
return;
}
console.log(resp);
});
});
Then you need to fetch Thread lists (100 per request (page)). And for each thread object you need to call get method for full thread. When using Gmail API authorize as user you want to fetch emails from. In request as userId use value 'me'.

Oauth2 flow without redirect_uri

I am creating an Android/iOS app which communicates with a Node.js server and would like to identify them securely on my server using Google (and/or Facebook) and OAuth2. I've looked at the following documentation: https://developers.google.com/+/web/signin/server-side-flow
I do not need authorization, I only need authentication (I only want to make sure that the person calling my Node.js service is the person they say they are). To achieve this, if I understand properly, I have to let the user log in using Google on the client side, this will give them an authorization_code which they can then give to my server. My server can then exchange that code for an access_token, and therefore retrieve information about the user. I am then guaranteed that the user is the person they say they are.
The Google documentations (link above) says: "In the Authorized redirect URI field, delete the default value. It is not used for this case.", however, for my server to exchange the authorization_code for an access_token, it needs to provide a redirect_uri, am I missing something?
The redirect_uri is useless for Unity games, for instance (since logging in with Google simply opens a new "window", which is closed when logged in, no redirection involved).
TL;DR
How do you use OAuth2 to authenticate users between my client and my server without redirection?
TL;DR How do you use OAuth2 to authenticate users between my client and my server without redirection?
You can't. OAuth requires that the user is directed to an authorization (and possibly login) screen, and then redirected back to your app.
EDIT 20/12/22. See comment below regarding latest status
Have you looked at this documentation? https://developers.google.com/accounts/docs/OAuth2InstalledApp#choosingredirecturi
Choosing a redirect URI
When you create a client ID in the Google Developers Console, two
redirect_uris are created for you: urn:ietf:wg:oauth:2.0:oob and
http://localhost. The value your application uses determines how the
authorization code is returned to your application.
http://localhost
This value signals to the Google Authorization Server that the
authorization code should be returned as a query string parameter to
the web server on the client. You may specify a port number without
changing the Google Developers Console configuration. To receive the
authorization code using this URL, your application must be listening
on the local web server. This is possible on many, but not all,
platforms. If your platform supports it, this is the recommended
mechanism for obtaining the authorization code.
I had this problem and it took me ages to find the "postmessage" solution that Nepoxx mentions in the comments of the accepted answer here.
For clarification, here's what worked for me.
Follow steps 1-6 here: https://developers.google.com/identity/sign-in/web/server-side-flow
Install googleapis library npm install --save googleapis
For the server-side token exchange do this:
var googleapis = require('googleapis');
var OAuth2 = googleapis.auth.OAuth2;
var oauth2Client = new OAuth2(
GOOGLE_SSO_CLIENT_ID,
GOOGLE_SSO_CLIENT_SECRET,
'postmessage' // this is where you might otherwise specifiy a redirect_uri
);
oauth2Client.getToken(CODE_FROM_STEP_5_OF_INSTRUCTIONS, function(err, tokens) {
// Now tokens contains an access_token and an optional refresh_token. Save them.
});
The redirect_uri can be a URL with a custom URL scheme for which the client registered a handler. This is described here: What's a redirect URI? how does it apply to iOS app for OAuth2.0?. It is not so much about "redirecting" it is about a callback endpoint to your app.
And it become really easy if you use VueJS with https://github.com/guruahn/vue-google-oauth2
Client side
import GAuth from 'vue-google-oauth2'
Vue.use(GAuth, {
clientId: 'xxxxxxx.apps.googleusercontent.com',
scope: 'profile',
})
async signWithGoogle() {
const code = await this.$gAuth.getAuthCode() //
console.log(code ) // { code: 'x/xxxxxxxxxx' }
// send the code to your auth server
// and retrieve a JWT or something to keep in localstorage
// to send on every request and compare with database
}
Server side
import { google } from 'googleapis'
const oauth2Client = new google.auth.OAuth2(GOOGLE_ID, GOOGLE_SECRET, 'postmessage')
google.options({ auth: oauth2Client })
async function getAccount(code) {
// the code you sent with the client
const { tokens } = await oauth2Client.getToken(code)
oauth2Client.setCredentials(tokens)
const oauth2 = google.oauth2({ version: 'v2' })
const { data: { id } } = await oauth2.userinfo.get()
// there you have the id of the user to store it in the database
// and send it back in a JWT
}

Resources