Google service account: The API returned an error: TypeError: source.hasOwnProperty is not a function after an hour - node.js

I have added google cloud service account in a project and its working. But the problem is that after an hour(i think), I get this error:
The API returned an error: TypeError: source.hasOwnProperty is not a function
Internal Server Error
and I need to restart the application to make it work.
Here in this StackOverflow post, I found this:
Once you get an access token it is treated in the same way - and is
expected to expire after 1 hour, at which time a new access token will
need to be requested, which for a service account means creating and
signing a new assertion.
but didn't help.
I'm using Node js and amazon secret service:
the code I have used to authorize:
const jwtClient = new google.auth.JWT(
client_email,
null,
private_key,
scopes
);
jwtClient.authorize((authErr) =>{
if(authErr){
const deferred = q.defer();
deferred.reject(new Error('Google drive authentication error, !'));
}
});
Any idea?
hint: Is there any policy in AWS secret to access a secret or in google cloud to access a service account? for example access in local or online?

[NOTE: You are using a service account to access Google Drive. A service account will have its own Google Drive. Is this your intention or is your goal to share your Google Drive with the service account?]
Is there any policy in AWS secret to access a secret or in google
cloud to access a service account? for example access in local or
online?
I am not sure what you are asking. AWS has IAM policies to control secret management. Since you are able to create a Signed JWT from stored secrets, I will assume that this is not an issue. Google does not have policies regarding accessing service accounts - if you have the service account JSON key material, you can do whatever the service account is authorized to do until the service account is deleted, modified, etc.
Now on to the real issue.
Your Signed JWT has expired and you need to create a new one. You need to track the lifetime of tokens that you create and recreate/refresh the tokens before they expire. The default expiration in Google's world is 3,600 seconds. Since you are creating your own token, there is no "wrapper" code around your token to handle expiration.
The error that you are getting is caused by a code crash. Since you did not include your code, I cannot tell you where. However, the solution is to catch errors so that expiration exceptions can be managed.
I recommend instead of creating the Google Drive Client using a Signed JWT that you create the client with a service account. Token expiration and refresh will be managed for you.
Very few Google services still support Signed JWTs (which your code is using). You should switch to using service accounts, which start off with a Signed JWT and then exchange that for an OAuth 2.0 Access Token internally.
There are several libraries that you can use. Either of the following will provide the features that you should be using instead of crafting your own Signed JWTs.
https://github.com/googleapis/google-auth-library-nodejs
https://github.com/googleapis/google-api-nodejs-client
The following code is an "example" and is not meant to be tested and debugged. Change the scopes in this example to match what you require. Remove the section where I load a service-account.json file and replace with your AWS Secrets code. Fill out the code with your required functionality. If you have a problem, create a new question with the code that you wrote and detailed error messages.
const {GoogleAuth} = require('google-auth-library');
const {google} = require('googleapis');
const key = require('service-account.json');
/**
* Instead of specifying the type of client you'd like to use (JWT, OAuth2, etc)
* this library will automatically choose the right client based on the environment.
*/
async function main() {
const auth = new GoogleAuth({
credentials: {
client_email: key.client_email,
private_key: key.private_key,
},
scopes: 'https://www.googleapis.com/auth/drive.metadata.readonly'
});
const drive = google.drive('v3');
// List Drive files.
drive.files.list({ auth: auth }, (listErr, resp) => {
if (listErr) {
console.log(listErr);
return;
}
resp.data.files.forEach((file) => {
console.log(`${file.name} (${file.mimeType})`);
});
});
}
main()

Related

What are REDIRECT_URI and REFRESH_TOKEN needed for in the Google Drive API?

I have doubts about how the Google Drive API works and how should I use it for my use cases. I have a REST API (based on NodeJS) which uploads documents in the drive. The only user that is actually accessing the drive is the one that runs the REST API (the root on the machine). As I understand, the OAuth2 protocol allows user to allow third-party applications to access it's information. So I'm not sure how my case is compatible with it. I have only one user that requires a one-time permission to be available forever.
Currently I have managed to use the Drive by doing authentication as follows:
const { google } = require('googleapis');
const oauth2Client = new google.auth.OAuth2(
CLIENT_ID,
CLIENT_SECRET,
REDIRECT_URI
);
oauth2Client.setCredentials({ refresh_token: REFRESH_TOKEN });
drive_instance = google.drive({
version: 'v3',
auth: oauth2Client,
});
For that, I registered the Google Drive through the Google Console. There I have the following fields:
{
"web": {
"client_id": "CILENT_ID",
"project_id": "project-id",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_secret": "CLIENT_SECRET"
"redirect_uris": [
]
}
}
Now, going over the docs, they suggest using their Playground. But the period of it being valid is one week.
I want to go production with my REST API but I'm not sure what REDIRECT_URI and REFRESH_TOKEN should be in that case. I actually want to have three instances:
Dev - For development.
Prod - For production.
Test - For integration tests.
I have the basic understanding of how the OAuth2 Protocol works but I'm not sure how to create those three instances, based no my use case. Basically I thought of keeping a config file, one for each mode, and the REST API will use it (based on the CLI options). Can someone please explain the general idea of all the options CLIENT_ID, CLIENT_SECRET, REDIRECT_URI, REFRESH_TOKEN? Do I actually need to "redirect" in each mode, even if it's one user?
If you are only accessing this one account, and you the develper control it. Then you should be using a serice account.
if you grant the service account access to the directory in drive you want to access then it will have access forever.
// service account key file from Google Cloud console.
const KEYFILEPATH = 'C:\\Youtube\\dev\\ServiceAccountCred.json';
// Request full drive access.
const SCOPES = ['https://www.googleapis.com/auth/drive'];
// Request full drive scope and profile scope, giving full access to google drive as well as the users basic profile information.
const SCOPES = ['https://www.googleapis.com/auth/drive', 'profile'];
// Create a service account initialize with the service account key file and scope needed
const auth = new google.auth.GoogleAuth({
keyFile: KEYFILEPATH,
scopes: SCOPES
});
Code shamelessly ripped from my tutorial Upload Image to Google drive with Node Js
Note for web app you are using.
The only reason the refresh token is expiring is because your app is still in test. If you move it to production in google developer console under the oauth consent screen the refresh token will stop expiring.
However there really is no reason to be using a web app if you can get away with using a service account.

Is there is a way to grant my nodejs application access to my google calendar?

so I have a nodejs application that sends invites to an event in google calendar to some people, for now I am using my gmail account and OAuth Playground to get a temporary Access token and it works, but the access token is just available for minutes and each time I need to refresh the Access token manually and give access to my google account calendar and this is the problem, now I want to make something dynamic without me interfering in the process.
This application is hosted in wix.
any suggestions ?
Thanks
IF you are only letting them access a calendar that you own and control then you can use a service account.
let google = require('googleapis');
let privatekey = require("./privatekey.json");
  Now let’s use the private key to configure a JWT auth client and authenticate our request.
// configure a JWT auth client
let jwtClient = new google.auth.JWT(
privatekey.client_email,
null,
privatekey.private_key,
['https://www.googleapis.com/auth/calendar']);
//authenticate request
jwtClient.authorize(function (err, tokens) {
if (err) {
console.log(err);
return;
} else {
console.log("Successfully connected!");
}
});
Create service account credentials in google developer console. then take the service account email address and share the calendar with it via the google calendar website. The rest of the code you have should be the same just swap out the auth section with this.
To access the calendar API please follow the Quickstart
The code provided creates a refresh token that will automatically generate a new access token for you, whenever the old one expires.
Be careful with unnecessary using service accounts, especially for adding invitees to a calendar event - there are currently issues with this feature.

Why my API calls using google-api-nodejs-client to Google Analytics are not working in production?

I'm calling the Google Analytics Reporting API using google-api-nodejs-client to show the number of visits inside a blog.
This blog is hosted inside Google App Engine Standard Environment.
In development, I'm authenticating my API calls using the Application Default Credentials. I downloaded the JSON file with the credentials from the account service I created exclusively for analytics purposes, set the file to the Google_Application_Credentials environment variable and everything worked. I'm able to get the data from Google Analytics and display it in the website.
But this is not working in production. I suppose getClient() it's not getting the credentials in that environment.
Things to note: 1) I did not upload the downloaded JSON file with the credentials from the service account (I think it would be counter intuitive and unsafe to do that, and from what I understood in the docs, GCP is able to deal automatically with the API authentications);
const {google} = require("googleapis");
async function main () {
// This method looks for the GCLOUD_PROJECT and GOOGLE_APPLICATION_CREDENTIALS
// environment variables.
const auth = await google.auth.getClient({
// Scope of the analytics reporting,
// with only reading access.
scopes: 'https://www.googleapis.com/auth/analytics.readonly',
});
// Create the analytics reporting object
const analyticsreporting = await google.analyticsreporting({
version: 'v4',
auth: auth,
});
// Fetch the analytics reporting
const res = await analyticsreporting.reports.batchGet({...});
return res.data;
}
I already run out of options. Can someone help me with this?
This is a problem with the default scopes and application default credentials. By default, if you don't create a new service account, you are going to get 'application default credentials' from the GCE metadata service:
https://cloud.google.com/docs/authentication/production#auth-cloud-implicit-nodejs
Those credentials usually only have the cloud-platform scope, and the set of scopes cannot be changed (as of today). To make this work, you have a few options.
You could create a new service account, download the service account key, and use the keyFile property in the getClient method options to reference the key. If you do it this way, the scopes you pass into getClient will be respected.
You could play with the scopes available to the service account under which your GAE application is running. I haven't personally tried that, but it theoretically should be possible.
Best of luck!

Authenticating a Google Cloud Function as a service account on other Google APIs

I have an HTTP-triggered function running on Google Cloud Functions, which uses require('googleapis').sheets('v4') to write data into a docs spreadsheet.
For local development I added an account via the Service Accounts section of their developer console. I downloaded the token file (dev-key.json below) and used it to authenticate my requests to the Sheets API as follows:
var API_ACCT = require("./dev-key.json");
let apiClient = new google.auth.JWT(
API_ACCT.client_email, null, API_ACCT.private_key,
['https://www.googleapis.com/auth/spreadsheets']
);
exports.myFunc = function (req, res) {
var newRows = extract_rows_from_my_client_app_request(req);
sheets.spreadsheets.values.append({
auth: apiClient,
// ...
resource: { values:newRows }
}, function (e) {
if (e) res.status(500).json({err:"Sheets API is unhappy"});
else res.status(201).json({ok:true})
});
};
After I shared my spreadsheet with my service account's "email address" e.g. local-devserver#foobar-bazbuzz-123456.iam.gserviceaccount.com — it worked!
However, as I go to deploy this to the Google Cloud Functions service, I'm wondering if there's a better way to handle credentials? Can my code authenticate itself automatically without needing to bundle a JWT key file with the deployment?
I noticed that there is a FUNCTION_IDENTITY=foobar-bazbuzz-123456#appspot.gserviceaccount.com environment variable set when my function runs, but I do not know how to use this in the auth value to my googleapis call. The code for google.auth.getApplicationDefault does not use that.
Is it considered okay practice to upload a private JWT token along with my GCF code? Or should I somehow be using the metadata server for that? Or is there a built-in way that Cloud Functions already can authenticate themselves to other Google APIs?
It's common to bundle credentials with a function deployment. Just don't check them into your source control. Cloud Functions for Firebase samples do this where needed. For example, creating a signed URL from Cloud Storage requires admin credentials, and this sample illustrates saving that credential to a file to be deployed with the functions.
I'm wondering if there's a better way to handle credentials? Can my
code authenticate itself automatically without needing to bundle a JWT
key file with the deployment?
Yes. You can use 'Application Default Credentials', instead of how you've done it, but you don't use the function getApplicationDefault() as it has been deprecated since this Q was posted.
The link above shows how to make a simple call using the google.auth.getClient API, providing the desired scope, and have it decide the credential type needed automatically. On cloud functions this will be a 'Compute' object, as defined in the google-auth-library.
These docs say it well here...
After you set up a service account, ADC can implicitly find your
credentials without any need to change your code, as described in the
section above.
Where ADC is Application Default Credentials.
Note that, for Cloud Functions, you use the App Engine service account:
YOUR_PROJECT_ID#appspot.gserviceaccount.com, as documented here. That is the one you found via the FUNCTION_IDENTITY env var - this rather tripped me up.
The final step is to make sure that the service account has the required access as you did with your spreadsheet.

Managing tokens a shared google account for file storage

I am creating an application in Node.js, hosted by Heroku. Using passport.js, I have already implemented sign-in authentication with a google account, integrated with a mySQL database.
In part of the application, I ask the applicant to upload a set of files. The way I would like to handle the uploading is through google drive APIs. Essentially, the user will be able to select files, which the back-end would then upload to a single-account google drive. Note: This will be a separate account from any of the applicant's accounts.
While I understand the process of uploading and retrieving the files, I am still unsure how the tokens work. From my research online I know:
Access tokens expire after a certain time
Refresh Tokens do not expire and are to be stored in the database
My question is how to manage these tokens in the backend. My current plan is to use the google oauth playground to get the access tokens for the app under the shared account. Then, every time I need to upload or access a file, I get a new access token using the refresh token, and then use that access token to do my API calls.
However, after doing some implementation testing, I have some confusion. I went through the Google Node.js Quickstart Guide and then modified the code to do a file-upload instead of a file reader. The modified code is below:
function fileUpload(auth) {
var drive = google.drive({version: 'v2'});
drive.files.insert({
auth: auth,
resource: {
title:'Test',
mimeType: 'text/plain'
},
media: {
mimeType: 'text/plain',
body: 'TEST'
}
}, function(err, response) {
if (err) {
console.log('The API returned an error ' + err);
return;
} else {
console.log('Inserted')
}
});
}
From my understanding, after the access token is expired, you cannot use it anymore. However, after I ran the code after the access token expired, it was still able to complete the process. Further, the access token did not change either. Hence, my confusion is how to manage these access tokens, specifically if I need to worry about access tokens expiring, or if they are valid once they are used once.

Resources