Application Default Credentials http trigger GCP function from local nodejs application - node.js

I want to trigger a GCP cloud function from a simple nodejs app running locally.
Reading the documentation it should be simple:
run gcloud auth application-default login to write ADC to file used by client libraries.
use google-auth-library to get a http client to use to trigger the function.
/**
* TODO(developer): Uncomment these variables before running the sample.
*/
// Example: https://my-cloud-run-service.run.app/books/delete/12345
// const url = 'https://TARGET_HOSTNAME/TARGET_URL';
// Example (Cloud Functions): https://project-region-projectid.cloudfunctions.net/myFunction
const targetAudience = 'https://<REGION>-<PROJECTID>.cloudfunctions.net/<FUNCTIONNAME>';
const { GoogleAuth } = require('google-auth-library');
const auth = new GoogleAuth();
const payload = {"prop1": "prop1Value"};
async function request() {
const client = await auth.getIdTokenClient(targetAudience);
const resp = await client.request({ url: targetAudience, method: 'POST', data: payload });
console.info(`Resp status: ${resp.status}; resp.data: ${resp.data}`);
}
(async () => {
await request();
})();
My understanding was that the google-auth-library would pick up the ADC from the file setup from running gcloud auth application-default login and everything would work.
My user has permission to invoke GCP functions as I can trigger the function using CURL with the header -H "Authorization:bearer $(gcloud auth print-identity-token)" \
However when I run this, it doesn't get past the line:
const client = await auth.getIdTokenClient(targetAudience);
Failing with:
Cannot fetch ID token in this environment, use GCE or set the GOOGLE_APPLICATION_CREDENTIALS environment variable t
o a service account credentials JSON file.
Using PubSub library works fine so expect ADC does work just not sure what am I missing when trying to trigger the GCP function.
Am I using the google-auth-library correctly here ?
Thanks

As mentioned in the thread:
gcloud auth activate-service-account --key-file is only for "you"
running gcloud commands, it won’t be picked up by "applications" that
need GOOGLE_APPLICATION_CREDENTIALS. As you can see from Invoke a
Google Cloud Run from java or How to call Cloud Run from outside
of Cloud Run/GCP?, you either need to have the JSON key file of
Service Account, or have to be running inside a GCE/GKE/Cloud Run/App
Engine/GCF instance.
For this to work on your local environment, I recommend logging in
with gcloud auth application-default login command (this command is
meant to work as if you’ve set GOOGLE_APPLICATION_CREDENTIALS
locally).
If that doesn't work, as a last resort you can refactor your code to
pick up identity token from an environment variable (if set) while
working locally,
such as: $ export ID_TOKEN="$(gcloud auth print-identity-token -q)" $ ./your-app
To know more about how the code does it with a JSON key file,refer to the link and similar implementation there.
For more information you can refer to a similar thread stated as :
Give the default service account access rights to Workspace
resource(s) you're attempting to access.
Use the JSON key file you set
up locally already, to have the Cloud Function run as the same user as
is happening when you run locally.
Essentially do a hybrid where you create a new service account that ONLY has the permissions you want (instead of using the default
service account or your personal user, both of which might have far
more permissions then desired for safety/security), use a key file to
run the Cloud Function under that identity, and only give the desired
permissions to that service account.

Related

Setting up Google Drive API on NodeJS using a service account

I'm trying to connect to the Google Drive API with a NodeJS server using a service account. The goal is for the server to be able to authenticate as the service account, retrieve relevant files from a drive, and send them back to the user, without the user needing to log in to Google directly. This would allow me to control file access through my web app instead of having to manually share and unshare files through Drive. From my understanding of the Google Drive API, this should all be possible. The problem is that I can't even figure out how to authenticate my server. The server runs on an AWS EC2 instance. To clarify, I do not want the user to have to authenticate using the frontend interface.
I've followed the quickstart guide and set up a service account & key as instructed here, but upon creating the key as instructed in the second link, it doesn't look like I have the correct credentials.json file. The JSON file I get after generating a key on the Google Developer Console has the following object keys (values intentionally removed):
type, project_id, private_key_id, private_key, client_email, client_id, auth_uri, token_uri, auth_provider_x509_cert_url, client_x509_cert_url
The quickstart guide suggests that this file should contain client_secret and redirect_uris within some installed object (const {client_secret, client_id, redirect_uris} = credentials.installed;):
Attempting to run this index.js quickstart file causes an error to be thrown, since installed does not exist within credentials.json. Where can I generate the necessary credentials file? Or am I on the wrong track completely?
Posts like this reference a similar issue on an older version of the quickstart documentation, but the solutions here don't help since there isn't a client_secret key in my credentials file.
When I saw the showing keys of your credentials.json file, I understood that the file is the credential file of the service account. If my understanding is correct, when I saw your showing script, it seems that the script is for OAuth2. In this case, this script cannot be used for the service account. I thought that this is the reason for your current issue.
In order to use Drive API using the service account, how about the following sample script?
Sample script:
Before you use this script, please set credentialFilename of the service account. In this case, please include the path.
const { google } = require("googleapis");
const credentialFilename = "credentials.json";
const scopes = ["https://www.googleapis.com/auth/drive.metadata.readonly"];
const auth = new google.auth.GoogleAuth({keyFile: credentialFilename, scopes: scopes});
const drive = google.drive({ version: "v3", auth });
// This is a simple sample script for retrieving the file list.
drive.files.list(
{
pageSize: 10,
fields: "nextPageToken, files(id, name)",
},
(err, res) => {
if (err) return console.log("The API returned an error: " + err);
const files = res.data.files;
console.log(files);
}
);
When this script is run, as a sample script, the file list is retrieved from the Google Drive of the service account. So, please modify this for your actual situation.
This sample script uses https://www.googleapis.com/auth/drive.metadata.readonly as the scope. Please modify this for your actual situation.
Reference:
Google APIs Node.js Client

How to authenticate with tokens in Nodejs to a private bucket in Cloud Storage

Usually in Python what I do, I get the application default credentials, I get the access token then I refresh it to be able to authenticate to a private environment.
Code in Python:
# getting the credentials and project details for gcp project
credentials, your_project_id = google.auth.default(scopes=["https://www.googleapis.com/auth/cloud-platform"])
#getting request object
auth_req = google.auth.transport.requests.Request();
print(f"Checking Authentication : {credentials.valid}")
print('Refreshing token ....')
credentials.refresh(auth_req)
#check for valid credentials
print(f"Checking Authentication : {credentials.valid}")
access_token = credentials.token
credentials = google.oauth2.credentials.Credentials(access_token);
storage_client = storage.Client(project='itg-ri-consumerloop-gbl-ww-dv',credentials=credentials)
I am entirely new to NodeJS, and I am trying to make the same thing.
My goal later is to create an app engine application that would expose an image that is found in a private bucket, so credentials are a must.
How it is done?
For authentication, you could rely on the default application credentials that are present within the GCP platform (GAE, Cloud Functions, VM, etc.). Then you could just run the following piece of code from the documentation:
const {Storage} = require('#google-cloud/storage');
const storage = new Storage();
const bucket = storage.bucket('albums');
const file = bucket.file('my-existing-file.png');
In most circumstances, there is no need to explicitly use authentication packages since they are already executed underneath the google-cloud/storage package in Nodejs. The same holds for the google-cloud-storage package in Python. It could help to look at the source code of both packages on Github. For me, this really helped to understand the authentication mechanism.
When I develop code on my own laptop, that interacts with google cloud storage, I first tell the gcloud SDK what my credentials are and on which GCP project I am working. I use the following commands for this:
gcloud config set project [PROJECT_ID]
gcloud auth application-default login
You could also set DEFAULT_APPLICATION_CREDENTIALS as an environment variable that points to a credentials file. Then within your code, you could pass the project name when initializing the client. This could be helpful if you are running your code outside of GCP on another server for example.

Why my API calls using google-api-nodejs-client to Google Analytics are not working in production?

I'm calling the Google Analytics Reporting API using google-api-nodejs-client to show the number of visits inside a blog.
This blog is hosted inside Google App Engine Standard Environment.
In development, I'm authenticating my API calls using the Application Default Credentials. I downloaded the JSON file with the credentials from the account service I created exclusively for analytics purposes, set the file to the Google_Application_Credentials environment variable and everything worked. I'm able to get the data from Google Analytics and display it in the website.
But this is not working in production. I suppose getClient() it's not getting the credentials in that environment.
Things to note: 1) I did not upload the downloaded JSON file with the credentials from the service account (I think it would be counter intuitive and unsafe to do that, and from what I understood in the docs, GCP is able to deal automatically with the API authentications);
const {google} = require("googleapis");
async function main () {
// This method looks for the GCLOUD_PROJECT and GOOGLE_APPLICATION_CREDENTIALS
// environment variables.
const auth = await google.auth.getClient({
// Scope of the analytics reporting,
// with only reading access.
scopes: 'https://www.googleapis.com/auth/analytics.readonly',
});
// Create the analytics reporting object
const analyticsreporting = await google.analyticsreporting({
version: 'v4',
auth: auth,
});
// Fetch the analytics reporting
const res = await analyticsreporting.reports.batchGet({...});
return res.data;
}
I already run out of options. Can someone help me with this?
This is a problem with the default scopes and application default credentials. By default, if you don't create a new service account, you are going to get 'application default credentials' from the GCE metadata service:
https://cloud.google.com/docs/authentication/production#auth-cloud-implicit-nodejs
Those credentials usually only have the cloud-platform scope, and the set of scopes cannot be changed (as of today). To make this work, you have a few options.
You could create a new service account, download the service account key, and use the keyFile property in the getClient method options to reference the key. If you do it this way, the scopes you pass into getClient will be respected.
You could play with the scopes available to the service account under which your GAE application is running. I haven't personally tried that, but it theoretically should be possible.
Best of luck!

Google Cloud Vision reverse image search fails on Azure App Service because GOOGLE_APPLICATION_CREDENTIALS file cannot be found

I am attempting to perform a Google reverse image search using Google Cloud Vision on an Azure app service web app.
I have generated a googleCred.json, which the Google client libraries use in order to construct API requests. Google expects it to be available from an environment variable named GOOGLE_APPLICATION_CREDENTIALS.
The Azure app service that runs the web app has settings that mimic environment variables for the Google client libraries. The documentation is here, and I have successfully set the variable here:
Furthermore, the googleCred.json file has been uploaded to the app service. Here is the documentation I followed to use FTP and FileZilla to upload the file:
Also, the file permissions are as open as they can be:
However, when I access the web app in the cloud, I get the following error message:
Error reading credential file from location D:\site\wwwroot\Statics\googleCred.json: Could not find a part of the path 'D:\site\wwwroot\Statics\googleCred.json'. Please check the value of the Environment Variable GOOGLE_APPLICATION_CREDENTIALS
What am I doing wrong? How can I successfully use the Google Cloud Vision API on an Azure web app?
This error message is usually thrown when the application is not being authenticated correctly due to several reasons such as missing files, invalid credential paths, incorrect environment variables assignations, among other causes.
Based on this, I recommend you to validate that the credential file and file path are being correctly assigned, as well as follow the Obtaining and providing service account credentials manually guide in order to explicitly specify your service account file directly into your code; In this way, you will be able to set it permanently and verify if you are passing the service credentials correctly.
Passing the path to the service account key in code example:
// Imports the Google Cloud client library.
const Storage = require('#google-cloud/storage');
// Instantiates a client. Explicitly use service account credentials by
// specifying the private key file. All clients in google-cloud-node have this
// helper, see https://github.com/GoogleCloudPlatform/google-cloud-node/blob/master/docs/authentication.md
const storage = new Storage({
keyFilename: '/path/to/keyfile.json'
});
// Makes an authenticated API request.
storage
.getBuckets()
.then((results) => {
const buckets = results[0];
console.log('Buckets:');
buckets.forEach((bucket) => {
console.log(bucket.name);
});
})
.catch((err) => {
console.error('ERROR:', err);
});
I'm writing here since i can't comment, but at a quick glance, is the "D:" in the path necessary? I assume you uploaded the file to the app service so try with this value for the path "\site\wwwroot\Statics\googleCred.json"

Firebase serve --only functions, admin.auth() locally does not work unless I manually set the key.json

I am trying to run cloud functions locally using firebase serve --only functions. It works if I manually specify the key.json that is downloaded when I create a service account through firebase console here: https://console.firebase.google.com/project/project-id/settings/serviceaccounts/adminsdk.
And then doing export GOOGLE_APPLICATION_CREDENTIALS=key.json. Why do I have to do this? Shouldn't firebase or gcloud handle this correctly? I thought credential in functions.config().firebase was ApplicationDefault. https://developers.google.com/identity/protocols/application-default-credentials. According to that, the env variable like above is checked first, so it works if that is there. But shouldn't #2 validate me correctly?
I have gcloud for app engine so I have done gcloud init and gcloud auth application-default login.
I initialize my app in index.js
const admin = require('firebase-admin');
const functions = require('firebase-functions');
admin.initializeApp(functions.config().firebase);
firebase serve --only functions deploys the functions fine. But when I execute one which uses this code admin.auth().getUser(uid)... it breaks with the following error:
Error: An internal error has occurred. Raw server response: "{"error":{"errors":[{"domain":"usageLimits","reason":"accessNotConfigured","message":"Access Not Configured. Google Identity Toolkit API has not been used in project 7640...50 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/identitytoolkit.googleapis.com/overview?project=7640...50 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.","extendedHelp":"https://console.developers.google.com/apis/api/identitytoolkit.googleapis.com/overview?project=7640...50"}],"code":403,"message":"Access Not Configured. Google Identity Toolkit API has not been used in project 7640...50 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/identitytoolkit.googleapis.com/overview?project=7640...50 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry."}}
The project number doesn't correspond to my firebase project or project id/number in console.cloud.google.com
Without gcloud, I get this error
Error: Credential implementation provided to initializeApp() via the "credential" property failed to fetch a valid Google OAuth2 access token with the following error: "Error fetching access token: invalid_grant (Token has been expired or revoked.)". There are two likely causes: (1) your server time is not properly synced or (2) your certificate key file has been revoked. To solve (1), re-sync the time on your server. To solve (2), make sure the key ID for your key file is still present at https://console.firebase.google.com/iam-admin/serviceaccounts/project. If not, generate a new key file at https://console.firebase.google.com/project/_/settings/serviceaccounts/adminsdk
similar to https://github.com/urish/firebase-server/issues/81
I already have a key file generated. I've tried deleting my existing ones and making new ones, the only solution is to download this key that is generated and manually export the environment variable
Is there any other way than that?
Technically yes, but you can build your code so that it uses a local key when testing locally, and uses the service key when running on the server.
For this I set the env var FB_SERVICE_ACCOUNT_KEY to the path of the json secret. When it is unset (i.e. deployed within Firebase functions), the standard init will be used.
Here is a basic example
if(process.env.FB_SERVICE_ACCOUNT_KEY){
var serviceAccount = require(process.env.FB_SERVICE_ACCOUNT_KEY);
admin.initializeApp({
credential: admin.credential.cert(serviceAccount),
databaseURL: `https://${process.env.GCLOUD_PROJECT}.firebaseio.com`
});
}
else{
admin.initializeApp(functions.config().firebase);
}

Resources