what I want to achieve, is that for my pods that live inside GKE to share files. So what I'm thinking is using the GoogleCloudStorage to write and read the files.
I have created a service account in my kubetcl
kubectl create serviceaccount myxxx-svc-account --namespace
myxxx
Then I also created the service account in my GCP console
Then, I added the roles of roles/iam.workloadIdentityUser in my GCP account
Next, I annotated my kubectl account with my GCP service account
kubectl annotate serviceaccount --namespace myxxx
myxxx-svc-account
iam.gke.io/gcp-service-account=myxxx-svc-account#myxxx-xxxxx.iam.gserviceaccount.com
I also added the roles of Storage Admin and Storage Object Admin in the GCP - IAM page
Then, in my deployment.yaml, I included my service account
spec:
serviceAccountName: myxxx-account
Bellow is how I try to upload a file to the storage
const {Storage} = require('#google-cloud/storage');
const storage = new Storage();
const bucket = storage.bucket('bucket-name');
const options = {
destination: '/folder1/folder2/123456789'
};
bucket.upload("./index.js", options, function(uploadError, file, apiResponse) {
console.log(uploadError.message)
console.log(uploadError.stack)
});
I deploy my node application to the GKE pods through docker. In the dockerFile, im using
FROM node
...
...
...
CMD ["node", "index.js"]
But I always get unauthorized 403 error
Could not refresh access token: A Forbidden error was returned while
attempting to retrieve an access token for the Compute Engine built-in
service account. This may be because the Compute Engine instance does
not have the correct permission scopes specified: Could not refresh
access token: Unsuccessful response status code. Request failed with
status code 403
Error: Could not refresh access token: A Forbidden
error was returned while attempting to retrieve an access token for
the Compute Engine built-in service account. This may be because the
Compute Engine instance does not have the correct permission scopes
specified: Could not refresh access token: Unsuccessful response
status code. Request failed with status code 403
at Gaxios._request (/opt/app/node_modules/gaxios/build/src/gaxios.js:130:23)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async metadataAccessor (/opt/app/node_modules/gcp-metadata/build/src/index.js:68:21)
at async Compute.refreshTokenNoCache (/opt/app/node_modules/google-auth-library/build/src/auth/computeclient.js:54:20)
at async Compute.getRequestMetadataAsync (/opt/app/node_modules/google-auth-library/build/src/auth/oauth2client.js:298:17)
at async Compute.requestAsync (/opt/app/node_modules/google-auth-library/build/src/auth/oauth2client.js:371:23)
at async Upload.makeRequest (/opt/app/node_modules/#google-cloud/storage/build/src/resumable-upload.js:574:21)
at async retry.retries (/opt/app/node_modules/#google-cloud/storage/build/src/resumable-upload.js:306:29)
at async Upload.createURIAsync (/opt/app/node_modules/#google-cloud/storage/build/src/resumable-upload.js:303:21)
What I'm doing wrong? seems like I have given the permission already? how can I troubleshoot it? Is it related with the docker image?
Related
I want to trigger a GCP cloud function from a simple nodejs app running locally.
Reading the documentation it should be simple:
run gcloud auth application-default login to write ADC to file used by client libraries.
use google-auth-library to get a http client to use to trigger the function.
/**
* TODO(developer): Uncomment these variables before running the sample.
*/
// Example: https://my-cloud-run-service.run.app/books/delete/12345
// const url = 'https://TARGET_HOSTNAME/TARGET_URL';
// Example (Cloud Functions): https://project-region-projectid.cloudfunctions.net/myFunction
const targetAudience = 'https://<REGION>-<PROJECTID>.cloudfunctions.net/<FUNCTIONNAME>';
const { GoogleAuth } = require('google-auth-library');
const auth = new GoogleAuth();
const payload = {"prop1": "prop1Value"};
async function request() {
const client = await auth.getIdTokenClient(targetAudience);
const resp = await client.request({ url: targetAudience, method: 'POST', data: payload });
console.info(`Resp status: ${resp.status}; resp.data: ${resp.data}`);
}
(async () => {
await request();
})();
My understanding was that the google-auth-library would pick up the ADC from the file setup from running gcloud auth application-default login and everything would work.
My user has permission to invoke GCP functions as I can trigger the function using CURL with the header -H "Authorization:bearer $(gcloud auth print-identity-token)" \
However when I run this, it doesn't get past the line:
const client = await auth.getIdTokenClient(targetAudience);
Failing with:
Cannot fetch ID token in this environment, use GCE or set the GOOGLE_APPLICATION_CREDENTIALS environment variable t
o a service account credentials JSON file.
Using PubSub library works fine so expect ADC does work just not sure what am I missing when trying to trigger the GCP function.
Am I using the google-auth-library correctly here ?
Thanks
As mentioned in the thread:
gcloud auth activate-service-account --key-file is only for "you"
running gcloud commands, it won’t be picked up by "applications" that
need GOOGLE_APPLICATION_CREDENTIALS. As you can see from Invoke a
Google Cloud Run from java or How to call Cloud Run from outside
of Cloud Run/GCP?, you either need to have the JSON key file of
Service Account, or have to be running inside a GCE/GKE/Cloud Run/App
Engine/GCF instance.
For this to work on your local environment, I recommend logging in
with gcloud auth application-default login command (this command is
meant to work as if you’ve set GOOGLE_APPLICATION_CREDENTIALS
locally).
If that doesn't work, as a last resort you can refactor your code to
pick up identity token from an environment variable (if set) while
working locally,
such as: $ export ID_TOKEN="$(gcloud auth print-identity-token -q)" $ ./your-app
To know more about how the code does it with a JSON key file,refer to the link and similar implementation there.
For more information you can refer to a similar thread stated as :
Give the default service account access rights to Workspace
resource(s) you're attempting to access.
Use the JSON key file you set
up locally already, to have the Cloud Function run as the same user as
is happening when you run locally.
Essentially do a hybrid where you create a new service account that ONLY has the permissions you want (instead of using the default
service account or your personal user, both of which might have far
more permissions then desired for safety/security), use a key file to
run the Cloud Function under that identity, and only give the desired
permissions to that service account.
I am building NodeJs server and trying to leverage FCM for push notifications. Locally everything works as expected, but in the K8S cluster I receive the following error
FirebaseAppError: Credential implementation provided to initializeApp() via the "credential" property failed to fetch a valid Google OAuth2 access token with the following error: "Error fetching access token: Missing error payload".
errorInfo: {
code: 'app/invalid-credential',
message: 'Credential implementation provided to initializeApp() via the "credential" property failed to fetch a valid Google OAuth2 access token with the following error: "Error fetching access token: Missing error payload".'
},
codePrefix: 'app'
I tried to initialize app without arguments admin.initializeApp(), keeping env var with path json key in a config object, the path is valid, the file is present, the env is printed out ok.
I tried like this const firebase = admin.initializeApp({ credential: admin.credential.applicationDefault(), }); which is obviuoely the same thing. I Tried to explicitly pass credential prop with json certificate like so const firebase = initializeApp({ credential: admin.credential.cert(cert) });
cert is present. I even tried to downgrade from firebase-admin from 10 to 9.
Node 12
please help
I found an issue. It was DNS settings of my K8S cluster. So it didn't have proper access to the outside world (check /etc/resolv.conf inside the pod)
Had to add dns settings to my deployment declaration like so
containers:
...
dnsPolicy: "None"
dnsConfig:
nameservers:
- 10.96.0.10 // for inner services lookup
- 8.8.8.8
searches:
- default.svc.cluster.local
- svc.cluster.local
- cluster.local
I'm having trouble on authenticating as a service account in my Next.js app hosted on Vercel. My code is working fine in my dev environment, but it fails with the following error message when I try to run it on Vercel Node.js v14 environment. I guess that it runs on my local machine because I'm logged in gcloud with my email (project owner).
This is the error I'm getting:
Error: Could not load the default credentials. Browse to https://cloud.google.com/docs/authentication/getting-started for more information.
at GoogleAuth.getApplicationDefaultAsync (/var/task/node_modules/google-auth-library/build/src/auth/googleauth.js:173:19)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
at async GoogleAuth.getClient (/var/task/node_modules/google-auth-library/build/src/auth/googleauth.js:551:17)
at async GrpcClient._getCredentials (/var/task/node_modules/google-gax/build/src/grpc.js:109:24)
at async GrpcClient.createStub (/var/task/node_modules/google-gax/build/src/grpc.js:252:23)
I've created the following service account to use it with my Next.js APIs.
It has all the necessary roles. I've created a JSON key and download it.
I'm using firebase-admin and this is how I'm initializing it:
export const initializeFirebaseAdmin = (): FirebaseAdmin => {
const account = getServiceAccount(); // THIS IS THE SERVICE ACCOUNT JSON KEY (ALREADY PARSED AS AN OBJECT)
if (!admin.apps.length) {
admin.initializeApp({
credential: admin.credential.cert(account),
});
}
return admin;
};
This is what I think it's happening:
From: https://cloud.google.com/docs/authentication/production#automatically
From the image above:
I'm not setting any GOOGLE_APPLICATION_CREDENTIALS environment variable
So I should be on number 2. It will "try to use the service account that is attached to the resource that is running your code."
It's obviously failing and I getting the error
But which resource is it referring to? Which code?
The resource that is running my code is Firebase Admin? Am I not initializing it correctly?
This code should work to authenticate the firebase-admin package:
export const initializeFirebaseAdmin = (): FirebaseAdmin => {
const account = getServiceAccount(); // THIS IS THE SERVICE ACCOUNT JSON KEY (ALREADY PARSED AS AN OBJECT)
if (!admin.apps.length) {
admin.initializeApp({
credential: admin.credential.cert(account),
});
}
return admin;
};
My problem was related to the fact that I was using a client exposed from the firebase-admin package, that doesn't have access to the authentication from the main package. So I needed to pass the credentials to it as well. Like:
const client = new admin.firestore.v1.FirestoreAdminClient({
credentials: SERVICE_ACCOUNT as CredentialBody // <<<<<< THIS IS THE SERVICE ACCOUNT JSON KEY
});
Refer also to: Trying to export backup via admin.firestore.v1.FirestoreAdminClient on a Next.js API route. Error: Could not load the default credentials
Error
When calling admin.auth().createCustomToken() I am getting the following error:
Error: The caller does not have permission; Please refer to https://firebase.google.com/docs/auth/admin/create-custom-tokens for more details on how to use and troubleshoot this feature.
The provided documentation leads me to believe that the service account I am initializing the Firebase Admin SDK with does not have sufficient permissions. I don't believe this to be the case, so I want to ask and see if I've missed anything.
Configuration
Firebase Admin SDK is initialized in the backend like so:
admin.initializeApp({
serviceAccountId: 'firebase-adminsdk-xxxxx#my-project-id.iam.gserviceaccount.com'
});
Technically the value is referenced from an env var, but I have confirmed this value to be correct.
The service account being used has the following roles:
roles/firebase.sdkAdminServiceAgent
roles/iam.serviceAccountTokenCreator
Per the documentation, the required permission for creating custom tokens is iam.serviceAccounts.signBlob. This permission is part of the iam.serviceAccountTokenCreator role as per this output:
❯ gcloud beta iam roles describe roles/iam.serviceAccountTokenCreator
description: Impersonate service accounts (create OAuth2 access tokens, sign blobs
or JWTs, etc).
etag: AA==
includedPermissions:
- iam.serviceAccounts.get
- iam.serviceAccounts.getAccessToken
- iam.serviceAccounts.getOpenIdToken
- iam.serviceAccounts.implicitDelegation
- iam.serviceAccounts.list
- iam.serviceAccounts.signBlob
- iam.serviceAccounts.signJwt
- resourcemanager.projects.get
- resourcemanager.projects.list
name: roles/iam.serviceAccountTokenCreator
stage: GA
title: Service Account Token Creator
Lastly, the code in question that is erroring out is as follows:
try {
const loginToken = await admin.auth().createCustomToken(uid);
return response(200).json({ loginToken });
} catch (err) {
...
}
The uid comes from signing in a user via a GoogleUser credential - the provided uid is confirmed to be accurate, and this flow works locally when referencing a JSON key file for the same service account.
Server is running on GKE, in case it could be a cluster permission error.
Any help would be greatly appreciated!
EDIT - RESOLVED
Hiranya's answer did the trick - the K8s deployment had been configured with a service account whose original intent was only to enable Cloud SQL Proxy. Giving this service account the serviceAccountTokenCreator role solved the issue.
You need to make sure the service account that the SDK is authorized with (not the one specified as serviceAccountId) has the token creator role. This is the service account auto-discovered by Google Application Default Credentials. In case of Cloud Functions this is the service account named {project-name}#appspot.gserviceaccount.com. You need to figure out the equivalent service account for GKE and grant it the token creator role.
I'm writing a cloud function in Nodejs (10), and trying to access a secret like so:
const [secret] = await new SecretManagerServiceClient().accessSecretVersion({
name: `projects/PROJECT_NUMBER/secrets/SECRET_NAME/versions/latest`
})
I created the secret in the web console and the name used in code matches that of the existing secret. On the page for the cloud function details, it states that the service account is PROJECT_ID#appspot.gserviceaccount,com, so I added the secretmanager.secretAccessor role to it. However, I'm still getting the same error every time:
Error: 7 PERMISSION_DENIED: Permission 'secretmanager.versions.access' denied for resource 'projects/PROJECT_NUMBER/secrets/SECRET_NAME/versions/latest' (or it may not exist).
It makes no difference if I specify a concrete version or just use latest.
HTTP cloud function code:
const { SecretManagerServiceClient } = require('#google-cloud/secret-manager');
const secretManagerServiceClient = new SecretManagerServiceClient();
const name = 'projects/shadowsocks-218808/secrets/workflow/versions/latest';
exports.testSecretManager = async (req, res) => {
const [version] = await secretManagerServiceClient.accessSecretVersion({ name });
const payload = version.payload.data.toString();
console.debug(`Payload: ${payload}`);
res.sendStatus(200);
};
Deploy:
gcloud functions deploy testSecretManager --runtime nodejs10 --trigger-http --allow-unauthenticated
Deploying function (may take a while - up to 2 minutes)...done.
availableMemoryMb: 256
entryPoint: testSecretManager
httpsTrigger:
url: https://us-central1-shadowsocks-218808.cloudfunctions.net/testSecretManager
ingressSettings: ALLOW_ALL
labels:
deployment-tool: cli-gcloud
name: projects/shadowsocks-218808/locations/us-central1/functions/testSecretManager
runtime: nodejs10
serviceAccountEmail: shadowsocks-218808#appspot.gserviceaccount.com
sourceUploadUrl: https://storage.googleapis.com/gcf-upload-us-central1-43476143-b555-4cb2-8f6f-1b2d1952a2d7/42c4cda4-98a8-4994-a3be-d2203b9e646a.zip?GoogleAccessId=service-16536262744#gcf-admin-robot.iam.gserviceaccount.com&Expires=1596513795&Signature=kbLw5teN8EoYmj4fEweKKiIaakxcrhlUg2GGHV4jWJjvmeEfXePpRNOn9yz2zLn%2Fba0UqM9qdJMXujs5afBk%2BVBmywPEiptAZe2qgmldpr%2BsYejFu0woNgsPHVqtJ0NoWDo6W2dq4CuNNwO%2BaQ89mnhahUUQTInkJ55Y3wCIe9smk%2BqWtcvta3zICiToA7RQvPKY5MS6NViyj5mLxuJtDlTY9IKPL%2BqG6JAaQJSFYKYVgLyb6JfirXk8Q7%2FMvnHPpXPlhvsBLQksbF6jDPeefp2HyW4%2FSIQYprfpwKV3hlEIQyRQllz5J9yF83%2FxDPh%2BQPc5QmswKP5XAvYaszJPEw%3D%3D
status: ACTIVE
timeout: 60s
updateTime: '2020-08-04T03:34:32.665Z'
versionId: '2'
Test:
gcloud functions call testSecretManager --data '{}'
Got error same as you:
error: |-
Error: function terminated. Recommended action: inspect logs for termination reason. Details:
7 PERMISSION_DENIED: Permission 'secretmanager.versions.access' denied for resource 'projects/shadowsocks-218808/secrets/workflow/versions/latest' (or it may not exist).
solution:
You can find the serviceAccountEmail: shadowsocks-218808#appspot.gserviceaccount.com from the deployment information details of cloud function.
go to IAM & Admin web UI, click ADD ANOTHER ROLE button, add Secret Manager Secret Accessor role to this service account.
Test again:
> gcloud functions call testSecretManager --data '{}'
executionId: 1tsatxl6fndw
result: OK
Read the logs for testSecretManager cloud function:
gcloud functions logs read testSecretManager
You will see the logs for the secret payload string.
I had the same issue and to solve it, I just had to:
Find the Service Account under General of my Google Cloud Function.
It looked like <project-name>#appspot.gserviceaccount.com
In IAM Admin, Add Secret Manager Secret Accessor Role to this Service Account.
After this, everything worked!
I have had similar issues working with secretmanager and the python google-cloud-secretmanager library (2.4). Specifically, after creating a secret and giving my service account the secretmanager.secretAccessor role on this secret (and nothing else, following the principle of least privilege), I was getting the following error when trying to access it:
details = "Permission 'secretmanager.versions.access' denied for resource 'projects/projectid/secrets/keyname/versions/latest' (or it may not exist)."
I could only make it work by also adding the secretmanager.viewer role at the project level, which as far as I can tell is not described in the documentation.
I had similar problem using terraform under gitlab.
I must add two authorizations to the service account which runs the pipeline:
resource "google_project_iam_policy" "gitlab" {
project = "secret_owner_project_id"
policy_data = data.google_iam_policy.iam.policy_data
}
data "google_iam_policy" "iam" {
binding {
role = "roles/secretmanager.secretAccessor"
members = [
"serviceAccount:project_accessing_secret#XYZ.iam.gserviceaccount.com",
]
}
binding {
role = "roles/viewer"
members = [
"serviceAccount:project_accessing_secret#XYZ.iam.gserviceaccount.com",
]
}
}
A bit late, but maybe this answer could be useful for future users. I encountered the same behavior only with Python. I tried lots of things but only thing that worked was creating new service account with zero roles(if I granted it secretmanager.secretAccessor role immediately, I got the same error). Then when empty service account is created, in IAM tab I press +Add, copy my empty service account adress and ONLY then I add secretmanager.secretAccessor role to it. Then I use this account as the account that will execute particular function. You of course may need to add other roles depending on what your function is intended to accomplish.
OAuth scope plays an important role here and please make sure the scope is defined correctly.
To use Secret Manager with workloads running on Compute Engine or GKE, the underlying instance or node must have the cloud-platform OAuth scope. If you receive an error with the following message, it means the instance or node was not provisioned with the correct OAuth scopes.
Request had insufficient authentication scopes
The required OAuth scope to use Secret Manager is:
https://www.googleapis.com/auth/cloud-platform
Example gcloud command to create dataproc with scope
gcloud dataproc clusters create xyz-pqr --region asia-south1 --subnet projects/xyz-pqr/regions/asia-south1/subnetworks/abc-serverless-vpc --zone asia-south1-b --master-machine-type n1-standard-4 --master-boot-disk-size 100 --num-workers 2 --worker-machine-type n1-standard-4 --worker-boot-disk-size 100 --image-version 2.0-debian10 --project xyz-development -scopes https://www.googleapis.com/auth/cloud-platform