I'm using MongoDB Atlas to host my MongoDB database and I want to use the MONGODB-AWS authentication mechanism for authentication. When I'm trying it locally with my personal IAM user it works as it should, however when it runs in production I get the error MongoError: bad auth : aws sts call has response 403. I run my Node.js application inside an AWS EKS cluster and I have added the NodeInstanceRole used in EKS to MonogDB Atlas. I use fromNodeProviderChain() from AWS SDK v3 to get my secret access key and access key id and have verified that I indeed get credentials.
Code to get the MongoDB URI:
import { fromNodeProviderChain } from '#aws-sdk/credential-providers'
async function getMongoUri(config){
const provider = fromNodeProviderChain()
const awsCredentials = await provider()
const accessKeyId = encodeURIComponent(awsCredentials.accessKeyId)
const secretAccessKey = encodeURIComponent(awsCredentials.secretAccessKey)
const clusterUrl = config.MONGODB_CLUSTER_URL
return `mongodb+srv://${accessKeyId}:${secretAccessKey}#${clusterUrl}/authSource=%24external&authMechanism=MONGODB-AWS`
}
Do I have to add some STS permissions for the node instance role or are the credentials I get from fromNodeProviderChain() not the same as the node instance role?
Related
I want to trigger a GCP cloud function from a simple nodejs app running locally.
Reading the documentation it should be simple:
run gcloud auth application-default login to write ADC to file used by client libraries.
use google-auth-library to get a http client to use to trigger the function.
/**
* TODO(developer): Uncomment these variables before running the sample.
*/
// Example: https://my-cloud-run-service.run.app/books/delete/12345
// const url = 'https://TARGET_HOSTNAME/TARGET_URL';
// Example (Cloud Functions): https://project-region-projectid.cloudfunctions.net/myFunction
const targetAudience = 'https://<REGION>-<PROJECTID>.cloudfunctions.net/<FUNCTIONNAME>';
const { GoogleAuth } = require('google-auth-library');
const auth = new GoogleAuth();
const payload = {"prop1": "prop1Value"};
async function request() {
const client = await auth.getIdTokenClient(targetAudience);
const resp = await client.request({ url: targetAudience, method: 'POST', data: payload });
console.info(`Resp status: ${resp.status}; resp.data: ${resp.data}`);
}
(async () => {
await request();
})();
My understanding was that the google-auth-library would pick up the ADC from the file setup from running gcloud auth application-default login and everything would work.
My user has permission to invoke GCP functions as I can trigger the function using CURL with the header -H "Authorization:bearer $(gcloud auth print-identity-token)" \
However when I run this, it doesn't get past the line:
const client = await auth.getIdTokenClient(targetAudience);
Failing with:
Cannot fetch ID token in this environment, use GCE or set the GOOGLE_APPLICATION_CREDENTIALS environment variable t
o a service account credentials JSON file.
Using PubSub library works fine so expect ADC does work just not sure what am I missing when trying to trigger the GCP function.
Am I using the google-auth-library correctly here ?
Thanks
As mentioned in the thread:
gcloud auth activate-service-account --key-file is only for "you"
running gcloud commands, it won’t be picked up by "applications" that
need GOOGLE_APPLICATION_CREDENTIALS. As you can see from Invoke a
Google Cloud Run from java or How to call Cloud Run from outside
of Cloud Run/GCP?, you either need to have the JSON key file of
Service Account, or have to be running inside a GCE/GKE/Cloud Run/App
Engine/GCF instance.
For this to work on your local environment, I recommend logging in
with gcloud auth application-default login command (this command is
meant to work as if you’ve set GOOGLE_APPLICATION_CREDENTIALS
locally).
If that doesn't work, as a last resort you can refactor your code to
pick up identity token from an environment variable (if set) while
working locally,
such as: $ export ID_TOKEN="$(gcloud auth print-identity-token -q)" $ ./your-app
To know more about how the code does it with a JSON key file,refer to the link and similar implementation there.
For more information you can refer to a similar thread stated as :
Give the default service account access rights to Workspace
resource(s) you're attempting to access.
Use the JSON key file you set
up locally already, to have the Cloud Function run as the same user as
is happening when you run locally.
Essentially do a hybrid where you create a new service account that ONLY has the permissions you want (instead of using the default
service account or your personal user, both of which might have far
more permissions then desired for safety/security), use a key file to
run the Cloud Function under that identity, and only give the desired
permissions to that service account.
I'm having trouble on authenticating as a service account in my Next.js app hosted on Vercel. My code is working fine in my dev environment, but it fails with the following error message when I try to run it on Vercel Node.js v14 environment. I guess that it runs on my local machine because I'm logged in gcloud with my email (project owner).
This is the error I'm getting:
Error: Could not load the default credentials. Browse to https://cloud.google.com/docs/authentication/getting-started for more information.
at GoogleAuth.getApplicationDefaultAsync (/var/task/node_modules/google-auth-library/build/src/auth/googleauth.js:173:19)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
at async GoogleAuth.getClient (/var/task/node_modules/google-auth-library/build/src/auth/googleauth.js:551:17)
at async GrpcClient._getCredentials (/var/task/node_modules/google-gax/build/src/grpc.js:109:24)
at async GrpcClient.createStub (/var/task/node_modules/google-gax/build/src/grpc.js:252:23)
I've created the following service account to use it with my Next.js APIs.
It has all the necessary roles. I've created a JSON key and download it.
I'm using firebase-admin and this is how I'm initializing it:
export const initializeFirebaseAdmin = (): FirebaseAdmin => {
const account = getServiceAccount(); // THIS IS THE SERVICE ACCOUNT JSON KEY (ALREADY PARSED AS AN OBJECT)
if (!admin.apps.length) {
admin.initializeApp({
credential: admin.credential.cert(account),
});
}
return admin;
};
This is what I think it's happening:
From: https://cloud.google.com/docs/authentication/production#automatically
From the image above:
I'm not setting any GOOGLE_APPLICATION_CREDENTIALS environment variable
So I should be on number 2. It will "try to use the service account that is attached to the resource that is running your code."
It's obviously failing and I getting the error
But which resource is it referring to? Which code?
The resource that is running my code is Firebase Admin? Am I not initializing it correctly?
This code should work to authenticate the firebase-admin package:
export const initializeFirebaseAdmin = (): FirebaseAdmin => {
const account = getServiceAccount(); // THIS IS THE SERVICE ACCOUNT JSON KEY (ALREADY PARSED AS AN OBJECT)
if (!admin.apps.length) {
admin.initializeApp({
credential: admin.credential.cert(account),
});
}
return admin;
};
My problem was related to the fact that I was using a client exposed from the firebase-admin package, that doesn't have access to the authentication from the main package. So I needed to pass the credentials to it as well. Like:
const client = new admin.firestore.v1.FirestoreAdminClient({
credentials: SERVICE_ACCOUNT as CredentialBody // <<<<<< THIS IS THE SERVICE ACCOUNT JSON KEY
});
Refer also to: Trying to export backup via admin.firestore.v1.FirestoreAdminClient on a Next.js API route. Error: Could not load the default credentials
I'm trying to figure out how to get postman to work with EKS. I have a simple nodejs app.
const express = require('express');
const app = express();
app.get('/', (req, res) => res.send('hello world'));
app.listen(3000, () => {
console.log('My REST API running on port 3000!');
});
Here's everything I've done so far:
I created a docker container and successfully pushed it to ECR.
Also I tested docker by running it locally and I was able to reach it and get hello world response so the docker container seems fine.
I created an EKS cluster with the docker container and have the api server endpoint
but when I try and make a call with postman, I get
I even tried adding access key and secret from IAM user that has access to EKS, but I get same error.
When I configured the cluster, I set it to public so I don't understand why Postman can't reach the API endpoint.
Also I added the following permissions to the IAM user I'm using in postman. I wasn't sure which one was correct so I added all of them. I also put the security credentials for that IAM user in postman.
What am I missing? I appreciate the help!
Actually, your Postman is reaching AWS EKS API endpoint, but you are getting authentication/authorization error - 403 Forbidden. I see OpenID Connect provider URL in the API config, so I would expect OIDC authentication and not AccessKey/SecretKey. Check AWS EKS documentation or contact your AWS support.
I have a node js application running on EC2. After a certain operation, I want to stop the EC2.
I am using this function to stop EC2
const stopInstance = () => {
// set the region
AWS.config.update({
accessKeyId: "MY ACCESS KEY",
secretAccesskey: "SECRET KEY",
region: "us-east-1"
})
// create an ec2 object
const ec2 = new AWS.EC2();
// setup instance params
const params = {
InstanceIds: [
'i-XXXXXXXX'
]
};
ec2.stopInstances(params, function(err, data) {
if (err) {
console.log(err, err.stack); // an error occurred
} else {
console.log(data); // successful response
}
});
}
When I am running it from EC2, it's giving error
UnauthorizedOperation: You are not authorized to perform this operation.
But when I am running the same code, using the same key and secret from my local machine, It's working perfectly.
Permissions I have
This will be down to the permissions of the IAM user being passed into the script.
Firstly this error message indicates that an IAM user/role was successfully used in the request, but failed to have permissions so that can be ruled out.
Assuming a key and secret are being successfully (looks like hard coded) you would be looking at further restrictions within the policy (such as principal or .
If the key and secret are not hard coded but instead passed in as environment variables, perform some debug to output the string values and validate these are what you expect. If they do not get passed into the SDK then it may be falling back to an instance role that is attached.
As a point of improvement, generally when interacting with the AWS SDK/CLI from within AWS (i.e. on an EC2 instance) you should use a IAM role over an IAM user as this will lead to less API credentials being managed/rotated. An IAM role will rotate temporary credentials for you every few hours.
If the same credentials are working on local machine then it's probably not a permission issue, but just to further isolate the issue, you can try to run the AWS-GetCallerIdentity to check the credentials that are being used.
https://docs.aws.amazon.com/cli/latest/reference/sts/get-caller-identity.html
In case if this does not help, create a new user and try giving full admin access and then using the credentials to see if this get's resolved. This will confirm whether we are facing a permission issue or not.
Using the Node sdk for AWS, I'm trying to use the credentials and permissions given by the IAM role that is attached to the EC2 instance that my Node application is running on.
According to the sdk documentation, that can be done using the EC2MetadataCredentials class to assign the configuration properties for the sdk.
In the file that I'm using the sdk in to access a DynamoDB instance, I have the configuration code:
import AWS from 'aws-sdk'
AWS.config.region = 'us-east-1'
AWS.config.credentials = new AWS.EC2MetadataCredentials({
httpOptions: { timeout: 5000 },
maxRetries: 10,
retryDelayOptions: { base: 200 }
})
const dynamodb = new AWS.DynamoDB({
endpoint: 'https://dynamodb.us-east-1.amazonaws.com',
apiVersion: '2012-08-10'
})
However, when I trying to visit the web application I always get an error saying:
Uncaught TypeError: d.default.EC2MetadataCredentials is not a constructor
Uncaught TypeError: _awsSdk2.default.EC2MetadataCredentials is not a constructor
Even though that is the exact usage from the documentation! Is there something small that I'm missing?
Update:
Removing the credentials and region definitions from the file result in another error that'll say:
Error: Missing region|credentials in config
I don't know if this is still relevant for you, but you do need to configure the EC2MetadataCredentials as it is not in the default ProviderChain ( search for new AWS.CredentialProviderChain([ in node_loader.js in the sdk).
It seems you might have an old version of aws_sdk as that code works for me:
import AWS from 'aws-sdk';
...
AWS.config.credentials = new AWS.EC2MetadataCredentials();
I was facing a similar issue where AWS SDK was not fetching credentials. According to the documentation, SDK should be able to automatically fetch the credentials.
If you configure your instance to use IAM roles, the SDK automatically selects the IAM credentials for your application, eliminating the need to manually provide credentials.
I was able to solve the issue by manually fetching the credentials, and providing them directly wherever required (For MongoDB Atlas in my case):
var AWS = require("aws-sdk");
AWS.config.getCredentials(function(err) {
if (err) console.log(err.stack);
// credentials not loaded
else {
console.log("Access key:", AWS.config.credentials.accessKeyId);
}
});
Source: https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/global-config-object.html
Although, why SDK is not not doing it automatically is still mystery to me. I will update the answer once I figure it out.