I have following use case. (Using AWS CodeBuild Node.js SDK)
I need to use a private GitHub repository as a source for CodeBuild project.
CASE no.1
When I go to CodeBuild project and "Edit Source" in place, I paste a personal access token and everything works as expected.
CASE no.2
The problem arises, when I want to achieve the very same thing, using AWS CodeBuild Node.js SDK.
I'm using the very same existing CodeBuild project and I set my AWS.CodeBuild.start params as following
{
projectName: CODEBUILD_PROJECT,
sourceLocationOverride: "https://github.com/owner/name",
buildspecOverride: buildspecYML,
sourceTypeOverride: "GITHUB",
sourceAuthOverride: {
// Deprecated, but TS requires it
type: "OAUTH",
resource: TOKEN
}
}
I get response:
InvalidInputException: No Access token found, please visit AWS CodeBuild console to connect to GitHub
My question is. Is my use-case supported with current state of the SDK? If yes, how do I achieve it?
One thing I noticed while going through TS type definitions is the comment that only OAUTH type is currently supported. For this reason, I'm not sure if there even is a way to achieve my use-case.
export interface SourceAuth {
/**
* This data type is deprecated and is no longer accurate or used. The authorization type to use. The only valid value is OAUTH, which represents the OAuth authorization type.
*/
type: SourceAuthType;
/**
* The resource value that applies to the specified authorization type.
*/
resource?: String;
}
Some more details:
What I'm trying to achieve here is building a simple CI, that will be managed from a GitHub App, so the token I will be passing is the users Access Token.
With a Github source, you have two options for authType:
OAUTH
PERSONAL_ACCESS_TOKEN
If using OAUTH, the only option to provide the credential is through AWS CodeBuild console. Start creating a project, under source, select 'Connect using OAuth', the click 'Connect to Github' and complete the process. You can cancel out of the Project after connecting to Github.
Confirm the credential is registered by executing the command:
$ aws codebuild list-source-credentials --region us-east-1
Output is something like:
{
"sourceCredentialsInfos": [
{
"arn": "arn:aws:codebuild:us-east-1:<ACCOUNTNUM>:token/github",
"serverType": "GITHUB",
"authType": "OAUTH"
},
{
"arn": "arn:aws:codebuild:us-east-1:<ACCOUNTNUM>:token/github_enterprise",
"serverType": "GITHUB_ENTERPRISE",
"authType": "PERSONAL_ACCESS_TOKEN"
}
]
}
Related
I am trying to use google text-to-speech and other translation service in my nodejs but when i connect to google api I get this error message
"Your application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the texttospeech.googleapis.com. We recommend configuring the billing/quota_project setting in gcloud or using a service account through the auth/impersonate_service_account setting. For more information about service accounts and how to use them in your application, see https://cloud.google.com/docs/authentication/. If you are getting this error with curl or similar tools, you may need to specify 'X-Goog-User-Project' HTTP header for quota and billing purposes. For more information regarding 'X-Goog-User-Project' header, please check https://cloud.google.com/apis/docs/system-parameters.",
metadata: Metadata {
internalRepr: Map(2) {
'google.rpc.errorinfo-bin' => [Array],
'grpc-status-details-bin' => [Array]
},
options: {}
},
note: 'Exception occurred in retry method that was not classified as transient'
}
so after many research i tried to verify that i am authenticating using my service account credentials. I ran this command
gcloud auth activate-service-account --key-file=./auth/service_acct_key.json
and it shows this
Activated service account credentials for: [firebase-adminsdk-uwecx#xxxxx.iam.gserviceaccount.com]
but when run the server again
node server.js
I still got the error
what is causing this error and how can authenticate correctly ?
With gcloud CLI, you have 2 level of authentication:
The CLI level
The Google Cloud Auth library level (Also named ADC, for Application Default Credential)
When you perform the command gcloud auth .... you are at the CLI level
When you perform the command gcloud auth application-default ... you are at the ADC level.
In your case, you only set the authentication at the CLI level, and, of course, that authentication isn't detected in your NODE app, that use Google Cloud libraries and search credential at ADC level.
When you use service account key file (that is a bad practice, but too often prosed and shared in tutorial, even on Google Cloud tutorials (...)), you have to set an environment variable GOOGLE_APPLICATION_CREDENTIALS with the value equals to the absolute path of your service account key file. Try that
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/auth/service_acct_key.json
node server.js
It should work.
Goal
get and set IAM Policies for auto-provisioned GCP Projects and Service Accounts within said projects using the Node.js Client Library for Google APIs. As well as give a the service account in the project the Dialogflow API Admin role (roles/dialogflow.admin)
Issue
I get the following error when I try to get the IAM policy for a project I just automatically created.
Error: 7 PERMISSION_DENIED: Permission 'resourcemanager.projects.getIamPolicy' denied on resource '//cloudresourcemanager.googleapis.com/projects/va-31b899e6' (or it may not exist).
at Object.callErrorFromStatus (/home/aeglad22/va-project-provisioning/node_modules/#grpc/grpc-js/build/src/call.js:31:26)
at Object.onReceiveStatus (/home/aeglad22/va-project-provisioning/node_modules/#grpc/grpc-js/build/src/client.js:180:52)
at Object.onReceiveStatus (/home/aeglad22/va-project-provisioning/node_modules/#grpc/grpc-js/build/src/client-interceptors.js:365:141)
at Object.onReceiveStatus (/home/aeglad22/va-project-provisioning/node_modules/#grpc/grpc-js/build/src/client-interceptors.js:328:181)
at /home/aeglad22/va-project-provisioning/node_modules/#grpc/grpc-js/build/src/call-stream.js:182:78
at processTicksAndRejections (node:internal/process/task_queues:78:11) {
code: 7,
details: "Permission 'resourcemanager.projects.getIamPolicy' denied on resource '//cloudresourcemanager.googleapis.com/projects/va-31b899e6' (or it may not exist).",
metadata: Metadata {
internalRepr: Map(3) {
'grpc-server-stats-bin' => [Array],
'google.rpc.errorinfo-bin' => [Array],
'grpc-status-details-bin' => [Array]
},
options: {}
},
note: 'Exception occurred in retry method that was not classified as transient'
}
Here is the function I am trying to do this in.
async function setServiceAccountRolesV2(projectID, serviceAccountID){
const authClient = await auth.getClient();
const resourcemanagerClient = new ProjectsClient();
var request = {
resource: "projects/"+projectID,
auth: authClient
}
await resourcemanagerClient.getIamPolicy(request, function(err, response) {
if (err) {
console.error(err);
return;
}
console.log(JSON.stringify(response, null, 2));
});
}
Authentication Info
I am using a service account key to authenticate all of my functions in this node app with. This service account has the following permissions granted at the organization level
This service account I am using to authenticate my app with succeeds at getIamPolicy when I try to get the policy of the project it was created in itself. But I get the error when I try to get the policy of new projects I have created using this "admin project" service account.
Summary
Why is permissions denied when trying to get the IAM Policy of projects I have created programmatically, but successful when getting the policy of the "admin" project that I have this service account and the node.js app running. I thought that if I granted my service account proper permissions at the organization level, and the projects I am creating programmatically were in that same organization, my authenticating service account should inherit all of the right permissions to grant service account roles and change IAM policy in these newly generated accounts.
A potential thought/gut feeling I have that could be completely wrong - is it possible these new projects I'm making don't have IAM Policies at all? so when I try to get and set them there's nothing to change?
Update for clarifications
I have a project that acts as an "administration project" which contain hosts the VM my Node.js app for provisioning GCP resources runs on.
This project is also where I created my service account that the Node.js app authenticates with.
I am creating new projects and service accounts within those projects with this Node.js app.
I have given the aforementioned service account the Owner permission at the organization level.
In my setServiceAccountRolesV2() method, I have tried making the resource my provisioned project manually, as opposed to passed as a parameter to make sure the the project is located correctly. I manually copy and pasted the project ID from one of the auto-provisioned projects into the resource field like this for example
resource: "projects/va-31b899e6",
and I get the same permission denied error (full error message shown above).
However when I try to use this getIamPolicy method with the "admin" project that my node.js app and service account were created in, I get a successful policy return.
resource: "projects/provisioning-admin-339515"
I don't understand why one works, and one doesn't while the service account I'm using to make the call has Owner role at the organization level. The va-31b899e6 project shown above is in fact under the same organization my admin project is.
When I run the gcloud command gcloud projects get-iam-policy va-31b899e6 --format=json > ~/policy.json to check the policy of the api-generated project (not the admin project), I get the following policy back:
{
"bindings": [
{
"members": [
"serviceAccount:tf-admin-sa#provisioner-admin-339515.iam.gserviceaccount.com" ],
"role": "roles/owner"
}
],
"etag": "ByXXh29efSc=",
"version": 1
}
This service account listed in the members is the service account I authenticate my Node app with. Again, Owner granted at the Org level. This to me looks like it should be able to use the get and setIamPolicy methods on this project, as well as any other project in my organization.
New edits to follow trouble shooting tips from answer.
1
Confirmed I am using the correct project in the api call:
async function setServiceAccountRolesV2(projectID, serviceAccountID){
const authClient = await auth.getClient();
const resourcemanagerClient = new ProjectsClient();
var request = {
resource: "projects/va-31b899e6",
auth: authClient
};
await resourcemanagerClient.getIamPolicy(request, function(err, response) {
if (err) {
console.error(err);
return;
}
console.log(JSON.stringify(response, null, 2));
});
}
(project ID copied from GCP Console) : resource: "projects/va-31b899e6",
2
I have verified my credentials are used correctly, I am using a json key file of the service account I created to create more projects and service accounts programmatically. This is how I am authenticating :
const auth = new google.auth.GoogleAuth({
keyFile: 'provisioner-admin-339515-411d1e284a77.json',
scopes: ['https://www.googleapis.com/auth/cloud-platform'],
});
Then in my function, I create a new instance of auth like this:
const authClient = await auth.getClient();
which is then sent in the request of the api call: auth: authClient
3
Verified permissions for my authenticating service account:
When I run
gcloud projects get-iam-policy va-31b899e6 \
--flatten="bindings[].members" \
--format="table(bindings.role)" \
--filter="bindings.members:tf-admin-sa#provisioner-admin-339515.iam.gserviceaccount.com"
I get the output ROLE: roles/owner
Your service account has too many roles. Most of the roles are redundant and included within other roles that you assigned. For example, Billing Account Administrator contains the permissions of Billing Account User. The role Owner possesses almost all of the roles in your screenshot.
Next, you need to understand the Principle of Least Privilege. Seth Vargo put together a good intro video. In summary, only grant the required privileges and no more. Your service account IAM roles are vast and a serious security weakness.
To solve the problem in your question, follow these steps:
STEP 1:
Confirm that the Project ID is correct in the API call. Make sure you are using the Project ID and not the Project Name. List the projects:
gcloud projects list
STEP 2:
Verify that your code is using the correct credentials (the ones you think you configured). Your question does not show how you are authorizing your code. You are using ADC (Application Default Credentials) which means the credentials could be found from several sources (CLI remembered credentials, the environment variable, metadata server).
If you are using the environment variable GOOGLE_APPLICATION_CREDENTIALS, open the file using the variable and make sure that it is a service account JSON key:
vi $GOOGLE_APPLICATION_CREDENTIALS
If you are using the CLI credentials, verify which identity is being used:
gcloud auth list
As a debugging test, clear the environment variable and use a user identity that has the role Owner and then login. Then retest your application.
unset GOOGLE_APPLICATION_CREDENTIALS
gcloud auth application-default login
STEP 3:
Once you have determined the correct Project ID and which service account your code is using, double-check the roles assigned to the service account at the project level. List the IAM roles with this command. Replace with your Project ID and Service Account Email address:
gcloud projects get-iam-policy <PROJECT_ID> \
--flatten="bindings[].members" \
--format="table(bindings.role)" \
--filter="bindings.members:<SERVICE_ACCOUNT_EMAIL>"
The service account needs one of these roles or similar to view IAM bindings:
roles/browser aka Browser
roles/iam.roleViewer aka Viewer
The service account needs this role or similar to modify IAM bindings:
roles/resourcemanager.projectIamAdmin aka Project IAM Admin
Manage access to projects, folders, and organizations
I am attempting to create a service fabric connection via the azure devops rest API, as documented here:
https://learn.microsoft.com/en-us/rest/api/azure/devops/serviceendpoint/endpoints/create?view=azure-devops-rest-6.0
When attempting to define the authorization parameters, I have the following fields in my request:
"authorization": {
"parameters":{
"certlookup":"Thumbprint",
"servercertthumbprint": "{{certificateThumbprint}}",
"certificate":"{{certname}}",
"certificatepassword":null,
},
"scheme":"certificate"
},
This will create a service connection to the cluster, however it does not look like the 'client certificate' parameter (as shown in screenshot bellow) is parsed in anyhwhere. I also cant seem to find anywhere in the documentation that says how to do this.
How can I pass in the "Client Certificate" Value when using the REST API?
Below is the detailed view of the parameters which you can use for creating Service Fabric Service Connection:
Pls try to make your request like below (un tested) to make it work:
"authorization": {
"parameters":{
"certlookup":"Thumbprint",
"servercertthumbprint": "{{certificateThumbprint}}",
"clientcertificatedata":"{{certname}}",
"password":null,
},
"scheme":"certificate"
},
Please refer to EndpointAuthorizationParameters Class for details about the parameters to use.
The correct parameter to specify this value is actually just "certificate".
The original request posted is correct, I was entering the name of the certificate rather than the encoding of it.
I am trying to use MSAL to add SSO for my single page app, for the purpose of getting an Oauth2 token to be used for resolving a Microsoft SaaS subscription. I have configured my code based on this sample repo: https://github.com/Azure-Samples/ms-identity-javascript-v2. This uses MSAL 2.0, instead of 1.X versions.
Here is the relevant code snippet, ran on the /signup page:
var config = {
auth: {
clientId: <MY_CLIENT_ID>,
authority: "https://login.microsoftonline.com/common",
redirectUri: <APP_URL> + "/signup",
},
cache: {
cacheLocation: "sessionStorage",
storeAuthStateInCookie: false,
}
};
var msalInstance = new msal.PublicClientApplication(config);
msalInstance.loginPopup(["openid", "profile", "User.Read"]).then((response) => {
console.log(response)
}).catch(error => {
console.error(error);
});
Calling this code inside my application, I successfully log in using the popup window, but get the following error: ServerError: invalid_client: 70002 - [2020-09-03 16:55:35Z]: AADSTS70002: The provided request must include a 'client_secret' input parameter.
I can see the underlying network call is https://login.microsoftonline.com/common/oauth2/v2.0/token with the following form data:
client_id: <MY_CLIENT_ID>
redirect_uri: <APP_URL>/signup
scope: openid profile
code: <some value>
code_verifier: <some value>
grant_type: authorization_code
client_info: 1
client-request-id: <some value>
Based on the documentation here https://learn.microsoft.com/en-us/advertising/guides/authentication-oauth-identity-platform?view=bingads-13#request-accesstoken, this call should have a configurable client_secret parameter, which results in the error I am getting I believe.
I have the required client secret from my App on Azure Portal. My question is, how can I configure the client secret in my config object? I have not been able to find any examples online, as well I tried adding clientSecret as a key under auth inside my config blindly, this did not pass it to the login.microsoftonline.com call.
UPDATE
After a bit more digging around, I realized the underlying issue of my setup was that I was trying to use the authorization grant flow to request a token but needed to use the client credentials flow. After moving the token acquisition to the backend (and using the client secret there), and some more help in this followup thread I was able to get SSO and token acquisition working as expected.
Instead of creating Web create Single-page-application this will solve you problem. See the example below
We have debugged your sample and we are able to use it with out any issue. After testing we believe the issue is occurring sine your code is not able to call Signin() method in '/authPopup.js'.You will not face any issue if this method call happens in your index.html.
Instead of using MSAL directly, consider looking into using the new SSO capabilities inside Teams - it wraps a lot of this for you and makes it easier to use. Here's an overview and here is a working sample.
Regarding the sample, it's in node, and I'm working on a dotnet version of the same. It looks like you use node, so you should rather refer to the original, but I think I found a few bugs so you can look at the client-side script in my version if you run into any issues - see here.
I have an HTTP-triggered function running on Google Cloud Functions, which uses require('googleapis').sheets('v4') to write data into a docs spreadsheet.
For local development I added an account via the Service Accounts section of their developer console. I downloaded the token file (dev-key.json below) and used it to authenticate my requests to the Sheets API as follows:
var API_ACCT = require("./dev-key.json");
let apiClient = new google.auth.JWT(
API_ACCT.client_email, null, API_ACCT.private_key,
['https://www.googleapis.com/auth/spreadsheets']
);
exports.myFunc = function (req, res) {
var newRows = extract_rows_from_my_client_app_request(req);
sheets.spreadsheets.values.append({
auth: apiClient,
// ...
resource: { values:newRows }
}, function (e) {
if (e) res.status(500).json({err:"Sheets API is unhappy"});
else res.status(201).json({ok:true})
});
};
After I shared my spreadsheet with my service account's "email address" e.g. local-devserver#foobar-bazbuzz-123456.iam.gserviceaccount.com — it worked!
However, as I go to deploy this to the Google Cloud Functions service, I'm wondering if there's a better way to handle credentials? Can my code authenticate itself automatically without needing to bundle a JWT key file with the deployment?
I noticed that there is a FUNCTION_IDENTITY=foobar-bazbuzz-123456#appspot.gserviceaccount.com environment variable set when my function runs, but I do not know how to use this in the auth value to my googleapis call. The code for google.auth.getApplicationDefault does not use that.
Is it considered okay practice to upload a private JWT token along with my GCF code? Or should I somehow be using the metadata server for that? Or is there a built-in way that Cloud Functions already can authenticate themselves to other Google APIs?
It's common to bundle credentials with a function deployment. Just don't check them into your source control. Cloud Functions for Firebase samples do this where needed. For example, creating a signed URL from Cloud Storage requires admin credentials, and this sample illustrates saving that credential to a file to be deployed with the functions.
I'm wondering if there's a better way to handle credentials? Can my
code authenticate itself automatically without needing to bundle a JWT
key file with the deployment?
Yes. You can use 'Application Default Credentials', instead of how you've done it, but you don't use the function getApplicationDefault() as it has been deprecated since this Q was posted.
The link above shows how to make a simple call using the google.auth.getClient API, providing the desired scope, and have it decide the credential type needed automatically. On cloud functions this will be a 'Compute' object, as defined in the google-auth-library.
These docs say it well here...
After you set up a service account, ADC can implicitly find your
credentials without any need to change your code, as described in the
section above.
Where ADC is Application Default Credentials.
Note that, for Cloud Functions, you use the App Engine service account:
YOUR_PROJECT_ID#appspot.gserviceaccount.com, as documented here. That is the one you found via the FUNCTION_IDENTITY env var - this rather tripped me up.
The final step is to make sure that the service account has the required access as you did with your spreadsheet.