What and how to pass credential using using Python Client Library for gcp compute API - python-3.x

I want to get list of all instances in a project using python google client api google-api-python-client==1.7.11
Am trying to connect using method googleapiclient.discovery.build this method required credentials as argument
I read documentation but did not get crdential format and which credential it requires
Can anyone explain what credentials and how to pass to make gcp connection

The credentials that you need are called "Service Account JSON Key File". These are created in the Google Cloud Console under IAM & Admin / Service Accounts. Create a service account and download the key file. In the example below this is service-account.json.
Example code that uses a service account:
from googleapiclient import discovery
from google.oauth2 import service_account
scopes = ['https://www.googleapis.com/auth/cloud-platform']
sa_file = 'service-account.json'
zone = 'us-central1-a'
project_id = 'my_project_id' # Project ID, not Project Name
credentials = service_account.Credentials.from_service_account_file(sa_file, scopes=scopes)
# Create the Cloud Compute Engine service object
service = discovery.build('compute', 'v1', credentials=credentials)
request = service.instances().list(project=project_id, zone=zone)
while request is not None:
response = request.execute()
for instance in response['items']:
# TODO: Change code below to process each `instance` resource:
print(instance)
request = service.instances().list_next(previous_request=request, previous_response=response)

Application default credentials are provided in Google API client libraries automatically. There you can find example using python, also check this documentation Setting Up Authentication for Server to Server Production Applications.

According to GCP most recent documentation:
we recommend you use Google Cloud Client Libraries for your
application. Google Cloud Client Libraries use a library called
Application Default Credentials (ADC) to automatically find your
service account credentials
In case you still want to set it manaully, you could, first create a service account and give all necessary permissions:
# A name for the service account you are about to create:
export SERVICE_ACCOUNT_NAME=your-service-account-name
# Create service account:
gcloud iam service-accounts create ${SERVICE_ACCOUNT_NAME} --display-name="Service Account for ai-platform-samples repo"
# Grant the required roles:
gcloud projects add-iam-policy-binding ${PROJECT_ID} --member serviceAccount:${SERVICE_ACCOUNT_NAME}#${PROJECT_ID}.iam.gserviceaccount.com --role roles/ml.developer
gcloud projects add-iam-policy-binding ${PROJECT_ID} --member serviceAccount:${SERVICE_ACCOUNT_NAME}#${PROJECT_ID}.iam.gserviceaccount.com --role roles/storage.objectAdmin
# Download the service account key and store it in a file specified by GOOGLE_APPLICATION_CREDENTIALS:
gcloud iam service-accounts keys create ${GOOGLE_APPLICATION_CREDENTIALS} --iam-account ${SERVICE_ACCOUNT_NAME}#${PROJECT_ID}.iam.gserviceaccount.com
Once it's done check whether the ADC path has been set properly by checking:
echo $GOOGLE_APPLICATION_CREDENTIALS
Having set the ADC path, you don't need to import from code the service access key, which undesirable, so the code looks as follows:
service = googleapiclient.discovery.build(<API>, <version>,cache_discovery=False)

Related

how to authenticate to google text-to-speech with service account

I am trying to use google text-to-speech and other translation service in my nodejs but when i connect to google api I get this error message
"Your application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the texttospeech.googleapis.com. We recommend configuring the billing/quota_project setting in gcloud or using a service account through the auth/impersonate_service_account setting. For more information about service accounts and how to use them in your application, see https://cloud.google.com/docs/authentication/. If you are getting this error with curl or similar tools, you may need to specify 'X-Goog-User-Project' HTTP header for quota and billing purposes. For more information regarding 'X-Goog-User-Project' header, please check https://cloud.google.com/apis/docs/system-parameters.",
metadata: Metadata {
internalRepr: Map(2) {
'google.rpc.errorinfo-bin' => [Array],
'grpc-status-details-bin' => [Array]
},
options: {}
},
note: 'Exception occurred in retry method that was not classified as transient'
}
so after many research i tried to verify that i am authenticating using my service account credentials. I ran this command
gcloud auth activate-service-account --key-file=./auth/service_acct_key.json
and it shows this
Activated service account credentials for: [firebase-adminsdk-uwecx#xxxxx.iam.gserviceaccount.com]
but when run the server again
node server.js
I still got the error
what is causing this error and how can authenticate correctly ?
With gcloud CLI, you have 2 level of authentication:
The CLI level
The Google Cloud Auth library level (Also named ADC, for Application Default Credential)
When you perform the command gcloud auth .... you are at the CLI level
When you perform the command gcloud auth application-default ... you are at the ADC level.
In your case, you only set the authentication at the CLI level, and, of course, that authentication isn't detected in your NODE app, that use Google Cloud libraries and search credential at ADC level.
When you use service account key file (that is a bad practice, but too often prosed and shared in tutorial, even on Google Cloud tutorials (...)), you have to set an environment variable GOOGLE_APPLICATION_CREDENTIALS with the value equals to the absolute path of your service account key file. Try that
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/auth/service_acct_key.json
node server.js
It should work.

How to authenticate with tokens in Nodejs to a private bucket in Cloud Storage

Usually in Python what I do, I get the application default credentials, I get the access token then I refresh it to be able to authenticate to a private environment.
Code in Python:
# getting the credentials and project details for gcp project
credentials, your_project_id = google.auth.default(scopes=["https://www.googleapis.com/auth/cloud-platform"])
#getting request object
auth_req = google.auth.transport.requests.Request();
print(f"Checking Authentication : {credentials.valid}")
print('Refreshing token ....')
credentials.refresh(auth_req)
#check for valid credentials
print(f"Checking Authentication : {credentials.valid}")
access_token = credentials.token
credentials = google.oauth2.credentials.Credentials(access_token);
storage_client = storage.Client(project='itg-ri-consumerloop-gbl-ww-dv',credentials=credentials)
I am entirely new to NodeJS, and I am trying to make the same thing.
My goal later is to create an app engine application that would expose an image that is found in a private bucket, so credentials are a must.
How it is done?
For authentication, you could rely on the default application credentials that are present within the GCP platform (GAE, Cloud Functions, VM, etc.). Then you could just run the following piece of code from the documentation:
const {Storage} = require('#google-cloud/storage');
const storage = new Storage();
const bucket = storage.bucket('albums');
const file = bucket.file('my-existing-file.png');
In most circumstances, there is no need to explicitly use authentication packages since they are already executed underneath the google-cloud/storage package in Nodejs. The same holds for the google-cloud-storage package in Python. It could help to look at the source code of both packages on Github. For me, this really helped to understand the authentication mechanism.
When I develop code on my own laptop, that interacts with google cloud storage, I first tell the gcloud SDK what my credentials are and on which GCP project I am working. I use the following commands for this:
gcloud config set project [PROJECT_ID]
gcloud auth application-default login
You could also set DEFAULT_APPLICATION_CREDENTIALS as an environment variable that points to a credentials file. Then within your code, you could pass the project name when initializing the client. This could be helpful if you are running your code outside of GCP on another server for example.

Azure Authentication and Authorization using java

How to authenticate azure using java with azure management or client libraries without directly using azure rest API's?
and what are the jars required for this?
Please help with samples.
If you want to use JAVA with Azure management for authentication, you can use the following two methods:
1.Create an instance of ApplicationTokenCredentials to supply the service principal credentials to the top-level Azure object from inside your code:
import com.microsoft.azure.credentials.ApplicationTokenCredentials;
import com.microsoft.azure.AzureEnvironment;
// ...
ApplicationTokenCredentials credentials = new ApplicationTokenCredentials(client,
tenant,
key,
AzureEnvironment.AZURE);
Azure azure = Azure
.configure()
.withLogLevel(LogLevel.NONE)
.authenticate(credentials)
.withDefaultSubscription();
2.File based authentication:
# sample management library properties file
subscription=########-####-####-####-############
client=########-####-####-####-############
key=XXXXXXXXXXXXXXXX
tenant=########-####-####-####-############
managementURI=https\://management.core.windows.net/
baseURL=https\://management.azure.com/
authURL=https\://login.windows.net/
graphURL=https\://graph.windows.net/
please check:here

How to keep google-cloud-auth.json securely in app.yaml as an environmental variable?

I'm new to deployment/securing keys, and I'm not sure how to securely store the google-cloud-auth.json (auth required for creating the API client) outside of source code to prevent leaking credentials.
I've currently secured my API keys and tokens in my app.yaml file specifying them as environmental variables which successfully work as expected and shown below.
accessruntime: nodejs10
env_variables:
SECRET_TOKEN: "example"
SECRET_TOKEN2: "example2"
However my google-cloud-auth.json is kept as its own file since the parameter used for creating the client requires a path string.
const {BigQuery} = require('#google-cloud/bigquery');
...
const file = "./google-cloud-auth.json";
// Creates a BigQuery client
const bigquery = new BigQuery({
projectId: projectId,
datasetId: datasetId,
tableId: tableId,
keyFilename: file
});
According to the Setting Up Authentication for Server to Server Production Applications:
GCP client libraries will make use of the ADC (Application Default Credentials) to find the credentials meant to be used by the app.
What ADC does is basically to check if the GOOGLE_APPLICATION_CREDENTIALS env variable is set with the path to a service account file.
In case the env variable is not set, ADC will use the default service account provided by App Engine.
With this information I can suggest a couple of solutions to provide these credentials safely:
If you require to use a specific service account, set the path to the file with the GOOGLE_APPLICATION_CREDENTIALS. This section explains how to do that.
If you are not a fan of moving credential files around, I would suggest trying to use the default service account provided by the App Engine.
I just created a new project and deployed a basic app by mixing these 2 guides:
BigQuery Client Libraries
Quickstart for Node.js in the App Engine Standard Environment
My app.yaml had nothing more than the runtime: nodejs10 line, and I was still able to query through the BigQuery client library, using the default service account.
This account comes with the Project/Editor role and you can add any additional roles you need.

Azure AD add keys via Azure CLI

I'm trying to add a key in my Azure AD application using Azure CLI.
But looking throught the Azure CLI API it seems that there is no such command.
For exmaple:
I'm trying to automate the task from the link below via Azure CLI:
http://blog.davidebbo.com/2014/12/azure-service-principal.html
I can create AD application, service principal, but I can't find a way to add key for newly create AD application.
I'll appreciate any ideas and directions :)
Thanks in advance !
For a new AD application, you can specify a key with -p while creating. For example,
azure ad app create -n <your application name> --home-page <the homepage of you application> -i <the identifier URI of you application> -p <your key>
For an existing AD application, surely the Graph API is able to update the AD Application Credential. Read this API reference, and you can see that the password credential is able to use "POST, GET, PATCH". However, it's too complicated to use the Graph API. I have check the Azure CLI. That functionality is not yet implemented, and the source is unreadable for me. Then, I took a look at Azure SDK for Python, because I am familiar with python, and I found out that they have already implemented it in 2.0.0rc2. See the GitHub Repo
I have written a python script. But, in order to use my script you need to install not only azure2.0.0rc2, but also msrest and msrestazure.
from azure.common.credentials import UserPassCredentials
from azure.graphrbac import GraphRbacManagementClient, GraphRbacManagementClientConfiguration
from azure.graphrbac.models import ApplicationCreateParameters, PasswordCredential
credentials = UserPassCredentials("<your Azure Account>", "<your password>")
subscription_id = "<your subscription id>"
tenant_id = "<your tenant id>"
graphrbac_client = GraphRbacManagementClient(
GraphRbacManagementClientConfiguration(
credentials,
subscription_id,
tenant_id
)
)
application = graphrbac_client.application.get('<your application object id>')
passwordCredential = PasswordCredential(start_date="2016-04-13T06:08:04.0863895Z",
end_date="2018-04-13T06:08:04.0863895Z",
value="<your new key>")
parameters = ApplicationCreateParameters(application.available_to_other_tenants,
application.display_name,
"<the homepage of your AD application>",
application.identifier_uris,
reply_urls=application.reply_urls,
password_credentials = [passwordCredential])
application = graphrbac_client.application.update('<your application object id>', parameters)
The only problem with this script is that you are only able to override all the existing keys of you AD application. You are not able to append a new key. This is a problem of the Graph API. The Graph API does not allow users to read an existing key. One possible solution would be storing your existing keys somewhere else. But, this will bring extra security risk.
I don't have any experience of automating adding the key, I'm not sure it's even possible to be honest. However have a look at the ApplicationEntity documentation in the Graph API, it might be possible using a POST request to the web service.

Resources