ResourceExhausted in Google Speech API - speech-to-text

this is my first post on StackOverflow. I get ResourceExhausted with Google Speech API - LongRunningRecognizeResponse. It says "e.g. check quota", but I'm not sure what it means exactly. How do I check quota and make necessary setups?
func sendRequestWithUri(uri string)
(*speechpb.LongRunningRecognizeResponse, error) {
log.Printf("Uri: %s", uri)
ctx := context.Background()
client, err := speech.NewClient(ctx)
if err != nil {
log.Fatalf("Failed to create client: %v", err)
}
// Create Request with config
req := &speechpb.LongRunningRecognizeRequest{
Config: &speechpb.RecognitionConfig{
Encoding: speechpb.RecognitionConfig_FLAC,
SampleRateHertz: 44100,
LanguageCode: "en-US",
},
Audio: &speechpb.RecognitionAudio{
AudioSource: &speechpb.RecognitionAudio_Uri{
Uri: uri,
},
},
}
log.Printf("Created request: %s", req)
op, err := client.LongRunningRecognize(ctx, req)
if err != nil {
return nil, err
}
return op.Wait(ctx)
}
The code takes the URL for a FLAC file uploaded on Google Cloud Storage. After attempts to try with files in different lengths, I always receive:
rpc error: code = ResourceExhausted desc = Resource has been exhausted (e.g. check quota).
exit status 1
Then I opened GCP console, and simply enabled Speech API on one of the projects, where I want to work with Speech API. On a local machine (macOS, bash), I have selected the target project via gcloud gcloud config configurations create myproject and gcloud auth login. It looks like project config is successfully created:
$ gcloud config list
[core]
account = mygmail
disable_usage_reporting = False
project = myproject
Your active configuration is: [myproject]
After enable and setup, I still get the same ResourceExhausted.
Do I miss anything? Thanks.

Related

How to specify x509 certificate for Azure SDK in Golang

I am trying to connect to use the Azure SDK for Golang to download files from a container online to my device and am using the connection string provided from azure to connect. For context this is running on a version of embedded Linux
I have two questions, first how do I pass a specific certificate to the azure SDK to use to connect, as currently when I connect, I get this issue
Get "https://transaction.blob.core.windows.net/transactions?comp=list&restype=container": x509: certificate signed by unknown authority
or failing that how do I generate the correct certificate to put it in /etc/ssl? Which I think is where go is looking for certificates as far as I understand.
Also second question what function from the azure sdk for go should I be using to download from a blob online if my folder structure looks like /transaction/my-libs/images/1.0.0/libimage.bin where transaction is my blob container.
func testConnection(){
Println("TESTING CONNECTION")
connStr := "..." // actual connection string hidden
serviceClient, err := azblob.NewServiceClientFromConnectionString(connStr, nil)
// crashes here <------------
//ctx := context.Background()
//container := serviceClient.NewContainerClient("transactions")
//
//_, err = container.Create(ctx, nil)
//
//blockBlob := container.NewBlockBlobClient("erebor-libraries")
//_, err = blockBlob.Download(ctx, nil)
//Open a buffer, reader, and then download!
downloadedData := &bytes.Buffer{}
reader := get.Body(RetryReaderOptions{}) // RetryReaderOptions has a lot of in-depth tuning abilities, but for the sake of simplicity, we'll omit those here.
_, err = downloadedData.ReadFrom(reader)
err = reader.Close()
if data != downloadedData.String() {
err := errors.New("downloaded data doesn't match uploaded data")
if err != nil {
return
}
}
pager := container.ListBlobsFlat(nil)
for pager.NextPage(ctx) {
resp := pager.PageResponse()
for _, v := range resp.ContainerListBlobFlatSegmentResult.Segment.BlobItems {
fmt.Println(*v.Name)
}
}
• You can use the following Azure SDK for Go command for passing a specific certificate to the Azure SDK to connect to other Azure resources by creating a service principal for it: -
‘ type ClientCertificateConfig struct {
ClientID string
CertificatePath string
CertificatePassword string
TenantID string
AuxTenants []string
AADEndpoint string
Resource string
} ‘
For more information on the creation of the client certificate and its usage, please refer to the documentation link below for more details: -
https://pkg.go.dev/github.com/Azure/go-autorest/autorest/azure/auth#ClientCertificateConfig
Also, even if your folder structure is ‘/transaction/my-libs/images/1.0.0/libimage.bin’, but the blob URL is unique with folder hierarchy mentioned in the blob URL, thus when connecting to the Azure storage account to download the blob, mention the URL in single inverted comma notation for the blob path to be specific.
Please refer to the sample code below for downloading the blobs through Azure SDK for Go: -
https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/storage/azblob#example-package
https://pkg.go.dev/github.com/Azure/azure-storage-blob-go/azblob#pkg-examples

Upload files to Azure Storage from Azure VM using SDK azblob and Managed Service Identity

I am trying to upload files to azure storage container using Go SDK for Azure storage from an Azure VM which has Azure Managed Identity attached to it. I am also using Azure auth to create a ServicePrincipalToken using MSIConfig. However I am receiving an error
RESPONSE Status: 400 Authentication information is not given in the correct format. Check the value of Authorization header.
Can someone please help me understand what I am missing?
Script I have used (modified form of the example ):
// main.go
package main
import (
"log"
"fmt"
"context"
"net/url"
"strings"
"github.com/Azure/azure-storage-blob-go/azblob"
"github.com/Azure/go-autorest/autorest/azure/auth"
)
func main() {
azureServicePrincipalToken, err := auth.NewMSIConfig().ServicePrincipalToken()
if err != nil {
log.Fatal(err)
}
accountName := "<TESTSA>"
containerName := "<TESTCONTAINER>"
// Create a BlockBlobURL object to a blob in the container (we assume the container already exists).
u, _ := url.Parse(fmt.Sprintf("https://%s.blob.core.windows.net/%s/readme.txt", accountName, containerName))
credential := azblob.NewTokenCredential(azureServicePrincipalToken.Token().AccessToken, nil)
if err != nil {
log.Fatal(err)
}
blockBlobURL := azblob.NewBlockBlobURL(*u, azblob.NewPipeline(credential, azblob.PipelineOptions{}))
log.Println(blockBlobURL)
ctx := context.Background() // This example uses a never-expiring context
// Perform UploadStreamToBlockBlob
bufferSize := 2 * 1024 * 1024
maxBuffers := 3
_, err = azblob.UploadStreamToBlockBlob(ctx, strings.NewReader("Hello azblob"), blockBlobURL,
azblob.UploadStreamToBlockBlobOptions{BufferSize: bufferSize, MaxBuffers: maxBuffers})
if err != nil {
log.Fatal(err)
}
}
When I execute go run main.go, I receive the following error:
2020/12/26 17:58:07 https://<TESTSA>.blob.core.windows.net/<TESTCONTAINER>/readme.txt
2020/12/26 17:58:07 write error: -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /home/<MYUSER>/go/pkg/mod/github.com/!azure/azure-storage-blob-go#v0.12.0/azblob/zc_storage_error.go:42
===== RESPONSE ERROR (ServiceCode=) =====
Description=Authentication information is not given in the correct format. Check the value of Authorization header.
RequestId:f30c063e-901e-0046-2cb0-db4781000000
Time:2020-12-26T17:58:07.7810745Z, Details:
Code: InvalidAuthenticationInfo
PUT https://<TESTSA>.blob.core.windows.net/<TESTCONTAINER>/readme.txt?blockid=j%2BItsAdqRN6EScZ3S2r8QwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA%3D%3D&comp=block&timeout=61
Authorization: REDACTED
Content-Length: [12]
User-Agent: [Azure-Storage/0.12 (go1.13.9; linux)]
X-Ms-Client-Request-Id: [21638ec4-138c-434d-4b53-d13924e51966]
X-Ms-Version: [2019-12-12]
--------------------------------------------------------------------------------
RESPONSE Status: 400 Authentication information is not given in the correct format. Check the value of Authorization header.
Content-Length: [298]
Content-Type: [application/xml]
Date: [Sat, 26 Dec 2020 17:58:07 GMT]
Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]
X-Ms-Request-Id: [f30c063e-901e-0046-2cb0-db4781000000]
exit status 1
I have also verified with the azcli command and I was able to upload a sample txt file helloworld to the storage container without any challenge. The commands I used:
az login --identity
az storage blob upload --container-name <TESTCONTAINER> --account-name <TESTSA> --name helloworld --file helloworld --auth-mode login
Response:
Finished[#############################################################] 100.0000%
{
"etag": "\"0x8D8A9CCDD921BA7\"",
"lastModified": "2020-12-26T18:34:22+00:00"
}
Thank you.
The code sample you refer to authorizes with Shared Key and Put Blob API, but not Azure AD.
credential, err := NewSharedKeyCredential(accountName, accountKey)
If you would like to authorize with Azure AD by ServicePrincipalToken, see Azure Active Directory authentication for Go.
applicationSecret := "APPLICATION_SECRET"
spt, err := adal.NewServicePrincipalToken(
*oauthConfig,
appliationID,
applicationSecret,
resource,
callbacks...)
if err != nil {
return nil, err
}
// Acquire a new access token
err = spt.Refresh()
if (err == nil) {
token := spt.Token
}

Authentication successful but unable to retrieve and list Azure AD Users using Azure Golang SDK

I am trying to use Azure Golang SDK to pull the list of Azure AD users.
Few things to Note:
1. Authentication is successful. "Authorization successful." is displayed when I use below code to fetch bearer token for Azure SPN.
This is pretty much vanilla code picked from Azure-Go-SDK repo.
//GetGraphAuthorizer gets an OAuthTokenAuthorizer for graphrbac API.
func GetGraphAuthorizer(fs auth.FileSettings) (autorest.Authorizer, error) {
if graphAuthorizer != nil {
return graphAuthorizer, nil
}
var a autorest.Authorizer
var err error
a, err = getAuthorizerForResource(grantType(), Environment().GraphEndpoint, fs)
if err == nil {
// cache
graphAuthorizer = a
fmt.Println("Authorization successful.")
} else {
graphAuthorizer = nil
fmt.Println ("Authorization failed.")
}
return graphAuthorizer, err
}
Defined a wrapper on around GetGraphAuthorizer function to instantiate userClient object:
func getADUserClient(fs auth.FileSettings) graphrbac.UsersClient {
userClient := graphrbac.NewUsersClient(azure.GetTenantId(fs))
a, _ := azure.GetGraphAuthorizer(fs)
userClient.Authorizer = a
userClient.AddToUserAgent(azure.UserAgent())
return userClient
}
Then I use the token to list users in Azure AD in below function:
adUserClient := getADUserClient(fs)
// if auth failed, then it should've displayed the failure message here but prints "Authorization successful instead"
for list, err := adUserClient.ListComplete(context.Background(), ""); list.NotDone(); err = list.Next() {
if err != nil {
fmt.Print("got error while traversing User list: ", err)
}
i := list.Value()
fmt.Println(*i.DisplayName)
fmt.Println(*i.GivenName)
}
No output what so ever!!
FYI:- I have users in Azure tenant.
I have granted SPN access to Graph API.
Any help is appreciated.
Not 100% sure but for some reason I had to grant Azure SPN "Delegated" permission to:
Directory.ReadWrite.All
Groups.ReadWrite.All
Surprisingly, Once the program listed users, I removed above "Delegated" permissions off SPN and just left the Application permissions and it continues to work!

How to setup Google Translate API for Node.js?

I want to use Googles Cloud Translation API in my Node.js application, however I'm getting a The request is missing a valid API key. error.
I have followed the Quickstart guide provided by Google.
I have created GCP project, downloaded the private key as JSON file and setup the environment variable in Powershell (img).
After that I've installed the library with
yarn add #google-cloud/translate
The code I'm running in my translate.js file comes from the Quickstart guide with additional try-catch blocks:
async function quickstart(
projectId = process.env.PROJECT_ID // Project Id from JSON file
) {
try {
// Imports the Google Cloud client library
const { Translate } = require('#google-cloud/translate');
// Instantiates a client
const translate = new Translate({ projectId });
// The text to translate
const text = 'Hello, world!';
// The target language
const target = 'ru';
// Translates some text into Russian
const [translation] = await translate.translate(text, target);
console.log('Text:', text);
console.log('Translation:', translation);
} catch (error) {
console.error(error);
}
}
quickstart();
When I then run node translate.js, I'll get an Error:
{ Error: The request is missing a valid API key.
...
code: 403,
errors:
[ { message: 'The request is missing a valid API key.',
domain: 'global',
reason: 'forbidden' } ],
response: undefined,
message: 'The request is missing a valid API key.' }
I am on Windows 10, Node v10.13.0.
Believe you would have missed this environment variable defining , before starting the node service
Replace [PATH] with the file path of the JSON file that contains your service account key, and [FILE_NAME] with the filename.
With PowerShell:
$env:GOOGLE_APPLICATION_CREDENTIALS="[PATH]"
For example:
$env:GOOGLE_APPLICATION_CREDENTIALS="C:\Users\username\Downloads\[FILE_NAME].json"

Bluemix - object storage - node.js - pkgcloud - openstack returns 401

I am trying to use pkgcloud (node.js) openstack with bluemix object storage, but when I put all the requested parameters as on official page, it always returns 401. I tried using postman as described on bluemix and it works.
I created a package, which is able to to authorize it right. It is just a copy of pkgcloud, with a few fixes.
EDIT: IT IS WORKING! The V2 supports was shot down by bluemix and it has only V3 support now, but I once again find the issues.
Remember to use newest version (2.0.0)
So this is how you can use it now :
var pkgcloud = require('pkgcloud-bluemix-objectstorage');
// Create a config object
var config = {};
// Specify Openstack as the provider
config.provider = "openstack";
// Authentication url
config.authUrl = 'https://identity.open.softlayer.com/';
config.region= 'dallas';
// Use the service catalog
config.useServiceCatalog = true;
// true for applications running inside Bluemix, otherwise false
config.useInternal = false;
// projectId as provided in your Service Credentials
config.tenantId = 'xxx';
// userId as provided in your Service Credentials
config.userId = 'xxx';
// username as provided in your Service Credentials
config.username = 'xxx';
// password as provided in your Service Credentials
config.password = 'xxx';
// This is part which is NOT in original pkgcloud. This is how it works with newest version of bluemix and pkgcloud at 22.12.2015.
//In reality, anything you put in this config.auth will be send in body to server, so if you need change anything to make it work, you can. PS : Yes, these are the same credentials as you put to config before.
//I do not fill this automatically to make it transparent.
config.auth = {
forceUri : "https://identity.open.softlayer.com/v3/auth/tokens", //force uri to v3, usually you take the baseurl for authentication and add this to it /v3/auth/tokens (at least in bluemix)
interfaceName : "public", //use public for apps outside bluemix and internal for apps inside bluemix. There is also admin interface, I personally do not know, what it is for.
"identity": {
"methods": [
"password"
],
"password": {
"user": {
"id": "***", //userId
"password": "***" //userPassword
}
}
},
"scope": {
"project": {
"id": "***" //projectId
}
}
};
console.log("config: " + JSON.stringify(config));
// Create a pkgcloud storage client
var storageClient = pkgcloud.storage.createClient(config);
// Authenticate to OpenStack
storageClient.auth(function (error) {
if (error) {
console.error("storageClient.auth() : error creating storage client: ", error);
}
else {
// Print the identity object which contains your Keystone token.
console.log("storageClient.auth() : created storage client: " + JSON.stringify(storageClient._identity));
}
});
PS : You should be able to connect to this service outside of bluemix, therefore you can test it on your localhost.
Lines below are for old content for version 1.2.3, read only if you want to use v2 version of pkgcloud which was working with bluemix before January 2016
EDIT: It looks like that bluemix shut down support for v2 openstack and only supports v3, which is not supported by pkgcloud at all. So this does not work anymore (at least for me).
The problem is actually between pkgcloud and bluemix authorization process. Bluemix is expecting a little diffent authorization. I created a package, which is able to to authorize it right. It is just a copy of pkgcloud, with a few fixes.
And this is how you can use it :
var pkgcloud = require('pkgcloud-bluemix-objectstorage');
// Create a config object
var config = {};
// Specify Openstack as the provider
config.provider = "openstack";
// Authentication url
config.authUrl = 'https://identity.open.softlayer.com/';
config.region= 'dallas';
// Use the service catalog
config.useServiceCatalog = true;
// true for applications running inside Bluemix, otherwise false
config.useInternal = false;
// projectId as provided in your Service Credentials
config.tenantId = 'xxx';
// userId as provided in your Service Credentials
config.userId = 'xxx';
// username as provided in your Service Credentials
config.username = 'xxx';
// password as provided in your Service Credentials
config.password = 'xxx';
// This is part which is NOT in original pkgcloud. This is how it works with newest version of bluemix and pkgcloud at 22.12.2015.
//In reality, anything you put in this config.auth will be send in body to server, so if you need change anything to make it work, you can. PS : Yes, these are the same credentials as you put to config before.
//I do not fill this automatically to make it transparent.
config.auth = {
tenantId: "xxx", //projectId
passwordCredentials: {
userId: "xxx", //userId
password: "xxx" //password
}
};
console.log("config: " + JSON.stringify(config));
// Create a pkgcloud storage client
var storageClient = pkgcloud.storage.createClient(config);
// Authenticate to OpenStack
storageClient.auth(function (error) {
if (error) {
console.error("storageClient.auth() : error creating storage client: ", error);
}
else {
// Print the identity object which contains your Keystone token.
console.log("storageClient.auth() : created storage client: " + JSON.stringify(storageClient._identity));
}
});

Resources