Can I use the Azure Speech resource without an endpoint?
I was working through the microsoft tutorials. Specifically, I was completing this lab. And in this lab the application client of the Speech resource does not use the endpoint, it only uses the key and location.
The same is told here in the documentation:
To find the keys and location/region of a completed deployment, follow these steps:
1. Sign in to the Azure portal using your Microsoft account.
2. Select All resources, and select the name of your Cognitive Services resource.
3. On the left pane, under RESOURCE MANAGEMENT, select Keys and Endpoint.
Each subscription has two keys; you can use either key in your application. To copy/paste a key to your code editor or other location, select the copy button next to each key, switch windows to paste the clipboard contents to the desired location.
Additionally, copy the LOCATION value, which is your region ID (ex. westus, westeurope) for SDK calls.
As you can see there is nothing told about the endpoint. Meaning that somehow the Speech resource client will know how to connect to the Speech resource by having just the key and location.
I am really confused, because I thought it should be impossible to do without an endpoint.
E.g. here the code samples which use the Speech resource do not use any endpoints (only a key and a location):
import os
from playsound import playsound
from azure.cognitiveservices.speech import SpeechConfig, SpeechRecognizer, AudioConfig
# Get spoken command from audio file
file_name = 'light-on.wav'
audio_file = os.path.join('data', 'speech', file_name)
# Configure speech recognizer
speech_config = SpeechConfig(cog_key, cog_location)
audio_config = AudioConfig(filename=audio_file) # Use file instead of default (microphone)
speech_recognizer = SpeechRecognizer(speech_config, audio_config)
# Use a one-time, synchronous call to transcribe the speech
speech = speech_recognizer.recognize_once()
# Play the original audio file
playsound(audio_file)
# Show transcribed text from audio file
print(speech.text)
I mean I can not even imagine how the Speech resource client (implemented by Microsoft) knows that it should connect to the resource in my Azure portal and not in some other portal without the endpoint. Looks like a magic to me, so I am definitely missing something here.
Thank you for trying Azure Speech service.
You're absolutely right - Speech service does use endpoints like any other cloud service of this kind.
If your code is using Speech SDK, then SDK provides the right endpoint for you, based upon the information you have provided, namely location.
I see from your code, that you are trying On-line transcription. Here you will find all regional endpoints used in this scenario.
There are other endpoints, like for Speech-to-text REST API V3 or Text-to-speech. They are all described in the documentation.
Related
I've got images stored as Blobs in Google Cloud Storage and I can see amazing capabilities offered by get_serving_url() which requires a blob_key as its first parameter. But I cannot see anyway to get a key from the Blob - am I mixing up multiple things called Blobs?
I'm using Python3 on GAE.
Try the code below but you'll need to first enable the Bundled Services API
from google.appengine.ext import blobstore
# Create the cloud storage file name
blobstore_filename = f"/gs/{<your_file_name>}"
# Now create the blob_key from the cloud storage file name
blob_key = blobstore.create_gs_key(blobstore_filename)
I'm traying to daploy an instance of the form recognizer app in Azure. For that I'm following the instructions in the documentation: https://learn.microsoft.com/en-us/azure/cognitive-services/form-recognizer/deploy-label-tool
I have created the docker instance and the connection, but the step to create the APP is failing.
This are the parameters I'm using:
Display Name: Test-form
Source Connection: <previuosly created connection>
Folder Path: None
Form Recognizer Service Uri: https://XXX-test.cognitiveservices.azure.com/
API Key: XXXXX
Description: None
And this is the error and getting:
I had the same error. It turned out to be due to incorrect SAS URI formatting because I generated and copied the SAS token via the Storage Accounts interface. It's much easier to get the correct format for the SAS URI if you generate it through the Storage Explorer (currently in Preview) as opposed to through the Storage Accounts.
If you read the documentation carefully it gives you a step by step guide
"To retrieve the SAS URL, open the Microsoft Azure Storage Explorer, right-click your container, and select Get shared access signature. Set the expiry time to some time after you'll have used the service. Make sure the Read, Write, Delete, and List permissions are checked, and click Create. Then copy the value in the URL section. It should have the form: https://.blob.core.windows.net/?"
Form Recognizer Documentation
The error messages point to a configuration issue with the AzureBlobStorageTemplate Thing. Most likely the containerName field for the Blob Storage Thing is empty or contains invalid characters
Ensure the containerName is a valid Azure storage container name.
Check https://learn.microsoft.com/en-us/rest/api/storageservices/Naming-and-Referencing-Containers--Blobs--and-Metadata for more information.
A container name must be a valid DNS name
The Connector loads and caches all configuration settings during startup. Any changes that you make to the configuration when troubleshooting are ignored until the Connector is restarted.
When creating the container connection, you must add the container into the SAS URI, such as
https://<storage-account>.blob.core.windows.net/<Enter-My-Container-Here>?<SAS Key>
You can also directly use the open source labeling tool, please see the section further down in the doc:
The OCR Form Labeling Tool is also available as an open-source project on GitHub. The tool is a web application built using React + Redux, and is written in TypeScript. To learn more or contribute, see OCR Form Labeling Tool.
Anyone can HELP? This one is really driving me crazy... Thank you!
I tried to use a google cloud platform API Speech-to-text.
Tools: WINDOWS 10 && GCP &&Python(Pycharm IDE)
I've created a service account as a owner for my speech-to-test project and generated a key from GCP console in json, then I set the environment variables.
Code I ran on WIN10 Powershell && CMD:
$env:GOOGLE_APPLICATION_CREDENTIALS="D:\GCloud speech-to-text\Speech
To Text Series-93e03f36bc9d.json"
set GOOGLE_APPLICATION_CREDENTIALS=D:\GCloud speech-to-text\Speech To
Text Series-93e03f36bc9d.json
PS: the added environment variables disappear in CMD and Powershell after reboot my laptop but do show in the env list if added again.
I've enabled the google storage api and google speech-to-text api in GCP console.
I've tried the explicitely showing credential method via python, same problem.
I've installed the google cloud SDK shell and initialized by using command to log in my account.
PYTHON SPEECH-TO-TEXT CODE(from GCP demo)
import io
import os
# Imports the Google Cloud client library
from google.cloud import speech
from google.cloud.speech import enums
from google.cloud.speech import types
# Instantiates a client
client = speech.SpeechClient()
# The name of the audio file to transcribe
file_name = os.path.join(
os.path.dirname(__file__),
'test_cre.m4a')
# Loads the audio into memory
with io.open(file_name, 'rb') as audio_file:
content = audio_file.read()
audio = types.RecognitionAudio(content=content)
config = types.RecognitionConfig(
encoding=enums.RecognitionConfig.AudioEncoding.LINEAR16,
sample_rate_hertz=16000,
language_code='en-US')
# Detects speech in the audio file
response = client.recognize(config, audio)
for result in response.results:
print('Transcript: {}'.format(result.alternatives[0].transcript))
----Expected to receive a "200OK" and the transcribed text when runing the code above (a demo of short speech to text api from GCP Document)
----But got:
D:\Python\main program\lib\site-packages\google\auth_default.py:66: UserWarning: Your application has authenticated using end user credentials from Google Cloud SDK. We recommend that most server applications use service accounts instead. If your application continues to use end user credentials from Cloud SDK, you might receive a "quota exceeded" or "API not enabled" error. For more information about service accounts, see https://cloud.google.com/docs/authentication/
warnings.warn(_CLOUD_SDK_CREDENTIALS_WARNING)
google.api_core.exceptions.ResourceExhausted: 429 Quota exceeded for quota metric 'speech.googleapis.com/default_requests' and limit 'DefaultRequestsPerMinutePerProject' of service 'speech.googleapis.com' for consumer 'project_number:764086051850'.
ANOTHER WEIRD THING: the error info shows that 'project_number:764086051850', which is different from my speech-to-text project_number on GCP (I do distinguish project number and project ID), the project_number shown in the error info also varies every time the code runs. It seems I was sending cloud requirement of the wrong project?
My GOOGLE_APPLICATION_CREDENTIALS system environment variables disappear after I restart my laptop next time. After adding again, it will appear in the env list but can't be stored after reboot again.
Appreciate it if someone can help, thank you!
try to do this:
Run gcloud init -> authenticate with your account and choose your project
Run gcloud auth activate-service-account <service account email> --key-file=<JSON key file>
Run gcloud config list to validate your configuration.
Run your script and see if it's better.
Else, try to do the same thing on a micro-vm for validating your code, service account and environment (and for validating that there is a problem only with Windows)
For Windows issues, I'm on ChromeBook, I can't test and help you on this. However, I checked about EnvVar on internet, and this update the registry. Check if you don't have stuff which protect Registry update (Antivirus,....)
D:\Python\main program\lib\site-packages\google\auth_default.py:66:
UserWarning: Your application has authenticated using end user
credentials from Google Cloud SDK. We recommend that most server
applications use service accounts instead. If your application
continues to use end user credentials from Cloud SDK, you might
receive a "quota exceeded" or "API not enabled" error. For more
information about service accounts, see
https://cloud.google.com/docs/authentication/
warnings.warn(_CLOUD_SDK_CREDENTIALS_WARNING)
This error means that your code is not using a service account. Your code is configured to use ADC (Application Default Credentials). Most likely your code is using the Google Cloud SDK credentials configured and stored by the CLI gcloud.
To determine what credentials the Cloud SDK is using, execute this command:
gcloud auth list
The IAM Member ID, displayed as ACCOUNT, with the asterisk is the account used by the CLI and any applications that do not specify credentials.
To learn more about ADC, read this article that I wrote:
Google Cloud Application Default Credentials
google.api_core.exceptions.ResourceExhausted: 429 Quota exceeded for
quota metric 'speech.googleapis.com/default_requests' and limit
'DefaultRequestsPerMinutePerProject' of service
'speech.googleapis.com' for consumer 'project_number:764086051850'.
The Cloud SDK has the concept of default values. Execute gcloud config list. This will display various items. Look for project. Most likely this project does not have the API Cloud Speech-to-Text enabled.
ANOTHER WEIRD THING: the error info shows that
'project_number:764086051850', which is different from my
speech-to-text project_number on GCP (I do distinguish project number
and project ID), the project_number shown in the error info also
varies every time the code runs. It seems I was sending cloud
requirement of the wrong project?
To see the list of projects, Project IDs and Project Numbers that your current credentials can see (access) execute:
gcloud projects list.
This command will display the Project Number given a Project ID:
gcloud projects list --filter="REPLACE_WITH_PROJECT_ID" --format="value(PROJECT_NUMBER)"
My GOOGLE_APPLICATION_CREDENTIALS system environment variables
disappear after I restart my laptop next time. After adding again, it
will appear in the env list but can't be stored after reboot again.
When you execute this command in a Command Prompt, it only persists for the life of the Command Prompt: set GOOGLE_APPLICATION_CREDENTIALS=D:\GCloud speech-to-text\Speech To
Text Series-93e03f36bc9d.json. When you exit the Command Prompt, reboot, etc. the environment variable is destroyed.
To create persistent environment variables on Windows, edit the System Properties -> Environment Variables. You can launch this command as follows from a Command Prompt:
SystemPropertiesAdvanced.exe
Suggestions to make your life easier:
Do NOT use long path names with spaces for your service account files. Create a directory such as C:\Config and place the file there with no spaces in the file name.
Do NOT use ADC (Application Default Credentials) when developing on your desktop. Specify the actual credentials that you want to use.
Change this line:
client = speech.SpeechClient()
To this:
client = speech.SpeechClient().from_service_account_json('c:/config/service-account.json')
Service Accounts have a Project ID inside them. Create the service account in the same project that you intend to use them (until you understand IAM and Service Accounts well).
I need to reed some CSV files that were shared which are in google Cloud Storage. My Script will run from another server outside from Google Cloud.
I am using this code:
from google.cloud import storage
client = storage.Client()
bucket = client.get_bucket('/stats/installs')
blob = storage.Blob('installs_overview.csv', bucket)
content = blob.download_as_string()
print(content)
Getting this error: Apparently I haven't specified the project but I don't have one
OSError: Project was not passed and could not be determined from the environment.
There are some wrong assumptions in the previous answers in this topic.
If it is a public bucket you do not have to worry about what project it is connected to. It is well documented how you, for example, can use a bucket to host a public website that browsers can access. Obviously the browser does not have to worry about what project it belongs to.
The code samples are a bit lacking on using public buckets & files,-- in all the examples you supply a project and credentials, which will
1) Bill bucket egress on the project you supply instead of the project the bucket is connected to
2) Assumes that you need to authenticate and authorise.
For a public file or bucket, however, all you have to worry about is the bucket name and file location.
You can
from google.cloud import storage
source="path/to/file/in/bucket.txt"
target="/your/local/file.txt"
client = storage.Client.create_anonymous_client()
# you need to set user_project to None for anonymous access
# If not it will attempt to put egress bill on the project you specify,
# and then you need to be authenticated to that project.
bucket = client.bucket(bucket_name="your-bucket", user_project=None)
blob = storage.Blob(source, bucket)
blob.download_to_filename(filename=target, client=client)
It is important that your file in the bucket has read access to "AllUsers"
First of all, I think there might be some confusion regarding Cloud Storage and how to access it. Cloud Storage is a Google Cloud Platform product, and therefore, to use it, a GCP Project must exist. You can find the project number and project ID for your project in the Home page of the Console, as explained in this documentation page.
That being said, let me refer you to the documentation page about the Python Cloud Storage Client Library. When you create the client to use the service, you can optionally specify the project ID and/or the credentials files to use:
client = storage.Client(project="PROJECT_ID",credentials="OAUTH2_CREDS")
If you do not specify the Project ID, it will be inferred from the environment.
Also, take into account that you must set up authentication in order to use the service. If you were running the application inside another GCP service (Compute Engine, App Engine, etc.), the recommended approach would be using the Application Default Credentials. However, given that that is not your case, you should instead follow this guide to set up authentication, downloading the key for the Service Account having permission to work with Cloud Storage and pointing to it in the environment variable GOOGLE_APPLICATION_CREDENTIALS.
Also, it looks like the configuration in your code is not correct, given that the bucket name you are using ('/stats/installs') is not valid:
Bucket names must be between 3 and 63 characters. A bucket name can
contain lowercase alphanumeric characters, hyphens, and underscores.
It can contain dots (.) if it forms a valid domain name with a
top-level domain (such as .com). Bucket names must start and end with
an alphanumeric character.
Note that you can see that the given bucket does not exist by working with exceptions, specifically google.cloud.exceptions.NotFound. Also, given that the files you are trying to access are public, I would not recommend to share the bucket and file names, you can just obfuscate them with a code such as <BUCKET_NAME>, <FILE_NAME>.
So, as a summary, the course of action should be:
Identify the project to which the bucket you want to work with belongs.
Obtain the right credentials to work with GCS in that project.
Add the project and credentials to the code.
Fix the code you shared with the correct bucket and file name. Note that if the file is inside a folder (even though in GCS the concept of directories itself does not exist, as I explained in this other question), the file name in storage.Blob() should include the complete path like path/to/file/file.csv.
I am not a google-cloud expert, but as some of the commentators have said, I think the problem will be that you haven't explicitly told the storage client which Project you are talking about. The error message implies that the storage client tries to figure out for itself which project you are referring to, and if it can't figure it out, it gives that error message. When I use the storage Client I normally just provide the project name as an argument and it seems to do the trick, e.g.:
client = storage.Client(project='my-uber-project')
Also, I just saw your comment that your bucket "doesn't have a project" - I don't understand how this is possible. If you log in to the google cloud console area and go to storage, surely your bucket is listed there and you can see your project name at the top of the page?
As #Mangu said, the bucket name in your code is presumably just to hide the real bucket name, as forward-slashes are not allowed in bucket names (but are allowed in blob names and can be used to represent 'folders').
I'm trying to build a small program to change the autoscale settings for our Azure WebApps, using the Microsoft.WindowsAzure.Management.Monitoring and Microsoft.WindowsAzure.Management.WebSites NuGet packages.
I have been roughly following the guide here.
However, we are interested in scaling WebApps / App Services rather than Cloud Services, so I am trying to use the same code to read the autoscale settings but providing a resource ID for our WebApp. I have already got the credentials required for making a connection (using a browser window popup for Active Directory authentication, but I understand we can use X.509 management certificates for non-interactive programs).
This is the request I'm trying to make. Credentials already established, and an exception is thrown earlier if they're not valid.
AutoscaleClient autoscaleClient = new AutoscaleClient(credentials);
var resourceId = AutoscaleResourceIdBuilder.BuildWebSiteResourceId(webspaceName: WebSpaceNames.NorthEuropeWebSpace, serverFarmName: "Default2");
AutoscaleSettingGetResponse get = autoscaleClient.Settings.Get(resourceId); // exception here
The WebApp (let's call it "MyWebApp") is part of an App Service Plan called "Default2" (Standard: 1 small), in a Resource Group called "WebDevResources", in the North Europe region. I expect that my problem is that I am using the wrong names to build the resourceId in the code - the naming conventions in the library don't map well onto what I can see in the Azure Portal.
I'm assuming that BuildWebSiteResourceId is the correct method to call, see MSDN documentation here.
However the two parameters it takes are webspaceName and serverFarmName, neither of which match anything in the Azure portal (or Google). I found another example which seemed to be using the WebApp's geo region for webSpaceName, so I've used the predefined value for North Europe where our app is hosted.
While trying to find the correct value for serverFarmName in the Azure Portal, I found the Resource ID for the App Service Plan, which looks like this:
/subscriptions/{subscription-guid}/resourceGroups/WebDevResources/providers/Microsoft.Web/serverfarms/Default2
That resource ID isn't valid for the call I'm trying to make, but it does support the idea that a 'serverfarm' is the same as an App Service Plan.
When I run the code, regardless of whether the resourceId parameters seem to be correct or garbage, I get this error response:
<string xmlns="http://schemas.microsoft.com/2003/10/Serialization/">
{"Code":"SettingNotFound","Message":"Could not find the autoscale settings."}
</string>
So, how can I construct the correct resource ID for my WebApp or App Service Plan? Or alternatively, is there a different tree I should be barking up to programatially manage WebApp scaling?
Update:
The solution below got the info I wanted. I also found the Azure resource explorer at resources.azure.com extremely useful to browse existing resources and find the correct names. For example, the name for my autoscale settings is actually "Default2-WebDevResources", i.e. "{AppServicePlan}-{ResourceGroup}" which I wouldn't have expected.
There is a preview service https://resources.azure.com/ where you can inspect all your resources easily. If you search for autoscale in the UI you will easily find the settings for your resource. It will also show you how to call the relevant REST Api endpoint to read or update that resorce.
It's a great tool for revealing a lot of details for your deployed resources and it will actually give you an ARM template stub for the resource you are looking at.
And to answer your question, you could programmatically call the REST API from a client with updated settings for autoscale. The REST API is one way of doing this, the SDK another and PowerShell a third.
The guide which you're following is based on the Azure Service Management model, aka Classic mode, which is deprecated and only exists mainly for backward compatibility support.
You should use the latest
Microsoft.Azure.Insights nuget package for getting the autoscale settings.
Sample code using the nuget above is as below:
using Microsoft.Azure.Management.Insights;
using Microsoft.Rest;
//... Get necessary values for the required parameters
var client = new InsightsManagementClient(new TokenCredentials(token));
client.AutoscaleSettings.Get(resourceGroupName, autoScaleSettingName);
Besides, the autoscalesettings is a resource under the "Microsoft.Insights" provider and not under the "Microsoft.Web" provider, which explains why you are not able to find it with your serverfarm resourceId.
See the REST API Reference below for getting the autoscale settings.
GET
https://management.azure.com/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/microsoft.insights/autoscaleSettings/{autoscale-setting-name}?api-version={api-version}