Azure speech to text with numbers - azure

A use case for my app is to convert speech (single word utterances) to text. I need to use Azure speech to text for this. Sometimes the speech needs to be converted into an integer - I need to submit the response as a quantity for example.
My question is is there anyway, via the REST API, to tell the speech to text service I want a numeric result? Currently it is returning things like 'one' instead of '1' and 'free' instead of '3'. I don't think there is a way to do this from the documentation but I wanted to see if anyone else has solved this problem before I think of a way around it.
This is the code I am using in my proof of concept project:
public static async Task SpeechToTextAsync(MemoryStream data, ISpeechResultCallback callBack)
{
string accessToken = await Authentication.GetAccessToken();
IToast toastWrapper = DependencyService.Get<IToast>();
if (accessToken != null)
{
toastWrapper.Show("Acquired token");
callBack.SpeechReturned("Acquired token");
using (var client = new HttpClient())
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create("https://westus.stt.speech.microsoft.com/speech/recognition/conversation/cognitiveservices/v1?language=en-GB&format=detailed");
request.SendChunked = true;
request.Accept = #"application/json;text/xml";
request.Method = "POST";
request.ProtocolVersion = HttpVersion.Version11;
request.Host = "westus.stt.speech.microsoft.com";
request.ContentType = #"audio/wav; codecs=audio/pcm; samplerate=16000";
// request.Headers["Ocp-Apim-Subscription-Key"] = Program.SubscriptionKey;
request.Headers.Add("Authorization", "Bearer " + accessToken);
request.AllowWriteStreamBuffering = false;
data.Position = 0;
byte[] buffer = null;
int bytesRead = 0;
using (Stream requestStream = request.GetRequestStream())
{
buffer = new Byte[checked((uint)Math.Min(1024, (int)data.Length))];
while ((bytesRead = data.Read(buffer, 0, buffer.Length)) != 0)
{
requestStream.Write(buffer, 0, bytesRead);
}
// Flush
requestStream.Flush();
}
try
{
string responseData = null;
using (WebResponse response = request.GetResponse())
{
var encoding = Encoding.GetEncoding(((HttpWebResponse)response).CharacterSet);
using (var responseStream = response.GetResponseStream())
{
using (var reader = new StreamReader(responseStream, encoding))
{
responseData = reader.ReadToEnd();
AzureSTTResults deserializedProduct = JsonConvert.DeserializeObject<AzureSTTResults>(responseData);
if(deserializedProduct == null || deserializedProduct.NBest == null || deserializedProduct.NBest.Length == 0)
{
toastWrapper.Show("No results");
callBack.SpeechReturned("No results");
}
else
{
toastWrapper.Show(deserializedProduct.NBest[0].ITN);
callBack.SpeechReturned(deserializedProduct.NBest[0].ITN);
}
}
}
}
}
catch (Exception ex)
{
toastWrapper.Show(ex.Message);
callBack.SpeechReturned(ex.Message);
}
}
}
else
{
toastWrapper.Show("No token required");
callBack.SpeechReturned("No token required");
}
}
And here is an example of the result that I would like to be '1':
{
"RecognitionStatus": "Success",
"Offset": 0,
"Duration": 22200000,
"NBest": [
{
"Confidence": 0.43084684014320374,
"Lexical": "one",
"ITN": "One",
"MaskedITN": "One",
"Display": "One."
}
]
}

I suggest to use this nuget from Microsoft. It works like a charm, here an example.
NumberRecognizer.RecognizeNumber("I have two apples", Culture.English)

According to the offical document Speech-to-text REST API, there is no option can help converting the numberic words to numbers.
Considering for the numberic words in English have the pattern in syntax, you can use a simple algorithm to implement the feature for converting words to numbers. As references, you can follow these below to write your own one in C# by yourself.
Converting words to numbers in c++
Translate (Convert) Words to Numbers RRS feed in SQL Server
Words in Numbers
Hope it helps.

Related

Access Microsoft.Storage/storageAccounts/blobServices metrics from code

I want to retrieve metrics related to Blob metric namespace from Azure Storage Account. I need to read the BlobCount value.
Initially I have tried like this:
var usedCapacityResults = await metricsClient.QueryResourceAsync(resourceId, new[] { "BlobCount1" },
new MetricsQueryOptions
{
MetricNamespace = "Blob",
Aggregations =
{
MetricAggregationType.Average
},
Granularity = TimeSpan.FromMinutes(5),
TimeRange = new QueryTimeRange(TimeSpan.FromMinutes(10))
});
if (usedCapacityResults.GetRawResponse().Status == StatusCodes.Status200OK)
{
var usedCapacityMetric = usedCapacityResults.Value.Metrics.FirstOrDefault(m => m.Name == "BlobCount" && m.Error == null);
var metricValue = usedCapacityMetric?.TimeSeries.FirstOrDefault();
if (metricValue != null && !metricValue.Values.IsNullOrEmpty())
{
var average = metricValue.Values[0].Average;
if (average != null) blobCount = (decimal)average;
}
}
But nothing gets returned.
Then I have tried to get the supported metric namespace, using this call:
GET https://management.azure.com/{resourceUri}/providers/microsoft.insights/metricNamespaces?api-version=2017-12-01-preview
and the only valid metric seems to be Microsoft.Storage/storageAccounts, that does not have the blob count metric.
Any idea how to read from code the BlobCount value?
There will be also the option to retrieve the list of containers and iterate though it to count the blobs, but this is something I want to avoid.
The working solution, with help from MS support:
This is the solution that was provided to me by MS Support team:
private async Task<decimal> GetStorageAccountBlobCount(MetricsQueryClient metricsClient, string resourceId)
{
var blobCount = (decimal)0.0;
try
{
resourceId = $"{resourceId}/blobServices/default";
var blobCountResult = await metricsClient.QueryResourceAsync(resourceId, new[] { "BlobCount" },
new MetricsQueryOptions
{
MetricNamespace = "Microsoft.Storage/storageAccounts/blobServices",
Aggregations =
{
MetricAggregationType.Average
},
Granularity = TimeSpan.FromHours(1),
TimeRange = new QueryTimeRange(TimeSpan.FromMinutes(60))
});
if (blobCountResult.GetRawResponse().Status == StatusCodes.Status200OK)
{
var blobCountMetric = blobCountResult.Value.Metrics.FirstOrDefault(m => m.Name == "BlobCount" && m.Error == null);
var metricValue = blobCountMetric?.TimeSeries.FirstOrDefault();
if (metricValue != null && !metricValue.Values.IsNullOrEmpty())
{
var average = metricValue.Values[0].Average;
if (average != null) blobCount = (decimal)average;
}
}
}
catch (Exception ex)
{
_logger.LogError($"Error on calculate blob count for {resourceId}", ex);
}
return blobCount;
}

What's wrong on using HttpWebRequest this way?

This is the way I do server request to some endpoints from Azure:
public T SingleRead<T>(string url, string method, object entity = null)
{
T returnValue = default(T);
var resp = GetRESTResponse(url, method, entity);
string responseText = GetResponseText(resp);
try
{
returnValue = JsonConvert.DeserializeObject<T>(responseText);
}
catch (Exception ex)
{
return default(T);
}
return returnValue;
}
private HttpWebResponse GetRESTResponse(string url, string method, object entity = null)
{
var address;
if (!url.StartsWith("http"))
{
if (!url.StartsWith("/")) url = $"/{url}";
address = baseAddress + url;
}
else
{
address = url;
}
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(address);
req.Method = method;
if (entity != null)
{
byte[] byteArray = Encoding.Default.GetBytes(JsonConvert.SerializeObject(entity));
req.ContentLength = byteArray.Length;
req.ContentType = "application/json";
Stream dataStream = req.GetRequestStream();
dataStream.Write(byteArray, 0, byteArray.Length);
dataStream.Flush();
dataStream.Close();
}
HttpWebResponse resp;
try
{
resp = (HttpWebResponse)req.GetResponse();
}
catch (WebException e)
{
Log(e.Reponse);
resp = (HttpWebResponse)e.Response;
}
return resp;
}
private static string GetResponseText(HttpWebResponse resp)
{
var encoding = Encoding.ASCII;
string responseText = ".";
using (var reader = new StreamReader(resp.GetResponseStream(), encoding))
{
responseText = reader.ReadToEnd();
}
if (resp.StatusCode == HttpStatusCode.InternalServerError || resp.StatusCode == HttpStatusCode.BadRequest || resp.StatusCode == HttpStatusCode.NotFound)
{
return "";
}
return responseText;
}
It works quite often. Sometimes, it doesn't, and I get the a "A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond" error from the Log.
Is there some wrong in the procedure, or it could be a "timeout" by endpoint?
The called server users says "we don't have any problem, and we don't get the request".
Not sure if its the fault of the code above (maybe some stream not closed?). But I don't see any problem. Do you see any trouble in this?
It can be a network issue, which fails quite often. You need to remember there are several switches until the request goes outside the datacenter, and there are countless requests happening at the same time (you, and all other Azure customers).
This can be a transient fault, which may work if you send another request. You need to implement some retry logic to identify if the failure is transient or not.
More info:
https://learn.microsoft.com/en-us/azure/architecture/patterns/retry
And here's a sample using Polly, which is strongly recommended and even used inside Azure SDKs:
https://stackoverflow.com/a/66554740/1384539
https://learn.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/implement-http-call-retries-exponential-backoff-polly
https://github.com/App-vNext/Polly

getFileAsync causing Excel to crash

I'm building an office-js add-in for Excel. I need to upload the current workbook to a back end server. I've implemented an example from the Micrsoft Documentation, which seems to work fine the first time I call it, but on subsequent calls, it causes Excel to crash. I'm using Excel 365 version 1812 (build 11126.20132)
Here is the link to the example in the MS docs:
https://learn.microsoft.com/en-us/javascript/api/office/office.document
There are many examples on this page, to find the one I'm working from search for "The following example gets the document in Office Open XML" I've included the example below for ease of reference.
The code just get's the current file and dumps the characters to the console's log. It works fine the first but crashes Excel the second time--after it has shown the length of FileContent.
export function getDocumentAsCompressed() {
Office.context.document.getFileAsync(Office.FileType.Compressed, { sliceSize: 65536 /*64 KB*/ },
function (result) {
if (result.status == "succeeded") {
// If the getFileAsync call succeeded, then
// result.value will return a valid File Object.
var myFile = result.value;
var sliceCount = myFile.sliceCount;
var slicesReceived = 0, gotAllSlices = true, docdataSlices = [];
console.log("File size:" + myFile.size + " #Slices: " + sliceCount);
// Get the file slices.
getSliceAsync(myFile, 0, sliceCount, gotAllSlices, docdataSlices, slicesReceived);
}else {
console.log("Error:", result.error.message);
}
});
}
function getSliceAsync(file, nextSlice, sliceCount, gotAllSlices, docdataSlices, slicesReceived) {
file.getSliceAsync(nextSlice, function (sliceResult) {
if (sliceResult.status == "succeeded") {
if (!gotAllSlices) { // Failed to get all slices, no need to continue.
return;
}
// Got one slice, store it in a temporary array.
// (Or you can do something else, such as
// send it to a third-party server.)
// console.log("file part",sliceResult.value.data)
docdataSlices[sliceResult.value.index] = sliceResult.value.data;
if (++slicesReceived == sliceCount) {
// All slices have been received.
file.closeAsync();
onGotAllSlices(docdataSlices);
}
else {
getSliceAsync(file, ++nextSlice, sliceCount, gotAllSlices, docdataSlices, slicesReceived);
}
}
else {
gotAllSlices = false;
file.closeAsync();
console.log("getSliceAsync Error:", sliceResult.error.message);
}
});
}
function onGotAllSlices(docdataSlices) {
var docdata = [];
for (var i = 0; i < docdataSlices.length; i++) {
docdata = docdata.concat(docdataSlices[i]);
}
var fileContent = new String();
for (var j = 0; j < docdata.length; j++) {
fileContent += String.fromCharCode(docdata[j]);
}
console.log("fileContent.length",fileContent.length)
// Now all the file content is stored in 'fileContent' variable,
// you can do something with it, such as print, fax...
}
Here is the result
File size:21489 #Slices: 1
fileContent.length 21489
Original example from Microsoft documentation (https://learn.microsoft.com/en-us/javascript/api/office/office.document)
// The following example gets the document in Office Open XML ("compressed") format in 65536 bytes (64 KB) slices.
// Note: The implementation of app.showNotification in this example is from the Visual Studio template for Office Add-ins.
function getDocumentAsCompressed() {
Office.context.document.getFileAsync(Office.FileType.Compressed, { sliceSize: 65536 /*64 KB*/ },
function (result) {
if (result.status == "succeeded") {
// If the getFileAsync call succeeded, then
// result.value will return a valid File Object.
var myFile = result.value;
var sliceCount = myFile.sliceCount;
var slicesReceived = 0, gotAllSlices = true, docdataSlices = [];
app.showNotification("File size:" + myFile.size + " #Slices: " + sliceCount);
// Get the file slices.
getSliceAsync(myFile, 0, sliceCount, gotAllSlices, docdataSlices, slicesReceived);
}
else {
app.showNotification("Error:", result.error.message);
}
});
}
function getSliceAsync(file, nextSlice, sliceCount, gotAllSlices, docdataSlices, slicesReceived) {
file.getSliceAsync(nextSlice, function (sliceResult) {
if (sliceResult.status == "succeeded") {
if (!gotAllSlices) { // Failed to get all slices, no need to continue.
return;
}
// Got one slice, store it in a temporary array.
// (Or you can do something else, such as
// send it to a third-party server.)
docdataSlices[sliceResult.value.index] = sliceResult.value.data;
if (++slicesReceived == sliceCount) {
// All slices have been received.
file.closeAsync();
onGotAllSlices(docdataSlices);
}
else {
getSliceAsync(file, ++nextSlice, sliceCount, gotAllSlices, docdataSlices, slicesReceived);
}
}
else {
gotAllSlices = false;
file.closeAsync();
app.showNotification("getSliceAsync Error:", sliceResult.error.message);
}
});
}
function onGotAllSlices(docdataSlices) {
var docdata = [];
for (var i = 0; i < docdataSlices.length; i++) {
docdata = docdata.concat(docdataSlices[i]);
}
var fileContent = new String();
for (var j = 0; j < docdata.length; j++) {
fileContent += String.fromCharCode(docdata[j]);
}
// Now all the file content is stored in 'fileContent' variable,
// you can do something with it, such as print, fax...
}
// The following example gets the document in PDF format.
Office.context.document.getFileAsync(Office.FileType.Pdf,
function(result) {
if (result.status == "succeeded") {
var myFile = result.value;
var sliceCount = myFile.sliceCount;
app.showNotification("File size:" + myFile.size + " #Slices: " + sliceCount);
// Now, you can call getSliceAsync to download the files,
// as described in the previous code segment (compressed format).
myFile.closeAsync();
}
else {
app.showNotification("Error:", result.error.message);
}
}
);
Since you're using Excel, have you tried the CreateWorkbork API? Might be a good workaround if the Document API has a bug, like Xuanzhou indicated earlier.
Here's a CreateDocument snippet that you can load into Script Lab. It shows how to create a Workbook copy based on an existing file.
Hope all that is helpful.
We already have a fix for it now. But the fix still need some time to go to production. Please try it several days later and let me know if the issue still exists. Thanks.

I want to send free form native query output to excel using a WebGrid

First I need to explain my problem, I have a native query application where someone types in "Select .... this and that" and the output is currently displayed in a grid that is paginated and I have been asked to add a button that will export the data to excel directly from an untyped datastream. my current code that I've found uses a grid and still doesn't prompt me to download a .xls file for some reason.
[Authorize(Roles = "Portal administrator")]
public void ExportExcel(NativeQueryVM model, int? page, string sort)
{
List<Dictionary<string, object>> result = null;
String vartimedate = DateTime.Now.ToString("yyyy-dd-M--HH-mm-ss");
try
{
var user = Membership.GetUser();
var grid = new System.Web.UI.WebControls.GridView();
if (page == null && sort == null)
{
if (model.QueryText.ToUpper().StartsWith("SELECT"))
{
UserQueryInput queryinput = new UserQueryInput();
queryinput.DatabaseId = model.selectedDatabase;
queryinput.QueryText = model.QueryText;
result = lookupProvider.GetSetNativeQuery((Guid)user.ProviderUserKey, user.UserName, queryinput, "Set");
model.QueryResultData = result;
ViewBag.SubmitType = "Select";
CreateDynamicResult(model.QueryResultData);
if (model == null || model.QueryResultData.Count == 0)
{
ViewBag.ResultMessage = "No Results Found";
}
else
{
WebGrid wd = new WebGrid(source: ViewBag.DynamicResult, canPage: false, canSort: false );
string griddata = wd.GetHtml().ToString();
string attachment = "attachment; filename=NativeQuery" + vartimedate + ".xls";
Response.ClearContent();
Response.AddHeader("content-disposition", attachment);
Response.ContentType = "application/excel";
StringWriter sw = new StringWriter();
HtmlTextWriter htw = new HtmlTextWriter(sw);
Response.Write(griddata);
Response.End();
}
}
}
else
{ //This part should come when page or sort is not null for Select. Should not be executed other than SELECT
result = lookupProvider.GetSetNativeQuery((Guid)user.ProviderUserKey, user.UserName, null, "Get");
model.QueryResultData = result;
ViewBag.SubmitType = "Select";
CreateDynamicResult(model.QueryResultData);
if (model == null || model.QueryResultData.Count == 0)
{
ViewBag.ResultMessage = "No Results Found";
}
else
{
}
}
}
catch (Exception ex)
{
logger.Log(new LogMessage()
.ForMethod("SubmitQuery")
.WithContext(Environment.MachineName)
.WithException(ex)
.WithDescription("An error occurred while submiting query to get result"));
}
}
Hi all that answered this problem, and I've found the answer and issues reside outside of the code that I've entered into the question. Jquery was intercepting the output of my file stream and on the other hand, if the button was created with runat=server the model is out of scope and the main part of the solution involved saving the model to a session and pulling it back in.
Thanks for your assistance all...
How are you calling this void function? The return paths include setting a ViewBag property with an error message along with modifying the response. It smells of bad design.
I'd create a hyperlink that links to a #Url.Action for something like this. Note that you'd need to return a FileContentResult with the data UTF8-encoded rather than modifying the Response directly:
public async Task<ActionResult> Test()
{
WebGrid wd = new WebGrid(source: ViewBag.DynamicResult, canPage: false, canSort: false );
string griddata = wd.GetHtml().ToString();
string attachment = "attachment; filename=NativeQuery" + vartimedate + ".xls";
Response.ClearContent();
Response.AddHeader("content-disposition", attachment);
Response.ContentType = "application/excel";
byte[] buffer = System.Text.UTF8Encoding.UTF8.GetBytes(griddata);
return File(buffer, "application/octet-stream");
}

Audible Audio (.aa) file spec?

Does anyone know of a good resource on the Audible Audio (.aa) file spec?
I'm trying to write a program that can use them, if no one knows of a resource, any tips on reverse engineering the spec my self? I opened it up in a Hex editor and poked around, looks like an MP3 but with a ton more header info.
This site provides some more info in regards to where certain chunks of data reside within the .aa file.
http://wiki.multimedia.cx/index.php?title=Audible_Audio
I have done some research into the Audible header to create a player for my car radio/computer. Basically there is a block of 3700 characters at the beginning of the file that encompasses a number of fields of interest, such as Title, Author, Narrator, etc. I have some limited parsing code in C# to display some of the basic info from the .aa file. as follows:
private void ParseFields(string fileName)
{
string aaHeader;
string tryDate;
if (fileName == "") return;
using (StreamReader sr = new StreamReader(fileName))
{
char[] buff = new char[3700];
sr.Read(buff, 0, buff.Length);
aaHeader = new string(buff);
}
try
{
_author = GetParsedItem(aaHeader, "author");
}
catch
{
_author = "?";
}
try
{
_title = GetParsedItem(aaHeader, "short_title");
}
catch
{
_title = "???";
}
try
{
_narrator = GetParsedItem(aaHeader, "narrator");
}
catch
{
_narrator = "?";
}
try
{
_description = GetParsedItem(aaHeader, "description");
}
catch
{
_description = "???";
}
try
{
_longDescription = GetParsedItem(aaHeader, "long_description");
}
catch
{
_longDescription = "";
}
try
{
tryDate = GetParsedItem(aaHeader, "pubdate");
if (tryDate != "")
_pubDate = Convert.ToDateTime(GetParsedItem(aaHeader, "pubdate"));
else
_pubDate = DateTime.Today;
}
catch
{
_pubDate = DateTime.Today;
}
}
private string GetParsedItem(string buffer, string fieldName)
{
if (buffer.Contains(fieldName))
{
int pos = buffer.IndexOf(fieldName);
pos += fieldName.Length;
int posEnd = buffer.IndexOf('\0',pos);
//if the value for the field is empty, skip it and look for another
if (pos == posEnd)
{
pos = buffer.IndexOf(fieldName, posEnd);
pos += fieldName.Length;
posEnd = buffer.IndexOf('\0', pos);
}
return buffer.Substring(pos, posEnd - pos);
}
else
return "(not found - " + fieldName + ")";
}
I think, there is no spec. Have a look at Wikipedia/Audible.com:
quote:
[...]
Audible introduced one of the first digital audio players in 1997.
The following year it published a Web site from which audio files in its
proprietary .aa format could be downloaded. Audible holds a number of patents
in this area.
[...]
summary: proprietary/patents

Resources