In a sample 1 minute audio clip of clear, good quality audio, I am seeing some strange transcription accuracy behaviour.
Accuracy is as expected for the first 45 seconds or so then all I get is a couple of stray words from the last 15 seconds. The audio is still clear and understandable.
BUT... If I clip those last 15 seconds and send it by itself it is transcribed accurately... so what the heck is going on here? is the context of the full clip somehow messing it up?
{
"created": "2017-07-28T11:53:29.584Z",
"id": "5f5f9f00-738b-11e7-8e0f-f3176d375982",
"updated": "2017-07-28T11:54:03.398Z",
"results": [{
"result_index": 0,
"results": [
{
"final": true,
"alternatives": [{
"transcript": "everybody's good to see so many folks in chambers today we never know who's going to join us so welcome I'm city councillor Marianne lead ward and I am %HESITATION the chair of today's committee of the whole %HESITATION first I will read a safety notice which I think is gonna be on your screens as well it there is %HESITATION in the event of an emergency please evacuate council chambers by the narrow staircase which is located through the doorway marked exit and obey all instructions given by the clerk ",
"confidence": 0.848
}]
},
{
"final": true,
"alternatives": [{
"transcript": "if assistance is required please see the clerk and once you've evacuated the building please gather in civic square outside of city hall ",
"confidence": 0.948
}]
},
{
"final": true,
"alternatives": [{
"transcript": "so I would like to thank the media who is here with us today %HESITATION in person and the public for attending and remind ",
"confidence": 0.959
}]
},
{
"final": true,
"alternatives": [{
"transcript": "person ",
"confidence": 0.608
}]
},
{
"final": true,
"alternatives": [{
"transcript": "the proceedings on ",
"confidence": 0.878
}]
},
{
"final": true,
"alternatives": [{
"transcript": "and they are also archived ",
"confidence": 0.675
}]
}
]
}],
"status": "completed"
}
{
"created": "2017-07-28T11:58:25.384Z",
"id": "0faf1e80-738c-11e7-8e0f-f3176d375982",
"updated": "2017-07-28T11:58:32.559Z",
"results": [{
"result_index": 0,
"results": [
{
"final": true,
"alternatives": [{
"transcript": "US today are in person and the public for attending and remind everyone that if you can't be with us out in person you can catch ",
"confidence": 0.945
}]
},
{
"final": true,
"alternatives": [{
"transcript": "the proceedings on live webcast ",
"confidence": 0.859
}]
},
{
"final": true,
"alternatives": [{
"transcript": "and they are also archive ",
"confidence": 0.731
}]
}
]
}],
"status": "completed"
}
Difficult to say what is going on without actually looking at the audio stream, can you please share that audio file with us through Bluemix customer support? https://support.eu-gb.bluemix.net/gethelp/ we will take a look at it.
Related
I'm using Microsoft.Skills.Text.EntityRecognitionSkill in my skillset which output "Person", "Location", "Organization".
however I want to only output Location that have a confident level > .5
is there a way to do that?
here is a snap of my code
{
"#odata.type": "#Microsoft.Skills.Text.EntityRecognitionSkill",
"categories": [
"Person",
"Location",
"Organization"
],
"context": "/document/finalText/pages/*",
"inputs": [
{
"name": "text",
"source": "/document/finalText/pages/*"
},
{
"name": "languageCode",
"source": "/document/languageCode"
}
],
"outputs": [
{
"name": "persons",
"targetName": "people"
},
{
"name": "locations"
},
{
"name": "namedEntities",
"targetName": "entities"
}
]
},
[Edited based on Mick's comment]
Yes, this should be possible by setting the minimumPrecision parameter of the entity recognition skill to 0.5, which will result in entities whose confidence is >= 0.5 to be returned.
The documentation for entity recognition skill is here: https://learn.microsoft.com/en-us/azure/search/cognitive-search-skill-entity-recognition
As Mick points out, the documentation says minimumPrecision is unused, however that documentation is out of date and I will fix it soon.
I'm made simple Media Player by using MediaResponse.
Google automatically handle : play, pause, stop, resume.
Since Google not supported : next function yet, I created custom Next Intent to handle next function.
I had 2 cases makes me confused after talk to Dr. Media then go :
MediaResponse show correctly, not playing yet (not click button Play yet).
Case 1 - Then I say next (or next song), it matched with my defined phrases in my custom intent. And Next function is OK.
Case 2 - I clicked Play button to play audio, then I say next (or next song), it didn't match with my defined phrases (it oftens say Okay and nothing happens), so Next function is not OK. (As images in below)
In case 2, How I can catch my phrases next, next song, play next ... when Media Player is playing? (In other words, these phrases can not trigger my custom intent)
Please help me, Thanks
p/s : It happens on Mobile phones
actions_intent_NEXT.json
{
"id": "01055a59-26a0-4f39-a770-6bc5404482d9",
"name": "actions_intent_NEXT",
"auto": true,
"contexts": [
"actions_capability_screen_output",
"actions_capability_media_response_audio"
],
"responses": [
{
"resetContexts": false,
"affectedContexts": [
{
"name": "actions_capability_screen_output",
"parameters": {},
"lifespan": 5
},
{
"name": "actions_capability_media_response_audio",
"parameters": {},
"lifespan": 5
}
],
"parameters": [
{
"id": "2ce8514c-a346-4251-9a35-cab32d2c9d7c",
"required": false,
"dataType": "#nextPlay",
"name": "nextPlay",
"value": "$nextPlay",
"isList": false
}
],
"messages": [
{
"type": 0,
"speech": []
}
],
"defaultResponsePlatforms": {},
"speech": []
}
],
"priority": 500000,
"cortanaCommand": {
"navigateOrService": "NAVIGATE",
"target": ""
},
"webhookUsed": true,
"webhookForSlotFilling": false,
"lastUpdate": 1542252675,
"fallbackIntent": false,
"events": [
{
"name": "actions_intent_NEXT"
}
],
"userSays": [
{
"id": "e28a4087-2bf9-494d-9790-c7347f870ee4",
"data": [
{
"text": "next song",
"alias": "nextPlay",
"meta": "#nextPlay",
"userDefined": true
}
],
"isTemplate": false,
"count": 0,
"updated": 1542249008,
"isAuto": false
},
{
"id": "7a8cf2a2-d131-490d-977d-1212f4642f52",
"data": [
{
"text": "next",
"alias": "nextPlay",
"meta": "#nextPlay",
"userDefined": true
}
],
"isTemplate": false,
"count": 0,
"updated": 1542248700,
"isAuto": false
}
],
"followUpIntents": [],
"liveAgentHandoff": false,
"endInteraction": false,
"templates": []
}
This is the input JSON packet. I'm writing the transformation query in stream analytics to get source and masterTag properties.
[{
"source": "xda",
"data":
[{
"masterTag": "UNIFY",
"speed": 180
}],
"EventEnqueuedUtcTime": "2018-07-20T19:28:18.5230000Z",
},
{
"source": "xda",
"data": [{
"masterTag": "UNIFY",
"speed": 214
}],
"EventEnqueuedUtcTime": "2018-07-20T19:28:20.5550000Z",
}
]
Here is what I've:
WITH data AS
(
SELECT
source,
GetArrayElement(data,0) as data_packet
FROM input
)
SELECT
source,
data_packet.masterTag
INTO
output
FROM data
However, this query only returns one element. How do I update this query to get all elements in the JSON packet?
I did test with your source data and query, it did get 2 output elements. So,I think you mean how to return all elements with your data array has more elements.
You could use below query:
SELECT
jsoninput.source,
arrayElement.ArrayValue.masterTag
INTO
output
FROM jsoninput
CROSS APPLY GetArrayElements(jsoninput.data) AS arrayElement
Json sample:
[{
"source": "xda",
"data":
[{
"masterTag": "UNIFY1",
"speed": 180
},
{
"masterTag": "UNIFY2",
"speed": 180
}],
"EventEnqueuedUtcTime": "2018-07-20T19:28:18.5230000Z",
},
{
"source": "xda",
"data": [{
"masterTag": "UNIFY3",
"speed": 214
},
{
"masterTag": "UNIFY4",
"speed": 180
}],
"EventEnqueuedUtcTime": "2018-07-20T19:28:20.5550000Z",
}
]
Output:
Hope it helps you.
Try the function GetArrayElements (plural) with a cross apply.
Examples
https://msdn.microsoft.com/en-us/azure/stream-analytics/reference/getarrayelements-azure-stream-analytics
https://msdn.microsoft.com/en-us/azure/stream-analytics/reference/complex-data-types-stream-analytics#array-data-types
We are having some issues with the Google NLP service. The service is intermittently refusing to return entities for certain terms. We use the NLP annotate API for free text answers to survey responses. A recent question was related to an image of a kids TV character in the UK called Zippy. Some example responses are below. Unfortunately we had thousands of responses like this and none of them detected "zippy" as an entity. Strangely "elmo", "zippie" and others were detected without any issue, only this specific set of chars ("zippy") returned with no entities. Any ideas why this might be?
{
"sentences": [{
"text": {
"content": "zippy",
"beginOffset": 0
},
"sentiment": {
"magnitude": 0.1,
"score": 0.1
}
}],
"tokens": [],
"entities": [],
"documentSentiment": {
"magnitude": 0.1,
"score": 0.1
},
"language": "en",
"categories": []
}
"rainbow" detected but not "zippy"
{
"sentences": [{
"text": {
"content": "zippy from rainbow",
"beginOffset": 0
},
"sentiment": {
"magnitude": 0.1,
"score": 0.1
}
}],
"tokens": [],
"entities": [{
"name": "rainbow",
"type": "OTHER",
"metadata": [],
"salience": 1,
"mentions": [{
"text": {
"content": "rainbow",
"beginOffset": 11
},
"type": "COMMON"
}]
}],
"documentSentiment": {
"magnitude": 0.1,
"score": 0.1
},
"language": "en",
"categories": []
}
"zippie" detected fine
{
"sentences": [{
"text": {
"content": "zippie",
"beginOffset": 0
},
"sentiment": {
"magnitude": 0,
"score": 0
}
}],
"tokens": [],
"entities": [{
"name": "zippie",
"type": "OTHER",
"metadata": [],
"salience": 1,
"mentions": [{
"text": {
"content": "zippie",
"beginOffset": 0
},
"type": "PROPER"
}]
}],
"documentSentiment": {
"magnitude": 0,
"score": 0
},
"language": "en",
"categories": []
}
"elmo" detected fine
{
"sentences": [{
"text": {
"content": "elmo",
"beginOffset": 0
},
"sentiment": {
"magnitude": 0.1,
"score": 0.1
}
}],
"tokens": [],
"entities": [{
"name": "elmo",
"type": "OTHER",
"metadata": [],
"salience": 1,
"mentions": [{
"text": {
"content": "elmo",
"beginOffset": 0
},
"type": "COMMON"
}]
}],
"documentSentiment": {
"magnitude": 0.1,
"score": 0.1
},
"language": "en",
"categories": []
}
Services like these are trained on a specific corpus of 'entity' values.
The service tokenizes/chunks, then uses part of speech tagging to identify noun phrases and checks against a giant index to see if that noun phrase is an entity.
Zippy must not be in the corpus. Not sure about google NLP, but Watson NLU comes with a GUI product for easily creating your own 'dictionary' of entity noun phrases.
Also very possible to create your own using NLTK or from scratch in python, but all require the effort of manually curating your own 'dictionary', unless you are able to get your hands on and adapt another.
How do I positive-boost the absence of certain terms? I've asked this question before here but the response was not satisfactory because it wasn't generalizable enough.
Lets try again, with more nuances.
I want to be able to distinguish laptops from their accessories. In human language this is done by the absense of terms. That is, when you say lenovo thinkpad you know that by omitting the word battery you mean you want the actual laptop. Compare this with when a person says lenovo thinkpad battery, where they mean the battery.
So suppose we have the index:
PUT test_index
{
"settings": {
"index": {
"number_of_shards": 1,
"number_of_replicas": 1
}
}
}
with mapping:
PUT test_index/_mapping/merchant
{
"properties": {
"title": {
"type": "string"
},
"category": {
"type": "string",
"index": "not_analyzed"
}
}
}
put two items into it:
PUT test_index/merchant/3
{
"title": "macbook battery",
"category": "laptops accessories"
}
PUT test_index/merchant/2
{
"title": "lenovo thinkpad battery",
"category": "laptops accessories"
}
PUT test_index/merchant/1
{
"title": "lenovo thinkpad white/black",
"category": "laptops"
}
Now search lenovo thinkpad:
POST test_index/_search
{
"query":{
"match": { "title": "lenovo thinkpad" }
}
}
The result is:
"hits": [
{
"_index": "test_index",
"_type": "merchant",
"_id": "2",
"_score": 0.70710677,
"_source": {
"title": "lenovo thinkpad battery",
"category": "laptops accessories"
}
},
{
"_index": "test_index",
"_type": "merchant",
"_id": "1",
"_score": 0.70710677,
"_source": {
"title": "lenovo thinkpad white/black",
"category": "laptops"
}
}
]
where notice that lenovo thinkpad battery is higher up than lenovo thinkpad white/black.
Now, I can see at least two reasonable ways to do this.
A) Use term frequency on a per-category basis to influence relevance of title match. For example, if for each category you extract the 95% percentile terms, you get that battery is a high frequency term in laptops accessories and so the word battery should be negative-boosted on all title queries.
B) Use term frequency on a per-category basis to influence relevance of category match. For example, in addition of the title match, you automatically negative-boost results whose categories have 95% percentile terms which aren't contained in your title match.
A and B aren't quite the same, but they both rely on the idea that certain absent words should be taken into account for relevance.
So...... thoughts?
My vote would be
C)
Fix the categories so that a battery doesn't have 'laptops' as a category (it's a 'laptopAccessory' or just 'accessory') Alternatively create an additional category (not called 'laptops') to indicate the actual machines themselves.
In your search, instead of trying to down-rank the accessories, you apply a boost to the 'laptops' category (no longer ambiguous). This will cause initial searches as in your example of 'lenovo thinkpad' to bring the actual machines up above the accessories. A more precise search ('lenovo thinkpad battery') will still work as you'd expect also.
Another nice UI/UX experience is to take the total set of categories returned in your results, and provide easy filter links. So if your initial search returns 'laptops' 'accessories' 'payment plans', then you'd have each of those as a link to a re-query that uses the original search plus a filter on that category.
Good luck!
Boost "that" category.
GET /test_index/merchant/_search
{
"from": 0,
"query": {
"bool": {
"must": [
{"match": {"title": "lenovo thinkpad"}}
],
"should": [
{
"match": {
"category": {
"boost": "2",
"query": "laptops"
}
}
}
]
}
},
"size": "10"
}
Result:
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 2,
"max_score": 1.573319,
"hits": [
{
"_index": "index",
"_type": "merchant",
"_id": "1",
"_score": 1.573319,
"_source": {
"title": "lenovo thinkpad white/black",
"category": "laptops"
}
},
{
"_index": "index",
"_type": "merchant",
"_id": "2",
"_score": 0.15889977,
"_source": {
"title": "lenovo thinkpad battery",
"category": "laptops accessories"
}
}
]
}
}
More on boosting, can be found here
We can update the absence of certain terms using boost property which was provided while query for that term.
Please check below query with boost property set to 10.
GET /test_index/students/_search
{
"from": 0,
"query": {
"bool": {
"must": [
{"match": {"age": "20"}}
],
"should": [
{
"match": {
"category": {
"boost": "10",
"query": "students"
}
}
}
]
}
},
"size": "10"
}