I am working on a new Azure Speech Cognitive Services Custom Commands based application. I have worked through most of the issues I have run into, but have been unable to select the Custom Voice used for text to speech output. I am able to test the voice in Speech Studio but the item/row selection does not persist. I am able to select a different voice programmatically when using SpeechConfig (outside of the Custom Command app) but do not have that option when using CustomCommandConfig with the Custom Commands app.
Working example using SpeechConfig for direct TTS:
var speechConfig = SpeechConfig.FromSubscription(_voiceConfig.AzureSpeechResourceId, _voiceConfig.AzureRegion);
var voiceName = "Microsoft Server Speech Text to Speech Voice (en-US, CoraNeural)";
speechConfig.SpeechSynthesisVoiceName = voiceName;
Config code for Custom Command application connection (no way to select voice):
var speechConfig = CustomCommandsConfig.FromSubscription(_voiceConfig.AzureApplicationId, _voiceConfig.AzureSpeechResourceId, _voiceConfig.AzureRegion);
speechConfig.SpeechSynthesisVoiceName = "<<<<-------THIS PROPERTY DOES NOT EXIST";
Screenshot of Custom Command UI within Speech Studio, where selection does not persist:
Custom Commands Screenshot
Any help or ideas you might have would be greatly appreciated!
Thanks in advance!
Related
Please I have struggling with consuming watson assistant log API. currently we use watson assistant log API for fetching daily conversation log between our chatbot and customer and feed the log into NLC for classifying the intents for precision analyst. Recently we observed that IBM has decomission the NLC and advice that we migrate to NLU. since then we've being having issues getting the component running.
right now the assistant log return empty log for this filter
authenticator = IAMAuthenticator('********')
assistant = AssistantV2(
version='2021-06-14',
authenticator = authenticator
)
assistant.set_service_url('https://api.eu-gb.assistant.watson.cloud.ibm.com')
print("The assistant URL ")
response=assistant.list_logs(
assistant_id='******', page_limit=100, filter='response_timestamp>2022-09-14T00:00:00.000Z',
cursor=cursor).get_result()
response=assistant.list_logs(
assistant_id='*******').get_result()
print(json.dumps(response, indent=2))
#print("the response is ",response)
this is returning an empty data
{
"logs": [],
"pagination": {}
}
Here are some reasons why this may occur.
You are using V2 API which requires an Enterprise account. If you only have Plus then you can use the V1 API. But there is no guarantee it will continue to work as expected in the future.
You don't have any logs for that date range. Check in the Analytics page.
An issue in your code. Try running the sample code on the REST API docs. If this works then compare against your code.
Your filter looks ok, but try this filter (change language and assistant ID):
language::en,request.context.system.assistant_id::<YOUR_ASSISTANT_ID>,response_timestamp>2022-09-14T00:00:00.000Z
Seeing as you have midnight as your time, you can also change that part to:
response_timestamp>2022-09-14
Working with the Azure Web App Bot SDK v3.
I try to output text and speech at the same time.
messageActivity.Text = information;
messageActivity.Speak = information;
messageActivity.Locale = "de-DE";
await context.PostAsync(messageActivity);
Cortana nor Direct Line nor the Bot emulator does speak something out. However, the Bot does receive the text to speak out loud.
In addition, even
await context.SayAsync(information, information);
isn't working. I seems like there is an issue with the localization or something. I ran out of ideas.
Direct line is configured as following
const speechOptions = {
speechRecognizer: new CognitiveServices.SpeechRecognizer({ subscriptionKey: 'SUB_KEY_XXX', locale: 'de-DE' }),
speechSynthesizer: new CognitiveServices.SpeechSynthesizer({
gender: CognitiveServices.SynthesisGender.Male,
subscriptionKey: 'SUB_KEY_AGAIN',
voiceName: 'Microsoft Server Speech Text to Speech Voice (de-DE, Michael)'
})
};
Sidenote: Voice to text works flawlessly.
EDIT: Direct Line does work now. While I was using a iFrame for demonstration purposes, the voice output only works if the input also was provided via voice input. However, you can change that behaviour as well.
You should try a well formed SSML wrapper if you want the informationText to speak in Cortana.
msg.speak = "<speak version=\"1.0\" xmlns=\"http://www.w3.org/2001/10/synthesis\" xml:lang=\"de-DE\"> <voice languages=\"de-DE\" gender=\"male\"> Haben Sie Wien schon bei Nacht gesehen? </voice><voice languages=\"de-DE it\" gender=\"female\"> Nein. </voice></speak>
Also note that Cortana only officially supports en-US for locale and market for third party skills. Though you can do a lot of cool things with other voices and locales, there are a few quirks and you can run into issues.
I have successfully hosted an instance of microsoft's Botframework web chat using directline on public domain, I want to make a chatbot in such a way that my customers can have their own channels completely separated from each other and I cannot find any kind of documentation anywhere, Kindly suggest me if this is possible and how?
I have written the complete code in Node.js and have very less idea about c#.
It seems that there is no such feature for uniform customized chat channel in bot framework. So we can leverage new builder.Message().address(address) to send messages to specific users from the official sample at https://github.com/Microsoft/BotBuilder-Samples/blob/master/Node/core-proactiveMessages/simpleSendMessage/index.js.
So I had a quick test which will save the users' addresses into an address list in server memory as a "customize channel", and trigger a key work to send message to these addresses in list as a broadcast in this "customize channel":
let channel_address = [];
bot.dialog('joinChannel',(session)=>{
channel_address.push(session.message.address);
}).triggerAction({
matches:/join/i
})
bot.dialog('broadcast',(session)=>{
channel_address.forEach((address)=>{
bot.send(
new builder.Message(session).address(address).text(session.message.text)
)
})
}).triggerAction({
matches:/^broadcast: .*/
})
Test Step:
Open two emulators connect to your local bot
in both emulators, type "join"
in either emulator, type text like broadcast: hi there
I'm developing a chatbot on azure using node.js. It's a data visualization bot which generates chart in html format using d3 library and display to user.
It seems that Microsoft bot builder doesn't support html format. But I have looked through this link:
https://blog.botframework.com/2017/09/07/html-not-supported-web-chat/
It says that there is a way to enable html content:
"If HTML rendering in Web Chat is a critical feature for your applications, you can clone or fork a copy of the Web Chat source code from GitHub, and enable it (on your own custom Web Chat client)."
I tried to clone the file and changed ‘html : false’ to ‘html : true’. But it's not working.
Can anyone tell me what I can do? Really appreciate it!!!
Depending on what data you are attempting to visualize, you might be able to use a service like Google Image Charts: https://developers.google.com/chart/image/docs/chart_playground
Using this service, with the following code:
// attach the card to the reply message
var msg = new builder.Message(session).addAttachment(createHeroCard(session));
session.send(msg);
function createHeroCard(session) {
return new builder.HeroCard(session)
.title('Months with Numbers Bar Chart')
.subtitle('Using a Chart as Image service...')
.text('Build and connect intelligent bots that have charts rendered as images.')
.images([
builder.CardImage.create(session, 'http://chart.googleapis.com/chart?cht=bvg&chs=250x150&chd=s:Monkeys&chxt=x,y&chxl=0:|Jan|Feb|Mar|Apr|May|Jun|Jul')
])
.buttons([
builder.CardAction.openUrl(session, 'https://learn.microsoft.com/bot-framework/', 'Get Started')
]);
}
Produces this hero card:
I'm trying to make a bot for Facebook Messenger using Microsoft's Bot Framework that will do this:
Get a user's voice message sent via Facebook Messenger
Convert speech to text
Do something with it
There's no problem with getting the voice message from Messenger (the URL can be extracted from the message the bot receives), and there's also no problem with converting an audio file to speech (using Bing Speech API or Google's similar API).
However, these APIs require PCM (WAV) files, while Facebook Messenger gives you an MP4 file.
Is there a popular/standard way of converting one format into another that is used in writing the bots?
So far my best idea is to run vlc.exe as a console job on my server and convert the file, but that doesn't sound like the best solution.
Developed a solution that works as follows:
Receive voice message from facebook
Download the MP4 file to local disk using the link inside Activity.Attachments
Use MediaToolKit (wrapper for FFMPEG) to convert MP4/AAC to WAV on local server
Send the WAV to Bing Speech API
So the answer to my question is: use MediaToolKit+ffmpeg to convert the file format.
Sample implementation and code here: https://github.com/J3QQ4/Facebook-Messenger-Voice-Message-Converter
public string ConvertMP4ToWAV()
{
var inputFile = new MediaFile { Filename = SourceFileNameAndPath };
var outputFile = new MediaFile { Filename = ConvertedFileNameAndPath };
using (var engine = new Engine(GetFFMPEGBinaryPath()))
{
engine.Convert(inputFile, outputFile);
}
return ConvertedFileNameAndPath;
}