Working with the Azure Web App Bot SDK v3.
I try to output text and speech at the same time.
messageActivity.Text = information;
messageActivity.Speak = information;
messageActivity.Locale = "de-DE";
await context.PostAsync(messageActivity);
Cortana nor Direct Line nor the Bot emulator does speak something out. However, the Bot does receive the text to speak out loud.
In addition, even
await context.SayAsync(information, information);
isn't working. I seems like there is an issue with the localization or something. I ran out of ideas.
Direct line is configured as following
const speechOptions = {
speechRecognizer: new CognitiveServices.SpeechRecognizer({ subscriptionKey: 'SUB_KEY_XXX', locale: 'de-DE' }),
speechSynthesizer: new CognitiveServices.SpeechSynthesizer({
gender: CognitiveServices.SynthesisGender.Male,
subscriptionKey: 'SUB_KEY_AGAIN',
voiceName: 'Microsoft Server Speech Text to Speech Voice (de-DE, Michael)'
})
};
Sidenote: Voice to text works flawlessly.
EDIT: Direct Line does work now. While I was using a iFrame for demonstration purposes, the voice output only works if the input also was provided via voice input. However, you can change that behaviour as well.
You should try a well formed SSML wrapper if you want the informationText to speak in Cortana.
msg.speak = "<speak version=\"1.0\" xmlns=\"http://www.w3.org/2001/10/synthesis\" xml:lang=\"de-DE\"> <voice languages=\"de-DE\" gender=\"male\"> Haben Sie Wien schon bei Nacht gesehen? </voice><voice languages=\"de-DE it\" gender=\"female\"> Nein. </voice></speak>
Also note that Cortana only officially supports en-US for locale and market for third party skills. Though you can do a lot of cool things with other voices and locales, there are a few quirks and you can run into issues.
Related
I am working on a new Azure Speech Cognitive Services Custom Commands based application. I have worked through most of the issues I have run into, but have been unable to select the Custom Voice used for text to speech output. I am able to test the voice in Speech Studio but the item/row selection does not persist. I am able to select a different voice programmatically when using SpeechConfig (outside of the Custom Command app) but do not have that option when using CustomCommandConfig with the Custom Commands app.
Working example using SpeechConfig for direct TTS:
var speechConfig = SpeechConfig.FromSubscription(_voiceConfig.AzureSpeechResourceId, _voiceConfig.AzureRegion);
var voiceName = "Microsoft Server Speech Text to Speech Voice (en-US, CoraNeural)";
speechConfig.SpeechSynthesisVoiceName = voiceName;
Config code for Custom Command application connection (no way to select voice):
var speechConfig = CustomCommandsConfig.FromSubscription(_voiceConfig.AzureApplicationId, _voiceConfig.AzureSpeechResourceId, _voiceConfig.AzureRegion);
speechConfig.SpeechSynthesisVoiceName = "<<<<-------THIS PROPERTY DOES NOT EXIST";
Screenshot of Custom Command UI within Speech Studio, where selection does not persist:
Custom Commands Screenshot
Any help or ideas you might have would be greatly appreciated!
Thanks in advance!
Following this guide:
https://actions-on-google.github.io/actions-on-google-nodejs/
I created an action for DialogFlow
import { dialogflow, Image, Conversation, BasicCard } from 'actions-on-google';
const app = dialogflow();
app.intent('test', (conv, input) => {
conv.contexts.set('i_see_context_in_web_demo', 1);
conv.ask(`i see this only into actions on google simulator`);
conv.ask(new Image({
url: 'https://developers.google.com/web/fundamentals/accessibility/semantics-builtin/imgs/160204193356-01-cat-500.jpg',
alt: 'cat',
}));
});
I then activated Web Demo integration
I saw that the Web Demo integration does not show the cards, the images. I hypothesize that it only shows text, no rich text
I understand that it elaborates only JSON like this:
{
"fulfillmentText": "Welcome!",
"outputContexts": []
}
But I did not find any method in the library used to enhance fulfillmentText
can you help me?
You're using the actions-on-google library, which is specifically designed to send messages that will be used by the Google Assistant. The Web Demo uses the generic messages that are available for Dialogflow. The actions-on-google library does not send these generic messages.
If you want to be able to create messages that are usable by both, you'll need to look into the dialogflow fulfillment library, which can create messages that are usable by the Google Assistant as well as other platforms. Be aware, however, that not all rich messages are available on all platforms, but that the basic text responses should be.
You also don't need to use a library - you can create the JSON response yourself.
I've created Bot using Microsoft's bot services ( Bot framework) using nodeJs.
Its working fine when for plain text, but when i tried to return HTML table using below method
eg.
var tableHTML = '<table style="padding:20px;border:1px solid black; border-collapse: collapse;" border=1 border-collapse=collapse><tr style="background-color:#c6c6c6"><th>Countries</th><th>Capitals</th><th>Population</th><th>Language</th></tr><tr><td>USA</td><td> Washington D.C.</td><td>309 million</td><td>English</td></tr><tr><td>Sweden</td><td>Stockholm</td><td>9 million</td><td>Swedish</td></tr></table>';
var message = {
type: 'message',
textFormat: 'xml',
text: tableHTML
};
session.send(message);
It worked for me on skype for business desktop APP , but not working on skype for business mobile APP ( it returns nothing )
Currently i'm facing issue while rending HTML table for mobile devices on skype for business
Skype for business mobile app supports HTML in bot ??
Please provide if anyone have solution for this
Thanks
Skype for business mobile clients do not support HTML or markdown formats. This is a pain point for many customers I encourage you to reach out to them via their uservoice and request this feature.
Format rendering will be channel-specific and you need to determine what formats a channel supports ahead of time.
I am building a bot service to communicate to various endpoints. I have Skype basic communication running through Microsoft Botbuilder in Nodejs. I can send text to the Skype client fine, however when I want to send rich messages with images (animated gifs) or videos, things start going haywire.
1) Sending a animated gif:
function createAnimationCard(session) {
return new builder.AnimationCard(session)
.title('Microsoft Bot Framework')
.subtitle('Animation Card')
.image(builder.CardImage.create(session, 'https://docs.botframework.com/en-us/images/faq-overview/botframework_overview_july.png'))
.media([
{ url: 'http://i.giphy.com/Ki55RUbOV5njy.gif' }
]);
}
I call this code from another function:
exports.sendGif = function sendGif(chatBotUserSession, picUrl, contentType) {
var card = createAnimationCard(chatBotUserSession);
var reply = new builder.Message(chatBotUserSession).addAttachment(card);
_bot.send(reply);
};
When I do this I get to see an image with a play button (in the desktop Skype, mobile client does not show image at all), however pressing the button shows an error "opening not possible". I have tried different gif sources in the url field, but none seem to work. Also tried https for source.
2) I then moved to trying to display a video:
function createVideoCard(session) {
return new builder.VideoCard(session)
.title('Big Buck Bunny')
.subtitle('by the Blender Institute')
.text('Big Buck Bunny (code-named Peach) is a short computer-animated comedy film by the Blender Institute, part of the Blender Foundation. Like the foundation\'s previous film Elephants Dream, the film was made using Blender, a free software application for animation made by the same foundation. It was released as an open-source film under Creative Commons License Attribution 3.0.')
.image(builder.CardImage.create(session, 'https://upload.wikimedia.org/wikipedia/commons/thumb/c/c5/Big_buck_bunny_poster_big.jpg/220px-Big_buck_bunny_poster_big.jpg'))
.media([
{ url: 'http://download.blender.org/peach/bigbuckbunny_movies/BigBuckBunny_320x180.mp4' }
])
.buttons([
builder.CardAction.openUrl(session, 'https://peach.blender.org/', 'Learn More')
]);
}
When I call this using:
exports.sendMovie = function sendMovie(chatBotUserSession, picUrl, contentType) {
var card = createVideoCard(chatBotUserSession);
var reply = new builder.Message(chatBotUserSession).addAttachment(card);
_bot.send(reply);
};
It all works... at least on my desktop Skype. My mobile skype (iOS) now crashes each time when I open the chat with my bot...
So two questions:
1) Does anyone know why I can't seem to send animated gifs that actually play?
2) Anyone have experience with bots on mobile Skype sending rich content and consequently crashing the app?
Thanks.
I'm trying the chrome text-to-speech API but even the demo provided by google
https://developer.chrome.com/trunk/extensions/examples/extensions/ttsdemo/ttsdemo.html
doesn't work for me, I can't hear any sound, do you?
I don't think it is a problem of my browser because google.translate.com (which I guess is based on the same technology) works for me if I try the listening mode.
Any idea?
Thanks
As of Chrome 33, Chrome's speech synthesis API is available in JavaScript.
Quick example:
window.speechSynthesis.speak(
new SpeechSynthesisUtterance('Oh why hello there.')
);
Details:
HTML5 Rocks: Introduction to the Speech Synthesis API
. . Hi, Eugenio.
. . This API is only available for extensions. You can port your logic to inside an extension (people would have to install it to use, of course), create an extension that exposes the functions to the "outside world" (people would still need to install the extension to use your app correctly) or simply use a client-side synthesizer (speak.js, for example).
. . You can use WebAudio API (or event tags) and calls to the Google Translate TTS endpoint, but that's not a Public API and it has no guarantees. It can simply stop working because of some limitation from Google, they can change the API or endpoints and yadda yadda. If it's only for testing, that'll probably do, but if it's a bigger project (or a comercial one), I strongly advise against that option.
. . Good luck.
Today (October 2015) there's 55% devices that have Speech Synthesis API support: http://caniuse.com/#feat=speech-synthesis
Here's the example:
// Create the utterance object
var utterance = new SpeechSynthesisUtterance();
utterance.text = 'Hello, World!';
// optional parameters
utterance.lang = 'en-GB'; // language, default is 'en-US'
utterance.volume = 0.5; // volume, from 0 to 1, default is 1
utterance.rate = 0.8; // speaking rate, default is 1
// speak it!
window.speechSynthesis.speak(utterance);
Just to add some links because I also was lost finding the right information.
You can use the so called "speech synthesis api" of Chrome, see demo: https://www.audero.it/demo/speech-synthesis-api-demo.html
Further info:
Talking Web Pages and the Speech Synthesis API
Getting Started with the Speech Synthesis API
Using HTML5 Speech Recognition and Text to Speech
Hope that helps, and hope the links will survive the future.