How do i implement watson text to speech while running a chatbot on a local host using nodejs?
My chatbot is already running on a localhost..I want to embed watson text to speech service. I have read that it could be done by websocket interfacing. I don t have any idea on that
Assuming that you have the Conversation Simple example built by IBM Developers using Node.js and Conversation Service, you can simply have your app submit an HTTP REST request by following this tutorial using Websocket or you can leverage a language-specific SDK, I'll paste in the links below.
So, a few months ago the #kane built one example that integrates the conversation simple example with text to speech, you can easily found them in this link.
You can check this commit for saw the changes and follow the logic to implement Text to Speech in your application. You'll see this code above calling the Text to Speech service with the Services credentials in the .env file, like the comments in the code:
const TextToSpeechV1 = require('watson-developer-cloud/text-to-speech/v1');
const textToSpeech = new TextToSpeechV1({
// If unspecified here, the TEXT_TO_SPEECH_USERNAME and
// TEXT_TO_SPEECH_PASSWORD env properties will be checked
// After that, the SDK will fall back to the bluemix-provided VCAP_SERVICES environment property
// username: '<username>',
// password: '<password>',
});
app.get('/api/synthesize', (req, res, next) => {
const transcript = textToSpeech.synthesize(req.query);
transcript.on('response', (response) => {
if (req.query.download) {
if (req.query.accept && req.query.accept === 'audio/wav') {
response.headers['content-disposition'] = 'attachment; filename=transcript.wav';
} else {
response.headers['content-disposition'] = 'attachment; filename=transcript.ogg';
}
}
});
transcript.on('error', next);
transcript.pipe(res);
});
// Return the list of voices
app.get('/api/voices', (req, res, next) => {
textToSpeech.voices(null, (error, voices) => {
if (error) {
return next(error);
}
res.json(voices);
});
});
Obs.: I suggest to you see the Commit and follow the same logic to make your changes in your app.
Node SDK for Watson Services.
API Reference for using Text to Speech with Node.js
Related
I have a Webhook that delivers a complex JSON payload to my Cloud Function URL, and writes that JSON to collections & documents within my Cloud Firestore.
I believe the Node.JS Runtime on Google Cloud Functions uses the Express Middleware HTTP framework.
I have a WooCommerce Webhook that wants me to send a JSON to a URL, I believe this is a POST http request.
I've used Webhook.site to test the Webhook, and it is displaying the correct JSON payload.
Developers have suggested I use cloud functions to receive the JSON, parse the JSON and write it to the Cloud Firestore.
// cloud-function-name = wooCommerceWebhook
exports.wooCommerceWebhook = functions.https.onRequest(async (req, res) => {
const payload = req.body;
// Write to Firestore - People Collection
await admin.firestore().collection("people").doc().set({
people_EmailWork: payload.billing.email,
});
// Write to Firestore - Volociti Collection
await admin.firestore().collection("volociti").doc("fJHb1VBhzTbYmgilgTSh").collection("orders").doc("yzTBXvGja5KBZOEPKPtJ").collection("orders marketplace orders").doc().set({
ordersintuit_CustomerIPAddress: payload.customer_ip_address,
});
// Write to Firestore - Companies Collection
await admin.firestore().collection("companies").doc().set({
company_AddressMainStreet: payload.billing.address_1,
});
return res.status(200).end();
});
I have the logs to my cloud function's failure to deploy if that is helpful.
Function cannot be initialized. Error: function terminated. Recommended action: inspect logs for termination reason. Additional troubleshooting documentation can be found at https://cloud.google.com/functions/docs/troubleshooting#logging
My package.json:
{
"name": "sample-http",
"version": "0.0.1"
}
You need to correctly define the dependency with the Firebase Admin SDK for Node.js, and initialize it, as shown below.
You also need to change the way you declare the function: exports.wooCommerceWebhook = async (req, res) => {...} instead of exports.wooCommerceWebhook = functions.https.onRequest(async (req, res) => {...});. The one you used is for Cloud Functions deployed through the CLI.
package.json
{
"name": "sample-http",
"version": "0.0.1",
"dependencies": { "firebase-admin": "^9.4.2" }
}
index.js
const admin = require('firebase-admin')
admin.initializeApp();
exports.wooCommerceWebhook = async (req, res) => { // SEE COMMENT BELOW
const payload = req.body;
// Write to Firestore - People Collection
await admin.firestore().collection("people").doc().set({
people_EmailWork: payload.billing.email,
});
// Write to Firestore - Volociti Collection
await admin.firestore().collection("volociti").doc("fJHb1VBhzTbYmgilgTSh").collection("orders").doc("yzTBXvGja5KBZOEPKPtJ").collection("orders marketplace orders").doc().set({
ordersintuit_CustomerIPAddress: payload.customer_ip_address,
});
// Write to Firestore - Companies Collection
await admin.firestore().collection("companies").doc().set({
company_AddressMainStreet: payload.billing.address_1,
});
return res.status(200).end();
};
I have a Bot running on botbuilder V3 where I am using a middleware explained here to intercept the messages.
bot.use({
botbuilder: function (session, next) {
myMiddleware.logIncomingMessage(session, next);
},
send: function (event, next) {
myMiddleware.logOutgoingMessage(event, next);
}
})
We are planning to migrate on sdk v4 so looking for similar capabilities in sdk v4. Are there any ?
I didn't find example on this page.
The BotAdapter base class exposes the use method to register middleware. So in your startup logic you'll create a specific implementation of a BotAdapter, typically BotFrameworkAdapter, and then add the middleware to it. Like so:
const botAdapter = new BotFrameworkAdapter( { /* credential stuff here*/ });
// Simple handler based
botAdapter.use(async (turnContext, next) => {
// pre logic
await next();
// post logic
});
// Or class based
botAdapter.use(new MyMiddleware());
i am struggling a bit with my google assistant action. Right now i am using Dialogflow and Firebase for my webhook. In my code i would like to get data from an API, for example this one: API. I am coding with Node.js by the way. Since Node is asynchronous i do not know how to get the Data. When i try to make an Callback it doesnt work e.g.:
app.intent(GetData, (conv) => {
var test= "error";
apicaller.callApi(answer =>
{
test = answer.people[0].name
go()
})
function go ()
{
conv.ask(`No matter what people tell you, words and ideas change the world ${test}`)
}
For some reason this works when i test it in an other application. With Dialogflow it does not work
I have also tried to use asynch for the function app.intent and tried it with await but this did not work too.
Do you have any idea how i could fix this?
Thank you in advance and best regards
Luca
You need to return Promise like
function dialogflowHanlderWithRequest(agent) {
return new Promise((resolve, reject) => {
request.get(options, (error, response, body) => {
JSON.parse(body)
// processing code
agent.add(...)
resolve();
});
});
};
See following for details:
Dialogflow NodeJs Fulfillment V2 - webhook method call ends before completing callback
If this works in another application then I believe you're getting an error because you're attempting to access an external resource while using Firebases free Spark plan, which limits you to Google services only. You will need to upgrade to the pay as you go Blaze plan to perform Outbound networking tasks.
Due to asynchronicity, the function go() is gonna be called after the callback of callapi been executed.
Although you said that you have tried to use async, I suggest the following changes that are more likely to work in your scenario:
app.intent(GetData, async (conv) => {
var test= "error";
apicaller.callApi(async answer =>
{
test = answer.people[0].name
await go()
})
async function go ()
{
conv.ask(`No matter what people tell you, words and ideas change the world ${test}`)
}
First follow the procedure given in their Github repository
https://github.com/googleapis/nodejs-dialogflow
Here you can find a working module already given in the README file.
You have to make keyFilename object to store in SessionsClient object creation (go at the end of post to know how to genearate credentials file that is keyFileName). Below the working module.
const express = require("express");
const bodyParser = require("body-parser");
const app = express();
app.use(bodyParser.urlencoded({ extended: true }));
app.use(bodyParser.json());
//////////////////////////////////////////////////////////////////
const dialogflow = require("dialogflow");
const uuid = require("uuid");
/**
* Send a query to the dialogflow agent, and return the query result.
* #param {string} projectId The project to be used
*/
async function runSample(projectId = "<your project Id>") {
// A unique identifier for the given session
const sessionId = uuid.v4();
// Create a new session
const sessionClient = new dialogflow.SessionsClient({
keyFilename: "<The_path_for_your_credentials_file>"
});
const sessionPath = sessionClient.sessionPath(projectId, sessionId);
// The text query request.
const request = {
session: sessionPath,
queryInput: {
text: {
// The query to send to the dialogflow agent
text: "new page",
// The language used by the client (en-US)
languageCode: "en-US"
}
}
};
// Send request and log result
const responses = await sessionClient.detectIntent(request);
console.log("Detected intent");
console.log(responses);
const result = responses[0].queryResult;
console.log(` Query: ${result.queryText}`);
console.log(` Response: ${result.fulfillmentText}`);
if (result.intent) {
console.log(` Intent: ${result.intent.displayName}`);
} else {
console.log(` No intent matched.`);
}
}
runSample();
Here you can obtain the keyFileName file that is the credentials file using google cloud platform using
https://cloud.google.com/docs/authentication/getting-started
For complete tutorial (Hindi language) watch youtube video:
https://www.youtube.com/watch?v=aAtISTrb9n4&list=LL8ZkoggMm9re6KMO1IhXfkQ
I want to create a skill for Amazon Alexa that - when triggered by voice commands - gets some information from an API via a HTTPS request and uses the result as spoken answer to the Alexa user.
There is a little challenge here (especially for node.js newbies) due to the event-driven concept of node.js and the internals of the Alexa Skills Kit for Node.js. And getting a hand on parameters from the user isn't that easy, either.
Can somebody provide some sample code to start with?
Preliminaries
To get started you need an Amazon account, and you must enable AWS for the account.
Then there is a nice step-by-step guide on the Amazon Website: https://developer.amazon.com/edw/home.html#/skills
It walks you through step-by-step through the process of creating a "skill". A skill is the ability for Alexa to answer questions in natural language.
During this process you also create a Lambda function (select to create one of the demo script applications, and you get all required libraries automatically)
Then you can then edit the code in the WebUI of the AWS Console).
The "skill" is automatically enabled on all your personal Alexa Devices, like my Amazon Echo dot at home.
remember that you can look at the console output in your AWS Cloudwatch section of the AWS console.
The two things I had to understand (and that others may run into, too)
While I created my first Alexa Skill I was new node.js, Lambda and the Alexa Skills SDK. So I ran into a few problems. I'd like to document the solutions here for the next person who runs into the same problem.
When you make an http get request in node.js using https.get() you need to provide a handler for the end callback like res.on('end', function(res) {});
The answer is sent back from the Lambda script to the Alexa Service when you call this.emit(':tell', 'blabla'); (this is what is used in the samples from AWS). But in the end-handler "this" isn't the right "this" anymore, you need to store the handle beforehand (I am doing this a little crookedly using mythis, I am sure there are smarter solutions, but it works).
I would have easily saved two hours of debugging had I had the following code snippet. :-)
The code
I my sample the lambda script already gets the preformatted text from the API. If you call an XML/JSON or whatever API you need to work with the answer a bit more.
'use strict';
const Alexa = require('alexa-sdk');
var https = require('https');
const APP_ID = ''; // TODO replace with your app ID (OPTIONAL).
const handlers = {
'functionwithoutdata': function() {
var responseString = '';
var mythis = this;
https.get('**YOURURL**?**yourparameters**', (res) => {
console.log('statusCode:', res.statusCode);
console.log('headers:', res.headers);
res.on('data', (d) => {
responseString += d;
});
res.on('end', function(res) {
const speechOutput = responseString;
console.log('==> Answering: ', speechOutput);
mythis.emit(':tell', 'The answer is'+speechOutput);
});
}).on('error', (e) => {
console.error(e);
});
},
'functionwithdata': function() {
var mydata = this.event.request.intent.slots.mydata.value;
console.log('mydata:', mydata);
var responseString = '';
var mythis = this;
https.get('**YOURURL**?**yourparameters**&mydata=' + mydata, (res) => {
console.log('statusCode:', res.statusCode);
console.log('headers:', res.headers);
res.on('data', (d) => {
responseString += d;
});
res.on('end', function(res) {
const speechOutput = responseString;
console.log('==> Answering: ', speechOutput);
mythis.emit(':tell', 'The answer is'+speechOutput);
});
}).on('error', (e) => {
console.error(e);
});
}
};
exports.handler = (event, context) => {
const alexa = Alexa.handler(event, context);
alexa.APP_ID = APP_ID;
alexa.registerHandlers(handlers);
alexa.execute();
};
How to create an Amazon Alexa bot from scratch?
If you’re looking for a way to create an Alexa voice enabled bot then you’re on the right place!
Let’s create an Amazon Alexa bot from scratch using node server running on our localhost and tunneled through ngrok.
Sign up for an Amazon developer account, if you don’t have one
Go to Alexa developer page
Go to the Alexa console
Click on Create skill
Give a name to the skill, I have named mine TestSkill and click on Next
Choose a model to add to your skill, I’ve selected custom for my experiments
Click on Create skill
This way you reach to the Alexa skill dashboard
Provide an invocation name, I’ve named it “give me pizza” and click on Save Model
Click on the Endpoint
Now, we need to provide the endpoint to Alexa console but first we need to setup an endpoint.
Creating a node server
Create a server which can accept POST requests on default location i.e. “/”.
There are so many techniques for creating a server, I personally prefer node.
I assume that you’ve node, npm and Visual studio code already installed
For the specific requirements of this tutorial, we will create a Hello World node app following the steps below:
Run npm init on a terminal and when asked for the package name Alexa
Follow the terminal wizard
Run cd Alexa
In the package.json file
Run npm i express http -save and this will add the following entry in the package.json file:
“dependencies”: {
“express”: “4.16.3”,
“http”: “0.0.0”
}
Set value to index.js of the main key in the package.json file
Add a file index.js on same level
Àdd the following code to the index.js file:
const express = require('express');
const app = express();
app.post('/', (req, res) =>
res.send({
version: '1.0',
response: {
shouldEndSession: false,
outputSpeech: {
type: 'SSML',
text: 'Hello World!',
ssml: 'Hello World!'
}
}
}));
app.listen(8080, () => console.log('Example app listening on port 8080!'));
Set value of scripts to { “start”: “node index.js” }
Run npm start on the terminal
Tunnel your localhost
Add a tunnel to the localhost on PORT 8080 running the node server using ngrok following the below steps:
Download and install ngrok, if not already done
Run ngrok http 8080 on a terminal
Copy the SSL enabled forwarded link, in my case https://6d0d6e60.ngrok.io is the SSL link
provide the link in the Enter URL link
Select HTTPS, and under SSL certificate type drop down select the 2nd option:
My development endpoint is a sub-domain of a domain that has a wildcard certificate from a certificate authority
Click Save Endpoints
Click on JSON editor and provide the following model:
{
"interactionModel": {
"languageModel": {
"invocationName": "get me pizza",
"intents": [
{
"name": "PizzaOrderIntent",
"slots": [],
"samples": [
"Give me a pizza"
]
},
{
"name": "AMAZON.FallbackIntent",
"samples": [
"I didn't understand"
]
},
{
"name": "AMAZON.CancelIntent",
"samples": [
"cancel plz"
]
},
{
"name": "AMAZON.HelpIntent",
"samples": [
"help"
]
},
{
"name": "AMAZON.StopIntent",
"samples": [
"stop"
]
}
],
"types": []
}
}
}
Click on Save Model and click on Build Model
Once, the skill model is build - we need to test it. Click on the Test tab and toggle ON the “Test is enabled for this skill”.
That’s it, you’ve created an Alexa bot connected to your locally running node project.
Questions? Comments? Do reach me at info#nordible.com
Read the full article
This is a sample code (not mine) that uses the alexa sdk which doesnt need to use awsLambda and works with just express, pure nodejs server
https://github.com/Glogo/alexa-skill-sample-nodejs-express
I have a client based on react and I bundle it with webpack 2. But the moment I import/require const SpeechToTextV1 = require('watson-developer-cloud/speech-to-text/v1'); I got some trouble. After I fixed it that it does not break the build, it still throws some warning like:
Module not found: Error: Can't resolve '../build/Release/validation' in '/Users/denbox/Desktop/schedulebot/web-interface/node_modules/websocket/lib'
# ./~/websocket/lib/Validation.js 9:21-59
# ./~/websocket/lib/WebSocketConnection.js
# ./~/websocket/lib/websocket.js
# ./~/websocket/index.js
# ./~/watson-developer-cloud/speech-to-text/recognize_stream.js
# ./~/watson-developer-cloud/speech-to-text/v1.js
# ./src/components/chat.jsx
# ./src/components/chat-page.js
# ./src/index.js
# multi (webpack)-dev-server/client?http://localhost:8080 ./src/index.js
Is it even possible to use the watson-developer-cloud node sdk for the speech-to-text service on the client or only directly on the nodejs server? Thank you.
The Watson Node.js SDK has growing compatibility for client-side usage, but it's not all the way there yet. However, for speech services, there is a separate SDK targeted at client-side usage: https://www.npmjs.com/package/watson-speech
I just added a Webpack example and confirmed that it works: https://github.com/watson-developer-cloud/speech-javascript-sdk/blob/master/examples/webpack.config.js
Update: I also went and added a Webpack example to the Node.js SDK - with the configuration there, it can build for the entire library, and actually works for a subset set of the modules as documented: https://github.com/watson-developer-cloud/node-sdk/tree/master/examples/webpack
Only in Node,js. The mechanism for using Speech To Text from the browser is to use websockets, but to do that you need a token, which will require a server side request. Once you have the token you can use the websockets interface.
With the answers above found a solution for my problem and it might help others who want to get started with the API:
import axios from 'axios';
import recognizeMicrophone from 'watson-speech/speech-to-text/recognize-microphone';
axios.get(`${BACKEND_ROOT_URL}/watsoncloud/stt/token`)
.then((res) => {
console.log('res:', res.data);
const stream = recognizeMicrophone({
token: res.data.token,
continuous: false, // false = automatically stop transcription the first time a pause is detected
});
stream.setEncoding('utf8');
stream.on('error', (err) => {
console.log(err);
});
stream.on('data', (msg) => {
console.log('message:', msg);
});
})
.catch((err) => {
console.log(`The following gUM error occured: ${err}`);
});
In the backend I create a proxy service that get's a token for the watson speech to text service so I don't have to save my credentials on the client:
const watson = require('watson-developer-cloud');
const express = require('express');
const cors = require('cors');
app.use(cors());
const stt = new watson.SpeechToTextV1({
// if left undefined, username and password to fall back to the SPEECH_TO_TEXT_USERNAME and
// SPEECH_TO_TEXT_PASSWORD environment properties, and then to VCAP_SERVICES (on Bluemix)
username: process.env.STT_SERVICE_USER,
password: process.env.STT_SERVICE_PW,
});
const authService = new watson.AuthorizationV1(stt.getCredentials());
// Endpoint to retrieve an watson speech to text api token
// Get token using your credentials
app.get('/watsoncloud/stt/token', (req, res, next) => {
// TODO check jwt at the auth service
authService.getToken((err, token) => {
if (err) {
next(err);
} else {
res.send({ token });
}
});
});
app.listen(port, (err) => {
if (err) {
console.log(`Error: ${err}`);
}
});