Sending Proactive Messages from Azure functions to botservice - node - node.js

I am using botframework v4, but coming over from v3, I have not found any documentation that is similar to the code I use below but for v4, regarding sending proactive messages from Azure Function App
Below is the code I previously used but am having trouble adapting:
var builder = require('botbuilder');
// setup bot credentials
var connector = new builder.ChatConnector({
appId: process.env.MICROSOFT_APP_ID,
appPassword: process.env.MICROSOFT_APP_PASSWORD
});
module.exports = function (context, req) {
if (req.body) {
var savedAddress = req.body.channelAddress;
var inMemoryStorage = new builder.MemoryBotStorage();
var bot = new builder.UniversalBot(connector).set('storage', inMemoryStorage);
sendProactiveMessage(savedAddress, bot)
}
};
function sendProactiveMessage(address, bot) {
var msg = new builder.Message().address(address);
msg.textLocale('en-US');
var img = {
attachments: [{
contentType: "image/jpg",
contentUrl: latestUrl,
}]
};
msg.addAttachment(img.attachments[0]);
msg.text('hello');
bot.send(msg);
}
This works fine with v3 but not v4.
If possible I would also like to find a way to log a user out:
await botAdapter.signOutUser(innerDc.context, this.connectionName);
This is how I do it in the bot itself, but doing so from Azure Functions again is proving difficult.
Any help would be appreciated.

Great that you are making the move from v3 to v4! Have you had a look at Send proactive notifications to users? This example is pretty straight forward and can be used within an Azure function.
First you retrieve the Conversation Reference in your bot by calling TurnContext.getConversationReference(context.activity);. This is the reference you could use in your proactive function to open the conversation. In your case you provide that via the request body to a proactive function, so I will do the same in my example.
My proactive endpoint example is written in Typescript, however it works the same way in plain Javascript. Create a HTTP trigger in Azure Functions and use the following code. I have added comments inline for clarity.
const { BotFrameworkAdapter } = require('botbuilder');
// Create adapter.
// If you need to share this adapter in multiple functions, you could
// instantiate it somewhere else and import it in every function.
const adapter = new BotFrameworkAdapter({
appId: process.env.MicrosoftAppId,
appPassword: process.env.MicrosoftAppPassword
});
module.exports = async function (context, req) {
// Validate if request has a body
if (!req.body) {
context.res = {
status: 400,
body: "Please pass a conversation reference in the request body"
};
return;
}
// Retrieve conversation reference from POST body
const conversationReference = req.body;
// Open the conversation and retrieve a TurnContext
await adapter.continueConversation(conversationReference, async turnContext => {
// Send a text activity
// Depending on the channel, you might need to use https://aka.ms/BotTrustServiceUrl
await turnContext.sendActivity('Proactive hello');
});
context.res = {
body: 'Message sent!'
};
};
In the end you could make a request to this Azure Function, where you pass the Conversation Reference as body of the type application/json.
Extending this example with features like signOutUser is simple. You can call all functions within the continueConversation function, just as in a normal bot. You can even receive the adapter object there if you wish.
await adapter.continueConversation(conversationReference, async turnContext => {
// Sign user out
turnContext.adapter.signOutUser(turnContext, 'connection-name');
});

Related

NodeJS Proactive Messaging in Bot-framework for Skill Bot

I am working on a requirement in NodeJS where I have to invoke proactive messages from Skill Bot. I have a set Interval loop which runs for every 8 seconds where I added adapter.ContinueConversation method. I have added the below code in skill bot on message method initially but I received 401 Unauthorized error.
await context.sendActivity(`Echo (JS) : '${ context.activity.text }'`);
await context.sendActivity('Say "end" or "stop" and I\'ll end the conversation and back to the parent.');
var adapter = context.adapter;
var conversationReference = TurnContext.getConversationReference(context.activity);
var refreshInterval = setInterval(async () => {
try {
await adapter.continueConversation(conversationReference, async turnContext => {
await turnContext.sendActivity('Are you still there?');
});
}
catch (error) {
console.log(error);
}
}, 8000,conversationReference);
After doing some research online, i have added additional parameters like claims identity and Root Bot's App Id as below but now I receive error that adapter.continueConversationAsync is not a function
var conversationReference = TurnContext.getConversationReference(stepContext.context.activity);
var claimsIdentity= stepContext.context.turnState.get(stepContext.context.adapter.BotIdentityKey);
var oAuthScope= stepContext.context.turnState.get(stepContext.context.adapter.OAuthScopeKey);
await adapter.continueConversationAsync(claimsIdentity, convRef, oAuthScope, async context => {
await context.sendActivity("hello");
})
Can anyone please help resolve the issue?
First, you're receiving the error, "adapter.continueConversationAsync is not a function" because your parameters are wrong. Per the docs there are two overloaded versions of the continueConversation method :
function continueConversation(reference: Partial<ConversationReference>, logic: (context: TurnContext) => Promise<void>): Promise<void>
and
function continueConversation(reference: Partial<ConversationReference>, oAuthScope: string, logic: (context: TurnContext) => Promise<void>): Promise<void>
Your initial implementation was fine, the problem isn't the lack of oAuthScope. I can think of three possible reasons why you get a 401 error :
Your appID and appPassword for your bot adapter might not be set correctly.
Verify your appId and appPassword is correct.
const adapter = new BotFrameworkAdapter({
appId: process.env.MicrosoftAppId,
appPassword: process.env.MicrosoftAppPassword
});
Try trusting the service URL of the conversation reference :
var conversationReference = TurnContext.getConversationReference(stepContext.context.activity);
await adapter.continueConversationAsync(conversationReference, async context => {
MicrosoftAppCredentials.trustServiceUrl(conversationReference.serviceUrl);
await context.sendActivity("hello");
})
And finally, try logging your conversationReference and verify the conversationReference itself is correct. Although since you're getting the conversationReference from the turncontext right before sending the proactive message this part shouldn't be the issue.

how to run the dialogflow node js?

i am still confused about the documentation on Dialogflow nodejs, can I get clear for this?
I have read the documentation this -> https://cloud.google.com/dialogflow/docs/reference/rest/v2/projects.agent.sessions/detectIntent
how to test my code and see the result of my query I input then ?
should I do POST via Axios inside this functions
async function runSample(projectId = 'your-project-id') {
// A unique identifier for the given session
const sessionId = uuid.v4();
// Create a new session
const sessionClient = new dialogflow.SessionsClient();
const sessionPath = sessionClient.sessionPath(projectId, sessionId);
// The text query request.
const request = {
session: sessionPath,
queryInput: {
text: {
// The query to send to the dialogflow agent
text: 'hello',
// The language used by the client (en-US)
languageCode: 'en-US',
},
},
};
// Send request and log result
const responses = await sessionClient.detectIntent(request);
console.log('Detected intent');
const result = responses[0].queryResult;
console.log(` Query: ${result.queryText}`);
console.log(` Response: ${result.fulfillmentText}`);
if (result.intent) {
console.log(` Intent: ${result.intent.displayName}`);
} else {
console.log(` No intent matched.`);
}
}
i still not get clear for this, so do we need use HTTP request also in local to check this?
I already follow the Dialogflow nodejs example, but what next?
it said on google we must POST to
POST https://dialogflow.googleapis.com/v2/{session=projects//agent/sessions/}:detectIntent
nd the req body is on the request variable inside that functions,
but I still not clear on dialogflow nodejs for the next step to run that method
You can run this directly using the express app. Just call this function from your express app route or you can call the given function in the file and use node <filename.js> to run the code. Make sure you add your google cloud credentials in the path name using process.env.GOOGLE_APPLICATION_CREDENTIALS = "<your_json_file_path>";
const sessionClient = new dialogflow.SessionsClient(
// below can be used to set the variable in code itself, if below is not working you can run following in the terminal
//export GOOGLE_APPLICATION_CREDENTIALS="/<path of the json file"
{
keyFilename: "/<path of the json file"
}
);

Possibility to send context data to dialogflow without webhook being called from Dialogflow, but from server itself

Currently I have a webhook which is being called from Dialogflow (fulfillment) and checks which intent is currently applicable. (I do everything via JSON) once I know which Intent, I send back (via the webhook) a JSON response. I was wondering if it is possible to initiate a JSON object and send it to Dialogflow. (I want to send context without letting the user type first and Dialogflow receiving a fallback) (NodeJS Server -> Dialogflow) Currently the only way I can do it is (Dialogflow -> NodeJS Server (JSON object with contex) -> Dialogflow)
CURRENT:
var dialog_flow = {
// Create context using Google dialogflow
createContext: async function(projectId, sessionId, contextId, parameters, lifespanCount){
console.log("params to be sent: ", parameters);
//Instantiates client to create the actual context before sending
const contextsClient = new dialogflow.ContextsClient({
keyFilename: './dialogflow_credentials.json'
});
const sessionPath = contextsClient.sessionPath(projectId, sessionId);
const contextPath = contextsClient.contextPath(
projectId,
sessionId,
contextId
);
const createContextRequest = {
parent: sessionPath,
context: {
name: contextPath,
lifespanCount: lifespanCount,
parameters: parameters
},
};
const response = await contextsClient.createContext(createContextRequest);
console.log(`Created ${response[0].name}`);
}
};
PARAMETERS:
let testobj = {
firstname: 'test name',
lastname: 'itworks!!',
dd: 'fgfgf',
ff: 'fgfg'
}
You can call directly Dialogflow from Node.js, without the need of receiving a webhook. For that you will need to use the dialogflow package.
const sessionClient = new dialogflow.SessionsClient();
const session = sessionClient.sessionPath(projectId, sessionId);
const request = {
session,
queryInput: {
text: {
text: 'Some text'
}
},
queryParams: {
contexts: [context] // here you send the data
}
};
sessionClient.detectIntent(request)
.then(console.log)
.catch(console.error);
In order to send a context, with data, you will need to use dialogflow.ContextsClient
I give a detailed explanation about that in these other questions:
set input or output context dialogflow nodejs v2
Dialogflow pass parameters through NodeJS
And in order to authenticate the client you can check:
Dialogflow easy way for authorization
UPDATE: 2 weeks ago, Dialogflow, removed the structjson.js sample from the repository, but in favor of an npm package to handle the encoding/decoding of the protobuf struct: pb-util

Is it possible to integrate bot-builder into an existing express app?

I have an existing node/express chatbot application that connects to several chat platforms using ExpressJS' next(), next() middleware design pattern. I send a 200 response at the very beginning to acknowledge the receipt of a message, and send a new POST request to send a message from my last middleware.
app.post("/bots", receiveMsg, doStuff, formatAndSendMsg, catchErrors);
Now I want to integrate Skype as a channel for my bot, but the NodeJS library for bot-framework has a totally different way of doing things, using events and such magic that I haven't fully understood yet:
var connector = new builder.ConsoleConnector();
app.post("/skype", connector.listen());
var bot = new builder.UniversalBot(connector, function (session) {
session.send("You said: %s", session.message.text);
});
It doesn't look like these are compatible ways to do things, so what is the best way to receive a message and then send a response to a user without having to change my express routing to fit bot-builder in? Can I get a Session object with Session.send() to respond to? Will I have to do all the addressing manually? Is there a method that resembles this:
app.post("/skype", (req, res, next) => {
const address = req.body.id;
const message = new builder.Message(address, messageBody).send()
}
Or:
app.post("/skype", connector.listen(), (req, res, next) => {
// (res.locals is available in every express middleware function)
const session = res.locals.botFrameworkSession;
// do stuff
session.send(message);
}
You can register bot application in your existing express applications. Bot builder SDK is also compatible in expressjs framework. You can refer to official sample which is also leveraging express.
Don't forget to modify the messsaging endpoint in your bot registration to your bot's endpoint, e.g.
https://yourdomain/stuff
in your scenario. Please refer to https://learn.microsoft.com/en-us/azure/bot-service/bot-service-quickstart-registration for more info.
Building messages, addressing them, and sending those messages are all possible using the official bot framework NodeJS library. What I couldn't do with that library was receive messages and verify their authenticity on my routes without making major changes to my design (using request middleware - next() - to process the incoming request) which is already in production with other bots and not easy to change.
Here's my workaround: First is this BotFrameworkAuthenticator class that I create based on the Microsoft documentation here: https://learn.microsoft.com/en-us/azure/bot-service/rest-api/bot-framework-rest-connector-authentication
It is instantiated with the appID and appPassword from your Bot Framework app.
import axios from "axios";
import * as jwt from "jsonwebtoken";
import * as jwkToPem from 'jwk-to-pem';
export class BotFrameworkAuthenticator {
private appId: string;
private appPassword: string;
private openIdMetadata: any;
// The response body specifies the document in the JWK format but also includes an additional property for each key: endorsements.
private validSigningKeys: any;
// The list of keys is relatively stable and may be cached for long periods of time (by default, 5 days within the Bot Builder SDK).
private signingKeyRefreshRate: number = 432000; // in seconds (432000 = 5 days)
constructor(appId, appPassword) {
this.appId = appId;
this.appPassword = appPassword;
this.getListOfSigningKeys();
}
// response data should contain "jwks_uri" property that contains address to request list of valid signing keys.
public async getOpenIdMetaData() {
// This is a static URL that you can hardcode into your application. - MS Bot Framework docs
await axios.get("https://login.botframework.com/v1/.well-known/openidconfiguration").then(response => {
this.openIdMetadata = response.data;
logger.info("OpenID metadata document recieved for Bot Framework.");
}).catch(err => {
logger.warn(err.message, "Could not get OpenID metadata document for Bot Framework. Retrying in 15 seconds...");
setTimeout(this.getListOfSigningKeys, 15000);
})
}
public async getListOfSigningKeys() {
await this.getOpenIdMetaData();
if (this.openIdMetadata && this.openIdMetadata.jwks_uri) {
// previous function getOpenIdMetaData() succeeded
await axios.get(this.openIdMetadata.jwks_uri).then(response => {
logger.info(`Signing keys recieved for Bot Framework. Caching for ${this.signingKeyRefreshRate / 86400} days.`);
this.validSigningKeys = response.data.keys;
setTimeout(this.getListOfSigningKeys, (this.signingKeyRefreshRate * 1000));
}).catch(err => {
logger.error(err.message, "Could not get list of valid signing keys for Bot Framework. Retrying in 15 seconds");
setTimeout(this.getListOfSigningKeys, 15000);
});
} else {
// previous function getOpenIdMetaData() failed, but has already queued this function to run again. Will continue until succeeds.
return;
}
}
/**
* Verifies that the message was sent from Bot Framework by checking values as specified in Bot Framework Documentation:
* https://learn.microsoft.com/en-us/azure/bot-service/rest-api/bot-framework-rest-connector-authentication#step-4-verify-the-jwt-token
* Retrieves the Bearer token from the authorization header, decodes the token so we can match the key id (kid) to a key in the OpenID
* document, then converts that key to PEM format so that jwt/crypto can consume it to verify that the bearer token is
* cryptographically signed.
* If the serviceUrl property in the token doe not match the serviceUrl property in the message, it should also be rejected.
*/
public verifyMsgAuthenticity(serviceUrl: string, headers: any) {
try {
const token = headers.authorization.replace("Bearer ", "");
const decoded = jwt.decode(token, { complete: true }) as any;
const verifyOptions = {
issuer: "https://api.botframework.com",
audience: this.appId,
clockTolerance: 300, // (seconds) The token is within its validity period. Industry-standard clock-skew is 5 minutes. (Bot Framework documentation);
}
const jwk = this.lookupKey(decoded.header.kid)
const pem = jwkToPem(jwk);
const verified = jwt.verify(token, pem, verifyOptions) as any;
if (!serviceUrl || serviceUrl !== verified.serviceurl) {
logger.warn("Non-matching serviceUrl in Bot Framework verified token!")
return false;
}
return true;
} catch (err) {
logger.warn("Received invalid/unsigned message on Bot Framework endpoint!", err.message)
return false;
}
}
// Finds the relevant key from the openID list. Does not transform the key.
private lookupKey(kid) {
const jwk = this.validSigningKeys.find((key) => {
return (key.kid === kid);
});
return jwk;
}
}
Use the BotFrameworkAuthenticator class like this at the very beginning of your express route to verify that all incoming requests are valid.
const botFrameworkAuthenticator = new BotFrameworkAuthenticator(appID, appPassword);
router.post("/", (req: Request, res: Response, next: NextFunction) => {
if (botFrameworkAuthenticator.verifyMsgAuthenticity(req.body.serviceUrl, req.headers) === true) {
res.status(200).send();
next();
} else {
// unsafe to process
res.status(403).send();
return;
}
});
And to send messages using the regular Bot Framework library without having a Session object that would normally be created by the Bot Framework library when it receives an incoming message:
import * as builder from "botbuilder";
// instantiate the chatConnector (only once, not in the same function as the sending occurs)
const botFrameworkSender = new builder.ChatConnector({ appId, appPassword });
//---------------------------------------------
const skypeMsg = req.body;
const address = {
channelId: skypeMsg.channelId,
user: skypeMsg.from,
bot: skypeMsg.recipient,
conversation: skypeMsg.conversation
};
const response = new builder.Message().text(someText).address(address).toMessage();
const formattedResponses = [response];
botFrameworkSender.send(formattedResponses, logErrorsToConsole);
Note that all of the builder.Message() -- .attachment(), .images(), etc.. -- functions can be used, not just the text()

Asynchronous Bot Framework Response possible?

Most node.js botframework examples involve the notion of the waterfall dialog model, which is a great way to structure conversations, but we don't need that since we have our own chat system. We simply receive messages via the webhook, process them and respond without the dialog system.
Also, and now we are getting to the heart of the matter;), the examples I have seen communicate back to the botframework in the web context:
var connector = new builder.ChatConnector({config});
var bot = new builder.UniversalBot(connector);
server.post('/api/messages', connector.listen());
bot.dialog('/', (session, args) => {
session.sendTyping();
session.send( 'Echo'+ session.message.text);
})
The above example simply responds with an 'echo' and before that set's the typing status.
Our system works asynchronous, the way we currently work is by bypassing the connector listen and dialog scheme. A simplified example of how we queue a botframework message.
server.post('/api/messages/:name', (req, res, next)=>{
queue.post('botframework',req.params.name,req.body)
.then(()=>res.sendStatus(200))
});
In the queue processing, we construct the botframework objects:
//the ':name' from the snippet above is used to identify
//the bot a retrieve credentials
const connector = new botbuilder.ChatConnector({
appId: bot.properties.botframework_id.value.appId,
appPassword: bot.properties.botframework_id.value.appPassword
});
const username=message.from.name.replace(/[\s-;:"';$]/g,"_")+"_skype";
var address = {
"channelId": message.channelId,
"user": message.from,
"bot": message.recipient,
"serviceUrl": message.serviceUrl,
"useAuth": true
};
let bot = new botbuilder.UniversalBot(connector);
let msg = new botbuilder.Message();
//processing...
//create the outgoing message...
bot.send(msg);
The problem here for us is that we simply don't know how to create a session object from a raw webhook message which is needed for the typing indicator and to ensure the order of messages when many messages are sent in quick succession.
Here is what we wish to accomplish:
//the ':name' from the snippet above is used to identify
//the bot a retrieve credentials
//the context is non HHTP
const connector = new botbuilder.ChatConnector({
appId: bot.properties.botframework_id.value.appId,
appPassword: bot.properties.botframework_id.value.appPassword
});
const username=message.from.name.replace(/[\s-;:"';$]/g,"_")+"_skype";
var address = {
"channelId": message.channelId,
"user": message.from,
"bot": message.recipient,
"serviceUrl": message.serviceUrl,
"useAuth": true
};
let bot = new botbuilder.UniversalBot(connector);
let session = bot.createSession();
session.sendTyping();
let message = getNextMessageFromChatServer(message);
session.send(message);
//more messages to be send?
//....
So the question: How can we create a session object from the raw data send to the botframework webhook?
You should be able to build a session using loadSession
var address = {
"channelId": message.channelId,
"user": message.from,
"bot": message.recipient,
"serviceUrl": message.serviceUrl,
"useAuth": true
};
let bot = new botbuilder.UniversalBot(connector);
let msg = new botbuilder.Message();
bot.loadSession(address, (err, session) => {
session.send("Message");
})

Resources