Waterfall in ComponentDialogs - node.js

I am having problems implementing WaterfallDialogs in ComponentDialogs trigered by LUIS in the BotFramework V4.2. After the first step, the dialog does not continue the Waterfall.
I don't really understand what should be the correct approach to persist the ComponentDialog: save the ConversationState? set the step.context? I've tried both but none worked for me so far 😣
This is my code for bot.js, where I have a LUIS instance to act as an orchestrator between dialogs:
async onTurn(turnContext) {
...
switch (dialogResult.status) {
case DialogTurnStatus.empty:
// Call to LUIS recognizer to get intent + entities
const results = await this.luisRecognizer.recognize(dc.context);
const topIntent = results.luisResult.topScoringIntent;
switch (topIntent.intent) {
case FMM_INTENT:
return await dc.beginDialog(FeedbackDialog.Name);
...
}
case DialogTurnStatus.waiting:
// The active dialog is waiting for a response from the user, so do nothing
break;
...
}
And this is my code por FeedbackDialog.js where I have the logic of the WaterfallDialog with Prompts:
module.exports = {
FeedbackDialog: class extends ComponentDialog {
static get Name() { return DIALOG_NAME; }
constructor(conversationState, botConfig, feedbackStateAccessor, dialogId) {
(dialogId === undefined) ? super(DIALOG_NAME) : super(dialogId);
this.conversationState = conversationState;
this.feedbackStateAccessor = feedbackStateAccessor;
this.addDialog(new WaterfallDialog(FBK_EVAL, [
this.feedbackPrompt.bind(this),
this.feedbackEvaluation.bind(this)
]));
this.addDialog(new ChoicePrompt(FBK_PROMPT));
}
async feedbackPrompt(step) {
let fmmState = await this.feedbackStateAccessor.get(step.context);
if (fmmState === undefined) {
await this.feedbackStateAccessor.set(step.context);
//opciones válidas, las mayúsculas las detecta también
const options = ['👍','👎'];
return await step.prompt(FBK_PROMPT, {
prompt: `¿Te ha resultado útil la respuesta?`,
retryPrompt: `¿Qué te ha parecido la respuesta?`,
choices: options
});
}
}
async feedbackEvaluation(step) {
let fmmState = await this.feedbackStateAccessor.get(step.context);
if (fmmState === undefined && step.result){
if (step.result == '👍'){
await step.context.sendActivity("¡Gracias por el feedback!");
return await step.endDialog();
}else {
await step.context.sendActivity("¡Revisaremos tu pregunta para seguir mejorando!");
return await this.conversationState.saveChanges(step.context);
}
} else {
return await step.next();
//next steps omitted since the code doesn't reach this step
}
}

I'm unsure what you're trying to achieve but if you're using a ChoicePrompt I believe the step result comes back into a value key, i.e. instead of
if (step.result == '👍')
try
if (step.result.value === '👍')
Then you can send a message back to the user and end the dialog and it should be fine:
await step.context.sendActivity("¡Gracias por el feedback!");
return step.endDialog();
or
await step.context.sendActivity("¡Revisaremos tu pregunta para seguir mejorando!");
return step.endDialog();
Also, remember to initialise the dialog correctly, i.e. in your bot file:
const { FeedbackDialog } = require('path/to/feedbackDialog')
const FEEDBACK_DIALOG = 'FeedbackDialog'
const FEEDBACK_PROPERTY = 'feedbackProperty'
and in the bot constructor (I use userState for state accessors, not conversation state):
this.dialogs = new DialogSet(this.dialogState)
this.feedbackAccessor = userState.createProperty(FEEDBACK_PROPERTY)
this.dialogs.add(new FeedbackDialog(
FEEDBACK_DIALOG,
this.feedbackAccessor
))
then call it as:
await dc.beginDialog(FEEDBACK_DIALOG);

Related

Executing nested WaterfallDialogs - nodejs

I'm trying to build a requirement system for order dialogs in our bot, so that we can reuse the main structure for different procedures.
enum DialogIds {
// Necessary Ids
oauthPrompt = "oauthPrompt",
// Requirement dialogs
itemWaterfallDialog = "itemWaterfallDialog",
// Reset Dialogs
summaryWaterfallDialog = "summaryWaterfallDialog",
// All other prompts
unrecognizedItemPrompt = "unrecognizedItemPrompt",
beneficiaryConfirmPrompt = "beneficiaryConfirmPrompt",
askBeneficiaryPrompt = "askBeneficiaryPrompt",
reasonPrompt = "reasonPrompt",
orderConfirm = "orderConfirm"
}
export class OrderDialog extends ComponentDialog {
private responseManager: ResponseManager;
private requirementManager: RequirementManager;
private luisResult: RecognizerResult | undefined = undefined;
// TODO: get userState and ConversationState
constructor(
private service: BotServices,
telemetryClient: BotTelemetryClient
) {
super(OrderDialog.name);
this.initialDialogId = OrderDialog.name;
// Response manager serving OrderResponses.json
this.responseManager = new ResponseManager(["fr-fr"], [OrderResponses]);
const routeWaterfallDialog: ((
sc: WaterfallStepContext
) => Promise<DialogTurnResult>)[] = [
this.route.bind(this)
];
this.telemetryClient = telemetryClient;
this.addDialog(
new WaterfallDialog(this.initialDialogId, routeWaterfallDialog)
);
/**
* Order specific dialogs and requirements
*/
const itemWaterfallDialog: WaterfallDialog = new WaterfallDialog(
DialogIds.itemWaterfallDialog,
[this.itemStep.bind(this), this.itemEndStep.bind(this)]
);
this.addDialog(itemWaterfallDialog);
const reqs = [
new Requirement<string>("claimant", false, undefined),
new Requirement<string>(
"item",
true,
undefined,
itemWaterfallDialog,
DialogIds.itemWaterfallDialog
),
];
// Create requirement manager for this dialog
this.requirementManager = new RequirementManager(reqs);
// Add all the prompt
this.addDialog(new ConfirmPrompt(DialogIds.beneficiaryConfirmPrompt));
this.addDialog(new TextPrompt(DialogIds.unrecognizedItemPrompt));
this.addDialog(new TextPrompt(DialogIds.askBeneficiaryPrompt));
this.addDialog(new TextPrompt(DialogIds.reasonPrompt));
this.addDialog(new ConfirmPrompt(DialogIds.orderConfirm));
}
/**
* We save the token, query graph is necessary and
* execute the next dialog if any, if not we'll
* execute the summary waterfallDialog.
* #param sc context
*/
async route(sc: WaterfallStepContext): Promise<DialogTurnResult> {
this.requirementManager.set("claimant", 'nothing');
let next = this.requirementManager.getNext();
while (next) {
await sc.beginDialog(next.dialogId!);
// Execute summary if there are no elements left
if (!this.requirementManager.getNextBool()) {
await sc.beginDialog(DialogIds.summaryWaterfallDialog);
}
next = this.requirementManager.getNext();
}
return sc.endDialog();
}
/**
* ITEM
* #param sc
*/
async itemStep(sc: WaterfallStepContext): Promise<DialogTurnResult> {
// Couldn't recgonize any item
if (this.luisResult!.entities.length === 0) {
await sc.context.sendActivity(
this.responseManager.getResponse(
OrderResponses.itemNotRecognized
)
);
// prompt user for the item again
return await sc.prompt(
DialogIds.unrecognizedItemPrompt,
this.responseManager.getResponse(OrderResponses.rePromptItem)
);
}
const entities = this.luisResult!.entities as generalLuis["entities"];
if (entities.PhoneItem || entities.ComputerItem) {
const item = entities.PhoneItem
? entities.PhoneItem
: entities.ComputerItem;
if (item) {
this.requirementManager.set("item", item[0][0]);
}
}
return await sc.next();
}
async itemEndStep(sc: WaterfallStepContext): Promise<DialogTurnResult> {
// Save result from itemStep(prompt triggered) if any
if (sc.result) {
await sc.context.sendActivity(
this.responseManager.getResponse(OrderResponses.thanksUser)
);
// retrieve item from result and save it
const item = sc.result as string;
this.requirementManager.set("item", item);
}
return sc.endDialog();
}
}
The line
const result = await sc.beginDialog(next.dialogId!);
Is starting a WaterfallDialog declared in the constructor of the Dialog, and the route method is also inside a general waterfallDialog.
The problem is that, when one of the child dialogs prompts the user, the code doesn't wait for the user response, and because of the way the route works it will call the same dialog again(if a value on an object is not filled it will call the indicated dialog, that's what the requirement manager does).
If saving the return from that line, we can see that the status is "waiting", how could I fix it, or should I create independent dialogs for each requirement, and not just waterfallDialogs?
Thanks.

Support for interruptions in bot

Do you know if there is way to implement global message handlers that can support commands like stop, bye, cancel, exit Virtual assistant bot ? I am trying to implement something like this.
I have a virtual assistant built already and it has couple of Skills or Skill Bots.
When user is in the multi turn conversation with a Skill, user should be able to exit out of skill by commands like stop, bye, cancel, exit.
I found old v3 doc but nothing for v4.
Check the documentations provided here Handling User Interruption They explain how to handle user interruption for SDK v4
Find below an example of how you can configure this in the Virtual Assistant.
In your MainDialog.cs
Add the following for your OnContinueDialogAsync: Keeping in mind that you can change and edit this as you see fit just be sure to check the OnInterruptDialogAsync result (status in this example) before you continue
protected override async Task<DialogTurnResult> OnContinueDialogAsync(DialogContext innerDc, CancellationToken cancellationToken = default(CancellationToken))
{
var status = await OnInterruptDialogAsync(innerDc, cancellationToken).ConfigureAwait(false);
if (status == InterruptionAction.Resume)
{
// Resume the waiting dialog after interruption
await innerDc.RepromptDialogAsync().ConfigureAwait(false);
return EndOfTurn;
}
else if (status == InterruptionAction.Waiting)
{
// Stack is already waiting for a response, shelve inner stack
return EndOfTurn;
}
else
{
var activity = innerDc.Context.Activity;
if (activity.IsStartActivity())
{
await OnStartAsync(innerDc).ConfigureAwait(false);
}
switch (activity.Type)
{
case ActivityTypes.Message:
{
// Note: This check is a workaround for adaptive card buttons that should map to an event (i.e. startOnboarding button in intro card)
if (activity.Value != null)
{
await OnEventAsync(innerDc).ConfigureAwait(false);
}
else
{
var result = await innerDc.ContinueDialogAsync().ConfigureAwait(false);
switch (result.Status)
{
case DialogTurnStatus.Empty:
{
await RouteAsync(innerDc).ConfigureAwait(false);
break;
}
case DialogTurnStatus.Complete:
{
// End active dialog
await innerDc.EndDialogAsync().ConfigureAwait(false);
break;
}
default:
{
break;
}
}
}
// If the active dialog was ended on this turn (either on single-turn dialog, or on continueDialogAsync) run CompleteAsync method.
if (innerDc.ActiveDialog == null)
{
await CompleteAsync(innerDc).ConfigureAwait(false);
}
break;
}
case ActivityTypes.Event:
{
//do something for event activity
break;
}
case ActivityTypes.Invoke:
{
// Used by Teams for Authentication scenarios.
break;
}
default:
{
await OnSystemMessageAsync(innerDc).ConfigureAwait(false);
break;
}
}
return EndOfTurn;
}
}
And override OnInterruptDialogAsync like below example:
This example includes LUIS but you can do whatever you want in OnInterruptDialogAsync
protected override async Task<InterruptionAction> OnInterruptDialogAsync(DialogContext dc, CancellationToken cancellationToken = default(CancellationToken))
{
var result = InterruptionAction.NoAction;
if (dc.Context.Activity.Type == ActivityTypes.Message && !string.IsNullOrEmpty(dc.Context.Activity.Text))
{
// get current activity locale
var localeConfig = _services.GetCognitiveModels();
// check general luis intent
localeConfig.LuisServices.TryGetValue("General", out var luisService);
if (luisService == null)
{
throw new Exception("The specified LUIS Model could not be found in your Skill configuration.");
}
else
{
var luisResult = await luisService.RecognizeAsync<GeneralLuis>(dc.Context, cancellationToken);
var topIntent = luisResult.TopIntent();
if (topIntent.score > 0.5)
{
switch (topIntent.intent)
{
case GeneralLuis.Intent.Cancel:
{
result = await OnCancel(dc);
break;
}
case GeneralLuis.Intent.Help:
{
result = await OnHelp(dc);
break;
}
case GeneralLuis.Intent.Logout:
{
result = await OnLogout(dc);
break;
}
}
}
}
}
return result;
}

Bot builder : Adaptive cards - call a method when submitting

I need to create a form in which the user has to fill it and to send it. So i have to create a submit button that calls another method but i couldn't find the link between the submit action and the call to another method.
The script of my form is :
public Attachment CreateAdaptiveCardwithEntry()
{
var submitActionData = JObject.Parse("{ \"Type\": \"SaveFunction\" }");
var card = new AdaptiveCard()
{
Body = new List<CardElement>()
{
// Hotels Search form
new TextBlock() { Text = "Titre de la note des frais" },
new TextInput()
{
Id = "titre",
Speak = "<s>Veuillez saisir le titre</s>",
Placeholder = "Veuillez saisir le titre",
Style = TextInputStyle.Text
},
},
Actions = new List<ActionBase>()
{
new SubmitAction()
{
DataJson = submitActionData.ToString()
}
}
};
The script of my card is :
var replyMessage = context.MakeMessage();
replyMessage.Attachments = new List<Attachment> { FraisDialog.CreateAdaptiveCardwithEntry() };
await context.PostAsync(replyMessage, CancellationToken.None);
context.Wait(MessageReceived);
the script in MessageReceivedAsync is :
public virtual async Task MessageReceivedAsync(IDialogContext context, IAwaitable<IMessageActivity> result)
{
var message = await result;
if (message.Value != null)
{
// Got an Action Submit
dynamic value = message.Value;
string submitType = value.Type.ToString();
switch (submitType)
{
case "SaveFunction":
await context.PostAsync("Please complete all the search parameters:\n");
return;
}
}
}
In this example i need to send the information with the Id = "titre" and pprocess it afterwards, i don't know how to send it(DataJson ?) and where(MessageReceivedAsync ?). Can someone help me ? do i need to create another dialog just for the card ?
Ps : all this code is in rootDialog.
i'm not getting the message 'Please complete all the search parameters'
If all of your code is in RootDialog then please use context.Wait(MessageReceivedAsync); after sending your attachment.
i need to send the information with the Id = "titre" and process it afterwards
When clicking the submit button, the form data is send to MessageReceived method as usual. If you want to just access the fields in the adaptive card you can access the dynamic variable value. Here is an example.
var message = await result;
if (message.Value != null)
{
// Got an Action Submit
dynamic value = message.Value;
string submitType = value.Type.ToString();
switch (submitType)
{
case "SaveFunction":
if(value.titre == "")
{
await context.PostAsync("Please complete all the search parameters:\n");
}
else
{
await context.PostAsync($"You entered {value.titre}");
}
return;
}
}

Speech is not being recognized with default dictation grammar in my UWP application.

Speech is not being recognized with default dictation grammar in my UWP application. However, it is perfectly recognized when I use programmatic list constraint. Below is the speech recognition part of my code for reference. If I do not comment the 5th line, this works fine. Am I doing something wrong below:
speechRecognizer = new SpeechRecognizer();
bool PermissionGained = await CheckMicrophonePermission();
if (PermissionGained)
{
//speechRecognizer.Constraints.Add(new SpeechRecognitionListConstraint(Grammar.GrammarCommands.GrammarConstraintList));
await speechRecognizer.CompileConstraintsAsync();
//recognize speech input at any point of time
speechRecognizer.ContinuousRecognitionSession.ResultGenerated +=
async (s, e1) =>
{
if ((e1.Result != null))
{
await this.Dispatcher.RunAsync(CoreDispatcherPriority.Normal,
async () =>
{
await ParsespeechCommand(e1.Result);
});
speechRecognizer.ContinuousRecognitionSession.Resume();
}
};
await speechRecognizer.ContinuousRecognitionSession.StartAsync(SpeechContinuousRecognitionMode.PauseOnRecognition);
}
As discussed I made a demo and made some changes from the codes.
Here are my codes:
private async void myBtn_Click(object sender, RoutedEventArgs e)
{
//here I remove the checkPermission process
var speechRecognizer = new SpeechRecognizer();
if (true)
{
//speechRecognizer.Constraints.Add(new SpeechRecognitionListConstraint(new List<string> { "winffee", "Elvis", "noob" }));
await speechRecognizer.CompileConstraintsAsync();
//recognize speech input at any point of time
speechRecognizer.ContinuousRecognitionSession.ResultGenerated +=
async (s, e1) =>
{
if ((e1.Result != null))
{
await this.Dispatcher.RunAsync(CoreDispatcherPriority.Normal,
() =>//here I remove the async
{
var result = e1.Result;//here I remove the method to focus on the e1.Result.
});
speechRecognizer.ContinuousRecognitionSession.Resume();
}
};
await speechRecognizer.ContinuousRecognitionSession.StartAsync(SpeechContinuousRecognitionMode.PauseOnRecognition);
}
}
The whole function is triggered by a button click event. And I made comment on every change position(totally 3 places).

Azure Graph api client remove member from group

I have trouble to remove user from group. I have no problem with adding a user. I do not recieve any error from myGroup.Members.Remove(user as DirectoryObject);. Is it a bug?
ActiveDirectoryClient client = AuthenticationHelper.GetActiveDirectoryClient();
User user = (User)await client.Users.GetByObjectId(userID).ExecuteAsync();
IGroup myIGroup = await client.Groups.GetByObjectId(objectId).ExecuteAsync();
Group myGroup = (Group)myIGroup;
if (myGroup != null && user != null)
{
try
{
switch (myAction)
{
case "Delete":
myGroup.Members.Remove(user as DirectoryObject);
break;
case "Add":
myGroup.Members.Add(user as DirectoryObject);
break;
}
await myGroup.UpdateAsync();
Try to add Expand(x => x.Members) while getting group. I've tried the following:
public async Task<Result> RemoveFromGroup(string upn, string groupId)
{
try
{
var group = (AD.Group) await ADClient.Groups
.Where(x => x.ObjectId == groupId)
.Expand(x => x.Members)
.ExecuteSingleAsync();
var user = (AD.User) await ADClient.Users
.Where(x => x.UserPrincipalName == upn)
.ExecuteSingleAsync();
group.Members.Remove(user);
await group.UpdateAsync();
return Result.Ok();
}
catch (Exception ex)
{
return Result.Fail(new Error(ex.Message, null, ex));
}
}
It works. The only difference - I get user by its principal name but it doesn't matter in that case.

Resources