advice on architecture for new user/entitlement checks in Bixby - bixby

I am getting ready to add a couple of persistence features to Bixby and want some advice on how to architect it. Specifically, I am going to add
1) check if user is new user
2) if existing user, check entitlements to content packages, each package has a string identifier
These are silent checks, they should be carried out prior to any utterance interaction.
My first thought was to create an action called Initialize with model, action, and code but no view and to call it as a require at the top of each action script. does that make sense? are there some good examples of how to do this?

This wasn't a very stack overflow-friendly question so I plowed ahead on my own. What I wound up doing to begin with is simply adding some code to the default-action.
module.exports.function = function (searchTerm, $vivContext) {
// check if user is already registered
// create query terms
bixbyUserId = $vivContext.bixbyUserId
const checkuserurl = properties.get("config", "baseUrl") + 'example-user-data'
const checkuserquery = {
apikey: properties.get("secret", "apiKey"),
q: "{\"" + properties.get("config", "userIdField") + "\":\"" + bixbyUserId + "\"}"
}
const checkuseroptions = {
format: "json",
query: checkuserquery,
cacheTime: 0
}
// submit query checking if user is in db
const checkuserresponse = http.getUrl(checkuserurl, checkuseroptions)
// if user exists in restdb, accept the userData
if (checkuserresponse && checkuserresponse.length === 1) {
var userData = checkuserresponse[0][properties.get("config", "userDataField")]
userData.$id = checkuserresponse[0]["_id"]
userData.bixbyuserid = bixbyUserId
console.log ('user exists and user id is', userData.bixbyuserid)
}
else {
// if user doesn't exist, create new user id in db with UserData
var userData = {}
userData.newuser = true
const createuserbody = {}
createuserbody[properties.get("config", "userIdField")] = bixbyUserId
createuserbody[properties.get("config", "userDataField")] = JSON.stringify(userData)
const createuserresponse = http.postUrl(checkuserurl, createuserbody, checkuseroptions)
console.log("user didn't exist in restdb so created one")
}
// get AltBrains
Needs testing and could benefit from a logic check.

Related

Is there any way to capture and save end-to-end conversation data into blob storage or cosmos DB in bot framework using SDK V4 Nodedjs

I want to store the conversation data to the storage account or cosmos DB. By trying this https://learn.microsoft.com/en-us/azure/bot-service/bot-builder-howto-v4-storage?view=azure-bot-service-4.0&tabs=javascript#using-blob-storage
I am able to send the utteranceslog into blob storage. But I want to store end-to-end conversation data which includes data of both users as well as bot responses using javascript.
I tried using saving user state and conversation state but didn't achieve the desired output.
I created a custom logger (based on an old botduilder-samples sample that isn't there anymore) that accomplishes this using TranscriptLoggerMiddleware. I chose CosmosDB instead of Blob Storage because I felt it was easier to store (and retrieve) as a JSON document. But you could tweak this concept to use any DB. Here is what I did.
First, create your custom logger code. As mentioned, I used CosmosDB so you might have to change some things if you're using a different DB. The timing of the activities was creating concurrency issues, so instead of working around that, I'm storing the transcript object locally and overwriting the DB object on each turn. Maybe not the most elegant, but it works. Also, I've found my wait function to be required. Otherwise you only get one side of the conversation. I've been told this type of wait function is not a best practice, but awaiting a promise or other methods of creating a delay did not work for me. Here is the code:
customerLogger.js
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.
const { CosmosDbStorage } = require('botbuilder-azure');
const path = require('path');
/**
* CustomLogger, takes in an activity and saves it for the duration of the conversation, writing to an emulator compatible transcript file in the transcriptsPath folder.
*/
class CustomLogger {
/**
* Log an activity to the log file.
* #param activity Activity being logged.
*/
// Set up Cosmos Storage
constructor(appInsightsClient) {
this.transcriptStorage = new CosmosDbStorage({
serviceEndpoint: process.env.COSMOS_SERVICE_ENDPOINT,
authKey: process.env.COSMOS_AUTH_KEY,
databaseId: process.env.DATABASE,
collectionId: 'bot-transcripts'
});
this.conversationLogger = {};
this.appInsightsClient = appInsightsClient;
this.msDelay = 250;
}
async logActivity(activity) {
if (!activity) {
throw new Error('Activity is required.');
}
// Log only if this is type message
if (activity.type === 'message') {
if (activity.attachments) {
var logTextDb = `${activity.from.name}: ${activity.attachments[0].content.text}`;
} else {
var logTextDb = `${activity.from.name}: ${activity.text}`;
}
if (activity.conversation) {
var id = activity.conversation.id;
if (id.indexOf('|') !== -1) {
id = activity.conversation.id.replace(/\|.*/, '');
}
// Get today's date for datestamp
var currentDate = new Date();
var day = currentDate.getDate();
var month = currentDate.getMonth()+1;
var year = currentDate.getFullYear();
var datestamp = year + '-' + month + '-' + day;
var fileName = `${datestamp}_${id}`;
var timestamp = Math.floor(Date.now()/1);
// CosmosDB logging (JK)
if (!(fileName in this.conversationLogger)) {
this.conversationLogger[fileName] = {};
this.conversationLogger[fileName]['botName'] = process.env.BOTNAME;
}
this.conversationLogger[fileName][timestamp] = logTextDb;
let updateObj = {
[fileName]:{
...this.conversationLogger[fileName]
}
}
// Add delay to ensure messages logged sequentially
await this.wait(this.msDelay);
try {
let result = await this.transcriptStorage.write(updateObj);
} catch(err) {
this.appInsightsClient.trackTrace({message: `Logger ${err.name} - ${path.basename(__filename)}`,severity: 3,properties: {'botName': process.env.BOTNAME, 'error':err.message,'callStack':err.stack}});
}
}
}
}
async wait(milliseconds) {
var start = new Date().getTime();
for (var i = 0; i < 1e7; i++) {
if ((new Date().getTime() - start) > milliseconds) {
break;
}
}
}
}
exports.CustomLogger = CustomLogger;
Now you need to attach this to the botframework adapter in your index.js file. The relevant pieces of code are:
index.js
const { TranscriptLoggerMiddleware } = require('botbuilder');
const { CustomLogger } = require('./helpers/CustomLogger');
//
//Your code to create your adapter, etc.
//
const transcriptLogger = new TranscriptLoggerMiddleware(new CustomLogger(appInsightsClient));
adapter.use(transcriptLogger);
I'm assuming here you already have your index.js file figured out, but if you need any assistance getting that set up and getting the transcript logger to work with it, just let me know.
EDIT: By request, here is what the object looks like in CosmosDB. Normally I would have the "from name" displayed, but because of the way I was testing the bot it came through "undefined".
{
"id": "2020-3-21_IfHK46rZV42KH5g3dIUgKu-j",
"realId": "2020-3-21_IfHK46rZV42KH5g3dIUgKu-j",
"document": {
"botName": "itInnovationBot",
"1584797671549": "Innovation Bot: Hi! I'm the IT Innovation Bot. I can answer questions about the innovation team and capture your innovation ideas. Let me know how I can help!",
"1584797692355": "undefined: Hello",
"1584797692623": "Innovation Bot: Hello.",
"1584797725223": "undefined: Tell me about my team",
"1584797725490": "Innovation Bot: The innovation team is responsible for investigating, incubating, and launching new technologies and applications. The innovation focus areas are:\n\n* Chatbots\n\n* Augmented Reality/Virtual Reality\n\n* Blockchain\n\n* Robotic Process Automation\n\n* AI & Machine Learning\n\nLet me know if you want to learn more about any of these technologies!",
"1584797746279": "undefined: Thanks",
"1584797746531": "Innovation Bot: You're welcome."
},
"_rid": "OsYpALLrTn2TAwAAAAAAAA==",
"_self": "dbs/OsYpAA==/colls/OsYpALLrTn0=/docs/OsYpALLrTn2TAwAAAAAAAA==/",
"_etag": "\"a4008d12-0000-0300-0000-5e7618330000\"",
"_attachments": "attachments/",
"_ts": 1584797747
}
To read the conversation back (even if still in the middle of the conversation), you just create a connector in your bot, recreate the key, and read the file as below (in this case id is passed into my function and is the conversation id):
const transcriptStorage = new CosmosDbStorage({
serviceEndpoint: process.env.COSMOS_SERVICE_ENDPOINT,
authKey: process.env.COSMOS_AUTH_KEY,
databaseId: process.env.DATABASE,
collectionId: 'bot-transcripts',
partitionKey: process.env.BOTNAME
});
// Get today's date for datestamp
var currentDate = new Date();
var day = currentDate.getDate();
var month = currentDate.getMonth()+1;
var year = currentDate.getFullYear();
var datestamp = year + '-' + month + '-' + day;
var filename = `${datestamp}_${id}`;
var transcript = await transcriptStorage.read([filename]);

How to interpolate variable into firebase snapshot?

I am using firebase functions to send users a notification. I can user string interpolation like so:
const original = change.after.val();
const lover = context.auth.uid;
const recipient_username = original.username;
const topic = `${recipient_username}_notifications`;
But now I need to make a database call to get the username from the user_id, and use the username to get the value of loves form the 'original' snapshot, but this does not work:
return db.ref('/users/' + lover).once('value', (snapshot) => {
var username = snapshot.val().username;
var love = original.loves.username // I need this to use the variable username, but it is just saying "username"
console.log("lover username")
console.log(username)
console.log("loves")
console.log(loves)
const payload = {
notification:{
title: "You've got Love!",
body: `${username} sent you Love! Refresh your inbox to recieve it, and why not send some back!`
}
};
How can I change var love = original.loves.username to be something like: var love = original.loves.${username}?
The database looks like this:
users/
username: usernamehere
love/
otherusername: 10 // the amount of love they sent.
You have called .val() on the original turning this into a Javascript object.
Traversing paths in Javascript objects can be done with the .loves helper functions or using string lookups. Try the following
var username = snapshot.val().username;
var love = original.loves[username];

How to create OutPutContext via V2 client library for node js

I am working on entity and intents creation in my agent using v2 client library for node.js . And for that i am going through this sample which is on git. And it says something related to session id and context id. Can anyone explain me what is sessionId and contextId. And also provide me link where i can read those thing in details.
I am unable to create context by following those example. How can i create context while creating intent at the same time.
The following is code to create a context. You cannot create a context and an intent in a single API call, you first need to create the context and then create the intent that uses the context. The response to the create context API call will return a context ID you can use in your intent.
const dialogflow = require('dialogflow');
// Instantiates clients
const entityTypesClient = new dialogflow.EntityTypesClient();
// The path to the agent the created entity type belongs to.
const agentPath = entityTypesClient.projectAgentPath(projectId);
const createEntityTypeRequest = {
parent: agentPath,
entityType: {
displayName: displayName,
kind: kind,
},
};
entityTypesClient
.createEntityType(createEntityTypeRequest)
.then(responses => {
console.log(`Created ${responses[0].name} entity type`);
})
.catch(err => {
console.error('Failed to create size entity type:', err);
});
Source: https://github.com/googleapis/nodejs-dialogflow/blob/master/samples/resource.js
Contexts are very closely associated with SessionID. Say for eg, you have a chatbot that gets spun up on two computers serving two different user's. Each user will have a respective session_id (If you're coding in NODE, when a new user fires the chatbot, you need to ensure he/she will get a unique session_id).
Now, every unique session id will have unique contexts. From above example, let's say user 1 will initialize an intent that has input context named 'abc' with lifespan of 2 and user 2 will initialize another intent that has input context named 'xyz' with lifespan of 5, these respective contexts gets recorded against each of these user's individual session id's. You can programatically control (edit) contexts and its lifecycle. This is the biggest advantage of code facilitated Dialogflow as opposed to using GUI. Using services like Firebase, you can also preserve session id's and its associated contexts so, next time same user sign's in again, they can start from where they had last left.
I can share a snippet from one of my previous projects where I was managing contexts programatically. Initialization script is as follows:
/**
* #author Pruthvi Kumar
* #email pruthvikumar.123#gmail.com
* #create date 2018-08-15 04:42:22
* #modify date 2018-08-15 04:42:22
* #desc Dialogflow config for chatbot.
*/
const dialogflow_config = {
projectId: 'xxx',
sessionId: 'chatbot-session-id', //This is default assignment. This will hve to be overridden by sessionId as obtained from client in order to main context per sessionId.
languageCode: 'en-US'
};
exports.configStoreSingleton = (function () {
let instanceStacks;
let instanceSessionId;
let contextStack = {};
let intentsStack = {};
let successfulIntentResponseStack = {};
function init() {
contextStack[dialogflow_config['sessionId']] = [];
intentsStack[dialogflow_config['sessionId']] = [];
successfulIntentResponseStack[dialogflow_config['sessionId']] = [];
return {
contextStack: contextStack,
intentsStack: intentsStack,
successfulIntentResponseStack: successfulIntentResponseStack
};
}
return {
init: function () {
if (!instanceStacks || (instanceSessionId !== dialogflow_config['sessionId'] && (!intentsStack[dialogflow_config['sessionId']]))) {
console.log('[dialogflow_config]: Singleton is not instantiated previously or New userSession is triggered! Fresh instance stack will be provisioned');
instanceStacks = init();
instanceSessionId = dialogflow_config['sessionId'];
}
return instanceStacks;
}
};
})();
exports.updateSessionIdOfDialogflowConfig = function (sessionId) {
if (typeof (sessionId) === 'string') {
dialogflow_config['sessionId'] = sessionId;
return true;
} else {
console.warn('[dialogflow_config]: SessionId must be of type STRING!');
return;
}
};
exports.getDialogflowConfig = function () {
return dialogflow_config;
};
And then, to programmatically manage contexts:
/**
* #author Pruthvi Kumar
* #email pruthvikumar.123#gmail.com
* #create date 2018-08-15 04:37:15
* #modify date 2018-08-15 04:37:15
* #desc Operate on Dialogflow Contexts
*/
const dialogflow = require('dialogflow');
const dialogflowConfig = require('../modules/dialogflow_config');
const structjson = require('./dialogflow_structjson');
const util = require('util');
const contextsClient = new dialogflow.ContextsClient();
exports.setContextHistory = function (sessionId, intent_name, context_payload, preservedContext=false) {
/* maintain context stack per session */
/* context_payload = {input_contexts: [], output_contexts = []}
*/
const contextStack = dialogflowConfig.configStoreSingleton.init().contextStack;
if (intent_name) {
contextStack[sessionId].push({
intent: intent_name,
contexts: context_payload,
preserveContext: preservedContext
});
} else {
console.warn('[dialogflow_contexts]: Intent name is not provided OR Nothing in context_payload to add to history!');
}
};
exports.getContextHistory = function () {
const contextStack = dialogflowConfig.configStoreSingleton.init().contextStack;
return contextStack;
}
exports.preserveContext = function () {
const contextStack = dialogflowConfig.configStoreSingleton.init().contextStack;
//Traverse contextStack, get the last contexts.
let context_to_be_preserved = contextStack[dialogflowConfig.getDialogflowConfig()['sessionId']][contextStack[dialogflowConfig.getDialogflowConfig()['sessionId']].length - 1];
//console.log(`context to be preserved is: ${util.inspect(context_to_be_preserved)}`);
return context_to_be_preserved['contexts'].map((context, index) => {
let context_id = exports.getContextId(context);
return exports.updateContext(context_id, true)
});
}
From here, you can reference this github resource to build your own contexts - https://github.com/googleapis/nodejs-dialogflow/blob/master/samples/resource.js
Happy creating digital souls!

Documentdb Failed to deserialize stored procedure response or convert it to my defined type

My Stored Procedure: (I created it via Azure Script Explorer)
function GetAllResources() {
var collection = getContext().getCollection();
// Query documents and take 1st item.
var isAccepted = collection.queryDocuments(
collection.getSelfLink(),
'SELECT * FROM MultiLanguage as m',
function (err, docs, options) {
if (err) throw err;
// Check the feed and if empty, set the body to 'no docs found',
// else take 1st element from feed
if (!docs || !docs.length) getContext().getResponse().setBody('no docs found');
else getContext().getResponse().setBody(JSON.stringify(docs));
});
if (!isAccepted) throw new Error('The query was not accepted by the server.');
}
The sproc can be executed successfully from script explorer.
My C# code to call the sproc:
public async Task<IHttpActionResult> GetReources() {
client = new DocumentClient(new Uri(ConfigurationManager.AppSettings["endpoint"]), ConfigurationManager.AppSettings["authKey"]);
var collectionLink = UriFactory.CreateDocumentCollectionUri(DatabaseId, CollectionId);
//var docs = await client.ReadDocumentFeedAsync(collectionLink, new FeedOptions { MaxItemCount = 10 });
//var docs = from d in client.CreateDocumentQuery<Models.Resource>(collectionLink)
// select d;
StoredProcedure storedProcedure = client.CreateStoredProcedureQuery(collectionLink).Where(c => c.Id == "GetAllResources").AsEnumerable().FirstOrDefault();
Models.Resource docs = await client.ExecuteStoredProcedureAsync<Models.Resource>(storedProcedure.SelfLink);
foreach (var d in docs) {
Models.Resource a = new Models.Resource();
a = docs;
//a.id = d.id;
//a.Scenario = d.Scenario;
//a.Translations = d.Translations;
//a.LastModified = d.LastModified;
//a.ModifiedBy = d.ModifiedBy;
//a.LastAccessed = d.LastAccessed;
resources.Add(a);
}
return Ok(resources);
}
First, there is an error for the "foreach..." like said
foreach cannot operate on variables of type Models.Resource because it
doesn't contain a public definition of GetEnumerator.
Then I tried to modify my sproc to only return 1 result and remove the foreach line, then I got error said
Failed to deserialize stored procedure response or convert it to type
'Models.Resource'
I just want to return the result of the stored procedure as my defined class (Models.Resource). How to do this?
It can be simpler to get sproc by name using CreateStoredProcedureUri, like this:
const string endpoint = "https://your.service.azure.com:443/";
const string authKey = "<your magic secret master key>==";
var client = new DocumentClient(new Uri(endpoint), authKey);
Uri sprocUri = UriFactory.CreateStoredProcedureUri("databaseName", "collectionName", "GetAllResources");
var result = await client.ExecuteStoredProcedureAsync<string>(sprocUri);
The stored procedure above serializes results of the query (docs array) to string, if you keep it this way, the result of sproc would be string, which I guess you would need to manually deserialize to objects. You can do this simpler, just return docs from sproc and have result as objects (like Models.Resource[]), serialization would happen automatically.
If you change the sproc to return just one doc (e.g. do __.response.setBody(docs[0]) and Models.Resource represent one item, then the call is correct:
Models.Resource doc = await client.ExecuteStoredProcedureAsync<Models.Resource>(sprocUri);
Also, to // Query documents and take 1st item, I wouldn't recommend to use script as script has overhead of running JavsScript engine. Scripts kick in when you have bulk operations (to optimize for network traffic) or have business logic which makes sense to run on the server. To take 1st item you can do query from client like this: SELECT TOP 1 * FROM c. Typically you would WHERE and ORDER BY clause to that.
There is a number of docdb samples on github, for instance, https://github.com/Azure/azure-documentdb-dotnet/tree/master/samples/code-samples/ServerSideScripts and https://github.com/Azure/azure-documentdb-dotnet/tree/master/samples/code-samples/Queries.
Thanks,
Michael
All right, let's make sure we are on the same page.
I am using the sproc same as above.
I am using client code like this:
class Models
{
// This would have more properties, I am just using id which all docs would have.
public class Resource
{
[JsonProperty("id")]
public string Id { get; set; }
}
}
public async Task<IHttpActionResult> GetResources()
{
const string endpoint = "https://myservice.azure.com:443/";
const string authKey = "my secret key==";
var client = new DocumentClient(new Uri(endpoint), authKey);
Uri sprocUri = UriFactory.CreateStoredProcedureUri("db", "c1", "GetAllResources");
var serializedDocs = await client.ExecuteStoredProcedureAsync<string>(sprocUri);
Models.Resource[] resources = JsonConvert.DeserializeObject<Models.Resource[]>(serializedDocs);
return Ok(resources);
}
It works fine. Is this what you are doing?

How to get and display the email using the Facebook provider in ASP.NET Identity and MVC 5

UPDATE: PLEASE SEE THIS POST: https://stackoverflow.com/a/20379623/687549
Been reading I think almost all questions on SO about external providers and how to get extra info/data/metadata/claims/orwhateveryouwannacallit in particular the email address which many use as the username on modern websites.
So the problem that I was facing was that I wanted to retrieve the email from the Facebook provider with as little code as possible. I thought to myself; the new ASP.NET Identity framework must have this buil-in and can do this with probably just a couple of lines of code. I searched and all I could find was these imensely large chunks of code and I thought to myself: there has got to be another more simpler way. And here it is, as an answer in this questionthread.
I managed to get this working with both Facebook and Google but what I'm concerned about is wheather or not I'm doing this right without any consequenses somewhere else in the code.
For instance: Do you really only need to specify the Scopes.Add("email") for it all to work or why haven't I been able to find more info about this on the interweb?
UPDATE: PLEASE SEE THIS POST: https://stackoverflow.com/a/20379623/687549
Startup.Auth.cs:
var facebookAuthenticationOptions = new FacebookAuthenticationOptions()
{
AppId = "myAppId",
AppSecret = "myAppSecret"
};
facebookAuthenticationOptions.Scope.Add("email");
app.UseFacebookAuthentication(facebookAuthenticationOptions);
AccountController (default mvc 5 template app stuff)
//
// GET: /Account/ExternalLoginCallback
[AllowAnonymous]
public async Task<ActionResult> ExternalLoginCallback(string returnUrl)
{
var loginInfo = await AuthenticationManager.GetExternalLoginInfoAsync();
if (loginInfo == null)
{
return RedirectToAction("Login");
}
// These next three lines is how I get the email from the stuff that gets returned from the Facebook external provider
var externalIdentity = HttpContext.GetOwinContext().Authentication.GetExternalIdentityAsync(DefaultAuthenticationTypes.ExternalCookie);
var emailClaim = externalIdentity.Result.Claims.FirstOrDefault(c => c.Type == ClaimTypes.Email);
var email = emailClaim.Value;
// Sign in the user with this external login provider if the user already has a login
var user = await UserManager.FindAsync(loginInfo.Login);
if (user != null)
{
await SignInAsync(user, isPersistent: false);
return RedirectToLocal(returnUrl);
}
else
{
// If the user does not have an account, then prompt the user to create an account
ViewBag.ReturnUrl = returnUrl;
ViewBag.LoginProvider = loginInfo.Login.LoginProvider;
// Populate the viewmodel with the email
return View("ExternalLoginConfirmation", new ExternalLoginConfirmationViewModel { UserName = email });
}
}
I have the same problem. You need to edit and add this code to ExternalLoginCallback in the AccountController
var loginInfo = await AuthenticationManager.GetExternalLoginInfoAsync();
if (loginInfo == null)
{
return RedirectToAction("Login");
}
// added the following lines
if (loginInfo.Login.LoginProvider == "Facebook")
{
var identity = AuthenticationManager.GetExternalIdentity(DefaultAuthenticationTypes.ExternalCookie);
var access_token = identity.FindFirstValue("FacebookAccessToken");
var fb = new FacebookClient(access_token);
dynamic myInfo = fb.Get("/me?fields=email"); // specify the email field
loginInfo.Email = myInfo.email;
}
Note the code dynamic myInfo = fb.Get("/me?fields=email"); this will work for facebook app with version 2.4, but for old version you can write this
dynamic myInfo = fb.Get("email");

Resources