Chatbot with large DB integration - bots

We want to build a Bot, which will ask questions to user and then
will record their response. This Q&A will be intelligently done based upon responses from user.
We have few initial questions at this moment;
There has been lots of API's available for bot creation like api.ai, wit.ai, botkit, IBM Watson, Microsoft Bot framework. Which one is best from development point of view in Java or Node.Js or Python. Also, from the point of having control on the data and complete flow.
We want to provide the questions from some DB[RDBMS or NOSQL] to our Bot as it will be large in future, which api would be best for the same.
We want to store the user response to the database with userid.
Based upon user response stored in DB, to the questions asked by bot we want to perform the analytics on it.
Can you please suggest, if this can be done using any one of the Bot API and which one should be preferred.
Thank you,
Amit

In your 4 conditions, all can be done using IBM Watson.
1:
Use Conversation Service for create a Chatbot, and you can save all user input with context variables.
IBM Watson provided some examples in Python, Node JS and Java SDK, just click in some Programming language for check the example and all codes.
2:
This example use Cloudant DB (nosql) from Conversation Simple Node.js link, but you can use other.
function log(input, output) {
if ( logs ) {
// If the logs db is set, then we want to record all input and responses
var id = uuid.v4();
logs.insert( {'_id': id, 'request': input, 'response': output, 'time': new Date()} );
}
}
if ( cloudantUrl ) {
// If logging has been enabled (as signalled by the presence of the cloudantUrl) then the
// app developer must also specify a LOG_USER and LOG_PASS env vars.
if ( !process.env.LOG_USER || !process.env.LOG_PASS ) {
throw new Error( 'LOG_USER OR LOG_PASS not defined, both required to enable logging!' );
}
// add basic auth to the endpoints to retrieve the logs!
var auth = basicAuth( process.env.LOG_USER, process.env.LOG_PASS );
// If the cloudantUrl has been configured then we will want to set up a nano client
var nano = require( 'nano' )( cloudantUrl );
// add a new API which allows us to retrieve the logs (note this is not secure)
nano.db.get( 'car_logs', function(err) {
if ( err ) {
console.error( err );
nano.db.create( 'car_logs', function(errCreate) {
console.error( errCreate );
logs = nano.db.use( 'car_logs' );
} );
} else {
logs = nano.db.use( 'car_logs' );
}
} );
3: All call conversation have some id, you can access this with context variable. Example (With IBM Watson conversation:
context.conversation_id
4: You can use other service from IBM Watson for it, but i recommend: AlchemyAPI or Discovery, depends on what you are really going to do. But take a look at both that I'm sure they poed to help you.

Related

Read data from Firebase in Dialogflow

I am building a chatbot but I got stuck fetching data from Firebase from Dialogflow.
I want a user to input his name, the bot to query Firebase and check if the name exists. If it does "user exists" else "user n not in the database".
[this is the code am using presently][1]
function SaveName(agent){
const number = agent.parameters.number; // when I input the number in Fialogflow
const docRef = db.collection('names').doc(sessionId);
return docRef.get()
.then(doc => {
if (!doc.exists) {
agent.add('No data found in the database!');
console.log(doc);
} else {
agent.add(doc.data().orders);
}
return Promise.resolve('Read complete');
}).catch(() => {
agent.add('Error reading entry from the Firestore database.');
agent.add('Please add a entry to the database first by saying, "Write <your phrase> to the database"');
});
}
This is possible by configuring the fulfillment for your agent.
Go to Intents, select your intent and then scroll down to the fulfillment section.
Click on Enable webhook call for this intent (see Custom webhook).
Create the webhook (you can enable the inline editor at Fulfillment - Inline Editor to do so).
In your webhook, query Firestore (see this answer).
Note that anyone will be able to know if a user exists or not in your db.
If you are building for the Google Assistant, have a look at account linking.
I was facing some similar issue where the fetched details were not being shown in the agent interaction. This happened with the template given here too. I then just removed the responses from the ReadFromFirestore Intent and was getting the expected fetched queries back. I again tried by putting the response back and calling the intent and the response was sort of alternating between the default response and the firestore query response.
Still, not keeping it worked for me and thought would help others too.

QnAmaker.ai with ms botframework node.js: welcome message and scoring answers

I am using QnAmaker.ai for FAQ bot developed using node.js msbotframework. I want to implement few more additional features:
Welcome message to the user when the bot connects.
Provide scores for every answer. For e.g. Was this useful "Yes" and "No". Is there a way to store this information in the QnAmaker KB.
Also, I have enabled chit-chat along with my custom set of question and answers. However answers from chit-chat are taking preference over the custom ones. I want the custom answers to override the chit-chat.
The code I am using as of now is very basic and picked up from the tutorial:
var previewRecognizer = new builder_cognitiveservices.QnAMakerRecognizer({
knowledgeBaseId: process.env.QnAKnowledgebaseId,
authKey: process.env.QnAAuthKey || process.env.QnASubscriptionKey
});
var basicQnAMakerPreviewDialog = new builder_cognitiveservices.QnAMakerDialog({
recognizers: [previewRecognizer],
defaultMessage: 'Sorry, I did not understand. Please say that again.',
qnaThreshold: 0.3
}
);
bot.dialog('basicQnAMakerPreviewDialog', basicQnAMakerPreviewDialog);
// Recognizer and and Dialog for GA QnAMaker service
var recognizer = new builder_cognitiveservices.QnAMakerRecognizer({
knowledgeBaseId: process.env.QnAKnowledgebaseId,
authKey: process.env.QnAAuthKey || process.env.QnASubscriptionKey, // Backward compatibility with QnAMaker (Preview)
endpointHostName: process.env.QnAEndpointHostName
});
var basicQnAMakerDialog = new builder_cognitiveservices.QnAMakerDialog({
recognizers: [recognizer],
defaultMessage: 'Sorry, I did not understand. Please say that again.',
qnaThreshold: 0.3
}
);
bot.dialog('basicQnAMakerDialog', basicQnAMakerDialog);
bot.dialog('/', //basicQnAMakerDialog);
[
function (session) {
var qnaKnowledgebaseId = process.env.QnAKnowledgebaseId;
var qnaAuthKey = process.env.QnAAuthKey || process.env.QnASubscriptionKey;
var endpointHostName = process.env.QnAEndpointHostName;
// QnA Subscription Key and KnowledgeBase Id null verification
if ((qnaAuthKey == null || qnaAuthKey == '') || (qnaKnowledgebaseId == null || qnaKnowledgebaseId == ''))
session.send('Please set QnAKnowledgebaseId, QnAAuthKey and QnAEndpointHostName (if applicable) in App Settings. Learn how to get them at https://aka.ms/qnaabssetup.');
else {
if (endpointHostName == null || endpointHostName == '')
// Replace with Preview QnAMakerDialog service
session.replaceDialog('basicQnAMakerPreviewDialog');
else
// Replace with GA QnAMakerDialog service
session.replaceDialog('basicQnAMakerDialog');
}
}
]);
Thanks
I see that you are using BotBuilder SDK v3 at the moment, where v4 is the newest release. My answers below are focussed on v3, however for such a simple bot upgrading to v4 wouldn't be really hard. v3 won't get feature updates from now on.
Welcome message to the user when the bot connects.
This functionality can differ on a per channel basis. You could listen to the conversationUpdate event to trigger a message or you could post an event activity in the WebChat for example. Links below describe both ways:
Handle user and conversation events (v3)
How to properly send a greeting message and common issues from customers
Provide scores for every answer
This isn't possible yet in QnaMaker at the moment. I would advise to use custom storage for this, like Application Insights or CosmosDB.
Take a look at the newly announced Active Learning possibilities of QnAMaker. They don't offer an SDK for Node yet, but this functionality might be of your interest if you are interested in getting more insights (and using it to train your model).
I want the custom answers to override the chit-chat
The Chit-chat feature just adds a pre-populated set of in your knowledge base, which isn't any different than the QnA sets you added yourself. It isn't possible to 'override' the ChitChat by your own answers by default.
You could remove the specific ChitChat QnA sets which you would like to override or you could correct the top scoring answer.

Getting the user id from a Firestore Trigger in Cloud Functions for Firebase?

In the example bellow, is there a way to get the user id (uid) of the user who wrote to 'offers/{offerId}'? I tried to do as described here but it doesn't work in Firestore.
exports.onNewOffer = functions.firestore
.document('offers/{offerId}')
.onCreate(event => {
...
});
I was struggling on this for a while and finally contacted the firebase Support:
The event.auth.uid is undefined in the event object for firestore database triggers. (It works for the realtime Database Triggers)
When I console.log(event) I can’t find any auth in the output.
The official support answer:
Sorry the auth is not yet added in the Firestore SDK. We have it listed in the next features.
Keep an eye out on our release notes for any further updates.
I hope this saves someone a few hours.
UPDATE:
The issue has been closed and the feature will never be implemeted:
Hi there again everyone - another update. It has been decided that unfortunately native support for context.auth for Firestore triggers will not be implemented due to technical constraints. However, there is a different solution in the works that hopefully will satisfy your use case, but I cannot share details. On this forum we generally keep open only issues that can be solved inside the functions SDK itself - I've kept this one open since it seemed important and I wanted to provide some updates on the internal bugs tracking this work. Now that a decision has been reached, I'm going to close this out. Thanks again for everyone's patience and I'm sorry I don't have better news. Please use the workaround referenced in here.
Summary of how I solved this / a workable solution:
On client
Add logged in/current user's uid (e.g. as creatorId) to entity they're creating. Access this uid by storing the firebase.auth().onAuthStateChanged() User object in your app state.
In Firebase Firestore/Database
Add a Security Rule to create to validate that the client-supplied creatorId value is the same as the authenticated user's uid; Now you know the client isn't spoofing the creatorId and can trust this value elsewhere.
e.g.
match /entity/{entityId} {
allow create: if madeBySelf();
}
function madeBySelf() {
return request.auth.uid == request.resource.data.creatorId;
}
In Firebase Functions
Add an onCreate trigger to your created entity type to use the client-supplied, and now validated, creatorId to look up the creating user's profile info, and associate/append this info to the new entity doc.
This can be accomplished by:
Creating a users collection and individual user documents when new accounts are created, and populating the new user doc with app-useful fields (e.g. displayName). This is required because the fields exposed by the Firebase Authentication system are insufficient for consumer app uses (e.g., displayName and avatarURL are not exposed) so you can't just rely on looking up the creating user's info that way.
e.g. (using ES6)
import * as functions from 'firebase-functions'
import * as admin from 'firebase-admin'
const APP = admin.initializeApp()
export const createUserRecord = functions.auth.user()
.onCreate(async (userRecord, context) => {
const userDoc = {
id: userRecord.uid,
displayName: userRecord.displayName || "No Name",
avatarURL: userRecord.photoURL || '',
}
return APP.firestore().collection('users').doc(userRecord.uid).set(userDoc)
})
Now that you have a validated creatorId value, and useful user objects, add an onCreate trigger to your entity type (or all your created entities) to look up the creating user's info and append it to the created object.
export const addCreatorToDatabaseEntry = functions.firestore
.document('<your entity type here>/{entityId}')
.onCreate(async (snapshot, context) => {
const userDoc = await APP.firestore().collection('users').doc(snapshot.data().creatorId).get()
return snapshot.ref.set({ creator: userDoc.data() }, { merge: true })
})
This clearly leads to a lot of duplicated user info data throughout your system -- and there's a bit of clean up you can do ('creatorId` is duplicated on the created entity in the above implementation) -- but now it's super easy to show who created what throughout your app, and appears to be 'the Firebase way'.
Hope this helps. I've found Firebase to be super amazing in some ways, and make some normally easy things (like this) harder than they 'should' be; on balance though am a major fan.
The documentation states clearly that the context.auth param is only available in the Realtime Database.
This field is only populated for Realtime Database triggers and
Callable functions. For an unauthenticated user, this field is null.
For Firebase admin users and event types that do not provide user
information, this field does not exist.
Personally I realized that I had the userId already in the path of my data.
export const onCreate = functions.firestore.document('docs/{userId}/docs/{docId}')
.onCreate((snapshot, context) => {
const userId = context.params.userId;
Until this is added to firestore functions a workaround is to add the user_id as a field when creating a document then deleting after. You can then grab it in the function onCreate then after you use it for what you need it for, while still in the function, just delete the field from that document.
As already suggested above, the workaround will be to add a user_id field with the data itself and then read it on the server.
The drawback with this approach will be a security loophole. As we are not verifying the user id on the server, any other user will be able to impersonate other users by sending their id with the data.
For security critical applications, the solution for this will be to use security rules to verify that the correct user_id has been sent with the data
allow write: if resource.data.user_id == request.auth.uid;
You could add your data via Callable function, from which you can access current user id:
exports.addNewOffer = functions.https.onCall(async (data, context) => {
const uid = context.auth.uid
const writeResult = await admin.firestore().collection('offers').add({ ... })
...
return { id: writeResult.id, ... }
})
What is about snap._fieldsProto.uid.stringValue
Example:
exports.hello = functions.firestore.document('hello/{worldId}').onCreate((snap, context) => {
console.log(snap._fieldsProto.uid.stringValue)
});
This should do the trick:
exports.newMessage = functions.firestore
.document('groups/messages/{messageId}')
.onCreate((snap, context) => {
var fromId = snap.data().fromId;
You can get the current signed-in user tokenId by calling getIdToken() on the User: https://firebase.google.com/docs/reference/functions/functions.auth.UserInfo

How to detect when bot was added to conversation and other events?

I am testing a bot that I am building using the Bot Framework. The emulator for local testing that Microsoft created has several events that can be provided to the bot to solicit a response.
I looked at the GitHub samples provided for Node.js here, but I can not find any example that responds to the different events within the Bot Framework Emulator.
The states are:
Bot Added to Conversation
Bot Removed from Conversation
User Added to Conversation
User Removed from Conversation
End of Conversation
Ping
Delete User Data
The API also does not make it clear how to achieve any of these actions.
Does anyone have any insight on where I should be looking for a example, or the API entries that I should be using?
In response to one of the answers, I did try code -
.onDefault(function (session) { console.log(session.message.type); }
But it only ever display "message" if a message was sent by the user.
The incoming message.type field will have "BotAddedToConversation" etc.
For the Node SDK, the botConnectorBot is able to trigger custom listeners on events using the on() handler.
Example
var builder = require('botbuilder');
var bot = new builder.BotConnectorBot({ appId: 'APPID', appSecret: 'APPSECRET' });
bot.on('DeleteUserData', function(message) {
// Handle Deleting User Data
});
More information can be found here.
You are also able to configure some standard messages using the configure() method.
Example
bot.configure({
userWelcomeMessage: "Hello... Welcome to the group.",
goodbyeMessage: "Goodbye..."
});
More information on what can be configured through options is located here.
Concerns
This is not part of the question, as the question was to identify how to listen to these events. But as a general concern, the event listener does not return a session object. It is unclear how to act once you handle the event.

Paypal: Transaction History with REST API / Nodejs SDK

I would like to display all the inbound/outbound transactions a user made/received to display it in a simple html list.
I'm using the recommended node.js module https://github.com/paypal/rest-api-sdk-nodejs and I'm trying to use this code to get the transactions:
// Get Transaction History
paypal.payment.list({
'count': 100,
'sort_by':'create_time',
'sort_order': 'asc',
'start_time': '2008-03-06T11:00:00Z'
}, function(error, payment_history){
if(error){
console.error('error', error);
response.data.payment_history = {};
} else {
console.log('history', payment_history);
response.data.payment_history = payment_history;
}
response.finish();
});
but payment_history gives me { count: 0 }. I'm pretty sure that I have transactions since 2008.
I'm not really sure what's the problem. The user is already logged in using the access_token and I can display user informations but I have no idea how to pull transaction informations.
Look into https://developer.paypal.com/docs/faq/#general-developer-questions for the answer to "Can I retrieve the history of transactions created using the Classic APIs using the REST API?".
I also ran into this issue. Strangely, It seems that the PayPal REST API only returns results for payments made through the REST API. See here: https://stackoverflow.com/a/18139747
Classic API will remain the way to do this until the REST API is improved.

Resources