Just getting started with nlp.js, and I'd like to be able to test out some ideas with their Express API server package.
As far as I can tell, there's no way to "intervene" in the QnA bot exchange. For instance, if I'd like to format the output to contain the user's name or a time or whatever.
Say my corpus was a tsv file with:
some question \t welcome, #name
And I wanted to swap out that #name tag? Right now, I just get that string exactly as is.
In the conf.json:
"api-server": {
"port": 3000,
"serveBot": true
}
Maybe there's a pipeline logic to do that?
Can't seem to find a lot of reference material on available events in the pipeline or how to intercede in the WebChat flow out there.
Related
I'm trying to make an app using DialogFlow which finds a specific object in a specific place.
This is a generic example.
The user would say something like "Where to I find Dog in Europe" and the app would reply with "Dog can be found in Europe via: breeding, finding it out in the wild or by buying it"
considering Dog as input1 and europe as input2
Ideally the app should be able to cross reference input1 and input2 to find the correct response. Can I implement a database like structure and do this?
You can't access a database from Dialogflow directly, but you can build your own fulfillment backend that can do anything you want. It communicates with Dialogflow via HTTP requests/responses in the Dialogflow Webhook format.
Here is an example fulfillment that reads data from Firebase database - https://github.com/actions-on-google/dialogflow-updates-nodejs
You can't access a database directly in Dialog flow, but you can build your own fulfillment back end. I have been using Airtable as a database and Integromat and Webhooks to query the database and parse the results back to Dialogflow. As a novice coder I found this to be the simnplest way.
KaySubb is right, you can make a fulfillment that reads data from a firebase database(or firestore).
You can do this turning on fulfillment at the bottom page of the intent page.
First go to https://console.firebase.google.com/ (login with google account) and you should be able to see your google cloud platform project.
To use firebase, you need to first install it. Get node.js as you need npm first. I'm not sure what OS you're on but go into command line or terminal and type.
npm install firebase --save
then type:
firebase login
this will authenticate your login and connect your project when you deploy.
Then use go to the directory you want to create your project in:
firebase init functions
Select your project and select javascript, install all dependencies
Now go to functions and open the index.js file. Here you can change you write code needed in js.
Write your functions and type:
firebase deploy
in the command line open in the file directory. When it completes, it will
give you a link. This as the webhook URL in dialogflow (it should start with
https://us-central). If you see only 1 link which says
console.firebase.google.com....... then open that link on a browser, click on
"functions" on the left side of the screen and get the link from there.
This should get you started with firebase, now you can link your project to firebase fulfillment. There is great firestore explanation here
https://www.youtube.com/watch?v=kdk6MhhI8oc
But I'll give you a brief explanation:
On the top of your index.js file you will need:
const functions = require('firebase-functions');
var admin = require("firebase-admin");
admin.initializeApp(functions.config().firebase);
var firestore = admin.firestore();
The basic code is here:
exports.webhook = functions.https.onRequest((request, response) => {
switch(request.body.result.action){
case 'saveData':
let params = request.body.result.parameters
firestore.collection('colName').doc('docName').add({
name:params.name
age:params.age
}).then(() => {
response.send({
speech:
`this is a response for "${params.name}".`
});
})
.catch((e => {
console.log('Error getting documents', e);
response.send({
speech:
`Sorry, something has gone wrong. Try again and if the problem persists, please report it.`
});
}))
break;
default:
}
})
I'll explain what it does:
You need the switch to decide which intent to do. request.body.result.action returns the action name (write this in dialogflow just above the parameters).
Once that is decided request.body.result.parameters give you the parameters from the intent. params.______ gives you the parameter.
I would definitely recommend reading the official documentation:
https://firebase.google.com/docs/firestore/quickstart
to help understand the data structure to help create the ideal database for you. Essentially a collection is a list and within that a doc is one entry. You can name them yourself of using the entries from param.
respond.send is what the bot will reply to the user, I've also shown how to use the parameters in the response.
.catch will just store any errors in the log, you can read the log in console.firebase.google.com.... open your project and click on function. There will be a place to read logs there. You can check any errors encountered over there.
default: will output whatever default response you wrote on dialogflow at the bottom of the intent.
Hope this helps,comment any questions. I have gone through a huge amount as concisely as I could. This will take some time to get used to and become good at, follow the docs and the youtube videos if you have a lot of trouble!
If you're having even more trouble, there is a slack that helps people that I can direct you to.
I have created a node/feathers project using this chat application guide as a base. It's working great, but now I would like to filter the results the api is giving. For example, when user makes GET request to /messages I would like the response to include only the messages that the authorized user has created, not anyone else's messages. Auth is working correctly in the api and message items have the userId who created the message, but I just don't understand what and where I'm supposed to do to filter the messages according to the user id. After hours of googling I couldn't find anything related to this or anyone even asking the question, so what am I missing here?
You can do a manual filtering. Both on before and after hooks. How to use hooks.
In before hooks you can create a function that update your query object to only get/find data it owns.
hook.params.query = { ... , ownedBy: hook.params.user._id }
Or do result filtering in after hooks, you have the hook.result which is the only thing you can manipulate in the after hooks. Then you can use Array.prototype.filter() to filter the results the user gets.
when trying to call application insights api service with this url it gave me 400 Bad request
https://api.applicationinsights.io/beta/apps/appID/events/pageViews?timespan=P30D&$filter=contains(pageView/url,'valid-url')&$count=true
appID and valid-url is set correctly and i delete them in this question to make it more easy to read
Is there any issue in using $filter=contains ???
the easiest way to verify this stuff is to use the api explorer, and the demo app:
https://dev.applicationinsights.io/apiexplorer/events
this url shows that startswith works fine:
https://dev.applicationinsights.io/apiexplorer/events?appId=DEMO_APP&apiKey=DEMO_KEY&eventType=pageViews×pan=P30D&$filter=startswith(pageView%2Furl%2C'http%3A%2F%2Faiconnect')
if you "need" something like contains, you can use $search (which looks across most fields, though, and has its own AND/OR text search logic)
https://dev.applicationinsights.io/apiexplorer/events?appId=DEMO_APP&apiKey=DEMO_KEY&eventType=pageViews×pan=P30D&$search=%22Customers%2FCreate%22
Issue Context:
I am using meteor js for a mobile app.
I have hooked it up with google analytics calls and basically I am using two type of calls:
Screen views
Events
Screen views are just fine, but I'm facing an issue with the events.
When I go to Behavior -> Events -> Screens, in the google analytics dashboard, I can see the URL of every page that has triggered an event under the Screen Name column. My problem is that the page URLs for my login page look something like this:
meteor.local/login?username=*******&password=+++++++&rememberMe=on
Where ******* is an actual username and +++++++ is the corresponding password!
Reason:
Since I have to share this analytics account with multiple people, I do not want this information to be available over here.
Clues:
CLUE 1:
I used to do GET http calls, but I have changed them all to POST and it still has not fixed the issue as I expected it not to pass plain parameters through URL anymore.
CLUE 2:
I've noticed that the default google analytics js framework is working with http and not https. I was wondering if it is calling the analytics server with a GET as well. If so, is there anyway to change that?
CLUE 3:
Here is how I am initiating the GA instance:
(function (i, s, o, g, r, a, m) {
i['GoogleAnalyticsObject'] = r;
i[r] = i[r] || function () {
(i[r].q = i[r].q || []).push(arguments)
}, i[r].l = 1 * new Date();
a = s.createElement(o),
m = s.getElementsByTagName(o)[0];
a.async = 1;
a.src = g;
m.parentNode.insertBefore(a, m)
})(window, document, 'script', '//www.google-analytics.com/analytics.js', 'ga');
ga('create', googleKey, 'auto');
CLUE 4:
I have also noticed that these URLs are getting captured very occasionally. E.g. in the pas 12,500 unique events (about 30,000 total events) it has captured just 9 URLs with the username and password. The remaining 12,491 events have
meteor.local/login
OR
meteor.local/--
OR
localhost/--
as the Screen Name.
CLUE 5:
I have also put 4 "search and replace" global filters on the analytics account to search for this string
meteor.local/.*
and replace it with this one
meteor.local/concealedURI
This does not seem to be working either.
I have added this filter on 4 different fields (Since I still really don't know where the URLs are coming from):
Host Name
Page Title
Referral
Request URI
CLUE 6:
This is how I am calling the GA instance to send the event:
ga('send', 'event', 'button', 'click', eventName);
Okay. So, I had to run a lot of experiments and try out different things to solve this issue.
After trying all the things that I have described in the question, I finally found a way to address this problem.
The main cause of this problem was that I was using a google analytics account set to track an App, to capture the data from an app that was built with meteor js (which basically utilizes cordova).
Using meteor means that my app's screens are actually web pages rendered as a mobile app. It seems like meteor uses URLs to navigate through these screens.
On the other hand, google analytics looks at (and captures) the screen name of an app's page, when an event is triggered from that page. In native apps this screen name will be something similar to "About us", "Contact Us", "Home", etc.
Now since a meteor app is not the same, the screen name returned by meteor is actually the URL of the page that has triggered the event.
This does not have anything to do with the http calls (Whether or not they are GET or POST), because it is the local URL used by meteor for navigating that is being passed down to google analytics and not any http calls.
Solutions
1.
If I had the google analytics account set as a web page tracker, I could have access to "Exclude URL Query Parameters" field and I could potentially exclude username and password as was suggested by #Mike and #PhilipPryde in the comments.
However, I needed to use google analytics set as an app tracker. So, this did not work for me.
Failed
2.
I did put a filter on the whole view in the google analytics and searched for meteor.local/.* and replaced that with hiddenURL. The filters on
Host Name
Page Title
Referral
Request URI
did not work.
But when I put the same filter on
Screen Label
field, it worked.
However, this only looked at the screen names returned by screen view hits and not the event. Thus, this did not actually solve my issue either.
Failed
Finally, I had to do this:
There is a method call on GA instance that lets you set different options up. I ended up using this:
ga('set', 'screenName', 'hiddenURL');
This changed the screen name to "hiddenURL". So, I used this before every event and it worked for me.
My code for sending events to google analytics looked like this:
ga('set', 'screenName', 'hiddenURL');
ga('send', 'event', 'button', 'click', eventName);
PS:
This changes the screen name that was showing up in real-time reports of google analytics to "hiddenURL", whenever someone triggered an event. But, it changes back to a screen name as soon as they go to another page. So, it would not also mess with any of your screen view data either, since it is not being captured as a screen view.
Of course that is because, I pass the screen name to my GA instance every time I send a screen view. So it looks like this:
sendScreenViewToGA = function (screenName) {
ga('send', 'screenview', {
'appName': 'Something',
'screenName': screenName,
'appVersion': x.x
});
}
If I had used the screen name, that is being set on the environment tight now, I would have ended up with all my screen names in analytics set to "hiddenURL".
I really hope this post will help others with same issues and save them some time.
I have a Logic App with Twitter connector and a Dropbox connector. The latter has repeater, which loops over the Twitter body and upload a text file in each iteration with Tweet_ID as file name. The Dropbox connector many times returns conflict errors, it seems Tweet connector keeps returning same tweets again and again, which had been already processed, which results in duplicate file names.
When I look at the output of the Dropbox connector, below is the body it returns.
"body": {
"status": 409,
"source": "api-content.dropbox.com",
"message": "conflict_file"
}
You have probably seen this page https://azure.microsoft.com/sv-se/documentation/articles/app-service-logic-use-logic-app-features/ where they show how to do this.
Have you checked that you don't supply the same Tweet_ID several times? The logic app json format it a bit tricky right now, with not so much documentation.
/dag
You are right. The twitter connector doesn't "remember" the tweets that are returned from a search. It will return the same again. (Just to be clear. We are discussing the Twitter Connector Action Search Tweets.)