Is it true Azure Luis only support up to 500 intents per application? https://learn.microsoft.com/en-us/azure/cognitive-services/luis/luis-limits
My requirement is more than 1000 intents. How can I use Luis to do that?
You should consider using Dispatch. It is a tool that was designed specifically for managing multiple LUIS models and/or QnA Maker knowledge bases that a bot needs to access.
You can find C#, Javascript, and Python samples on the BotBuilder-Samples repo, for reference, titled "14.nlp-with-dispatch".
In your case, this tool is provides a means for overcoming LUIS intent limitations by allowing you to create multiple models to draw from. Dispatch negotiates these models by creating a single LUIS app that then routes the requests to the appropriate model.
Hope of help!
To answer your question, yes it is true. But it is plenty. We have some of the biggest company using our platfomr to test their training data on LUIS/WATSON/DF... and it is extremely rare to pass the 500 and still get top performance.
We typically advice anyway to fine tune your training data for 200 or 300 intents max and if you have more, look into a Model controller architecture with several slave (specific) models
So you might be sure you have 1000 intents, can you reduce it with Entity?
Related
Our teams create individual bots for themselves and now we want to integrate them into the Enterprise Assistant.
What I need help with is Multi Bot Orchestration.
Has anyone been able to do this efficiently?
Basically, the user asks a question in the Enterprise Assistant and then the bot gets the answer from the respective child/skill QnA bot.
I am able to add skills like calendar, people, SAP, etc. but dealing with QnA bots is proving to be an impossible challenge.
To create a multi bot orchestration, we need to include multiple levels of dispatchers, pipelines for exception handling and fallbacks. It needs an expert system type agent who can handle multiple NLP models. A chat bot is designed based on NLP modelling. The NLP implementation will be depending on number of operations that enterprise application will handle. The following are the different NLP models required to handle a perfect enterprise application.
General QnA
Billing information
Calendar
Leave Planning
Employee storage disk
On demand service to the clients
Fault tolerance
Saving the QnA pattern of chat between bot and human
Calculating the accuracy
Taking users own request and performing NLP
like this we have so many requirements to implement a Bot. All the models will be trained and connected to the dispatcher.
The following are the things followed in bot orchestration.
It consists of 5 different implementations:
Ingress Pipeline
Egress Pipeline
Dispatcher
Exception pipeline
Fallback pipeline
Ingress Pipeline:
The ingress pipeline will be detection few things like
Language detect and translate
Entity recognition
User input redaction
Egress Pipeline:
the following are requirement and operations in egress pipeline
3rd party tools for analytics
Managed responses (default responses)
Data service redaction
Dispatcher:
The dispatcher will be having two different implementations.
Policies and rules,
NLU (Natural Language Understanding) strategy.
Policies and Rules
The policies and rules of the organization must be trained to the model. Because, if the user is asking for the financial status of the orgzanition, the bot must understand the policies and rules it must follow in order to answer for the question.
Natual Language Understanding
Natural Language Understanding (NLU) — The computer’s ability to understand what we say. The context what user is typing in his/her own words is the procedure of NLU.
The dispatcher will be communicating with the egress pipeline and ingress pipeline as it is the parent handler of all the operations and model training results.
To address the exceptions and rollbacks, we need to have exception handlers who are also expert system agents. An agent is having some expertise in handling the exceptions.
Example:
Question: Give the details of pithon projects of this monthh.
Handled Exceptions: In the above question "python" spelt like "pithon" and "month" was spelt like "monthh". NLP and NLU have to understand these cluases and have to handle in exception of mis-spelt words.
The following link can explain in detailed architecture of Multi-bot orchestration.
https://servisbot.com/multi-bot-orchestration-architecture/
I am trying to implement a chatbot using linked data. In a paper, I read that I can use SVM for the intent classification which is basically the first stage in conversational systems. It is the process of mapping queries to a predefined class. I wanted to know whether there is any other way or applications / programs that I can use to do this. Could someone please advise me?
Thank you in advance.
Rasa is an open-source platform that can be used to easily set up a chatbot, and through it you can set up several kinds of intent classification (including with an SVM).
It also allows you to annotate example utterances and map them to intents within the tool itself.
These tasks are certainly doable on your own, but if speed and easy integration with a working chatbot are priorities, then I recommend using a platform like that.
I am new in microservices. I am coming from monolithic background in current environment i have different kinds services for different purposes like search, file, email, notification. I have taken so many courses but in that the instructor separate each entity and make it's own database also create API for that(like separate shopping cart entity, product entity) it makes no sense, I am not getting what is real world use of microservices or how to make separate component to build it's own microservice.
Can anyone give Real Project example?
Thanks in advance
Read this and this. Also look here and here. I don't think that anyone will give a link to the real working project, so you can try this.
I am not getting what is real world use of microservices
mostly as you heard in all of those tutorials the microservices architecture leverage advantages of:
the smaller services are easy to maintain and develop
easily can scale specific services rather than the whole project(monolith). for example you scale service-1 to 4 instances that request traffic split into these 4 instance and service-2 to 2 instances and go on (load balance). and these services may distributed in to different servers and locations.
if one service failed to work it does not terminate the whole system since they are independent.
services can be reusable for other scenarios or features.
small team can works for each services and its easy to manage both project and development flow.
and also it suffer from disadvantages of
services are simple and small but all as a whole system is complex so designing part are very critical.
poor performance and it requires do some extras to improve the performance (different types of caching on different levels).
transactions are complex and its developments are time costly. imagine simple update should be projected to other services if its required and you have to consider failure and rollback strategy ( SAGA ).
how to make separate component to build it's own microservice
this is the most challenging part of microservices. you need deep study on Domain driven design DDD.
Decompose by subdomain
Decompose by Business Capabilities
Can anyone give Real Project example?
there are many projects the develop microservices with different patterns. I think you have to start your own and make your hands dirty.
I am trying to understand the different model in building a bot using dialogflow and came across this 2 methods.
Fulfillment model (with webhook enabled) documentation here
API Interactions documentation here
I understand that both of this models have their own pros and cons, and I understand how they both work. Most online examples are showing the fulfillment method (I guess that's more common?)
However, I would still like to ask what reason will it be to choose one or the other? If anyone had used either model before, what limitations are there?
p/s: I've look through quite a number of tutorials, and read through the dialogflow documentation.
the integration by fulfillment is indeed the default approach because you use DialogFlow to design your conversation flow and (big bonus) manage the integration with the various channels (ie Telegram, Facebook).
It is the easiest way to design a fully fledge conversation, you only need to worry about the post hooks that are sent to your backend to either save the data or alter the conversation (add contexts or trigger events).
Important remark: all user traffic (who says what) goes via Dialogflow cloud
The API interaction becomes a good option when you have already an existing frontend (say an existing application or web sites) and you want to plug in DialogFlow NLP capabilities.
I have done something like that to create a FAQ chatbot that called DialogFlow to identify which intent would match a certain phrase while the BOT was deployed in MS Teams.
The architecture would indeed look like the one in the documentation: MS Team ecosystem is the "End-User" part, then my Java app ("Your System") would use the API to call DialogFlow.
Important remark: only given statements (the ones you send) go to Dialogflow cloud
I want to build a chatbot for a customer service application. I tried SaaS services like Wit.Ai, Motion.Ai, Api.Ai, LUIS.ai etc. These cognitive services find the "intent" and "entities" when trained with the typical interactions model.
I need to build chatbot for on-premise solution, without using any of these SaaS services.
e.g Typical conversation would be as following -
Can you book me a ticket?
Is my ticket booked?
What is the status of my booking BK02?
I want to cancel the booking BK02.
Book the tickets
StandFord NLP toolkit looks promising but there are licensing constraints. Hence I started experimenting with the OpenNLP. I assume, there are two OpenNLP tasks involved -
Use 'Document Categorizer' to find out the intent
Use 'Named Entity Recognition' to find out entities
Once the context is identified, I will call my application APIS to build the response.
Is it a right approach?
How good OpenNLP is in parsing the text?
Can I use Facebook FASTTEXT library for Intent identification?
Is there any other open source library which can be helpful in building the BOT?
Will "SyntaxNet" be useful for my adventure?
I prefer to do this in Java. BUT open to node or python solution too.
PS - I am new to NLP.
Have a look at this. It says it is an Open-source language understanding for bots and a drop-in replacement for popular NLP tools like wit.ai, api.ai or LUIS
https://rasa.ai/
Have a look at my other answer for a plan of attack when using Luis.ai:
Creating an API for LUIS.AI or using .JSON files in order to train the bot for non-technical users
In short use Luis.ai and setup some intents, start with one or two and train it based on your domain. I am using asp.net to call the Cognitive Service API as outlined above. Then customize the response via some JQuery...you could search a list of your rules in a javascript array when each intent or action is raised by the response from Luis.
If your Bot is english based, then I would use OpenNLP's sentence parser to dump the customer input into a database (I do this today). I then use the OpenNLP tokenizer and push the keywords (less the stop words) and Parts of Speech into a database table for keyword analysis. I have a custom Sentiment model built for OpenNLP that will tag each sentence with a Pos, Neg, Neutral sentiment...You can then use this to identify negative customer service feedback. To build your own Sentiment model have a look at SentiWord.net and download their domain agnostic data file to build and train an OpenNLP model or have a look at this Node version...
https://www.npmjs.com/package/sentiword
Hope that helps.
I'd definitely recommend Rasa, it's great for your use case, working on-premise easily, handling intents and entities for you and on top of that it has a friendly community too.
Check out my repo for an example of how to build a chatbot with Rasa that interacts with a simple database: https://github.com/nmstoker/lockebot
I tried RASA, But one glitch I found there was the inability of Rasa to answer unmatched/untrained user texts.
Now, I'm using ChatterBot and I'm totally in love with it.
Use "ChatterBot", and host it locally using - 'flask-chatterbot-master"
Links:
ChatterBot Installation: https://chatterbot.readthedocs.io/en/stable/setup.html
Host Locally using - flask-chatterbot-master: https://github.com/chamkank/flask-chatterbot
Cheers,
Ratnakar
With the help of the RASA and Botkit framework we can build the onpremise chatbot and the NLP engine for any channel. Please follow this link for End to End steps on building the same. An awsome blog that helped me to create a one for my office
https://creospiders.blogspot.com/2018/03/complete-on-premise-and-fully.html
First of all any chatbot is going to be the program that runs along with the NLP, Its the NLP that brings the knowledge to the chatbot. NLP lies on the hands of the Machine learning techniques.
There are few reasons why the on premise chatbots are less.
We need to build the infrastructure
We need to train the model often
But using the cloud based NLP may not provide the data privacy and security and also the flexibility of including my business logic is very less.
All together going to the on premise or on cloud is based on the needs and the use case of the requirements.
How ever please refer this link for end to end knowledge on building the chatbot on premise with very few steps and easily and fully customisable.
Complete On-Premise and Fully Customisable Chat Bot - Part 1 - Overview
Complete On-Premise and Fully Customisable Chat Bot - Part 2 - Agent Building Using Botkit
Complete On-Premise and Fully Customisable Chat Bot - Part 3 - Communicating to the Agent that has been built
Complete On-Premise and Fully Customisable Chat Bot - Part 4 - Integrating the Natural Language Processor NLP
Disclaimer: I am the author of this package.
Abodit NLP (https://nlp.abodit.com) can do what you want but it's .NET only at present.
In particular you can easily connect it to databases and can provide custom Tokens that are queries against a database. It's all strongly-typed and adding new rules is as easy as adding a method in C#.
It's also particularly adept at turning date time expressions into queries. For example "next month on a Thursday after 4pm" becomes ((((DatePart(year,[DATEFIELD])=2019) AND (DatePart(month,[DATEFIELD])=7)) AND (DatePart(dw,[DATEFIELD])=4)) AND DatePart(hour,[DATEFIELD])>=16)