Building a Facebook messenger bot using Claudia JS and plan on hosting on AWS Lambda.
I want to ask the user a series of questions.
When a user responds with an answer, I need to save that for later and once I have all the information I need, I will pass the answers to a function.
What is the best way to save this information?
I was thinking some caching layer such as redis but because that is stored in RAM I will lose it when lamda server shuts down. Mongodb apparently has a lot of overheads when connecting but will at least be persistent.
Perhaps just a simple mySQL server?
How does everybody else do it? I feel like there is a simple solution that I am missing.
I will first answer the part about how I'm doing it: I'm using a MongoDB. I toyed with the ideas you mentioned, but quickly crossed out in-memory solutions (Memcached, Redis) with the same reason. My final solution came down to either a relational DB or a noSQL like MongoDB. To be honest, at my project's scale, I did not think about robustly comparing performance between DB types.
With my particular feature "roadmap," I decided to go with Mongo to approach a more "OOP" style when dealing with the user "object" without having to explicitly define a user class, thanks to the normalized structure of Mongo. I understand the same could be done for MySQL, too, just that processing json data is more "object-like" for me and flask, i.e. user = getUserFromMongo, which gives me a dict in Python then I can just do user['first_name']. The codes belows will explain this simplicity:
(Somehow this was feeling like... not having to write SQL commands for simple database interaction in Rails)
My user object data on MongoDB
Finally, as to how I manage user input, I adopted Wit.ai's concept of context. I don't know how they do it exactly, but a context to me is the type of conversation purpose that is going on. I use it like a stack, and as soon as the current context is done, pop it off the context data of the user. For every message the bot receives, the program will get the current context and direct the flow. Whenever an unknown error occurs (exceptions handling), most likely because the user is saying something the bot doesn't understand, I clear the context data, too.
The good part about MongoDB is that I can shape the context however I want and treat it just as an object. A simple one is like {name: yelp-search, stage:ask-for-user-location}, and I imagine complex ones could be built on that structure, too. Of course, a stack implementation of the context does not deal with complex conversation with complex past reference.
I put my project on Github if you want to take a look at it.
i have also used mysql for chatbot but i have used NodeJS for the backend app.For that mysql module would be very helpful.
You need to store users' current state for the question answer session and also store the answer itself from the user and you need to make a switch or if-else-if case for asking questions to user based on its state as switch(state) and in cases of switch just update it's state.and you have user's facebook-id in event object of chatbot so that you can store data of each user individually with their state and question-answer in different table.
For e.g. define flags{1,2,3}
user's state will be 1 in begining so ask him for e.g. question-1
only,and store this as answer-1, you can do this by it's state
checking, and after this update status to 2.
so,in this way you can ask each individual student question as per
their state and answer him.
I've done the same in exact above manner.
Hope this would be helpful to you.
Related
This is more of a pragmatical question in regards to implementing pagination with the usage of Cassandra. I have read the documentation in regards to the Page state and that it should not be exposed to end user as it is not encrypted and creates the possibility of unwanted tampering.
So, with all that said, my question is what are the pragmatics and best practices in terms of using the PageState object when implementing an API that needs to fetch the next records in line upon scrolling?
After re-reading your question, I think I understand now what you're trying to achieve.
You are correct in saying that you shouldn't expose the page state to end-users of your application. I don't think there will be anything catastrophic if the page state is modified/tampered by a user but if they do, it will either return unpredictable results or be unusable and the driver will throw an exception.
On your question of what is best practice when implementing an API to retrieve the next "page" of results, I would recommend separating the implementation of your API from the logic which requests the records from the DB.
For example, don't use the driver's page state in your API but instead implement an option page=next so the HTTP endpoint looks like /api/users?page=next. Cheers!
I am quite sure that I am not the first person on the planet trying to implement the following, but I am sure that I am not able to find a good guide how to do it.
Our node backend is setup quite like a MVC to say so.
View = Express Server offering our api
Controller = Library, a set of controller functions to manage our data
Model = Our mysql database, it's Javascript DAO respectively (since our usecase is quite unique we need to write own DAO's and can not go let's say for js-data.
The challenge we face now, is:
As a developer, I want to keep our library clean from overhead for developers.
On the other side, as a database administrator I clearly want to know who did what modification and so on
Until now I tried to keep the 'user' object out of the library, since I do not want all controller functions to look like
function ctrl(param1, param2, param..., user)
Going for this would mean, we have to pass around User objects all the time, which would make it a pain to code inside the libraries.
On the other hand, I can not find any other approach in node/express to somehow get knowledge about the user without passing it (since we do not really have sessions, at least not yet in our code).
TL:DR; I do want to action log all database modifications, but do not want to pass around a User object all the time.
Is there any known approach for that challenge which does scale and is 'best practice'?
Thanks in advance
I am building a webservice with node.js and I am registering users in my service. I am using node.js + mongodb for my db and once I create a new user I also want to create a unique id for them and send that back as a response, just like all the great services like fb that send u back the facebook id. I do not want to send back the _id from mongodb, so how do I generate a unique id for every user in node, also is it better to do it this way or just send back the mongo _id.
is it better to do it this way or just send back the mongo _id
If you have to ask, it is better to just send back the mongo _id. Unless you have iron-clad reasoning behind why that simple, straightforward, make-life-easy-for-everyone technique is problematic, by all means, just send back the mongo _id.
If you decide the mongo _id is not good enough for you (probably due to FUD as opposed to any realistic reasoning), you have the following extra challenges you are adopting without any benefit:
you have to think more carefully about your indexes
Helper libraries functions like findById don't work for you anymore
Now you have 2 huge, hard-to-eyeball IDs to deal with on every record
helper libraries like mongoose are also going to be challenging to leverage
You will have to be mapping back and forth between _id and mySuperAwesomeExtraneousId constantly during debugging for the entire lifetime of your app
K.I.S.S.
That said, you can always just use an additional mongo ObjectId as they are perfectly valid unique Ids:
const mongo = require('mongodb')
let mySuperAwesomeExtraneousId = new mongo.ObjectID()
Use coupon-code.This is simple to use and solves most of the use cases for generation of unique ids.
https://github.com/appsattic/node-coupon-code
https://github.com/broofa/node-uuid
There are numerous good reasons for not sending back the _id created by Mongoose by default for example. For one, those IDs could easily be guessed by a rogue entity or hacker. And it's also never a good idea to expose your database ids anyway.
For an app that relies solely on a unique ID for a password-less account access, generating a highly unique hash id may be the best option. The perfect node module for that is hashids
You may also give crypto a try:
require('crypto').randomBytes(48, function(ex, buf) {
var token = buf.toString('hex');
});
good luck!
Use http://mongoosejs.com as an abstraction layer to MongoDB. It will ensure that you always have the _id available. It also will manage many other things, such as validation, connections, etc.
Pros: It is simpler to use than raw drivers, while at the same time maintaining all the power of a raw MongoDB driver. You will be up and running in about 20 minutes after visiting the mongoose website. Its one of the few times you don't trade off power for simplicity. It is a thin abstraction layer, you will have a net performance gain in code by having this single, robust layer in place then by trying to re-invent the wheel on every single part of your code that needs to access the DB. It supports every single MongoDB feature. It provides validation. It provides a simple interface to managing indexes. It automatically creates databases and collections. Built by the same guys that made ExpressJS, Socket.IO and Mocha for node.js. The list goes on.
Cons: Does not support multiple MongoDB connections. This is usually not a problem though because you will most likely use MongoDB's sharding features before you need to create multiple connections to multiple MongoDB clusters. It has a silly name.
We have been using Mongoose in production environments for quite sometime. If you want to see it in action, look at ZingProject.com. Its entirely node.js + mongoose over MongoDB. Its lightning fast.
I am currently using J Oliver's EventStore which as I understand uses Guids for the Stream ID and this is what is used to build my aggregate root.
From a CQRS point of view and a DDD perspective, I should be thinking about the domain and not GUIDs.
So, if I do a GET (Mvc client), am I right to say that my URL should have the identity of my domain object (aggregate root) and from that I get the GUID from the read store which is then used to build my aggregate root from the event store?
Or should I pass the GUID around to my forms and pass them back as hidden form variables? At least this way I know the aggregate root id and do not have to query the read store?
I suppose the first way is the correct way (not using GUIDs in forms) as then all my Gets and POSTS deal with identities of domain objects rather than GUIDSs which the client will not know.
I suppose this also allows me to build as REST based API which focuses on resources and their identities rather than system generated GUIDS. Please correct me if I am wrong
In my opinion you are on the right track here. The ui should rely solely on the read model and not really care about the aggregates. Then when you need to send a command you can use the read model to get the id of the aggregate you are interested in. The read model should be very fast to read from anyway (that's the whole reason behind using different models for reads and writes) and very easy to cache if you need to. This will also give you much nicer urls.
I have a custom ASP.NET Membership Provider that I am trying to add password history functionality to. User's passwords expire after X days. Then they have to change their password to one that has not been used in their past X changes.
I already had the User entity, which has a password attribute for their current password. This maps to the User table in the db. Since I needed a list of previous passwords I created a UserPassword table to store this information with a FK reference to the UserId.
Since passwords are value objects, and have no meaning outside of the user, they belong inside the User aggregate, with the User as the root. But here in lies my dilemma. When I retrieve a User from the repository do I always have to get all of their previously used passwords? 99% of the time I don't care about their old passwords, so retrieving them each time I need a User entity seems like a dumb thing to do for db performance. I can't use lazy loading because the User entity is disconnected from the context.
I was thinking of creating a PasswordHistory entity but for the reason stated above, passwords aren't really entities.
How would you DDD experts out there handle this situation?
Thanks.
Edit 1: After considering this some more, I realized this is essentially a question about Lazy Loading. More specifically, how do you handle lazy-loading in a disconnected entity?
Edit 2: I am using LINQ to SQL. The entities are completely detached from the context using this from CodePlex.
It is hard to fully answer this question because you do not specify a platform, so I cannot be exactly sure what you even mean by "disconnected". With Hibernate "disconnected" means you have an object in a valid session but the database connection is not currently open. That is trivial, you simply reconnect and lazy load. The more complicated situation is where you have an object which is "detached" i.e no longer associated with an active session at all and in that case you cannot simply reconnect, you have to either get a new object or attach the one you have to an active session.
Either way, even in the more complicated scenarios, there is still not a whole lot to lazy loading strategies because the requirements are so inflexible: You have to be "connected" to load anything, lazy or otherwise. Period. I will assume "disconnected" means the same thing as detached. Your strategy comes down to two basic scenarios: is this a situation where you probably need to just reconnect/attach on the fly to lazy load, or is it a scenario where you want to make a decision to sometimes conditionally load additional objects before you disconnect in the first place?
Sometimes you may in fact need to code for both possibilities.
In your case you also have to be connected not only to lazy load the old passwords but to update the User object in the first place. Also since this is ASP.NET you might be using session per request, in which case your option is now basically down to only one - conditionally lazy load before your disconnect and that is about it.
The most common scenario would be a person logs in and the system determines they are required to change their password, and asks them to do so before proceeding. In that case you might as well just take care of it immediately after login and keep the User connected. But you are probably using session per request, so what you could do is in the first request process the time limit and if it is expired, you are still connected here so go ahead and return a fully loaded User (assuming you are using the historic passwords in some kind of client side script validation). Then on the submit trip you could reattach or just get a new User instance and update that.
Then there is always the possibility you also have to provide them with the option to change their password at any time. They are already logged in. Does not matter much here, you have a User but the request ended long ago and it does not have passwords loaded. Here, I would probably just write a service method where when they invoke a change password function the service gets a second copy of the User object with the full history for update purposes only, then updates the password, and then discards that object without ever even using it for session or authentication purposes. Or if you are using Session per request you have to do the equivalent - get a fully initialized object for client side validation purposes, then when the data is submitted you can either reattach either one you already have or just get yet a third instance to actually do the update.
If the password is needed after beginning an authenticated session, you could still do the same things and either replace the local User or update the local User's in memory password version as well.
If you have too much stuff going on with multiple levels of authentication most likely you are going to have to require them to logoff and do a full log back in after a password change anyway, so the state of the User does not matter much once they request a password change.
In any case if you are using session per request and your objects become fully detached after every request, in the first scenario you can still lazy load while you are on the server on the original request to return data for client side validation. In the second scenario you have to make another trip (there really is no such thing as lazy loading here). In both case though you have to weigh your two update options because you are always disconnected before an update. You can either just get a second instance from the database on the submit trip to update, or you can reattach the one you already have. It depends on what is optimal/easiest - does saving a db round trip for an uncommon event really matter? Does reattaching using your ORM of choice possibly hit the database again anyway? I would probably not bother to reattach and instead just get a new instance for the actual update as I needed it.