Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I m looking to store the profile details of the users in my app like their date of birth, country, first name, last name, date on which they joined, location of their profile_pic etc.
I am using mongoose in Expressjs app. I will also be storing the username and hashed password and if they have activated from their email address. I have a few questions that I am getting confused and not really sure what to do?
Should I store all the users in one mongoose model and have a column for activated to c if the user has activated from its email link or not OR Should I have two different tables and once the user has activated I move him from the unactivated table to activated table.
Second thing is want to know should i store the details of profile as columns in same model or create another model for the profile details using population. I am thinking this as these details have anything I need to query about. They are just basically for read and write. I dont query using those parameters. So I was thinking having them in a different model would be slight be better as always I would be querying for the username or password only. One other option I think possible is having them in sub documents.
I will also be storing user preferences of my app and I have the same prob as above.
Please help me as to what option should I choose. I have done a lot of reading but not sure as of now. Please help me with whats the standard thing to do and what would be better
Thanks
Yes, store all users in one mongoose model, with an activated field. That way you simplify some likely database queries, such as getting a list of all your users (activated or not) or checking whether a username has been taken.
You should also store the details of the profile as fields in the same model. If you don't query using those parameters, don't index them. Putting them as subdocuments, in my opinion, is not that useful, and it only makes sense if you plan on having sets of profile details (more than one) per user.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
How do one implement live like and dislike [or say views count] in couchdb/couchbase in the most efficient way.
Yeah one can use reduce to calculate count each time and on front end only use increment and decrement to one API call to get results.
But for every post there will be say millions of views, like and dislikes.
If we will have millions of such post [in a social networking site], the index will be simply too big.
In terms of Cloudant, the described use case requires a bit of care:
Fast writes
Ever-growing data set
Potentially global queries with aggregations
The key here is to use an immutable data model--don't update any existing documents, only create new ones. This means that you won't have to suffer update conflicts as the load increases.
So a post is its own document in one database, and the likes stored separately. For likes, you have a few options. The classic CouchDB solution would be to have a separate database with "likes" documents containing the post id of the post they refer to, with a view emitting the post id, aggregated by the built-in _count. This would be a pretty efficient solution in this case, but yes, indexes do occupy space on Couch-like databases (just like as with any other database).
Second option would be to exploit the _id field, as this is an index you get for free. If you prefix the like-documents' ids with the liked post's id, you can do an _all_docs query with a start and end key to get all the likes for that post.
Third - recent CouchDBs and Cloudant has the concept of partitioned databases, which very loosely speaking can be viewed as a formalised version of option two above, where you nominate a partition key which is used to ensure a degree of storage locality behind the scenes -- all documents within the same partition are stored in the same shard. This means that it's faster to retrieve -- and on Cloudant, also cheaper. In your case you'd create a partitioned "likes" database with the partition key being the post-id. Glynn Bird wrote up a great intro to partitioned DBs here.
Your remaining issue is that of ever-growth. At Cloudant, we'd expect to get to know you well once your data volume goes beyond single digit TBs. If you'd expect to reach this kind of volume, it's worth tackling that up-front. Any of the likes schemes above could most likely be time-boxed and aggregated once a quarter/month/week or whatever suits your model.
I think this question has been asked but there are no clear answers.
The question is simple.
Can you have an entity list on the server.
For example I have a list of Product names on my database which can be really big. I want the intent to recognise these entities based on a list on the server.
The other thing I would like to do is filter an entity list.
e.g. I have a list of stores. I want it to be filtered by location, say by distance and lat long showing only stores near you when I ask a question.
Things which are so easy to do in apps seem so difficult in Dialogflow.
Please do not provide solutions which can be done on the server through webhooks. I already know about that and have used it.
I just want a better way to use entities so that the NLP can become more powerful.
The best way to do will be using Entities with webhook.
You may enable slot filling for the parameters.
In the webhook, have a set of stores based on locations and hashmap with the location as key and set of stores as value.
when the location is provided, fetch the corresponding set of stores.
when the store is provided, see if that store is present in the set.
reprompt if the information is not correct by resetting the context if required.
UPDATE
You may ask the user for the product names. Match the entity name with the names in DB. If present, use it if not, provide the user with some option from the DB that may match with what the user is saying and ask them to choose one. You need to think from a conversation point of view how two people communicate with each other.
I'm building a small Node/Mongo app that serves users with up to 3 questions per day. Users can only answer yes or no and the correct answer will be determined at a later time (these questions are closer to predictions). Currently, I have these documents:
User
id
Question
id
QuestionAnswer
id
question_id (ref)
UserAnswer
id
question_id (ref)
user_id (ref)
What is the most efficient way to query the db so I get today's questions but also check whether that user has answered that question already? I feel like I'm overthinking it. I've tried a couple ways that seem to be overkill.
It's good to put them all in one schema since we don't have joins in mongodb.
It is faster than using relations.
Also for keeping your query small, take a look at this.
You should stay away from relations till you have a good reason for using them. So, what you need is only one schema.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
I come from mysql background, and I am aware of typical security concerns when using mysql.
Now, I am using mongodb (java driver).
What are the security concerns, and what are possible ways of avoiding security problems?
Specifically these areas:
1) Do I need to do anything for each get/post?
2) I store cookies from my application on client side and read those later (currently the only information I store is user's location, no sensitive information), Anything I should be careful about?
3) I have text boxes, text areas in my forms which users submit. Do I need to check for anything before saving data in mongo?
Can anybody provide any instances of security problems with existing applications in production?
It is in fact possible to perform injections with Mongo. My experience with it is in Ruby, but consider the following:
Request: /foo?id=1234
id = query_param["id"]
collection.find({_id: id})
# collection.find({_id: 1234})
Seems innocuous enough, right? Depending on your HTTP library, though, you may end up parsing certain query strings as data structures:
Request: /foo?id[$gt]=0
# query_param["id"] => {"$gt": 0}
collection.find({_id: id})
# collection.find({_id: {"$gt": 0}})
This is likely less of a danger in strongly typed languages, but it's still a concern to watch out for.
The typical rememdy here is to ensure that you always cast your inbound parameter data to the type you expect it to be, and fail hard when you mismatch types. This applies to cookie data, as well as any other data from untrusted sources; aggressive casting will prevent a clever user from modifying your query by passing in operator hashes in stead of a value.
The MongoDB documentation similarly says:
Field names in MongoDB’s query language have semantic meaning. The dollar sign (i.e $) is a reserved character used to represent operators (i.e. $inc.) Thus, you should ensure that your application’s users cannot inject operators into their inputs.
You might also get some value out of this answer.
Regarding programming:
When you come from a mysql background, you are surely thinking about SQL Injections and wonder if there is something like that for MongoDB.
When you make the same mistake of generating commands as strings and then sending them to the database by using db.command(String), you will have the same security problems. But no MongoDB tutorial I have ever read even mentions this method.
When you follow the usually taught practice of building DBObjects and passing them to the appropriate methods like collection.find and collection.update, it's the same as using parameterized queries in mysql and thus protects you from most injection attempts.
Regarding configuration:
You need, of course, make sure that the database itself is configured properly to not allow unauthorized access. Note that the out-of-the-box configuration of MongoDB is usually not safe, because it allows non-authorized access from anywhere. Either enable authentication, or make sure that your network firewalls are configured to only allow access to the mongodb port from within the network. But this is a topic for dba.stackexchange.com
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
I am developing a search engine for my website and i want to add following features to it;
Full text search
Did you mean feature
Data store in MongoDB
I want to make a restful backend. I will be add data to mongodb manually and it will be indexed (which one i can prefer? Mongodb indexing or some other search indexing libraries like Lucene). I also want to use node.js. These are what i found from my researches. Any idea would be appreciated for the architecture
Thanks in advance
I'm using Node.js / MongoDB / Elasticsearch (based on Lucene). It's an excellent combination. The flow is stunning as well, since all 3 packages (can) deal with JSON as their native format, so no need for transforming DTO's etc.
Have a look:
http://www.elasticsearch.org/
I personally use Sphinx and MongoDb, it is a great pair and I have no problems with it.
I back MongoDB onto a MySQL instance which Sphinx just quickly indexes. Since you should never need to actively index _id, since I have no idea who is gonna know the _id of one of your objects to search for, you can just stash it in MySQL as a string field and it will work just fine.
When I pull the results back out of Sphinx all I do is convert to (in PHP) a new MongoId, or in your case a ObjectId and then simply query on this object id for the rest of the data. It couldn't be simpler, no problems, no hassle, no nothing. And I can spin off the load of reindexing delta indexes off to my MySQL instance keeping my MongoDB instance dealing with what it needs to: serving up tasty data for the user.