Sequelize query multiple times - node.js

I'm trying to improve the search on my website, this is how it currently looks (I use NodeJS, Sequelize and PostgreSQL):
db.food.findAll({
where: {
nameFood: {
$iLike: '%' + queryName + '%'
}
}
}).then(function (foods) {
foods.sort(compareFood);
res.json(foods);
}, function (e) {
res.status(500).send();
});
I think it is pretty self explanatory, but if something isn't clear just ask me on the comments.
Now, this search algorithm takes into consideration the WHOLE parameter, so searching for "chicken eggs" would return nothing since on my database they're saved as "eggs".
My idea for a quick improvement would be to split the query looking for spaces and then query each keyword, something like:
var keywords = queryName.split(' ');
With that I have the keywords, now how can I query a variable number of times and join the result in an array to be returned like the one in foods?
I checked the documentation and some questions in here but couldn't find anything, any help is appreciated, thank you so much.

You can use the $or property for querying anything matching "eggs" or "chicken".
Here's an example:
// split queryName by spaces
var qry = queryName.split(' ');
// convert each item into a $iLike object
// ['chicken', 'eggs'] -> [{ $iLike: '%chicken%' }, { $iLike: '%eggs%' }]
qry = qry.map(function(item) {
return {
$iLike: '%' + item + '%';
};
});
db.food.findAll({
where: {
nameFood: { $or: qry }
}
}).then(function (foods) {
foods.sort(compareFood);
res.json(foods);
}).catch(function (e) {
res.status(500).send();
});
Hope that answers your question but before you go I've also got a nice tip for you that you might find helpful.
One of them is using the .catch function for receiving any errors. Rather then using .then for both results and errors, you can leave .then to handling the results and .catch to handling the errors.
db.food.findAll()
.then(function(results) {
})
.catch(function(err) {
});
instead of
db.food.findAll()
.then(function(results) {
}, function(err) {
});

Related

Update a field in an array of subdocuments for ALL records

When I first created my site I didn't know MongoDB did case sensitive searching, among many other things, and I'm trying to fix the problems.
I've already updated the code to alter the user input before it's saved so it is lowercase AND trimmed of extra spaces.
I can easily update a collection of users from the mongo shell, and then I'm good to go there, but I have another collection that needs updating and it looks like this:
Registration Data Object:
{
event: string,
field: string,
users: [string],
players: [
first: string,
last: string,
email: string
]
}
If it's possible to update the players.email field for ALL registrations by trimming it and making it lowercase from the mongo shell, I'd love to do that. But I don't think it is, and I've had trouble doing it from mongoose (using Node.js backend, AngularJS frontend).
I was thinking something like this, but I get Type error: Registration.save is not a function:
module.exports.regUpdateAll = function(req, res) {
console.log('UPDATE ALL REGISTRATIONS');
Registration.find().exec((err, reg) => {
reg.forEach((reg) => {
for(var i = 0; i < reg.players.length; i++) {
reg.players[i].email = reg.players[i].email.toLowerCase().trim();
}
});
console.log(reg);
Registration.save(reg).then((err, response) => {
if(!err) {
res.status(200).send(response);
} else {
console.log(err);
res.status(500).send(err);
}
});
});
};
How can I fix this to work?
You need to call .save() from the object of Registration model. for example
Registration.find().exec((err, regtrations) => {
regtrations.forEach((reg) => {
reg.players =
reg.players.map( p=> {
p.email = p.email.toLowerCase().trim();
return p;
})
console.log(reg);
reg.markModified("players");
reg.save( (error)=> {
//do something
})
})
});
I GOT IT! This took ALL DAY, but I finally got it to work. I could NOT get it to work based on the value of has_signed, but I was able to do it this way:
this.adminService.GetUnsignedWaivers()
.subscribe((data: []) => {
this.players = data;
this.players.forEach((p) => {
p.waivers =
p.waivers.map( w=> {
if(w.signatureUrl.length>0) {
w.url = w.signatureUrl;
w.message = "View Waiver";
}
return w;
});
});
this.size = this.players.length;
console.log(this.players);
});
I just don't understand WHY.

how to write a query where it uses two fields in the document

The title is not enough maybe. Here let me explain.
Assume I have database structure as
{
name: "alex",
age: "21",
location: "university-alex"
}
I am aware that this database structure is not rational but I just want to show my problem in a shortest way.
I want to get the documents where location includes name field value. Here university-alex includes ,alex so it should return as a result of the query.
What have I done so far?
I wrote this query but couldn't get a result.
db.collection.find({location: {$regex: "$name"}})
How can I edit it?
I think what you're trying to achieve can be done with the $where operator.
According to this part of the documentation
db.collection.find({ $where: function() {
return (this.location.includes(this.name));
} } );
Or you can just pass the JS expression to the find method :
db.collection.find(function() {
return (this.location.includes(this.name));
});
Hope it helps,
best regards
A part of #boehm_s answer that's viable and quite clear you can also create a multiple field index that you can use to find what you need. This kind of index is very useful if you want to perform a combined query approach, e.g. looking in more than one string field if a passed parameters match their content or is contained into them. You can take a look at this doc page about Text Indexes. I don't know if you are using Mongoose or not but this answer may be useful.
Pay attention to the fact that this approach of mine would return ALL the documents that contain, in one or both field,s the "word" you are looking for.
After the creation of the index on your fields
db.collection.createIndex(
{
name: "text",
location: "text"
}
)
Let's suppose you've named this index txtIndex, then, you can do
Mongo Driver for Node Way
. . .
let userProjection = {
"name": 1,
"age": 1,
"location": 1
};
/**
* #param req Contains information to find a user
* #param req.findMe Contains name concatenated to location
*/
let findUsers = (req) => {
letUsers = db.collection('users');
return new Promise((resolve, reject) => {
User.findOne({'txtIndex': params.body.findMe}, {fields: userProjection},(err, user) => {
if (err) {
return reject({message: 'MongoDB Error', err: err});
}
if (!user) {
return reject({message: 'User not found!'});
}
return resolve(user);
});
});
}
Mongoose Way
let Users = require('./users-model.js);
/**
* #param req Contains information to find a user
* #param req.findMe Contains name concatenated to location
*/
let findUsers = (req) => {
Users.findOne({txtIndex: req.body.FindMe}).then( function (err, user) {
if (err) {
return reject({message: 'MongoDB Error', err: err});
}
if (!user) {
return reject({message: 'User not found!'});
}
return resolve(user);
});
}

Get total count along with Mongoose Query skip & limt

I have a json data which contains many objects. I want to limit the data for pagination and I need the total items count. Please help.
Model.find().skip((pageNumber-1)*limit).limit(limit).exec()
I want the count and skipped data in response.
You can use async library for running 2 queries at once. In your case you can run one query to get the number of documents and another for pagination.
Example with 'User' model:
var async = require('async');
var User = require('./models/user');
var countQuery = function(callback){
User.count({}, function(err, count){
if(err){ callback(err, null) }
else{
callback(null, count);
}
}
};
var retrieveQuery = function(callback){
User.find({}).skip((page-1)*PAGE_LIMIT)
.limit(PAGE_LIMIT)
.exec(function(err, doc){
if(err){ callback(err, null) }
else{
callback(null, doc);
}
}
};
async.parallel([countQuery, retrieveQuery], function(err, results){
//err contains the array of error of all the functions
//results contains an array of all the results
//results[0] will contain value of doc.length from countQuery function
//results[1] will contain doc of retrieveQuery function
//You can send the results as
res.json({users: results[1], pageLimit: PAGE_LIMIT, page: page, totalCount: results[0]});
});
async allows you to run a number of queries in parallel depending on the hardware you are using. This would be faster than using 2 independent queries to get count and get the required documents.
Hope this helps.
I have solved it with $facet and aggregate the following way in mongoose v3+:
const [{ paginatedResult, [{ totalCount }] }] = await Model.aggregate([{
$facet: {
paginatedResult: [
{ $match: query },
{ $skip: skip },
{ $limit: limit }
],
totalCount: [
{ $match: query },
{ $count: 'totalCount' }
]
}
}])
where the totalCount refers the total number of records matching the search query while the paginatedResult is only the paginated slice of them.
The problem with these solutions is that for every request you are doing two queries. This becomes problematic when you have a complex data structure and large data set as performance becomes an issue. Consider instead creating a special function that listens for the /resource?count=true or /resource/count GET methods and returns only the count.
You need to perform 2 queries to achieve that. One to get results and another to get total items amount with .count().
For example code you can watch at on of "paginator" for mongoose mongoose-paginate.
To performe only one query, you may use the find() method associated with promises and array slices. A small example would be:
getPaginated(query, skip, limit){
return this.model.find(query)
.lean()
.then((value)=>{
if (value.length === 0) return {userMessage: 'Document not found'};
const count = value.length;
//skip===0 must be handled
const start = parseInt(limit)*parseInt(skip - 1);
const end = start + parseInt(reqQuery.pagesize);
//slicing the array
value = value.slice(start,end);
//could return it another way...
value.push( { 'querySize': count });
return value;
})
.catch((reason)=>{
//...handling code
});
}

How to retrieve query data output from Dynamodb using Node.js

I am having difficulty to understand where is my query stored in which array variable. I know the "var params" contains all the query conditions and "db.query" will perform the actual query action. But how do I get my query result which match my search condition?
At the end, I want to pass my query output parameters (title, postedby) to html or jade for display so I need to understand where all the query data are stored at first place and how to retrieve them. Some sample code to demo this part would be greatly appreciated. Thanks
var params = {
TableName: tablename,
AttributesToGet: [
'title',
'postedBy'
],
Limit: 20,
ScanIndexForward: true,
KeyConditions: { /* required */
key : {
ComparisonOperator: 'EQ', /* required */
AttributeValueList: [
{
'N' : 'some value'
}
/* more items */
},
},
};
db.query(params, function(err, data) {
if (err) {
console.log('show error here');
} else {
console.log('Success');
}
});
I found out that all my query data is stored inside "data" from function(err, data). But i am having problem passing "data" to the following export function which later can be used as a local variable inside my webpage. Then I realized the "data" part is always empty {} inside export function even i am sure the above db.query can return the right result. So i believe somehow the export.list function is executed before the db.query happen so I am not getting any data assigned. How can I overcome this part? Thanks.
exports.list = function(req, res){
res.render('post', {title:'Title', posts : data });
console.log("Result: %j", data);
};
Note: Console.log("Result: ", data) is showing {}
I figured it out. Use the callback function due to the nature of javascript. Thanks.

mongoosejs query fails when filtering using a string of keys as selector

I was attempting to query docs from a mongodb and have the system return only a few fields from the documents matching the query. I first tried the syntax listed below for the first query and it failed to return i.e. the callback was never called.
I then experimented some more with alternative syntax and was able get results from the second query listed below. I'd like to understand why the first query didn't work - have I misinterpreted the syntax?
this is mongoose 3.6.8 and mongo 2.4
TIA
First query
query.find({
category:{
$in: categoryArray
}
,expiration:{$gte: new Date()}
,place:queryVal.destination
}
,'_id expiration url'
,function (err, docs) {
if (err) {
console.log(err);
} else {
console.log('queryJoin returned ' + docs.length + 'entries');
}
}
);
Second query
query.find({
category:{$in: categoryArray}
,expiration:{$gte: new Date()}
,place:queryVal.destination
})
.select({_id:1, expiration:1, url:1})
.exec(function(err, docs) {
console.log('queryJoin returns');
if (err) {
console.log(err);
} else {
console.log('queryJoin returned ' + docs.length + 'entries');
}
});
Your first attempt used Model.find syntax but you were trying to use it with Query#find which doesn't support a fields parameter. As such, Mongoose interpreted your field selection string as a callback which is why your actual callback didn't get called.

Resources