how to use groupBy in Bookshelf JS, here is my Controller code.
router.route('/fetchStudentAttendance')
.post(function(req, res) {
StudentAttendance
.query(function (qb){
qb.where('date', '>=', req.body.date);
qb.groupBy("students_id");
})
.where({'class_id': req.body.class_id, 'section_id': req.body.section_id})
.fetchAll({ withRelated: [{'StudentRef':function(qb) {qb.column('id', 'first_name', 'middle_name', 'last_name')}}]})
.then(studentAttendance => {
let content = {
data: studentAttendance,
success: true,
message: 'Record Not Found',
};
return res.send(content);
})
.catch(error => {
let content = {
data: error,
success: false,
message: 'Error while fetching Student Attendance.',
};
return res.send(content);
});
});
when i am trying to "groupBy" employee_id it will give Error like this.
code:"42703"
file:"parse_relation.c"
length:110
line:"3194"
name:"error"
position:"127"
routine:"check_ungrouped_columns_walker"
severity:"ERROR"
TL;DR
Use Bookshelf's built-in Collection manipulation to do custom groupBy after the .fetchAll()
OR
Use raw Knex to generate your query and results as needed, since groupBy will require some SQL aggregation definitions. This can be a Knex query result, or a Bookshelf object.
More Words
Bookshelf isn't exactly a query generator; not like the query libray KnexJS it is built off of. Bookshelf is made as a way to collect rows from your DB as Object Models. A SQL "groupBy" clause is kind of custom to the query, and Bookshelf itself implements lodash methods, like lodash's groupBy as stated in their docs. However, that is a server-side grouping on the queried data, it is not using SQL to do so.
When using your Model in Bookshelf, you should use new Model or Model.forge() before you fetch (or in case of many, .fetchAll()). You can start a Knex query, in a way, by using Model.query (see docs), but be aware that is returning Bookshelf objects. I typically use .query(function (qb) {}) to do some custom knex, like a custom WHERE clause or an qb.orderBy('prop'). Since groupBy would include 'many', you would want Model.query().fetchAll(), and be aware of custom aggregation in that query (Bookshelf will likely handle it, but it wouldn't be exactly like the model you defined, especially with custom methods). Using Knex directly might be a good option too.
Related
I am new to Javascript.
I am developping an API to retrieve information from MySQL with :
Node.js
Express
Sequelize
I developed the following controller based on a Sequelize finder (findByPk, to retrieve information by a primary key). There are several nested includes.
exports.findByPk = (req, res) => {
Poi.findByPk(req.params.id,{
attributes: [
'name',
'description'
],
include: [{
model: Source,
attributes: ['originalId'],
where: {source: 'google'}
},{
...
}]
})
.then(data => {
res.send(data);
})
};
As far as I understand, the finder outputs a relatively complex Sequelize instance object data which is passed in res.send (but the API gives access to a proper Json).
I need to postprocess data before passing the result in res.send :
I need to retreive information from a Google API using the data fields, and add this information to the API output
I need to compute some values based on the data fields, and add these values to the API output (long story short, it cannot be part of the Sequelize query because of a known issue in Sequelize)
How to retrieve the values I need from data ? Should I treat it as a regular javascript object, understand its structure, and manually retrieve the values I need ? Or are there methods for that ?
After that, how to add information to data ? Should I actually add information to data, or create a new object ? Should it be a Sequelize object or can it be a regular json-like javascript object ?
Thank you very much
You can just turn data (that is actually an instance of the Poi Sequelize model) into a plain JS object and afterwards work with it in a regular way:
.then(data => {
const plainObject = data.get({ plain: true });
plainObject.newProp = 1
// some work here including accessing Google API
...
res.json(plainObject);
I'm currently playing around with NestJS and am using MongoDB with TypeORM, and I couldn't find something similar to the .populate() method in Mongoose, is there a way to do it with TypeORM or should I stick to Mongoose?
For example, here is a route I created with Express + Mongoose, and I want to recreate it with NestJS + TypeORM:
route.get('/:slug', async (req, res) => {
const collection = await Collection.findOne({slug: req.params.slug}).populate('products');
res.json(collection);
});
In TypeORM there is something called relations which you can use to populate the docs from other(related) collections.
Here is an example from TypeORM docs :
createConnection(/*...*/).then(async connection => {
/*...*/
let photoRepository = connection.getRepository(Photo);
let photos = await photoRepository.find({ relations: ["metadata"] });
}).catch(error => console.log(error));
You can read more about it here on TypeORM docs.
According to how you design collection/schema in TypeORM, your query might look like this, i havent tried the query but you can make it work like this:
Collection.find({slug: req.params.slug}, {relations : ['products']});
Note, there are also functions like .innerJoinAndSelect and .leftJoinAndSelect which you can use with QueryBuilder(.createQueryBuilder) and populate documents from other collection just like mongoose .populate()
I suggest that you read the TypeORM, there are many examples given as well, it will help you build the kind of query you need.
I'm trying to build an application, using MongoDB and Node.JS. I have 3 models: User, Ride, Participating.
Participating contains a userID and a rideID. It is almost as with a SQL logic: Participating links the two others models.
I'd like to, using a userID, return every Ride thanks to Participating Model
I tried to use a forEach, as the first request returns an array.
router.get('/getAllRide/:userID',function(req,res){
let userID = req.params.userID
let return = []
Participating.find({_idUser: userID })
.then(participating => {
participating.forEach(element => {
Ride.find({_id: element._id})
.exec()
.then(ride => {
retour.push(ride)})
});
res.status(200).json(return)
});
At the end of this code, the array return is empty, while it is supposed to contain every Ride whose _id is in an entity Participating.
OK, there are a couple of issues here:
return is a keyword. You probably shouldn't be using it as a variable name.
Database calls are asynchronous. forEach loops are synchronous. This means that you're immediately going to be returning retour (which looks undefined).
Mongoose has tools to populate nested relationships -- it's best not to do it in application code. Even if you are doing this in application code, it's likely best not to iterate over your results & do new finds -- instead, it's better to construct a single find query that returns all of the new documents you need.
If you did want to do this in application code, you'd want to either use async/await or Promise.all:
const toReturn = [];
const findPromises = participating.map(element => {
return Ride.find({_id: element._id})
.exec()
.then(result => toReturn.push(result)
});
return Promise.all(findPromises).then(() => res.status(200).json(toReturn));
(note: rather than using Promise.all, if you're using Bluebird you could instead use Promise.map.
In Mongoose, I can use a query populate to populate additional fields after a query. I can also populate multiple paths, such as
Person.find({})
.populate('books movie', 'title pages director')
.exec()
However, this would generate a lookup on book gathering the fields for title, pages and director - and also a lookup on movie gathering the fields for title, pages and director as well. What I want is to get title and pages from books only, and director from movie. I could do something like this:
Person.find({})
.populate('books', 'title pages')
.populate('movie', 'director')
.exec()
which gives me the expected result and queries.
But is there any way to have the behavior of the second snippet using a similar "single line" syntax like the first snippet? The reason for that, is that I want to programmatically determine the arguments for the populate function and feed it in. I cannot do that for multiple populate calls.
After looking into the sourcecode of mongoose, I solved this with:
var populateQuery = [{path:'books', select:'title pages'}, {path:'movie', select:'director'}];
Person.find({})
.populate(populateQuery)
.execPopulate()
you can also do something like below:
{path:'user',select:['key1','key2']}
You achieve that by simply passing object or array of objects to populate() method.
const query = [
{
path:'books',
select:'title pages'
},
{
path:'movie',
select:'director'
}
];
const result = await Person.find().populate(query).lean();
Consider that lean() method is optional, it just returns raw json rather than mongoose object and makes code execution a little bit faster! Don't forget to make your function (callback) async!
This is how it's done based on the Mongoose JS documentation http://mongoosejs.com/docs/populate.html
Let's say you have a BookCollection schema which contains users and books
In order to perform a query and get all the BookCollections with its related users and books you would do this
models.BookCollection
.find({})
.populate('user')
.populate('books')
.lean()
.exec(function (err, bookcollection) {
if (err) return console.error(err);
try {
mongoose.connection.close();
res.render('viewbookcollection', { content: bookcollection});
} catch (e) {
console.log("errror getting bookcollection"+e);
}
//Your Schema must include path
let createdData =Person.create(dataYouWant)
await createdData.populate([{path:'books', select:'title pages'},{path:'movie', select:'director'}])
I have a database which have approx 600000 records.
I 'm using the sails.js but
when I'm fetching the data with waterline approach It takes very long time to fetch 600000 records (approx 17sec) and It has limited query i.e It doesn't have Accessing Join Tables. so I join take the result of two query and then filter the data that's by It take lot's of time.
So I decided to use MongoDB with sails instead of waterline and I'm wondering if I can somehow use the Blueprint API without being linked to a Waterline model.
How to use the MongoDB instead of the waterline?
If you want to use the Sails models api, you can overwrite the blueprint methods in the controller.
So to overwrite, say, User Model, create the following functions in UserController.js:
find, create, update, destroy
find will override 'get api/v1/user'
create will override 'post api/v1/user'
update will override 'put api/v1/user'
destroy will override 'delete api/v1/user'
Once inside the controller, you can run a native query on Mongo like so:
In UserControllelr.js
find: function (req, res) {
var packet = req.params.all();
// packet now has all url and posted parameters
User.native(function (err, UserCollection) {
if(err) {
//handle error
}
else {
// here you can run any native mongo Query using UserCollection
// eg:
UserCollection.aggregate(
{"$match": {"gender": "Male"} },
{"$group": { "_id": "$socialNetwork", "count": {"$sum":1} } },
function (err, results) {
//do stuff with results
})
}
})
}
Hope this helps.