I have the following kind of setup: I have a model called Person, which is related to other Persons (eg. neighbours) through a many-to-many association. The interpretation is that a person can have many neighbours, and a person can be the neighbour of many other people. The model attributes look just about like this:
...
attributes: {
name: 'string',
neigh: {
collection: 'Person',
via: 'neighOf',
dominant: true
},
neighOf: {
collection: 'Person',
via: 'neigh'
}
}
...
Now imagine us having two people, Person A and Person B. A is a neighbor of B, and B is a neighbor of A. A moves away, so they're no longer neighbors. Now when I remove B from the neighs of A, A will not automatically be removed from the neighs of B. This is a problem.
I tried to address the problem by creating an afterUpdate function to the Person model, in which I would check every person in the neighOf field of A for whether A still has them as a neighbour, and if not, would update the Persons that no longer were neighbours of A. I also wanted to do the opposite, so if A became a neighbour of B again (eg. the neigh field of B was updated), the neigh field of A should update automatically as well. Here's what I came up with:
afterUpdate: function(updatedPerson, cb) {
Person.findOne(updatedPerson.id)
.populate('neigh')
.populate('neighOf')
.then(function(person) {
updatedPerson = person;
var ids = _.pluck(updatedPerson.neigh, 'id');
return Person.find(ids).populate('neigh');
})
.then(function(people) {
neigh_of_updated = people;
var ids = _.pluck(updatedPerson.neighOf, 'id');
return Person.find(ids).populate('neigh');
})
.then(function(people) {
neighOf_of_updated = people;
var updates = [];
_.forEach(neighOf_of_updated, function(person) {
if (_.find(person.neigh, function(neigh) {return neigh.id === updatedPerson.id})
&& !_.find(updatedPerson.neigh, function(neigh) {return neigh.id === person.id})) {
var neighIds = _.pluck(person.neigh, 'id');
_.remove(neighIds, function(id) {
return id === updatedPerson.id;
});
updates.push(Person.update(person.id, {neigh: neighIds}));
}
})
return Promise.all(updates);
})
.then(function() {
var updates = [];
_.forEach(neighOfUpdated, function(person) {
if (!_.find(person.neigh, function(neigh) { return neigh.id === updatedPerson.id; })) {
var neighIds = _.pluck(person.neigh, 'id');
neighIds.push(updatedPerson.id);
updates.push(Person.update({id: Person.id}, {neigh: neighIds}))
}
})
return Promise.all(updates);
})
.then(function() {
cb();
})
.catch(function(error) {
cb(error);
})
}
So far I haven't been able to get both working. I can get them working separately, but when combined, the code is looping itself endlessly due to some updates happening in the middle of it all, causing new and new afterUpdates to be called.
I feel like this is a pretty common use case, so I'm guessing there should be an easier way to do all of this - or if there isn't, at least some way to get this working. As it is, I feel like I'm hitting my head against a brick wall in a matter that should be an easy task. Anyone have experience of similar associations and how to handle them?
Related
I am attempting to create an instance and multiple related instances with a many to many relation using a junction table.
While creating the multiple related instances, I need to add a value to a property on the junction table as well. I don't know if it is my lack of knowledge of sequelize or promises that is causing my problem.
The code I am using is below. This code does add the items to the database, but I need to redirect after the operation has completed, which is not working.
Basically, I need to create a Recipe. Once that is created, I need to create Ingredients and relate them to that Recipe. The ingredients are stored in an array coming from a form on an HTML page. While relating the Ingredients, I need to add the ingredient_quantity to the RecipeIngredients table, which is the through part of the relationship (the junction table).
global.db.Recipe.belongsToMany(
global.db.Ingredient,
{
as: 'Ingredients',
through: global.db.RecipeIngredients,
foreignKey: 'recipe_id'
});
global.db.Ingredient.belongsToMany(
global.db.Recipe,
{
as: 'Recipes',
through: global.db.RecipeIngredients,
foreignKey: 'ingredient_id'
});
router.post('/new', ensureLoggedIn, bodyParser.json(), function (req, res) {
var recipeName = req.body.recipe_name;
var steps = req.body.steps;
var ingredients = req.body.ingredients;
var ingredientQty = {};
var currentIngredient;
var ingredientsToAdd = [];
db.Recipe.create({
recipe_name: recipeName,
directions: steps,
FamilyId: req.user.FamilyId,
CreatedBy: req.user._id
})
.then(function (recipe) {
for (var i = 0; i < ingredients.length; i++) {
currentIngredient = ingredients[i];
ingredientQty[currentIngredient.ingredient_name] =
currentIngredient.quantity;
db.Ingredient.findOrCreate({
where: {
ingredient_name: currentIngredient.ingredient_name,
FamilyId: req.user.FamilyId
}
})
.spread(function (ingredient, created) {
if (created) {
console.log("Added Ingredient to DB: " +
currentIngredient.ingredient_name);
}
ingredient.Recipes = {
ingredient_quantity:
ingredientQty[ingredient.ingredient_name]
};
ingredient.CreatedBy = req.user._id;
recipe.addIngredient(ingredient)
.then(function () {
console.log("Added Ingredient " + ingredient.ingredient_name
+ " to Recipe " + recipe.recipe_name);
});
})
}
})
.finally(function(recipe){
res.redirect('/recipes');
});
});
Any help would be greatly appreciated. I know that I am running into issues because of trying to use promises inside of a loop, I just don't know how else I can accomplish this.
Using sequelize, you can create objects along with its associated objects in one step, provided all objects that you're creating are new. This is also called nested creation. See this link and scroll down to section titled "Creating with associations"
Coming to your issue, you've a many-to-many relationship between Recipe and Ingredient, with RecipeIngredients being the join table.
Suppose you've a new Recipe object which you want to create, like:
var myRecipe = {
recipe_name: 'MyRecipe',
directions: 'Easy peasy',
FamilyId: 'someId',
CreatedBy: 'someUserId'
}
And an array of Ingredient objects, like:
var myRecipeIngredients = [
{ ingredient_name: 'ABC', FamilyId: 'someId'},
{ ingredient_name: 'DEF', FamilyId: 'someId'},
{ ingredient_name: 'GHI', FamilyId: 'someId'}]
// associate the 2 to create in 1 step
myRecipe.Ingredients = myRecipeIngredients;
Now, you can create myRecipe and its associated myRecipeIngredients in one step as shown below:
Recipe.create(myRecipe, {include: {model: Ingredient}})
.then(function(createdObjects){
res.json(createdObjects);
})
.catch(function(err){
next(err);
});
And that is all !!
Sequelize will create 1 row in Recipe, 3 rows in Ingredient and 3 rows in RecipeIngredients to associate them.
I was able to fix the problem that I was having. The answers here helped me come up with my solution.
I am posting the solution below in case anyone else runs into a similar issue. I created a variable to store the Promise from Recipe.create(), I used Promise.map to findOrCreate all of the ingredients from the form data. Because findOrCreate returns an array containing Promise and a boolean for if the item was created, I then had to get the actual ingredients out of the results of the Promise.map function. So I used the JavaScript array.map() function to get the first item from the arrays. And finally, use Promise.map again to add each Ingredient to the Recipe
var ingredients = req.body.ingredients,
recipeName = req.body.recipeName,
ingredientsQty = {}; // Used to map the ingredient and quantity for the
// relationship, because of the Junction Table
var recipe = models.Recipe.create({recipe_name: recipeName});
// Use Promise.map to findOrCreate all ingredients from the form data
Promise.map(ingredients, function(ing){
ingredientsQty[ing.ingredient_name] = ing.ingredient_quantity;
return models.Ingredient.findOrCreate({ where: { ingredient_name: ing.ingredient_name}});
})
// Use JavaScript's Array.prototype.map function to return the ingredient
// instance from the array returned by findOrCreate
.then(function(results){
return results.map(function(result){
return result[0];
});
})
// Return the promises for the new Recipe and Ingredients
.then(function(ingredientsInDB){
return Promise.all([recipe, ingredientsInDB]);
})
// Now I can use Promise.map again to create the relationship between the /
// Recipe and each Ingredient
.spread(function(addedRecipe, ingredientsToAdd){
recipe = addedRecipe;
return Promise.map(ingredientsToAdd, function(ingredientToAdd){
ingredientToAdd.RecipeIngredients = {
ingredient_quantity: ingredientsQty[ingredientToAdd.ingredient_name]
};
return recipe.addIngredient(ingredientToAdd);
});
})
// And here, everything is finished
.then(function(recipeWithIngredients){
res.end
});
I am using NodeJS, PostgreSQL and the amazing pg-promise library. In my case, I want to execute three main queries:
Insert one tweet in the table 'tweets'.
In case there is hashtags in the tweet, insert them into another table 'hashtags'
Them link both tweet and hashtag in a third table 'hashtagmap' (many to many relational table)
Here is a sample of the request's body (JSON):
{
"id":"12344444",
"created_at":"1999-01-08 04:05:06 -8:00",
"userid":"#postman",
"tweet":"This is the first test from postman!",
"coordinates":"",
"favorite_count":"0",
"retweet_count":"2",
"hashtags":{
"0":{
"name":"test",
"relevancetraffic":"f",
"relevancedisaster":"f"
},
"1":{
"name":"postman",
"relevancetraffic":"f",
"relevancedisaster":"f"
},
"2":{
"name":"bestApp",
"relevancetraffic":"f",
"relevancedisaster":"f"
}
}
All the fields above should be included in the table "tweets" besides hashtags, that in turn should be included in the table "hashtags".
Here is the code I am using based on Nested transactions from pg-promise docs inside a NodeJS module. I guess I need nested transactions because I need to know both tweet_id and hashtag_id in order to link them in the hashtagmap table.
// Columns
var tweetCols = ['id','created_at','userid','tweet','coordinates','favorite_count','retweet_count'];
var hashtagCols = ['name','relevancetraffic','relevancedisaster'];
//pgp Column Sets
var cs_tweets = new pgp.helpers.ColumnSet(tweetCols, {table: 'tweets'});
var cs_hashtags = new pgp.helpers.ColumnSet(hashtagCols, {table:'hashtags'});
return{
// Transactions
add: body =>
rep.tx(t => {
return t.one(pgp.helpers.insert(body,cs_tweets)+" ON CONFLICT(id) DO UPDATE SET coordinates = "+body.coordinates+" RETURNING id")
.then(tweet => {
var queries = [];
for(var i = 0; i < body.hashtags.length; i++){
queries.push(
t.tx(t1 => {
return t1.one(pgp.helpers.insert(body.hashtags[i],cs_hashtags) + "ON CONFLICT(name) DO UPDATE SET fool ='f' RETURNING id")
.then(hash =>{
t1.tx(t2 =>{
return t2.none("INSERT INTO hashtagmap(tweetid,hashtagid) VALUES("+tweet.id+","+hash.id+") ON CONFLICT DO NOTHING");
});
});
}));
}
return t.batch(queries);
});
})
}
The problem is with this code I am being able to successfully insert the tweet but nothing happens then. I cannot insert the hashtags nor link the hashtag to the tweets.
Sorry but I am new to coding so I guess I didn't understood how to properly return from the transaction and how to perform this simple task. Hope you can help me.
Thank you in advance.
Jean
Improving on Jean Phelippe's own answer:
// Columns
var tweetCols = ['id', 'created_at', 'userid', 'tweet', 'coordinates', 'favorite_count', 'retweet_count'];
var hashtagCols = ['name', 'relevancetraffic', 'relevancedisaster'];
//pgp Column Sets
var cs_tweets = new pgp.helpers.ColumnSet(tweetCols, {table: 'tweets'});
var cs_hashtags = new pgp.helpers.ColumnSet(hashtagCols, {table: 'hashtags'});
return {
/* Tweets */
// Add a new tweet and update the corresponding hash tags
add: body =>
db.tx(t => {
return t.one(pgp.helpers.insert(body, cs_tweets) + ' ON CONFLICT(id) DO UPDATE SET coordinates = ' + body.coordinates + ' RETURNING id')
.then(tweet => {
var queries = Object.keys(body.hashtags).map((_, idx) => {
return t.one(pgp.helpers.insert(body.hashtags[i], cs_hashtags) + 'ON CONFLICT(name) DO UPDATE SET fool = $1 RETURNING id', 'f')
.then(hash => {
return t.none('INSERT INTO hashtagmap(tweetid, hashtagid) VALUES($1, $2) ON CONFLICT DO NOTHING', [+tweet.id, +hash.id]);
});
});
return t.batch(queries);
});
})
.then(data => {
// transaction was committed;
// data = [null, null,...] as per t.none('INSERT INTO hashtagmap...
})
.catch(error => {
// transaction rolled back
})
},
NOTES:
As per my notes earlier, you must chain all queries, or else you will end up with loose promises
Stay away from nested transactions, unless you understand exactly how they work in PostgreSQL (read this, and specifically the Limitations section).
Avoid manual query formatting, it is not safe, always rely on the library's query formatting.
Unless you are passing the result of transaction somewhere else, you should at least provide the .catch handler.
P.S. For the syntax like +tweet.id, it is the same as parseInt(tweet.id), just shorter, in case those are strings ;)
For those who will face similar problem, I will post the answer.
Firstly, my mistakes:
In the for loop : body.hashtag.length doesn't exist because I am dealing with an object (very basic mistake here). Changed to Object.keys(body.hashtags).length
Why using so many transactions? Following the answer by vitaly-t in: Interdependent Transactions with pg-promise I removed the extra transactions. It's not yet clear for me how you can open one transaction and use the result of one query into another in the same transaction.
Here is the final code:
// Columns
var tweetCols = ['id','created_at','userid','tweet','coordinates','favorite_count','retweet_count'];
var hashtagCols = ['name','relevancetraffic','relevancedisaster'];
//pgp Column Sets
var cs_tweets = new pgp.helpers.ColumnSet(tweetCols, {table: 'tweets'});
var cs_hashtags = new pgp.helpers.ColumnSet(hashtagCols, {table:'hashtags'});
return {
/* Tweets */
// Add a new tweet and update the corresponding hashtags
add: body =>
rep.tx(t => {
return t.one(pgp.helpers.insert(body,cs_tweets)+" ON CONFLICT(id) DO UPDATE SET coordinates = "+body.coordinates+" RETURNING id")
.then(tweet => {
var queries = [];
for(var i = 0; i < Object.keys(body.hashtags).length; i++){
queries.push(
t.one(pgp.helpers.insert(body.hashtags[i],cs_hashtags) + "ON CONFLICT(name) DO UPDATE SET fool ='f' RETURNING id")
.then(hash =>{
t.none("INSERT INTO hashtagmap(tweetid,hashtagid) VALUES("+tweet.id+","+hash.id+") ON CONFLICT DO NOTHING");
})
);
}
return t.batch(queries);
});
}),
I'm developing a small NodeJS web app using Mongoose to access my MongoDB database. A simplified schema of my collection is given below:
var MySchema = mongoose.Schema({
content: { type: String },
location: {
lat: { type: Number },
lng: { type: Number },
},
modifierValue: { type: Number }
});
Unfortunately, I'm not able to sort the retrieved data from the server the way it is more convenient for me. I wish to sort my results according to their distance from a given position (location) but taking into account a modifier function with a modifierValue that is also considered as an input.
What I intend to do is written below. However, this sort of sort functionality seems to not exist.
MySchema.find({})
.sort( modifierFunction(location,this.location,this.modifierValue) )
.limit(20) // I only want the 20 "closest" documents
.exec(callback)
The mondifierFunction returns a Double.
So far, I've studied the possibility of using mongoose's $near function, but this doesn't seem to sort, not allow for a modifier function.
Since I'm fairly new to node.js and mongoose, I may be taking a completely wrong approach to my problem, so I'm open to complete redesigns of my programming logic.
Thank you in advance,
You might have found an answer to this already given the question date, but I'll answer anyway.
For more advanced sorting algorithms you can do the sorting in the exec callback. For example
MySchema.find({})
.limit(20)
.exec(function(err, instances) {
let sorted = mySort(instances); // Sorting here
// Boilerplate output that has nothing to do with the sorting.
let response = { };
if (err) {
response = handleError(err);
} else {
response.status = HttpStatus.OK;
response.message = sorted;
}
res.status(response.status).json(response.message);
})
mySort() has the found array from the query execution as input and the sorted array as output. It could for instance be something like this
function mySort (array) {
array.sort(function (a, b) {
let distanceA = Math.sqrt(a.location.lat**2 + a.location.lng**2);
let distanceB = Math.sqrt(b.location.lat**2 + b.location.lng**2);
if (distanceA < distanceB) {
return -1;
} else if (distanceA > distanceB) {
return 1;
} else {
return 0;
}
})
return array;
}
This sorting algorithm is just an illustration of how sorting could be done. You would of course have to write the proper algorithm yourself. Remember that the result of the query is an array that you can manipulate as you want. array.sort() is your friend. You can information about it here.
I would like baffle.where({id: 1}).fetch() to always get typeName attribute as a part of baffle model, without fetching it from baffleType explicitly each time.
The following works for me but it seems that withRelated will load relations if baffle model is fetched directly, not by relation:
let baffle = bookshelf.Model.extend({
constructor: function() {
bookshelf.Model.apply(this, arguments);
this.on('fetching', function(model, attrs, options) {
options.withRelated = options.withRelated || [];
options.withRelated.push('type');
});
},
virtuals: {
typeName: {
get: function () {
return this.related('type').attributes.typeName;
}
}
},
type: function () {
return this.belongsTo(baffleType, 'type_id');
}
});
let baffleType = bookshelf.Model.extend({});
What is the proper way to do that?
Issue on repo is related to Fetched event, However Fetching event is working fine (v0.9.2).
So just for example if you have a 3rd model like
var Test = Bookshelf.model.extend({
tableName : 'test',
baffleField : function(){
return this.belongsTo(baffle)
}
})
and then do Test.forge().fetch({ withRelated : ['baffleField']}), fetching event on baffle will fire. However ORM will not include type (sub Related model) unless you specifically tell it to do so by
Test.forge().fetch({ withRelated : ['baffleField.type']})
However I would try to avoid this if it is making N Query for N records.
UPDATE 1
I was talking about same thing that you were doing on fetching event like
fetch: function fetch(options) {
var options = options || {}
options.withRelated = options.withRelated || [];
options.withRelated.push('type');
// Fetch uses all set attributes.
return this._doFetch(this.attributes, options);
}
in model.extend. However as you can see, this might fail on version changes.
This question is super old, but I'm answering anyway.
I solved this by just adding a new function, fetchFull, which keeps things pretty DRY.
let MyBaseModel = bookshelf.Model.extend({
fetchFull: function() {
let args;
if (this.constructor.withRelated) {
args = {withRelated: this.constructor.withRelated};
}
return this.fetch(args);
},
};
let MyModel = MyBaseModel.extend({
tableName: 'whatever',
}, {
withRelated: [
'relation1',
'relation1.related2'
]
}
);
Then whenever you're querying, you can either call Model.fetchFull() to load everything, or in cases where you don't want to take a performance hit, you can still resort to Model.fetch().
I have two different subscriptions in my app:
Meteor.subscribe('collection');
and
Meteor.subscribe('filtered-collection',param1,param2);
I want to supply the data to different templates through different template helpers, say allResults and filteredResults respectively.
Since $geoWithin doesn't work at the client side and I need to use it for filtering, I cannot just filter the first subscription by
filteredResults = Collection.find(selector);`
Therefore, I need a separate subscription for it.
So, the question is: how to find the result set from respective subscription and pass it through a helper?
I finally solved the problem. I don't think the solution is ideal though.
At Server:
Collection = new Meteor.Collection('collection');
Meteor.publish('collection',function(){
return Collection.find();
});
Meteor.publish('filteredCollection',function(loc, radius){
var selector = {};
if (radius === undefined)
radius = 100;
if (loc !== undefined && !(isNaN(loc[0]) || isNaN(loc[1]))) {
selector.loc = {
$geoWithin: {
$centerSphere: [loc, radius / 6371]
}
};
}
var sub = this,
handle = null;
var handle = Collection.find(selector).observeChanges({
added: function(id, fields) {
sub.added("filteredCollection", id, fields);
},
changed: function(id, fields) {
sub.changed("filteredCollection", id, fields);
},
removed: function(id) {
sub.removed("filteredCollection", id);
}
});
sub.ready();
this.onStop(function() {
handle.stop();
});
});
At client:
Collection = new Meteor.Collection('collection');
FilteredCollection = new Meteor.Collection('filteredCollection');
Meteor.subscribe('collection');
Meteor.subscribe('filteredCollection',loc,radius);
Template.collection.helpers({
collection: function(){
return Collection.find();
},
filteredCollection: function(){
return FilteredCollection.find();
}
});
At the client, Collection and FilteredCollection are two different subsets of the same underlying collection at the server. But whether the two subsets are dependent on each other in terms of caching and persistence, is (I think) a different question altogether.