This is a contrived example of what I would like to do:
Suppose I have a database of teams and players:
team:
->id
->color
->rank
->division
player:
->id
->team_id
->number
->SECRET
And the following bookshelf models:
var Base = require('./base');
const Player = Base.Model.extend(
{
tableName: "players",
},
nonsecretdata: function() {
return this.belongsTo('Team')
},
{
fields: {
id: Base.Model.types.integer,
team_id: Base.Model.types.integer,
number: Base.Model.types.integer,
SECRET: Base.Model.types.string,
}
}
);
module.exports = Base.model('Player', Player);
And
var Base = require('./base');
const Team = Base.Model.extend(
{
tableName: "teams",
},
{
fields: {
id: Base.Model.types.integer,
color: Base.Model.types.string,
rank: Base.Model.types.integer,
division: Base.Model.types.string,
}
}
);
module.exports = Base.model('Team', Team);
My question is, how can I limit the scope of player such that SECRET is not grabbed by calls to join player and team with callback nonsecretdata?
I am new to Bookshelf so if any other information is needed, please let me know. Thank you
++++++++++
Edit: Do I need to create a separate model?
The only way to do this using bookshelf would be to delete the individual fields from the object after fetching the entire model.
A potentially better solution for this use case would be to define a custom Data Access Object class that uses a SQL query for the information that would like to be obtained and then use that DOA instead of using bookshelf. That way the SQL code is still abstracted away from the code that is requesting the information and the SECRET or any other potential sensitive information that is added to the table will not be included in the fetch.
Related
I have managed to get my include statements working with my foreign keys however when I try to add a 'where' statement to the findAll statement I am getting the below error. I have checked my foreign keys and models numerous times and I can't see any problems with them.
Do I have the syntax correct in the below API? The data in the keys is important so I don't want to have to spend more time on a workaround. I have seen other examples that have the where statement inside one of the include keys but the fields I want pulled are on the main table so I don't think this is the same situation.
Unhandled rejection Error: Include unexpected. Element has to be either a Model, an Association or an object.
API
const sequelize = require("pg");
const { Sequelize, Op, Model, DataTypes } = require("sequelize");
exports.schoolDiveLogApproval = (req, res) => {
try {
const schoolID = req.params.schoolID;
diveLog.findAll({
include: [{
model: diveType, as: 'diveTypeID_FK2',
}, {
model: diveSchool, as: 'diveSchoolID_FK',
}, {
model: current, as: 'currentID_FK',
}, {
model: visibility, as: 'visibilityID_FK',
}, {
model: user, as: 'userID_FK1',
}, {
model: diveSpot, as: 'diveSpotID_FK2'
}],
where: {
[Op.and]: [
{diveSchoolID: schoolID},
{diveVerifiedBySchool: false}
]
},
})
...........
Update
Error message
{
"message": "missing FROM-clause entry for table \"diveLog\""
}
Controller API with raw SQL
exports.schoolDiveLogApproval = async (req, res) => {
try {
const schoolID = req.params.schoolID;
const { QueryTypes } = require('sequelize');
await diveLog.sequelize.query(
'SELECT * '+
'FROM "diveLogs" '+
'LEFT OUTER JOIN "diveTypes" AS "diveTypeID_FK2" ON "diveLogs"."diveTypeID" = "diveTypeID_FK2"."diveTypeID" ' +
'LEFT OUTER JOIN "diveSchools" AS "diveSchoolID_FK" ON "diveLog"."diveSchoolID" = "diveSchoolID_FK"."diveSchoolID" ' +
'LEFT OUTER JOIN "currents" AS "currentID_FK" ON "diveLog"."currentID" = "currentID_FK"."currentID" ' +
'LEFT OUTER JOIN "visibilities" AS "visibilityID_FK" ON "diveLog"."visibilityID" = "visibilityID_FK"."visibilityID" ' +
'LEFT OUTER JOIN "userLogins" AS "userID_FK" ON "diveLog"."userID" = "userID_FK1"."userID" ' +
'LEFT OUTER JOIN "diveSpots" AS "diveSpotID_FK2" ON "diveLog"."diveSpotID" = "diveSpotFK2"."diveSpotID" ' +
'WHERE "diveLogs"."diveSchoolID" = ? ' +
'AND "diveLogs"."diveVerifiedBySchool" = ?',
{
replacements: [schoolID, false],
type: QueryTypes.SELECT
}
)
All the table and column names look correct as per the pgAdmin database so I can't see where the problem could be other than the syntax.
I have went off the way the SQL is executed in the IDE terminal so it should execute in theory. Is this more likely be to do with the way I pass the id?
Well, I've really tried some minutes around this sequelize, but I'd really suggest you to learn and use raw SQL, its way easier and universal than that.
After half an hour of trying to set up and understand this sqlize and see so many weird issues like these:
selecting non-existing columns like id, adding an "s" out of nowhere to the name of the table "user_infoS(?)".. so I gave up and came up with a query that might help you, if I understood your problem correctly.
If you send proper schema I can help you further, but yeah, sorry I can't spend more time in this sequelize, this thing is atrocious.
const { QueryTypes } = require('sequelize');
await sequelize.query(
'SELECT *'+
'FROM diveLog l'+
'JOIN diveType t ON l.diveTypeId = t.id'+
'JOIN diveSchool s ON l.diveSchoolId = s.id'+
'JOIN current c ON l.currentId = c.id'+
'JOIN visibility v ON l.visibilityId = v.id'+
'JOIN user u ON l.userId = u.id'+
'JOIN diveSpot sp ON l.diveSpotId = sp.id'+
'WHERE diveSchoolID = ?'+
'AND diveVerifiedBySchool = ?',
{
replacements: [schoolID, false],
type: QueryTypes.SELECT
}
);
With regards to the first error in your API, ensure you are requiring your models like below:
const { diveLog, diveType, diveSchool, diveSchool, current, visibility, user, diveSpot } = require('../models');
The second error message:
{
"message": "relation "divelogs" does not exist"
}
Indicates that the table "divelogs" cannot be found in your database, double check if the table exists, if it does not exist, run your migrations to create the table. Please note that, if enabled Sequelize will automatically make your table names plural. If you define a model called User, Sequelize will create the table as Users
When I went back to using the original Sequelize with the include / where clauses I removed the [Op and] from the API, then re-declared each of the models at the top of the file and it worked.
exports.schoolDiveLogApproval = async (req, res) => {
try {
const schoolID = req.params.schoolID;
diveLog.findAll({
include: [{
model: diveType, as: 'diveTypeID_FK2',
}, {
model: diveSchool, as: 'diveSchoolID_FK',
}, {
model: current, as: 'currentID_FK',
}, {
model: visibility, as: 'visibilityID_FK',
}, {
model: user, as: 'userID_FK1',
}, {
model: diveSpot, as: 'diveSpotID_FK2'
}],
where: [{
diveSchoolID: req.params.schoolID,
diveVerifiedBySchool: false
}],
})
.then((diveLog) => {
const schoolDiveLogList = [];
for (i = 0; i < diveLog.length; i++) {
Second error: you are committing a typo on aliases using sometimes "devLogS" and other times "devLoG" (in the singular).
Try more before asking here, we won't do your work for you.
I tried to find a way to copy/clone instances in Sequelize but without success. Is there any way to do it with a built-in function or without? What I want is to simply copy rows in the database and the new item should have only a different id.
There is no such direct function for that , What you can do is :
Fetch the object that you want to clone/copy
Remove Primary Key from it
Make a new entry from it
model.findOne({ //<---------- 1
where : { id : 1 } ,
raw : true })
.then(data => {
delete data.id; //<---------- 2
model.create(data); //<---------- 3
})
As said, there is no such direct function for that (thanks Vivek)
If you find useful, place the following code on your model class:
async clone() {
let cData = await THISISMYMODEL.findOne({
where: { id: this.id},
raw: true,
});
delete cData.id;
return await THISISMYMODEL.create(data);
}
Take into account that "THISISMYMODEL" should be the model class defined and "id" the primary key attribute used.
Also take into account the use of Foreign Keys (relations with other models), it will use the same keys. Otherwise you should clone those instances too.
You may need to update the name though or some other field to identify it as a copy,
const data = await model.findOne({ where: {id: 1}, raw: true, attributes: { exclude: ['id'] } });
data.name = data.name + '(copy)';
const newRecord = await model.create(data);
Write a Model.create(data) function inside Model.js and call this function from inside of a loop, as many times you need it will create the copy of the same data.
I am attempting to create an instance and multiple related instances with a many to many relation using a junction table.
While creating the multiple related instances, I need to add a value to a property on the junction table as well. I don't know if it is my lack of knowledge of sequelize or promises that is causing my problem.
The code I am using is below. This code does add the items to the database, but I need to redirect after the operation has completed, which is not working.
Basically, I need to create a Recipe. Once that is created, I need to create Ingredients and relate them to that Recipe. The ingredients are stored in an array coming from a form on an HTML page. While relating the Ingredients, I need to add the ingredient_quantity to the RecipeIngredients table, which is the through part of the relationship (the junction table).
global.db.Recipe.belongsToMany(
global.db.Ingredient,
{
as: 'Ingredients',
through: global.db.RecipeIngredients,
foreignKey: 'recipe_id'
});
global.db.Ingredient.belongsToMany(
global.db.Recipe,
{
as: 'Recipes',
through: global.db.RecipeIngredients,
foreignKey: 'ingredient_id'
});
router.post('/new', ensureLoggedIn, bodyParser.json(), function (req, res) {
var recipeName = req.body.recipe_name;
var steps = req.body.steps;
var ingredients = req.body.ingredients;
var ingredientQty = {};
var currentIngredient;
var ingredientsToAdd = [];
db.Recipe.create({
recipe_name: recipeName,
directions: steps,
FamilyId: req.user.FamilyId,
CreatedBy: req.user._id
})
.then(function (recipe) {
for (var i = 0; i < ingredients.length; i++) {
currentIngredient = ingredients[i];
ingredientQty[currentIngredient.ingredient_name] =
currentIngredient.quantity;
db.Ingredient.findOrCreate({
where: {
ingredient_name: currentIngredient.ingredient_name,
FamilyId: req.user.FamilyId
}
})
.spread(function (ingredient, created) {
if (created) {
console.log("Added Ingredient to DB: " +
currentIngredient.ingredient_name);
}
ingredient.Recipes = {
ingredient_quantity:
ingredientQty[ingredient.ingredient_name]
};
ingredient.CreatedBy = req.user._id;
recipe.addIngredient(ingredient)
.then(function () {
console.log("Added Ingredient " + ingredient.ingredient_name
+ " to Recipe " + recipe.recipe_name);
});
})
}
})
.finally(function(recipe){
res.redirect('/recipes');
});
});
Any help would be greatly appreciated. I know that I am running into issues because of trying to use promises inside of a loop, I just don't know how else I can accomplish this.
Using sequelize, you can create objects along with its associated objects in one step, provided all objects that you're creating are new. This is also called nested creation. See this link and scroll down to section titled "Creating with associations"
Coming to your issue, you've a many-to-many relationship between Recipe and Ingredient, with RecipeIngredients being the join table.
Suppose you've a new Recipe object which you want to create, like:
var myRecipe = {
recipe_name: 'MyRecipe',
directions: 'Easy peasy',
FamilyId: 'someId',
CreatedBy: 'someUserId'
}
And an array of Ingredient objects, like:
var myRecipeIngredients = [
{ ingredient_name: 'ABC', FamilyId: 'someId'},
{ ingredient_name: 'DEF', FamilyId: 'someId'},
{ ingredient_name: 'GHI', FamilyId: 'someId'}]
// associate the 2 to create in 1 step
myRecipe.Ingredients = myRecipeIngredients;
Now, you can create myRecipe and its associated myRecipeIngredients in one step as shown below:
Recipe.create(myRecipe, {include: {model: Ingredient}})
.then(function(createdObjects){
res.json(createdObjects);
})
.catch(function(err){
next(err);
});
And that is all !!
Sequelize will create 1 row in Recipe, 3 rows in Ingredient and 3 rows in RecipeIngredients to associate them.
I was able to fix the problem that I was having. The answers here helped me come up with my solution.
I am posting the solution below in case anyone else runs into a similar issue. I created a variable to store the Promise from Recipe.create(), I used Promise.map to findOrCreate all of the ingredients from the form data. Because findOrCreate returns an array containing Promise and a boolean for if the item was created, I then had to get the actual ingredients out of the results of the Promise.map function. So I used the JavaScript array.map() function to get the first item from the arrays. And finally, use Promise.map again to add each Ingredient to the Recipe
var ingredients = req.body.ingredients,
recipeName = req.body.recipeName,
ingredientsQty = {}; // Used to map the ingredient and quantity for the
// relationship, because of the Junction Table
var recipe = models.Recipe.create({recipe_name: recipeName});
// Use Promise.map to findOrCreate all ingredients from the form data
Promise.map(ingredients, function(ing){
ingredientsQty[ing.ingredient_name] = ing.ingredient_quantity;
return models.Ingredient.findOrCreate({ where: { ingredient_name: ing.ingredient_name}});
})
// Use JavaScript's Array.prototype.map function to return the ingredient
// instance from the array returned by findOrCreate
.then(function(results){
return results.map(function(result){
return result[0];
});
})
// Return the promises for the new Recipe and Ingredients
.then(function(ingredientsInDB){
return Promise.all([recipe, ingredientsInDB]);
})
// Now I can use Promise.map again to create the relationship between the /
// Recipe and each Ingredient
.spread(function(addedRecipe, ingredientsToAdd){
recipe = addedRecipe;
return Promise.map(ingredientsToAdd, function(ingredientToAdd){
ingredientToAdd.RecipeIngredients = {
ingredient_quantity: ingredientsQty[ingredientToAdd.ingredient_name]
};
return recipe.addIngredient(ingredientToAdd);
});
})
// And here, everything is finished
.then(function(recipeWithIngredients){
res.end
});
I have defined several models that use a Datasource "db" (mysql) for my environment.
Is there any way to have several datasources attached to those models, so I would be able to perform REST operations to different databases?
i.e:
GET /api/Things?ds="db"
GET /api/Things?ds="anotherdb"
GET /api/Things (will use default ds)
As #superkhau pointed above, each LoopBack Model can be attached to a single data-source only.
You can create (subclass) a new model for each datasource you want to use. Then you can either expose these per-datasource models via unique REST URLs, or you can implement a wrapper model that will dispatch methods to the correct datasource-specific model.
In my example, I'll show how to expose per-datasource models for a Car model that is attached to db and anotherdb. The Car model is defined in the usual way via common/models/car.json and common/models/car.js.
Now you need to define per-datasource models:
// common/models/car-db.js
{
"name": "Car-db",
"base": "Car",
"http": {
"path": "/cars:db"
}
}
// common/models/car-anotherdb.js
{
"name": "Car-anotherdb",
"base": "Car",
"http": {
"path": "/cars:anotherdb"
}
}
// server/model-config.json
{
"Car": {
"dataSource": "default"
},
"Car-db": {
"dataSource": "db"
},
"Car-anotherdb": {
"dataSource": "anotherdb"
}
}
Now you have the following URLs available:
GET /api/Cars:db
GET /api/Cars:anotherdb
GET /api/Cars
The solution outlined above has two limitations: you have to define a new model for each datasource and the datasource cannot be selected using a query parameter.
To fix that, you need a different approach. I'll again assume there is a Car model already defined.
Now you need to create a "dispatcher".
// common/models/car-dispatcher.json
{
"name": "CarDispatcher",
"base": "Model", //< important!
"http": {
"path": "/cars"
}
}
// common/models/car-dispatcher.js
var loopback = require('loopback').PersistedModel;
module.exports = function(CarDispatcher) {
Car.find = function(ds, filter, cb) {
var model = this.findModelForDataSource(ds);
model.find(filter, cb);
};
// a modified copy of remoting metadata from loopback/lib/persisted-model.js
Car.remoteMethod('find', {
isStatic: true,
description: 'Find all instances of the model matched by filter from the data source',
accessType: 'READ',
accepts: [
{arg: 'ds', type: 'string', description: 'Name of the datasource to use' },
{arg: 'filter', type: 'object', description: 'Filter defining fields, where, orderBy, offset, and limit'}
],
returns: {arg: 'data', type: [typeName], root: true},
http: {verb: 'get', path: '/'}
});
// TODO: repeat the above for all methods you want to expose this way
Car.findModelForDataSource = function(ds) {
var app = this.app;
var ds = ds && app.dataSources[ds] || app.dataSources.default;
var modelName = this.modelName + '-' + ds;
var model = loopback.findModel(modelName);
if (!model) {
model = loopback.createModel(
modelName,
{},
{ base: this.modelName });
}
return model;
};
};
The final bit is to remove Car and use CarDispatcher in the model config:
// server/model-config.json
{
"CarDispatcher": {
dataSource: null,
public: true
}
}
By default, you can only attach data sources on a per-model basis. Meaning you can attach each model to a different data source via datasources.json.
For your use case, you will to add a remote hook to each endpoint you want for multiple data sources. In your remote hook, you will do something like:
...
var ds1 = Model.app.dataSources.ds1;
var ds2 = Model.app.dataSources.ds2;
//some logic to pick a data source
if (context.req.params...
...
See http://docs.strongloop.com/display/LB/Remote+hooks for more info.
For anyone still looking for a working answer to this, the solution for switching databases on the fly was to write a middleware script that examined the request path and then created a new DataSource connector, passing in a variable based on the req.path variable. For example, if the request path is /orders, then "orders" as a string would be saved in a variable, then we attached a new Datasource, passing in that variable for "orders". Here's the complete working code.
'use strict';
const DataSource = require('loopback-datasource-juggler').DataSource;
const app = require('../server.js');
module.exports = function() {
return function datasourceSelector(req, res, next) {
// Check if the API request path contains one of our models.
// We could use app.models() here, but that would also include
// models we don't want.
let $models = ['offers', 'orders', 'prducts'];
// $path expects to be 'offers', 'orders', 'prducts'.
let $path = req.path.toLowerCase().split("/")[1];
// Run our function if the request path is equal to one of
// our models, but not if it also includes 'count'. We don't
// want to run this twice unnecessarily.
if (($models.includes($path, 0)) && !(req.path.includes('count'))) {
// The angular customer-select form adds a true value
// to the selected property of only one customer model.
// So we search the customers for that 'selected' = true.
let customers = app.models.Customer;
// Customers.find() returns a Promise, so we need to get
// our selected customer from the results.
customers.find({"where": {"selected": true}}).then(function(result){
// Called if the operation succeeds.
let customerDb = result[0].name;
// Log the selected customer and the timestamp
// it was selected. Needed for debugging and optimization.
let date = new Date;
console.log(customerDb, $path+req.path, date);
// Use the existing veracore datasource config
// since we can use its environment variables.
let settings = app.dataSources.Veracore.settings;
// Clear out the veracore options array since that
// prevents us from changing databases.
settings.options = null;
// Add the selected customer to the new database value.
settings.database = customerDb;
try {
let dataSource = new DataSource(settings);
// Attach our models to the new database selection.
app.models.Offer.attachTo(dataSource);
app.models.Order.attachTo(dataSource);
app.models.Prduct.attachTo(dataSource);
} catch(err) {
console.error(err);
}
})
// Called if the customers.find() promise fails.
.catch(function(err){
console.error(err);
});
}
else {
// We need a better solution for paths like '/orders/count'.
console.log(req.path + ' was passed to datasourceSelector().');
}
next();
};
};
I have an emberjs application backed by a nodejs server and mongodb. Currently my database is sending documents with an '_id' field. I have the followign code to force Ember to treat '_id' as the primary key:
App.ApplicationSerializer = DS.RESTSerializer.extend({
primaryKey: '_id'
});
On the other hand i have two models related by a 'hasMany' relationship as such:
App.Player = DS.Model.extend({
name: DS.attr('string'),
thumbLink: DS.attr('string'),
activeGame: DS.belongsTo('game', { async: true }),
email: DS.attr('string'),
firstName: DS.attr('string'),
lastName: DS.attr('string'),
admin: DS.attr('boolean')
});
App.Game = DS.Model.extend({
name: DS.attr('string'),
active: DS.attr('boolean'),
players: DS.hasMany('player', { async: true })
});
The problem is that when i try to save the model ( this.get('model').save() )on my controller the ids are not serialized and the result is ember sending the following:
{"game":{"name":"Indie/Rock","active":false,"players":[],"_id":"53cbbf43daa978983ee0b101"}}
As you can see the players array is empty, and as a result, the server is saving that empty array which in fact is not correct. I am aware that it is possible to use { embedded: true } on the models and return the models with embedded documents from the server, but i want to preserve the async feature.
I have tried to extend the game serializer from EmbeddedRecordsMixing as the following:
App.GameSerializer = DS.ActiveModelSerializer
.extend(DS.EmbeddedRecordsMixin)
.extend({
attrs: {
players: {serialize: 'ids', deserialize: 'ids'},
}
});
But when i do so i get the following error from ember even though the ApplicationSerializer is suppossedly telling Ember to user _id as primary key:
Assertion Failed: Error: Assertion Failed: You must include an `id` for App.Game in a hash passed to `push`
My question is if it is possible to maintain the async features of ember-data while being able to serialize the document with the ids on it's relation and using _id as a primary key.
Thank you.
Ember Data is stupid in this aspect, if it's a ManyToOne relationship, it only includes the id from the belongsTo side. Honestly I've had it on my bucket list to submit a PR, but time is limited.
https://github.com/emberjs/data/commit/7f752ad15eb9b9454e3da3f4e0b8c487cdc70ff0#commitcomment-4923439
App.ApplicationSerializer = DS.RESTSerializer.extend({
serializeHasMany: function(record, json, relationship) {
var key = relationship.key;
var payloadKey = this.keyForRelationship ? this.keyForRelationship(key, "hasMany") : key;
var relationshipType = RelationshipChange.determineRelationshipType(record.constructor, relationship);
if (relationshipType === 'manyToNone' || relationshipType === 'manyToMany'
|| relationshipType === 'manyToOne') { // This is the change
json[payloadKey] = get(record, key).mapBy('id');
// TODO support for polymorphic manyToNone and manyToMany relationships
}
},
});