I need to delete and insert the same data to the table.
But currently ,after the delete processes it seems that the data are still in the database. I think this was a soft-delete only. I cannot insert the same data to the DB since there are items that are supposed to be unique and it is not deleted from the DB.
After the deleteResults function, I assumed that the items are now deleted to the database, but when the insertQuestions is called, it encounters an error which is some data are the same with the data in the DB.
My model is not set as paranoid so I cannot use the force property.
How to hard-delete in Sequelize?
Here is the code for delete and insert implementation:
const res = await <<Model>>.destroy({
where: {
id: id
},
transaction: transaction
})
// This contains the destroy calls
const deleteResults = await deleteQuestions(questionnaireResult.id, questionnaireResult.qneQuestions, transaction);
//Insert to DB
const insertQuestionsResult = await insertQuestions(questionnaireResult.id, null, qsReponseObj.questions, answerTypes, languages, sortValuesQuestion, transaction);
Note that they are using the same transaction.
You are using transaction to do these two operations but seems did not call commit() function. I did a simple test on my local, before I run my code, data in users table:
After I run the code below, delete the record with id of value 2 and insert a new record with id 2 but new user name:
class Users extends Model {}
Users.init({
name: DataTypes.STRING
}, { sequelize, modelName: 'users' });
(async () => {
const t = await sequelize.transaction();
await Users.destroy({
where: {
id: 2
},transaction: t
});
await Users.create({
id:2,
name:"a new user name"
},{transaction:t})
await t.commit();
})().then(()=>{sequelize.close()})
.catch(error =>{console.log(error)})
Result:
Related
I have a recurring problem when deleting items with NodeJs and Sequelize MySQL DB that the shopping cart is not cleared after an order has been placed on my site.
I want to delete the items in the OrderItem table and the Cart table when the order is created.
const createOrder = await ShopOrder.create({
orderId: cart.id,
username,
phone_number,
address,
status,
totalPrice: totalPrices
});
createOrder.save()
Cart.destroy({ where: { id: cart.id } });
res.cookie('cart', '', { maxAge: 1 });
OrderItem.destroy({ where: {}, truncate: true });
res.status(201).json(createOrder);
return createOrder;
But when I run the destroy command on creation I get the success 201 that the command was created but when I query the database all the tables are empty
I have a local postgres database running on my machine. I use node.js to access it. I have a table called 'friends' where every row is a user and a friend. I also have a table called 'users' where every row has all basic info about a user(e.g name and such).
When I want to remove a friendship between two users I have to remove two rows from the 'friends' table. I do this with this function:
const removeFriend = async (clientId, friendId) => {
// initiate transaction
return db
.transaction((trx) => {
// remove friendship from client
trx('friends')
.where({ user_id: clientId, friend_id: friendId })
.del()
.then(() => {
// remove friendship from friend
return trx('friends').where({ user_id: friendId, friend_id: clientId }).del();
})
// if all good then commit
.then(trx.commit)
// if bad then rollback
.catch(trx.rollback);
})
.catch(() => 'error');
};
I call the removeFriend function this way removeFriend(clientId, friendId)
Then when i want to get a list of all friends with their names from the database i use this function:
const getUserFriends = async (clientId) => {
// get friends
return db('friends')
.where({ user_id: clientId })
.join('users', 'users.id', 'friends.friend_id')
.select('friends.friend_id', 'users.name')
.then((friends) => friends)
.catch(() => 'error');
};
I call the getUserFriends function this way await getUserFriends(clientId)
The problem is that when I use removeFriend function and then directly use the getUserFriends function i get a list where the users are still friends. However, If i look in the database the rows have been deleted so naturaly i should get a list where the users are not friends. Do I use the await wrong or something?
I'm building an API with Restify which is based on Express. I'm using Typeorm and I'm wondering what is the best way to update different properties which came from user input.
Essentially I have a route like this:
server.put('/users/:id', errorHandler(update));
which fire this method:
const update = async (req: Request, res: Response) => {
const user = { ...req.body, id: req.params.id } as User;
res.send(await userService.update(user));
}
as you can see I used the spread operator to create an User entity. Then, inside userService.update I have the following:
export const update = async (user: User): Promise<User> => {
const repository = getRepository(User);
const entity = await repository.findOne({ id: user.id });
if (!entity) throw new errors.ResourceNotFoundError(`There is no user with id of ${user.id}`);
Object.assign(entity, user, { id: entity.id, chat_id: entity.chat_id, project_id: entity.project_id, deleted: false });
return await repository.save(entity);
}
as you can see, I want prevent that the data provided by the API consumer will replace some important properties like: id, chat_id, project_id, deleted, so I used the method Object.assign to achieve this.
Is this a good way? What do you suggest for improve this?
You can use update method of typeorm like this, it will partially update the values that you give as a second argument.
// this will find a user with id ${user.id} and will only
// change the fields that is specified in the user object
await repository.update(user.id, user);
// check if updated for debugging
const updatedUser = await repository.findOne(user.id);
console.log(updatedUser, null, 2)
If you want to create a new record of the existing user in db, then you only need to change it's id. To do that
Deep clone the object so there will be another user object with new reference
Remove id field from the deep cloned object and use insert afterwards
// Deep clone user object
const clonedUser = JSON.parse(JSON.stringify(user))
// Delete id field from the deep clone
delete clonedUser.id;
// create a new user with different id
await repository.insert(clonedUser);
You can filter your important properties.
And pass the user id to the update method of your userService.
const { id, chat_id, project_id, deleted, ...user } = req.body;
const { id } = req.params;
res.send(await userService.update(id, user));
This will make sure user object don't have the properties(that is important).
And you can change your update method like below:
export const update = (userId: string, user: User): Promise<User> => {
return getRepository(User).update(userId, user);
}
Due to a management decision, we are using userId for the users collection, postId for the posts collection, and topicId for the topics collection, instead of '_id' for each collection as the unique identifier.
This causes a few problems getting started - one of the problems I have encountered is with upserts -
Using Mongoose, we have a schema that restricts userId to be a unique value - but when doing an update on a user model, with upsert set to true, MongoDB appears to only look at the ObjectIds of a collection to see if the same one exists - it doesn't check to see if a model already exists with the same userId - therefore Mongo does an insert instead of an update.
let me illustrate this with some data:
let's say the user's collection has one document:
{
_id:'561b0fad638e99481ab6d84a'
userId:3,
name:'foo'
}
we then run:
User.update({userId:3},{"$set":{name:'bar'},{upsert:true},function(err,resp){
if(err){
// "errMessage": "insertDocument :: caused by :: 11000 E11000 duplicate key error index: app42153482.users.$userId_1 dup key: { : 3 }",
}
});
one would think that MongoDB would find the existing document with userId:3 and udpate it, so there must be something I am doing wrong since it's giving me the duplicate key error?
Typically the default value ObjectId is more ideal for the _id. Here, in this situation you can either override the default _id or you can have your own field for id(like userId in your case).
Use a separate counters collection to track the last number sequence used. The _id field contains the sequence name and the seq field contains the last value of the sequence.
Insert into the counters collection, the initial value for the userid:
db.counters.insert( {
_id: "userid",
seq: 0 } )
Create a getNextSequence function that accepts a name of the sequence. The function uses the findAndModify() method to atomically increment the seq value and return this new value:
function getNextSequence(name) {
var ret = db.counters.findAndModify(
{
query: { _id: name },
update: { $inc: { seq: 1 } },
new: true
}
);
return ret.seq;
}
Use this getNextSequence() function during insert().
db.users.insert(
{
_id: getNextSequence("userid"),
name: "Sarah C."
}
)
db.users.insert(
{
_id: getNextSequence("userid"),
name: "Bob D."
}
)
This way you can maintain as many sequences as you want in the same counter collection. For the upsert issue, check out the Optimistic Loop block in this link Create an auto-increment sequence field.
The second approach is to use a mongoose middleware like mongodb-autoincrement.
Hope it helps.
I don't know which versions of MongoDB and Mongoose you are using, but I couldn't reproduce your problem with MongoDB 3.0 and Mongoose 4.1.10.
I made a sample for you which will create and save a new user, update (using upsert) it, and create another one through an upsert. Try running this code:
"use strict";
var mongoose=require("mongoose");
var Schema = require('mongoose').Schema;
var ObjectId = mongoose.Schema.Types.ObjectId;
// Connect to test
mongoose.connect("mongodb://localhost:27017/test");
// Lets create your schema
var userSchema = new Schema({
_id: ObjectId,
userId: {type: Number, unique: true },
name: String
});
var User = mongoose.model("User", userSchema, "Users");
User.remove() // Let's prune our collection to start clean
.then( function() {
// Create our sample record
var myUser = new User({
_id:'561b0fad638e99481ab6d84a',
userId:3,
name:'foo'
});
return myUser.save();
})
.then( function() {
// Now its time to update (upsert userId 3)
return User.update({userId:3},{"$set":{name:'bar'}},{upsert:true});
})
.then( function() {
// Now its time to insert (upsert userId 4)
return User.update({userId:4},{"$set":{name:'bee'}},{upsert:true});
})
.then( function() {
// Lets show what we have inserted
return User.find().then(function(data) {console.log(data)});
})
.catch( function(err) {
// Show errors if anything goes wrong
console.error("ERROR", err);
})
.then( function() {
mongoose.disconnect();
});
Following the documentation (of MongoDB 3.0) upsert:true will only not insert a non-existing document if your query conditions match on the _id field.
See: https://docs.mongodb.org/manual/reference/method/db.collection.update/#mongodb30-upsert-id
Why are you not using the user_name for a user as unique id?
Because auto-incrementing fields as ids are a bad practice to use in a mongodb environment, especially if you want to use sharding
=> all your inserts will occur on the latest shard
=> the mongodb cluster will have to rebalance often / redistribute the data around.
(Currently this will not occur on your system as you still use the generated _id field)
You can off course also create a unique index on the user_id field:
https://docs.mongodb.org/manual/core/index-unique/#index-type-unique
I have the following code:
#!/usr/bin/env node
'use strict';
var Sequelize = require('sequelize');
var sequelize = new Sequelize('sqlite:file.sqlite');
var User = sequelize.define('User', { email: Sequelize.STRING});
var Thing = sequelize.define('Thing', { name: Sequelize.STRING});
Thing.belongsTo(User);
sequelize.sync({force: true}).then(function () {
return User.create({email: 'asdf#example.org'});
}).then(function (user) {
return Thing.create({
name: 'A thing',
User: user
}, {
include: [User]
});
}).then(function (thing) {
return Thing.findOne({where: {id: thing.id}, include: [User]});
}).then(function (thing) {
console.log(JSON.stringify(thing));
});
I get the following output:
ohnobinki#gibby ~/public_html/turbocase1 $ ./sqltest.js
Executing (default): INSERT INTO `Users` (`id`,`email`,`updatedAt`,`createdAt`) VALUES (NULL,'asdf#example.org','2015-12-03 06:11:36.904 +00:00','2015-12-03 06:11:36.904 +00:00');
Executing (default): INSERT INTO `Users` (`id`,`email`,`createdAt`,`updatedAt`) VALUES (1,'asdf#example.org','2015-12-03 06:11:36.904 +00:00','2015-12-03 06:11:37.022 +00:00');
Unhandled rejection SequelizeUniqueConstraintError: Validation error
at Query.formatError (/home/ohnobinki/public_html/turbocase1/node_modules/sequelize/lib/dialects/sqlite/query.js:231:14)
at Statement.<anonymous> (/home/ohnobinki/public_html/turbocase1/node_modules/sequelize/lib/dialects/sqlite/query.js:47:29)
at Statement.replacement (/home/ohnobinki/public_html/turbocase1/node_modules/sqlite3/lib/trace.js:20:31)
It seems that specifying {include: [User]} instructs Sequelize to create a new User instance matching the contents of user. That is not my goal. In fact, I find it hard to believe that such behaviour would ever be useful—I at least have no use for it. I want to be able to have a long-living User record in the database and at arbitrary times create new Things which refer to the User. In my shown example, I wait for the User to be created, but in actual code it would likely have been freshly loaded through User.findOne().
I have seen other questions and answers say that I have to explicitly specify the implicitly-created UserId column in my Thing.create() call. When Sequelize provides an API like Thing.belongsTo(User), I shouldn’t have to be aware of the fact that a Thing.UserId field is created. So what is the clean API-respecting way of creating a new Thing which refers to a particular User without having to guess the name of the UserId field? When I load a Thing and specify {include: [User]}, I access the loaded user through the thing.User property. I don’t think I’m supposed to know about or try to access a thing.UserId field. In my Thing.belongsTo(User) call, I never specify UserId, I just treat that like an implementation detail I shouldn’t care about. How can I continue to avoid caring about that implementation detail when creating a Thing?
The Thing.create() call that works but looks wrong to me:
Thing.create({
name: 'A thing',
UserId: user.id
});
Option 1 - risks DB inconsistency
Sequelize dynamically generates methods for setting associations on instances, e.g. thing.setUser(user);. In your use case:
sequelize.sync({force: true})
.then(function () {
return Promise.all([
User.create({email: 'asdf#example.org'}),
Thing.create({name: 'A thing'})
]);
})
.spread(function(user, thing) {
return thing.setUser(user);
})
.then(function(thing) {
console.log(JSON.stringify(thing));
});
Option 2 - does not work/buggy
It isn't documented, but from a code dive I think the following should work. It doesn't but that seems to be because of a couple of bugs:
// ...
.then(function () {
return models.User.create({email: 'asdf#example.org'});
})
.then(function(user) {
// Fails with SequelizeUniqueConstraintError - the User instance inherits isNewRecord from the Thing instance, but it has already been saved
return models.Thing.create({
name: 'thingthing',
User: user
}, {
include: [{
model: models.User
}],
fields: ['name'] // seems nec to specify all non-included fields because of line 277 in instance.js - another bug?
});
})
Replacing models.User.create with models.User.build doesn't work because the built but not saved instance's primary key is null. Instance#_setInclude ignores the instance if its primary key is null.
Option 3
Wrapping the Thing's create in a transaction prevents an inconsistent state.
sq.sync({ force: true })
.then(models.User.create.bind(models.User, { email: 'asdf#example.org' }))
.then(function(user) {
return sq.transaction(function(tr) {
return models.Thing.create({name: 'A thing'})
.then(function(thing) { return thing.setUser(user); });
});
})
.then(print_result.bind(null, 'Thing with User...'))
.catch(swallow_rejected_promise.bind(null, 'main promise chain'))
.finally(function() {
return sq.close();
});
I have uploaded a script demo'ing option 2 and option 3 here
Tested on sequelize#6.5.1 sqlite3#5.0.2 I can use User.associations.Comments.foreignKey as in:
const Comment = sequelize.define('Comment', {
body: { type: DataTypes.STRING },
});
const User = sequelize.define('User', {
name: { type: DataTypes.STRING },
});
User.hasMany(Comment)
Comment.belongsTo(User)
console.dir(User);
await sequelize.sync({force: true});
const u0 = await User.create({name: 'u0'})
const u1 = await User.create({name: 'u1'})
await Comment.create({body: 'u0c0', [User.associations.Comments.foreignKey]: u0.id});
The association is also returned during creation, so you could also:
const Comments = User.hasMany(Comment)
await Comment.create({body: 'u0c0', [Comments.foreignKey]: u0.id});
and on many-to-many through tables you get foreignKey and otherKey for the second foreign key.
User.associations.Comments.foreignKey contains the foreignKey UserId.
Or analogously with aliases:
User.hasMany(Post, {as: 'authoredPosts', foreignKey: 'authorId'});
Post.belongsTo(User, {as: 'author', foreignKey: 'authorId'});
User.hasMany(Post, {as: 'reviewedPosts', foreignKey: 'reviewerId'});
Post.belongsTo(User, {as: 'reviewer', foreignKey: 'reviewerId'});
await sequelize.sync({force: true});
// Create data.
const users = await User.bulkCreate([
{name: 'user0'},
{name: 'user1'},
])
const posts = await Post.bulkCreate([
{body: 'body00', authorId: users[0].id, reviewerId: users[0].id},
{body: 'body01', [User.associations.authoredPosts.foreignKey]: users[0].id,
[User.associations.reviewedPosts.foreignKey]: users[1].id},
])
But that syntax is so long that I'm tempted to just hardcode the keys everywhere.