How to delete property from object received in sequelize? - node.js

I have a model which requires a date input in the where clause for a query.
const Model = sequelizeTwo.define('model', {
A: Sequelize.BIGINT,
B: Sequelize.DOUBLE,
C: Sequelize.DOUBLE,
D: Sequelize.DOUBLE,
E: Sequelize.DOUBLE,
F: Sequelize.DOUBLE,
DATE: Sequelize.DATEONLY,
newCol: Sequelize.VIRTUAL
},{
tableName: "Model",
timestamps: false,
freezeTableName: true
})
DATE here is used as a params for the query to display information on the client. But that is the end of it's use. I don't want to send DATE to the client back but I see no way to remove it.
If i remove it from the model, it gives a timezone error which is another problem as well.
app.get('/api/:date', (req, res) => {
var date = req.params.date
Model.findAll({
where: {
DATE: {
[Op.eq]: date
}
},
order: [
['A', 'ASC']
]
}).then(result => {
...
for (i; i < result.length; i++) {
...
delete result[i].DATE
console.log(result[i].DATE)
result[i]["newCol"] = values;
}
res.json(result);
})
})
I have tried using the delete operator inside and outside the loop, but it's of no use. It still retains the property and sends it back to the client
delete result[i].DATE //Inside loop
delete result.DATE //before loop
The values do they get updated when assignment is done but the property/key can't be modified.
result[i].DATE = null or undefined
What i want to achieve here is that I just want to send an object back which has all the properties in the sequelize model except DATE

I think you should use excludes feature for this case:
Model.findAll({
where: {
DATE: {
[Op.eq]: date
}
},
order: [
['A', 'ASC']
],
attributes: { exclude: ['DATE'] }
})
More information here: sequelize documentation

I think that the best practice is to create another model for the view as a data transfer object (DTO), and to map only the desired properties to it (without the Date).
Further read: DTO.
Pseudocode example:
Model.findAll({
//...
}).then(result => {
...
let resultDTO = mapToDTO(result);
res.json(resultDTO);
})

Related

Using updateOne method to update an object field inside array - throws error "Cannot create field 'url' in element"

I have MongoDB database (with Mongoose) containing a collection of Products (among others), which looks like this:
[
{
name: 'Product A',
url: 'product-a',
category: 'accesory',
price: 12,
shortDescription: ['example description'],
technicalSpecs: [{ speed: 10, weight: 20 }],
images: [],
reviews: [],
relatedProducts: [
{
url: 'product-b',
name: 'Product B',
// to be added in Update query
//id: id_of_related_product
}
]
} /* other Product objects */
]
As every MongoDB document is provided with _id property by default, but within the relatedProducts array i only have url and name properties, i want to add the id property (associated with corresponding Product) for each object in the relatedProducts array, so i will be able to conveniently query and process those related products.
I came up with an idea to query all Products to get only those, which have non-empty relatedProducts array. Then i loop them and i search for Product model, which has specific url and name properties - this let's me get it's true (added by MongoDB) _id. At the end i want to add this _id to matching object inside relatedProducts array.
My code:
async function assignIDsToRelatedProducts(/* Model constructor */ Product) {
const productsWithRelatedOnes = await Product.find(
{ relatedProducts: { $ne: [] }}, ['relatedProducts', 'name', 'url']
);
for (const productItem of productsWithRelatedOnes) {
for (const relatedProduct of productItem.relatedProducts) {
const product = await Product.findOne(
{ url: relatedProduct.url, name: relatedProduct.name },
'_id'
);
// throws error
await productItem.updateOne(
{ 'relatedProducts.url': relatedProduct.url },
{ $set: { 'relatedProducts.$.id': product._id } }
);
}
}
}
However it throws the following error:
MongoError: Cannot create field 'url' in element {relatedProducts: [ /* array's objects here */ ]}
I don't know why MongoDB tries to create field 'url', as i use it to project/query url field (not create it) in updateOne method. How to fix this?
And - as i am newbie to MongoDB - is there a simpler way of achieving my goal? I feel that those two nested for..of loops are unnecessary, or even preceding creation of productsWithRelatedOnes variable is.
Is it possible to do with Mongoose Virtuals? I have tried it, but i couldn't match virtual property within the same Product Model - attach it to each object in relatedProducts array - after calling .execPopulate i received either an empty array or undefined (i am aware i should post at-the-time code of using Virtual, but for now i switched to above solution).
Although i didn't find solution or even reason of my problem, i solved it with a slightly other approach:
async function assignIDsToRelatedProducts(Product) {
const productsHavingRelatedProducts = Product.find({ relatedProducts: { $ne: [] }});
for await (const withRelated of productsHavingRelatedProducts) {
for (const relatedProductToUpdate of withRelated.relatedProducts) {
const relatedProduct = await Product
.findOne(
{ url: relatedProductToUpdate.url, name: relatedProductToUpdate.name },
['url', '_id']
);
await Product.updateMany(
{ 'relatedProducts.url': relatedProduct.url },
{ $set: { 'relatedProducts.$.id': relatedProduct._id } }
);
}
}
const amountOfAllProducts = await Product.find({}).countDocuments();
const amountOfRelatedProductsWithID = await Product
.find({ 'relatedProducts.id': { $exists: true } }).countDocuments();
console.log('All done?', amountOfAllProducts === amountOfRelatedProductsWithID);
}
Yet, i still suppose it can be done more concisely, without the initial looping. Hopefully somebody will suggest better solution. :)

Mongoose: updateMany() is not working as expected

I'm using mongoose to handle my DB queries. I'm trying to update a set of records entirely using this method. Mode code looks like this:
// prepare database query
const filter = { type: 'company' };
const update = req.body.payload; // payload contains the array of objects (i.e. updated records)
const options = { new: true, runValidators: true }
// find and update the taxonomy record
await Taxonomy.updateMany(filter, update, options);
But whenever I run this query I'm getting following error in the console:
Error [MongooseError]: Invalid update pipeline operator: "_id"
I suppose there is something wrong in my update payload. The req.body.payload looks like this:
[
{
_id: '5ef3d08c745428001d92f896',
type: 'company',
name: 'Company Size',
__v: 0
},
{
_id: '5ef3cdc5745428001d92f893',
type: 'company',
name: 'Company Industry',
__v: 0
}
]
Can you please tell me what actually is wrong here?
This is not the right usage of updateMany() - it is aimed to update many documents with a single change.
To update many documents use bulkwrite() (docs) :
async function myUpdateMany(Model, objectsArray) {
try {
let ops = []
for (let obj of (objectsArray || [])) {
ops.push({
updateOne: {
filter: { platformId: obj.platformId },
update: obj,
upsert: false, // set "true" if you want to add a new document if it doesn't exist
}
})
}
Model.bulkWrite(ops, { ordered: false });
} catch (err) {
throw Error("myUpdateMany error: " + err)
}
}
Regarding runValidators, according to this, it seems to work by default.

AWS DynamoDB Transact Write using IN operator on Array

I am working on one query where I need to use TrasactWrite of dynamoDB and perform update query on one of my table.
Scenario is I am having array of unique IDs and I need to perform same update operation on each ID in array and change one flag in table without using Loop or map.
But I am having difficulty in finding example or docs related to how do I use IN operator on IDs which are primary keys in a Key parameter of transactWrite.
Here is my sample code:
let IDs = await find('table', { type: 'some_type' }, 'type-index}
let params = {
TransactItems: [{
Update: {
TableName: 'table',
Key: '#_id IN :IDs,
UpdateExpression: 'set #flag = :flag',
ExpressionAttributeNames: {
'#flag': 'flag',
'#_id': '_id'
},
ExpressionAttributeValues: {
':flag': false,
":IDs": IDs
}
}
}]
}
Already tried: Answer Its not same question as mine
I will answer if anyone has any question. Any help will be really helpful. Thank you
You can't use IN operator in this case - Update. With update operator you have to put key value to Update object. A key includes partitionKey and sortKey, I think in your case, you just set partitionKey.
To update more than one item by ID, you can put many Update object to TransactItems.
let IDs = await find('table', { type: 'some_type' }, 'type-index');
let params = {
TransactItems: [], // init empty array
}
IDs.forEach((ID) => {// loop though IDs array
params.TransactItems.push({
Update: {
TableName: 'table',
Key: {
'_id': ID, // ID value
},
UpdateExpression: 'set #flag = :flag',
ExpressionAttributeNames: {
'#flag': 'flag',
// '#_id': '_id' // remove this line
},
ExpressionAttributeValues: {
':flag': false,
// ":IDs": IDs // remove this line
}
}
})
});
// await client.transactWrite(params).promise()

MongoDB - find one and add a new property

Background: Im developing an app that shows analytics for inventory management.
It gets an office EXCEL file uploaded, and as the file uploads the app convert it to an array of JSONs. Then, it comapers each json object with the objects in the DB, change its quantity according to the XLS file, and add a timestamp to the stamps array which contain the changes in qunatity.
For example:
{"_id":"5c3f531baf4fe3182cf4f1f2",
"sku":123456,
"product_name":"Example",
"product_cost":10,
"product_price":60,
"product_quantity":100,
"Warehouse":4,
"stamps":[]
}
after the XLS upload, lets say we sold 10 units, it should look like that:
{"_id":"5c3f531baf4fe3182cf4f1f2",
"sku":123456,
"product_name":"Example",
"product_cost":10,
"product_price":60,
"product_quantity":90,
"Warehouse":4,
"stamps":[{"1548147562": -10}]
}
Right now i cant find the right commands for mongoDB to do it, Im developing in Node.js and Angular, Would love to read some ideas.
for (let i = 0; i < products.length; i++) {
ProductsDatabase.findOneAndUpdate(
{"_id": products[i]['id']},
//CHANGE QUANTITY AND ADD A STAMP
...
}
You would need two operations here. The first will be to get an array of documents from the db that match the ones in the JSON array. From the list you compare the 'product_quantity' keys and if there is a change, create a new array of objects with the product id and change in quantity.
The second operation will be an update which uses this new array with the change in quantity for each matching product.
Armed with this new array of updated product properties, it would be ideal to use a bulk update for this as looping through the list and sending
each update request to the server can be computationally costly.
Consider using the bulkWrite method which is on the model. This accepts an array of write operations and executes each of them of which a typical update operation
for your use case would have the following structure
{ updateOne :
{
"filter" : <document>,
"update" : <document>,
"upsert" : <boolean>,
"collation": <document>,
"arrayFilters": [ <filterdocument1>, ... ]
}
}
So your operations would follow this pattern:
(async () => {
let bulkOperations = []
const ids = products.map(({ id }) => id)
const matchedProducts = await ProductDatabase.find({
'_id': { '$in': ids }
}).lean().exec()
for(let product in products) {
const [matchedProduct, ...rest] = matchedProducts.filter(p => p._id === product.id)
const { _id, product_quantity } = matchedProduct
const changeInQuantity = product.product_quantity - product_quantity
if (changeInQuantity !== 0) {
const stamps = { [(new Date()).getTime()] : changeInQuantity }
bulkOperations.push({
'updateOne': {
'filter': { _id },
'update': {
'$inc': { 'product_quantity': changeInQuantity },
'$push': { stamps }
}
}
})
}
}
const bulkResult = await ProductDatabase.bulkWrite(bulkOperations)
console.log(bulkResult)
})()
You can use mongoose's findOneAndUpdate to update the existing value of a document.
"use strict";
const ids = products.map(x => x._id);
let operations = products.map(xlProductData => {
return ProductsDatabase.find({
_id: {
$in: ids
}
}).then(products => {
return products.map(productData => {
return ProductsDatabase.findOneAndUpdate({
_id: xlProductData.id // or product._id
}, {
sku: xlProductData.sku,
product_name: xlProductData.product_name,
product_cost: xlProductData.product_cost,
product_price: xlProductData.product_price,
Warehouse: xlProductData.Warehouse,
product_quantity: productData.product_quantity - xlProductData.product_quantity,
$push: {
stamps: {
[new Date().getTime()]: -1 * xlProductData.product_quantity
}
},
updated_at: new Date()
}, {
upsert: false,
returnNewDocument: true
});
});
});
});
Promise.all(operations).then(() => {
console.log('All good');
}).catch(err => {
console.log('err ', err);
});

How to add dynamic additional attributes into a junction table with ManyToMany association in Sequelize

I created a many-to-many association by sequelize in my koa app. But I had no idea on how to create additional attributes in the junction table. Thanks.
I referred to the official doc of sequelize but didn't find a solution. In brief:
"an order can have many items"
"an item can exist in many orders"
Then I created OrderItems as junction table.
But I have trouble in inserting value into the junction table
// definitions
const Item = sequelize.define('item', itemSchema);
const Order = sequelize.define('order', orderSchema);
// junction table
const OrderItems = sequelize.define('order_item', {
item_quantity: { type: Sequelize.INTEGER } // number of a certain item in a certain order.
});
// association
Item.belongsToMany(Order, { through: OrderItems, foreignKey: 'item_id' });
Order.belongsToMany(Item, { through: OrderItems, foreignKey: 'order_id' });
// insert value
const itemVals = [{ name: 'foo', price: 6 }, { name: 'bar', price: 7 }];
const orderVals = [
{
date: '2019-01-06',
items: [{ name: 'foo', item_quantity: 12 }]
},
{
date: '2019-01-07',
items: [{ name: 'foo', item_quantity: 14 }]
}
]
items = Item.bulkCreate(itemVals)
orders = Order.bulkCreate(orderVals)
//Questions here: create entries in junction table
for (let order of orders) {
const itemsInOrder = Item.findAll({
where: {
name: {
[Op.in]: order.items.map(item => item.name)
}
}
})
order.addItems(itemsInOrder, {
through: {
item_quantity: 'How to solve here?'
}
})
}
// my current manual solution:
// need to configure column names in junction table manually.
// Just like what we do in native SQL.
const junctionValList =[]
for (let orderVal of orderVals) {
orderVal.id = (await Order.findOne(/* get order id */)).dataValues.id
for (let itemVal of orderVal.items) {
itemVal.id = (await Item.findOne(/* get item id similarly */)).dataValues.id
const entyInJunctionTable = {
item_id: itemVal.id,
order_id: orderVal.id,
item_quantity: itemVal.item_quantity
}
junctionValList.push(entyInJunctionTable)
}
}
OrderItems.bulkCreate(junctionValList).then(/* */).catch(/* */)
In case that this script it's for seeding purpose you can do something like this:
/*
Create an array in which all promises will be stored.
We use it like this because async/await are not allowed inside of 'for', 'map' etc.
*/
const promises = orderVals.map((orderVal) => {
// 1. Create the order
return Order.create({ date: orderVal.date, /* + other properties */ }).then((order) => {
// 2. For each item mentioned in 'orderVal.items'...
return orderVal.items.map((orderedItem) => {
// ...get the DB instance
return Item.findOne({ where: { name: orderedItem.name } }).then((item) => {
// 3. Associate it with current order
return order.addItem(item.id, { through: { item_quantity: orderedItem.item_quantity } });
});
});
});
});
await Promise.all(promises);
But it's not an efficient way to do it in general. First of all, there are a lot of nested functions. But the biggest problem is that you associate items with the orders, based on their name and it's possible that in the future you will have multiple items with the same name.
You should try to use an item id, this way you will be sure about the outcome and also the script it will be much shorter.

Resources