Related
I have created add product API like this. This is working fine. I'm posting successfully varient data by sharing product id as a foreign key, but I'm confused about how can I update product data. Can I update data by using this code?
try {
const { name, description, photo, tag, productId ,lableType,reccomendedProduct, varientDetails, isTex,GSTrate,GSTtyp, HSNcode, categoryId, subCategoryId, videoUpload,} = req.body;
const data= db.product.findOne({ where: { id: productId },
include: [{ model: db.tagModel, attributes: ["id","name","productId"]}, { model: db.reccomendProduct, attributes: ["id", "productName","productId"]},
{ model: db.varientModel, attributes: ["id", "sort","sku","productId","waightunitno","unit","mrp","discount","price","stock","minstock","outofstock"]}]
}).then(product => {
if (product) {
db.product.update({
categoryId: categoryId ? categoryId : product.categoryId,
subCategoryId: subCategoryId ? subCategoryId : product.subCategoryId,
name:name,
description:description,
lableType:lableType,
isTex:isTex,
// photo:req.file ? req.file.location:'',
photo:photo,
GSTrate:GSTrate,
GSTtyp:GSTtyp,
HSNcode:HSNcode,
categoryId:categoryId,
subCategoryId:subCategoryId,
videoUpload:videoUpload }, { where: { id: product.id }
})
}
if(varientDetails ) {
db.varientModel.findAll ({ where: { productId:productId }})
.then(varient => {
console.log(varient+data)
for (let i=0; i < varientDetails.length; i++) {
db.varientModel.update({
productId:productId,
sort:varientDetails[i].sort,
sku: varientDetails[i].sku,
waightunitno:varientDetails[i].waightunitno,
unit:varientDetails[i].unit,
mrp:varientDetails[i].mrp,
discount: varientDetails[i].discount,
price: varientDetails[i].price,
stock: varientDetails[i].stack,
minstock: varientDetails[i].minstock,
outofstock: varientDetails[i].outofstock
}, { where: { productId:productId[i] }
})
}
})
}
Yes, there are ways to do it.
I don't find them as expressive and as clear as multiple one.
1. Creating Query on own
You can create function like this
function updateUsers(updateFirstValue, updateSecondValue, productIds) {
let query = "";
for (let index = 0; index < productIds.length; index++) {
query += `update tableName set firstUpdateValue="${updateFirstValue[index]}",secondUpdateValue="${updateSecondValue[index]}" where productId="${productIds[index]}";`;
}
return query;
}
//This is vulnerable to SQL Injection so change according to your needs
//It's just idea of doing it
let firstUpdates = [800, 900, 10];
let secondUpdates = [1.23, 2.23, 8.97];
let productIds = [1, 9, 3];
let generatedQuery = updateUsers(firstUpdates, secondUpdates, productIds);
console.log(generatedQuery);
// to run this with sequelize we can execute plain query with this
//sequelize.query(generatedQuery);
2. Using bulkCreate and updateOnDuplicate
let updatingValue = [
{productId:1, sort:100,sku:800},
{productId:2, sort:800,sku:8.27},
{productId:3, sort:400,sku:7.77}
];
model.bulkCreate(updatingValue,{
fields:["productid","sort","sku"],
updateOnDuplicate: ["sort","sku"]
}
// these are the column name that gets updated if primaryKey(productId) gets matched you have to update these accordingly
)
It had problem before but is updated now this PR
Other methods but quite complicated are also here.
I am using Node library for integration with BigQuery. Wanted to know how I can store Geography datatype values in Table using Table.insert method. I tried few thing like below but gives error
const options = {
datasetId: 'mydataset',
tableId: 'mytable',
rows: [
{
rec1:"raj",
rec2:"geography::Point('-6', '6', 4326)"
}
] }
const table = await bigqueryClient
.dataset(options.datasetId)
.table(options.tableId)
table.insert(options.rows, (err, resp) => {
if (err) {
console.log('*****err', err)
} else {
console.log('*****resp', resp)
} })
Or only way to achieve this is using query method on BigQuery client with query in below format ?
insert `bigqueryproject1-279307.mydataset.mytable` (rec1,rec2) values ('raj', ST_GEOGPOINT(-6, 6) )
Try to create table with schema first. Something like this
const schema = [
{
name: 'rec1',
type: 'STRING',
},
{
name: 'rec2',
type: 'GEOGRAPHY',
}
];
const options = {
schema: schema,
};
const [table] = await bigquery
.dataset('mydataset')
.createTable('mytable', options);
Then try to insert data with GEOG data types. I recommend to use geom data in format of geojson while wkt format is sometimes not imported correctly (from my experience)
{
"type": "Point",
"coordinates": [
-6,
6
]
}
I have query results from MongoDB as an array of documents with nested subdocuments and arrays of subdocuments.
[
{
RecordID: 9000,
RecordType: 'Item',
Location: {
_id: 5d0699326e310a6fde926a08,
LocationName: 'Example Location A'
}
Items: [
{
Title: 'Example Title A',
Format: {
_id: 5d0699326e310a6fde926a01,
FormatName: 'Example Format A'
}
},
{
Title: 'Example Title B',
Format: {
_id: 5d0699326e310a6fde926a01,
FormatName: 'Example Format B'
}
}
],
},
{
RecordID: 9001,
RecordType: 'Item',
Location: {
_id: 5d0699326e310a6fde926a08,
LocationName: 'Example Location C'
},
Items: [
{
Title: 'Example Title C',
Format: {
_id: 5d0699326e310a6fde926a01,
FormatName: 'Example Format C'
}
}
],
}
]
Problem
I need to export the results to XLSX in column order. The XLSX library is working to export the top-level properties (such as RecordID and RecordType) only. I also need to export the nested objects and arrays of objects. Given a list of property names e.g. RecordID, RecordType, Location.LocationName, Items.Title, Items.Format.FormatName the properties must be exported to XLSX columns in the specified order.
Desired result
Here is the desired 'flattened' structure (or something similar) that
I think should be able to convert to XLSX columns.
[
{
'RecordID': 9000,
'RecordType': 'Item',
'Location.LocationName': 'Example Location A',
'Items.Title': 'Example Title A, Example Title B',
'Items.Format.FormatName': 'Example Format A, Example Format B',
},
{
'RecordID': 9001,
'RecordType': 'Item',
'Location.LocationName': 'Example Location C',
'Items.Title': 'Example Title C',
'Items.Format.FormatName': 'Example Format C',
}
]
I am using the XLSX library to convert the query results to XLSX which works for top-level properties only.
const worksheet: XLSX.WorkSheet = XLSX.utils.json_to_sheet(results.data);
const workbook: XLSX.WorkBook = { Sheets: { 'data': worksheet }, SheetNames: ['data'] };
const excelBuffer: any = XLSX.write(workbook, { bookType: 'xlsx', type: 'array' });
const data: Blob = new Blob([excelBuffer], { type: EXCEL_TYPE });
FileSaver.saveAs(data, new Date().getTime());
POSSIBLE OPTIONS
I am guessing I need to 'flatten' the structure either using aggregation in the query or by performing post-processing when the query is returned.
Option 1: Build the logic in the MongoDB query to flatten the results.
$replaceRoot might work since it is able to "promote an existing embedded document to the top level". Although I am not sure if this will solve the problem exactly, I do not want to modify the documents in place, I just need to flatten the results for exporting.
Here is the MongoDB query I am using to produce the results:
records.find({ '$and': [ { RecordID: { '$gt': 9000 } } ]},
{ skip: 0, limit: 10, projection: { RecordID: 1, RecordType: 1, 'Items.Title': 1, 'Items.Location': 1 }});
Option 2: Iterate and flatten the results on the Node server
This is likely not the most performant option, but might be the easiest if I can't find a way to do so within the MongoDB query.
UPDATE:
I may be able to use MongoDB aggregate $project to 'flatten' the results. For example, this aggregate query effectively 'flattens' the results by 'renaming' the properties. I just need to figure out how to implement the query conditions within the aggregate operation.
db.records.aggregate({
$project: {
RecordID: 1,
RecordType: 1,
Title: '$Items.Title',
Format: '$Items.Format'
}
})
UPDATE 2:
I have abandoned the $project solution because I would need to change the entire API to support aggregation. Also, I would need to find a solution for populate because aggregate does not support it, rather, it uses $lookup which is possible but time consuming because I would need to write the queries dynamically. I am going back to look into how to flatten the object by creating a function to iterate the array of objects recursively.
Below is a solution for transforming the Mongo data on the server via a function flattenObject which recursively flattens nested objects and returns a 'dot-type' key for nested paths.
Note that the snippet below contains a function that renders and editable table to preview, however, the important part you want (download the file), should be triggered when you run the snippet and click the 'Download' button.
const flattenObject = (obj, prefix = '') =>
Object.keys(obj).reduce((acc, k) => {
const pre = prefix.length ? prefix + '.' : '';
if (typeof obj[k] === 'object') Object.assign(acc, flattenObject(obj[k], pre + k));
else acc[pre + k] = obj[k];
return acc;
}, {});
var data = [{
RecordID: 9000,
RecordType: "Item",
Location: {
_id: "5d0699326e310a6fde926a08",
LocationName: "Example Location A"
},
Items: [{
Title: "Example Title A",
Format: {
_id: "5d0699326e310a6fde926a01",
FormatName: "Example Format A"
}
},
{
Title: "Example Title B",
Format: {
_id: "5d0699326e310a6fde926a01",
FormatName: "Example Format B"
}
}
]
},
{
RecordID: 9001,
RecordType: "Item",
Location: {
_id: "5d0699326e310a6fde926a08",
LocationName: "Example Location C"
},
Items: [{
Title: "Example Title C",
Format: {
_id: "5d0699326e310a6fde926a01",
FormatName: "Example Format C"
}
}]
}
];
const EXCEL_MIME_TYPE = `application/vnd.ms-excel`;
const flattened = data.map(e => flattenObject(e));
const ws_default_header = XLSX.utils.json_to_sheet(flattened);
const ws_custom_header = XLSX.utils.json_to_sheet(flattened, {
header: ['Items.Title', 'RecordID', 'RecordType', 'Location.LocationName', 'Items.Format.FormatName']
});
const def_workbook = XLSX.WorkBook = {
Sheets: {
'data': ws_default_header
},
SheetNames: ['data']
}
const custom_workbook = XLSX.WorkBook = {
Sheets: {
'data': ws_custom_header
},
SheetNames: ['data']
}
const def_excelBuffer = XLSX.write(def_workbook, {
bookType: 'xlsx',
type: 'array'
});
const custom_excelBuffer = XLSX.write(custom_workbook, {
bookType: 'xlsx',
type: 'array'
});
const def_blob = new Blob([def_excelBuffer], {
type: EXCEL_MIME_TYPE
});
const custom_blob = new Blob([custom_excelBuffer], {
type: EXCEL_MIME_TYPE
});
const def_button = document.getElementById('dl-def')
/* trigger browser to download file */
def_button.onclick = e => {
e.preventDefault()
saveAs(def_blob, `${new Date().getTime()}.xlsx`);
}
const custom_button = document.getElementById('dl-cus')
/* trigger browser to download file */
custom_button.onclick = e => {
e.preventDefault()
saveAs(custom_blob, `${new Date().getTime()}.xlsx`);
}
/*
render editable table to preview (for SO convenience)
*/
const html_string_default = XLSX.utils.sheet_to_html(ws_default_header, {
id: "data-table",
editable: true
});
const html_string_custom = XLSX.utils.sheet_to_html(ws_custom_header, {
id: "data-table",
editable: true
});
document.getElementById("container").innerHTML = html_string_default;
document.getElementById("container-2").innerHTML = html_string_custom;
<script src="https://cdnjs.cloudflare.com/ajax/libs/xlsx/0.14.3/xlsx.full.min.js"></script>
<head>
<title>Excel file generation from JSON</title>
<meta charset="utf-8" />
<style>
.xport,
.btn {
display: inline;
text-align: center;
}
a {
text-decoration: none
}
#data-table,
#data-table th,
#data-table td {
border: 1px solid black
}
</style>
</head>
<script>
function render(type, fn, dl) {
var elt = document.getElementById('data-table');
var wb = XLSX.utils.table_to_book(elt, {
sheet: "Sheet JS"
});
return dl ?
XLSX.write(wb, {
bookType: type,
bookSST: true,
type: 'array'
}) :
XLSX.writeFile(wb, fn || ('SheetJSTableExport.' + (type || 'xlsx')));
}
</script>
<div>Default Header</div>
<div id="container"></div>
<br/>
<div>Custom Header</div>
<div id="container-2"></div>
<br/>
<table id="xport"></table>
<button type="button" id="dl-def">Download Default Header Config</button>
<button type="button" id="dl-cus">Download Custom Header Config</button>
<script src="https://cdnjs.cloudflare.com/ajax/libs/FileSaver.js/1.3.8/FileSaver.min.js"></script>
I wrote a function to iterate all object in the results array and create new flattened objects recursively. The flattenObject function shown here is similar to the previous answer and I took additional inspiration from this related answer.
The '_id' properties are specifically excluded from being added to the flattened object, since ObjectIds are still being returned as bson types even though I have the lean() option set.
I still need to figure out how to sort the objects such that they are in the order given e.g. RecordID, RecordType, Items.Title. I believe that might be easiest to achieve by creating a separate function to iterate the flattened results, although not necessarily the most performant. Let me know if anyone has any suggestions on how to achieve the object sorting by a given order or has any improvements to the solution.
const apiCtrl = {};
/**
* Async array iterator
*/
apiCtrl.asyncForEach = async (array, callback) => {
for (let index = 0; index < array.length; index++) {
await callback(array[index], index, array)
}
}
// Check if a value is an object
const isObject = (val) => {
return typeof val == 'object' && val instanceof Object && !(val instanceof Array);
}
// Check if a value is a date object
const isDateObject = (val) => {
return Object.prototype.toString.call(val) === '[object Date]';
}
/**
* Iterate object properties recursively and flatten all values to top level properties
* #param {object} obj Object to flatten
* #param {string} prefix A string to hold the property name
* #param {string} res A temp object to store the current iteration
* Return a new object with all properties on the top level only
*
*/
const flattenObject = (obj, prefix = '', res = {}) =>
Object.entries(obj).reduce((acc, [key, val]) => {
const k = `${prefix}${key}`
// Skip _ids since they are returned as bson values
if (k.indexOf('_id') === -1) {
// Check if value is an object
if (isObject(val) && !isDateObject(val)) {
flattenObject(val, `${k}.`, acc)
// Check if value is an array
} else if (Array.isArray(val)) {
// Iterate each array value and call function recursively
val.map(element => {
flattenObject(element, `${k}.`, acc);
});
// If value is not an object or an array
} else if (val !== null & val !== 'undefined') {
// Check if property has a value already
if (res[k]) {
// Check for duplicate values
if (typeof res[k] === 'string' && res[k].indexOf(val) === -1) {
// Append value with a separator character at the beginning
res[k] += '; ' + val;
}
} else {
// Set value
res[k] = val;
}
}
}
return acc;
}, res);
/**
* Convert DB query results to an array of flattened objects
* Required to build a format that is exportable to csv, xlsx, etc.
* #param {array} results Results of DB query
* Return a new array of objects with all properties on the top level only
*/
apiCtrl.buildExportColumns = async (results) => {
const data = results.data;
let exportColumns = [];
if (data && data.length > 0) {
try {
// Iterate all records in results data array
await apiCtrl.asyncForEach(data, async (record) => {
// Convert the multi-level object to a flattened object
const flattenedObject = flattenObject(record);
// Push flattened object to array
exportColumns.push(flattenedObject);
});
} catch (e) {
console.error(e);
}
}
return exportColumns;
}
I am dealing with a query with a criteria object that is being passed as the first argument to this query:
module.exports = (criteria, sortProperty, offset = 0, limit = 20) => {
// write a query that will follow sort, offset, limit options only
// do not worry about criteria yet
console.log(criteria);
const query = Artist.find({ age: { $gte: 19, $lte: 44 } })
.sort({ [sortProperty]: 1 })
.skip(offset)
.limit(limit);
return Promise.all([query, Artist.count]).then(results => {
return {
all: results[0],
count: results[1],
offset: offset,
limit: limit
};
});
};
By default, the criteria object has a single name property that is an empty string.
The age property points to an object that has both min and max values assigned to it. I also have a yearsActive property inside of the criteria object and that also has a min and max value.
So three different properties: age, name and yearsActive.
This has been an extremely challenging one for me and if you look above that's as far as I got.
When my criteria property is console logged it only has a name { name: "" }. It has no yearsActive or age by default when it first starts. So that is where the point of the sliders come in. When I start moving these sliders around on the frontend, then it gets the age and yearsActive appended to the criteria object.
So I need to figure out how to update the query to consider for example the different ages and I have been considering using an if conditional inside a helper function.
Regarding to the comment that I left you.
You have three states at least one when you retrieve the data to the UI. In this case, I would recommend you use aggregation in order to retrieve the data as a model as your business.
For example, the problem as you have is that sometimes you don't know about the max or min value for age or yearsActive, but also you should have an identifier that could be an ObjectId which will be used to update the model identified by that property.
Artist.aggregate([
{
$match: { age: { $gte: 19, $lte: 44 } }
},
{
$sort: { yourProperty: 1 }
},
{
$skip: 10
},
{
$limit: 10
},
{
$project: {
// You set your properties to retrieve with the 1 as flag
propertieX: 1,
"another.property": 1,
"age.max": {
$cond: {
if: { $eq: [ "", "$age.max" ] },
then: 0, // Or the value that you want to set it
else: "$age.max"
}
}
}
}]);
The other state is when you do the query according to the parameters that you're submitting from the form.
If you assurance to retrieve a model with the logic as you want. For example you should return this model in every request using $project and applying the default values when doesn't exist the manipulation in the front-end side as in the searching should be easy to manage.
{
ObjectId: YOUR_OBJECT_ID,
age: {
min: YOUR_MIN_VALUE,
max: YOUR_MAX_VALUE
},
yearsActive: {
min: YOUR_MIN_VALUE,
max: YOUR_MAX_VALUE
}
}
Finally, when you would send the data to save it you should sent the entire model that you returned but the must important thing is identify only that element by the ObjectId to do the update.
NOTE: This is an approach that I will do according with the information that I understand from your question, If I'm bad with me interpretation let me know, and if you want to share more information or open a repository to understand in code, should more easy to me understand the problem.
So what I decided to do since the code would look messy to throw all inside the Artist.find({}) was to create a separate helper function:
const buildQuery = (criteria) => {
console.log(criteria);
};
This helper function is being called with the criteria object and I have to form up the object in such a way that it will represent the query the way in which I want to search the Artist collection.
What made this difficult to wrap my head around was the not very well formed object for searching over a collection with its random properties such as age which has a min and a max which Mongo does not know how to deal with by default. MongoDB does not know what min and max mean exactly.
So inside the helper function I made a separate object to return from this function thats going to represent the actual query that I want to send off to Mongo.
const buildQuery = (criteria) => {
console.log(criteria);
const query = {};
};
I am not modifying the object in anyway, I am just reading some of the desired search results or what the user wants to see from this UI object and so I made this object called query and I added the idea of age.
const buildQuery = (criteria) => {
console.log(criteria);
const query = {};
query.age = {};
};
I decided to do an if conditional inside of the helper function for the specific age range that I want to find.
const buildQuery = (criteria) => {
console.log(criteria);
const query = {};
if (criteria.age) {
query.age = {};
}
};
So this is where the Mongo query operators come into play. The two operators I want to be concerned with is the greater than or equal to ($gte) and the less than or equal to ($lte) operators.
This is how I actually implemented in practice:
const buildQuery = (criteria) => {
console.log(criteria);
const query = {};
if (criteria.age) {
query.age = {
$gte: criteria.age.min,
$lte: criteria.age.max
};
}
};
The query object here will eventually be returned from the buildQuery function:
const buildQuery = (criteria) => {
console.log(criteria);
const query = {};
if (criteria.age) {
query.age = {
$gte: criteria.age.min,
$lte: criteria.age.max
};
}
return query;
};
That query object will be passed off to the find operation:
module.exports = (criteria, sortProperty, offset = 0, limit = 20) => {
// write a query that will follow sort, offset, limit options only
// do not worry about criteria yet
const query = Artist.find(buildQuery(criteria))
.sort({ [sortProperty]: 1 })
.skip(offset)
.limit(limit);
return Promise.all([query, Artist.count]).then(results => {
return {
all: results[0],
count: results[1],
offset: offset,
limit: limit
};
});
};
const buildQuery = (criteria) => {
console.log(criteria);
const query = {};
if (criteria.age) {
query.age = {
$gte: criteria.age.min,
$lte: criteria.age.max
};
}
return query;
};
So what I am doing here is to get the equivalent of Artist.find({ age: { $gte: minAge, $lte: maxAge }).
So for yearsActive I decided to implement something that is nearly identical:
const buildQuery = criteria => {
console.log(criteria);
const query = {};
if (criteria.age) {
query.age = {
$gte: criteria.age.min,
$lte: criteria.age.max
};
}
if (criteria.yearsActive) {
}
return query;
};
So if the user changes the slider, I am going to expect my criteria object to have a yearsActive property defined on it like so:
const buildQuery = criteria => {
console.log(criteria);
const query = {};
if (criteria.age) {
query.age = {
$gte: criteria.age.min,
$lte: criteria.age.max
};
}
if (criteria.yearsActive) {
query.yearsActive = {
$gte: criteria.yearsActive.min,
$lte: criteria.yearsActive.max
}
}
return query;
};
What I'm trying to do is to get rows in between two dates by using Sequelize ORM in Node.js. I'm using PostgreSQL. The problem is that the request that I'm making is interpreted incorrectly by Sequelize.
Here is the code that I'm using to make request
const dbresp = await Table.findAll({
attributes: [...],
where: {
...
createdAt: {
$between: [new Date(Date(startDate)), new Date(Date(endDate))],
// same effect
// $lte: new Date(startDate),
// $gte: new Date(endDate),
},
},
logging: console.log,
raw: true,
order: [['createdAt', 'ASC']],
// limit: count,
});
By logging raw SQL request it is obvious that request is incorrect
SELECT ...
FROM "table" AS "table"
WHERE "table"."createdAt" = '2019-02-05 21:00:00.000 +00:00'
"table"."createdAt" = '2019-02-05 21:00:00.000 +00:00'
ORDER BY "table"."createdAt" ASC;
What is a proper way to make such a request? Should I use a raw query?
I've googled this issue but no StackOverflow nither GitHub did help.
Ok, IDK what causes this issue but I fixed it by using Op object of Sequelize like this.
const Op = require('./models').Sequelize.Op;
const dbresp = await Table.findAll({
attributes: [...],
where: {
...
createdAt: {
[Op.between]: [startDate, endDate],
},
},
logging: console.log,
raw: true,
order: [['createdAt', 'ASC']],
// limit: count,
});
Seems like $between operator does not work
Try using the code shared below:
[Op.between]: [date1, date2]
Or the raw query:
SELECT ...
FROM table t
WHERE t.createdAt BETWEEN '2019-02-05' AND '2019-02-07'
ORDER BY t.createdAt ASC;
This should work.
Sequelize allows you to create aliases.
https://sequelize.org/v5/manual/querying.html
const Sequelize = require('sequelize');
const op = Sequelize.Op;
const operatorsAliases = {
$between: op.between, //create an alias for Op.between
}
const connection = new Sequelize(db, user, pass, { operatorsAliases })
const dbresp = await Table.findAll({
attributes: [...],
where: {
...
createdAt: {
$between: [new Date(Date(startDate)), new Date(Date(endDate))],
// same effect
// $lte: new Date(startDate),
// $gte: new Date(endDate),
},
},
logging: console.log,
raw: true,
order: [['createdAt', 'ASC']],
// limit: count,
});