Joi Array Validation - node.js

Consider this code snippet. I want to query 'Markers' by 1 or many colors...
module.exports = [
{
method: 'GET',
path: '/search/markers',
options: {
auth,
description: 'Get Markers',
notes: 'Returns all marker info',
tags: ['api', 'markers'],
validate: {
query: {
color: Joi.array().items(Joi.string().valid('Red', 'Blue')).single().allow(''),
}
},
response: {
schema: MarkersValidation.getSearchMarkers
}
},
handler: SearchController.searchMarkers
}
];
exports.searchMarkers = async (request, h) => {
try {
const Markers = h.models().Markers;
const queryMarkers = Markers.query()
.select(h.knex().raw([
'markers.color'
]))
if (request.query.color) {
queryMarkers.andWhere(h.knex().raw(`color && '{${request.query.color}}'::text[]`));
}
}
}
catch (e) {
//error handler
}
}
However, I get the error below from Postman when I try to apply the query params of color with both Red, Blue. When I apply a single color param, for example: Red, it works fine.
Error
child "color" fails because [single value of "color" fails because ["color" must be one of [Red, Blue]]]
URL
{{url}}/search/markers?color=Red, Blue
Note
I tried removing .single(), but when I do, I get this error:
child "color" fails because ["color" must be an array]
Question
How do I resolve this?
Suppose I wanted a list of available colors to query by: 'Green', 'Purple', 'Yellow', 'Red', 'Blue'.
How would I add 1 or all of the options to the query?
Examples
{{url}}/search/markers?color=Red,
{{url}}/search/markers?color=Red, Blue, Yellow
This is my current code, but obviously doesn't work, any thoughts?
const myColors = ['Green', 'Purple', 'Yellow', 'Red', 'Blue'];
validate: {
query: {
color: Joi.array().items(Joi.string().valid(myColors)).single().allow(''),
}
}
UPDATED - SIMPLIFIED
In this image, I have an array with ".valid()" and the validation fails.
In this image, I have an array with no ".valid()" and the validation passes.
My Question
How can I add ".valid()" or something like it, to my Joi.array so that only the values I set are valid for this query?

You must pass the color query parameters as an array in your URL.
{{url}}/search/markers?color[]=Red&color[]=Blue
The joi schema to validate the color array:
const myColors = ['Green', 'Purple', 'Yellow', 'Red', 'Blue'];
joi.object().keys({
color: joi.array().items(myColors.map(col => col)),
});

const myColors = require('../data/colors');
const validateColors = (colors) => {
if (!colors) {
return true;
}
const colorsArray = colors.split(',');
const colorsSchema = Joi.array().items(Joi.string().valid(myColors)).allow('')
return Joi.validate(colorsArray, colorsSchema);
}
query: {colors: Joi.string().allow('')}

Related

Nodejs Mongoose how do I update an object in an array based the objects ID

Here is my an example of a document from my "Cart" schema:
{
_id: 1,
line_items: [
{ item_id: 101, color: 'white', quantity: 1 },
{ item_id: 101, color: 'green', quantity: 1 },
{ item_id: 102, color: 'white', quantity: 1 },
]
}
I want to increase the quantity of the item which is uniquely identified by item_id = 101 and color = 'white'
I want it to be increase by x amount where x can be any number. How do I do this?
(edit 1)
Here is what I have tried
await Cart.findById(1).then(doc => {
const i = doc.line_items.findIndex(elm => {
return (elm.item == item && elm.color == color) // (edit 2) item is a variable = 101 and color is a variable = 'white'
});
doc.line_items[i].quantity += quantity;
doc.save();
}).catch(err => {
throw err;
});
However, this isnt working because the changes in quantity aren't being saved to the database for some reason even though it is being updated when I console.log() it inside of this code.
I also beleive it should be possible to solve my problem with a single findByIdAndUpdate function though, but I have no idea how to do that
I think you have a problem getting the index. I tried your code and it worked fine.
await Cart.findById
.findById(1)
.then((doc) => {
const i = doc.line_items.findIndex((elm) => {
//here is elm.item_id
return elm.item_id == item && elm.color == color;
});
doc.line_items[i].quantity += 10;
doc.save();
})
.catch((err) => {
throw err;
});
So apparently the reason my attempt did not work was because doc.save() was not being run because mongoose did not recognize any changes to the document. Why this is the case, I don't know, but by adding the line doc.markModified('line_items'); before saving, it is now working.
await Cart.findById(1).then(doc => {
const i = doc.line_items.findIndex(elm => {
return (elm.item == 101 && elm.color == 'white')
});
doc.line_items[i].quantity += 10;
doc.markModified('line_items'); // <-- solution
doc.save();
}).catch(err => {
throw err;
});

Filter and show only SVG images in ApostropheCMS

I created a new filter named svg to show only svg images or non-svg images.
But I don't understand where can I set the filter value to true or false?
The code is shown below.
Widget index file:
module.exports = {
label: 'Section SVG Images',
addFields: [
{
name: 'svg-images',
label: 'SVG Images',
type: 'singleton',
widgetType: 'apostrophe-images',
filters: {
svg: true
},
required: true
},
]
};
Custom cursor filter:
module.exports = {
construct: function(self, options) {
self.addFilter('svg', {
finalize: function() {
var svg = self.get('svg'); // <--- HARE svg is olways 'undefined'
if (typeof svg == 'undefined') {
return;
}
if (svg) {
var criteria = {
'attachment.extension': 'svg'
};
} else {
var criteria = {
'attachment.extension': { $ne: 'svg' }
};
}
self.and(criteria);
},
safeFor: 'public',
launder: function(a) {
return self.apos.launder.boolean(a);
}
});
}
};
filter is not a top level option for singleton fields in Apostrophe. That is unique to join-type fields. You could have this widget extend apostrophe-pieces-widgets, then you could probably set that same filters object directly on the new pieces widget. That should register the cursor filter, at least.
It's worth noting that there is currently a PR in progress to add filtering by both file type and orientation to Apostrophe core!

How to identify one polygon from a multipolygon in Mapbox?

I am plotting a multipolygon in mapbox, that means even though the polygons are visibly separate but they are linked to the same source. If I add a click handler to this, and then click on any one of the polygons from the multipolygon, it effects all other polygons in the multipolygon equally.
Now my problem is, that I want to identify which specific polygon was clicked from the multipolygons.
Suppose I call a function when someone clicks on the source/layer, I want to send a unique identifier which denotes which polygon was clicked from among the multipolygons.
How can I achieve this?
Below is snippet of code that might be relevant:
for (let k = 0; k < sectionResult.data.response.length; k++) {
let features = sectionResult.data.response[k].coordinates.map((item) => {
return {
type: 'Feature',
geometry: {
type: 'Polygon',
coordinates: item
}
};
});
map.addSource(sectionResult.data.response[k].name, {
type: 'geojson',
data: {
type: 'FeatureCollection',
features: features
}
});
map.addLayer({
id: sectionResult.data.response[k].name,
type: 'fill',
source: sectionResult.data.response[k].name,
paint: {
'fill-color': '#00e',
'fill-opacity': 0.3
}
});
map.addLayer({
id: `${sectionResult.data.response[k].name}-labels`,
type: 'symbol',
source: sectionResult.data.response[k].name,
layout: {
'text-field': sectionResult.data.response[k].name,
'text-size': 20
},
paint: {
'text-color': '#fff308'
}
});
map.addLayer({
id: `${sectionResult.data.response[k].name}-borders`,
type: 'line',
source: sectionResult.data.response[k].name,
layout: {},
paint: {
'line-color': '#fff308',
'line-width': 3
}
});
map.on('mousemove', sectionResult.data.response[k].name, function (e) {
map.getCanvas().style.cursor = 'pointer';
let mapLayer = map.getLayer(
`${sectionResult.data.response[k].name}-borders-onHover`
);
if (typeof mapLayer === 'undefined') {
map.addLayer({
id: `${sectionResult.data.response[k].name}-borders-onHover`,
type: 'line',
source: sectionResult.data.response[k].name,
layout: {},
paint: {
'line-color': '#fff308',
'line-width': 3
}
});
}
});
map.on('mouseleave', sectionResult.data.response[k].name, function (e) {
map.getCanvas().style.cursor = '';
let mapLayer = map.getLayer(
`${sectionResult.data.response[k].name}-borders-onHover`
);
if (typeof mapLayer !== 'undefined') {
map.removeLayer(`${sectionResult.data.response[k].name}-borders-onHover`);
}
});
map.on('click', sectionResult.data.response[k].name, function (e) {
functionCall(); //I want to pass the unique identifier of the polygon that was clicked.
});
}
I think what you are looking for is queryRenderedFeatures this will return the features in a concrete filter, including the mouse position.
map.on('mousemove', function (e) {
var features = map.queryRenderedFeatures(e.point);
// Limit the number of properties we're displaying for
// legibility and performance
var displayProperties = [
'type',
'properties',
'id',
'layer',
'source',
'sourceLayer',
'state'
];
var displayFeatures = features.map(function (feat) {
var displayFeat = {};
displayProperties.forEach(function (prop) {
displayFeat[prop] = feat[prop];
});
return displayFeat;
});
document.getElementById('features').innerHTML = JSON.stringify(
displayFeatures,
null,
2
);
});
You have a full example here

How to get raw sql with binds in sequelize?

I'm trying to find a way to get raw SQL with binds.
The following works with Sequelize model:
let conditions = {
color: { [Sequelize.Op.in]: ['blue', 'yellow'] },
type: 'active'
};
let options = { limit: 10 };
let sql = await sequelize.dialect.QueryGenerator.selectQuery('widgets',
{
where: conditions,
...options
});
the above outputs
SELECT *
FROM "widgets"
WHERE "widgets"."color" IN ('blue', 'yellow') AND "widgets"."type" = 'active'
LIMIT 10;
How can we get sql with binding for raw SQL query?
sequelize.query(`
SELECT *
FROM widgets
WHERE color IN (:colors)
`,
{
raw: true,
type: QueryTypes.SELECT,
replacements: {
colors: ['blue', 'yellow']
}
});
We can generate where clause by using sequelize getWhereConditions method
let sql = await sequelize.dialect.QueryGenerator.getWhereConditions(
conditions,
'widgets',
);

NodeJS MongoDB Mongoose export nested subdocuments and arrays to XLSX columns

I have query results from MongoDB as an array of documents with nested subdocuments and arrays of subdocuments.
[
{
RecordID: 9000,
RecordType: 'Item',
Location: {
_id: 5d0699326e310a6fde926a08,
LocationName: 'Example Location A'
}
Items: [
{
Title: 'Example Title A',
Format: {
_id: 5d0699326e310a6fde926a01,
FormatName: 'Example Format A'
}
},
{
Title: 'Example Title B',
Format: {
_id: 5d0699326e310a6fde926a01,
FormatName: 'Example Format B'
}
}
],
},
{
RecordID: 9001,
RecordType: 'Item',
Location: {
_id: 5d0699326e310a6fde926a08,
LocationName: 'Example Location C'
},
Items: [
{
Title: 'Example Title C',
Format: {
_id: 5d0699326e310a6fde926a01,
FormatName: 'Example Format C'
}
}
],
}
]
Problem
I need to export the results to XLSX in column order. The XLSX library is working to export the top-level properties (such as RecordID and RecordType) only. I also need to export the nested objects and arrays of objects. Given a list of property names e.g. RecordID, RecordType, Location.LocationName, Items.Title, Items.Format.FormatName the properties must be exported to XLSX columns in the specified order.
Desired result
Here is the desired 'flattened' structure (or something similar) that
I think should be able to convert to XLSX columns.
[
{
'RecordID': 9000,
'RecordType': 'Item',
'Location.LocationName': 'Example Location A',
'Items.Title': 'Example Title A, Example Title B',
'Items.Format.FormatName': 'Example Format A, Example Format B',
},
{
'RecordID': 9001,
'RecordType': 'Item',
'Location.LocationName': 'Example Location C',
'Items.Title': 'Example Title C',
'Items.Format.FormatName': 'Example Format C',
}
]
I am using the XLSX library to convert the query results to XLSX which works for top-level properties only.
const worksheet: XLSX.WorkSheet = XLSX.utils.json_to_sheet(results.data);
const workbook: XLSX.WorkBook = { Sheets: { 'data': worksheet }, SheetNames: ['data'] };
const excelBuffer: any = XLSX.write(workbook, { bookType: 'xlsx', type: 'array' });
const data: Blob = new Blob([excelBuffer], { type: EXCEL_TYPE });
FileSaver.saveAs(data, new Date().getTime());
POSSIBLE OPTIONS
I am guessing I need to 'flatten' the structure either using aggregation in the query or by performing post-processing when the query is returned.
Option 1: Build the logic in the MongoDB query to flatten the results.
$replaceRoot might work since it is able to "promote an existing embedded document to the top level". Although I am not sure if this will solve the problem exactly, I do not want to modify the documents in place, I just need to flatten the results for exporting.
Here is the MongoDB query I am using to produce the results:
records.find({ '$and': [ { RecordID: { '$gt': 9000 } } ]},
{ skip: 0, limit: 10, projection: { RecordID: 1, RecordType: 1, 'Items.Title': 1, 'Items.Location': 1 }});
Option 2: Iterate and flatten the results on the Node server
This is likely not the most performant option, but might be the easiest if I can't find a way to do so within the MongoDB query.
UPDATE:
I may be able to use MongoDB aggregate $project to 'flatten' the results. For example, this aggregate query effectively 'flattens' the results by 'renaming' the properties. I just need to figure out how to implement the query conditions within the aggregate operation.
db.records.aggregate({
$project: {
RecordID: 1,
RecordType: 1,
Title: '$Items.Title',
Format: '$Items.Format'
}
})
UPDATE 2:
I have abandoned the $project solution because I would need to change the entire API to support aggregation. Also, I would need to find a solution for populate because aggregate does not support it, rather, it uses $lookup which is possible but time consuming because I would need to write the queries dynamically. I am going back to look into how to flatten the object by creating a function to iterate the array of objects recursively.
Below is a solution for transforming the Mongo data on the server via a function flattenObject which recursively flattens nested objects and returns a 'dot-type' key for nested paths.
Note that the snippet below contains a function that renders and editable table to preview, however, the important part you want (download the file), should be triggered when you run the snippet and click the 'Download' button.
const flattenObject = (obj, prefix = '') =>
Object.keys(obj).reduce((acc, k) => {
const pre = prefix.length ? prefix + '.' : '';
if (typeof obj[k] === 'object') Object.assign(acc, flattenObject(obj[k], pre + k));
else acc[pre + k] = obj[k];
return acc;
}, {});
var data = [{
RecordID: 9000,
RecordType: "Item",
Location: {
_id: "5d0699326e310a6fde926a08",
LocationName: "Example Location A"
},
Items: [{
Title: "Example Title A",
Format: {
_id: "5d0699326e310a6fde926a01",
FormatName: "Example Format A"
}
},
{
Title: "Example Title B",
Format: {
_id: "5d0699326e310a6fde926a01",
FormatName: "Example Format B"
}
}
]
},
{
RecordID: 9001,
RecordType: "Item",
Location: {
_id: "5d0699326e310a6fde926a08",
LocationName: "Example Location C"
},
Items: [{
Title: "Example Title C",
Format: {
_id: "5d0699326e310a6fde926a01",
FormatName: "Example Format C"
}
}]
}
];
const EXCEL_MIME_TYPE = `application/vnd.ms-excel`;
const flattened = data.map(e => flattenObject(e));
const ws_default_header = XLSX.utils.json_to_sheet(flattened);
const ws_custom_header = XLSX.utils.json_to_sheet(flattened, {
header: ['Items.Title', 'RecordID', 'RecordType', 'Location.LocationName', 'Items.Format.FormatName']
});
const def_workbook = XLSX.WorkBook = {
Sheets: {
'data': ws_default_header
},
SheetNames: ['data']
}
const custom_workbook = XLSX.WorkBook = {
Sheets: {
'data': ws_custom_header
},
SheetNames: ['data']
}
const def_excelBuffer = XLSX.write(def_workbook, {
bookType: 'xlsx',
type: 'array'
});
const custom_excelBuffer = XLSX.write(custom_workbook, {
bookType: 'xlsx',
type: 'array'
});
const def_blob = new Blob([def_excelBuffer], {
type: EXCEL_MIME_TYPE
});
const custom_blob = new Blob([custom_excelBuffer], {
type: EXCEL_MIME_TYPE
});
const def_button = document.getElementById('dl-def')
/* trigger browser to download file */
def_button.onclick = e => {
e.preventDefault()
saveAs(def_blob, `${new Date().getTime()}.xlsx`);
}
const custom_button = document.getElementById('dl-cus')
/* trigger browser to download file */
custom_button.onclick = e => {
e.preventDefault()
saveAs(custom_blob, `${new Date().getTime()}.xlsx`);
}
/*
render editable table to preview (for SO convenience)
*/
const html_string_default = XLSX.utils.sheet_to_html(ws_default_header, {
id: "data-table",
editable: true
});
const html_string_custom = XLSX.utils.sheet_to_html(ws_custom_header, {
id: "data-table",
editable: true
});
document.getElementById("container").innerHTML = html_string_default;
document.getElementById("container-2").innerHTML = html_string_custom;
<script src="https://cdnjs.cloudflare.com/ajax/libs/xlsx/0.14.3/xlsx.full.min.js"></script>
<head>
<title>Excel file generation from JSON</title>
<meta charset="utf-8" />
<style>
.xport,
.btn {
display: inline;
text-align: center;
}
a {
text-decoration: none
}
#data-table,
#data-table th,
#data-table td {
border: 1px solid black
}
</style>
</head>
<script>
function render(type, fn, dl) {
var elt = document.getElementById('data-table');
var wb = XLSX.utils.table_to_book(elt, {
sheet: "Sheet JS"
});
return dl ?
XLSX.write(wb, {
bookType: type,
bookSST: true,
type: 'array'
}) :
XLSX.writeFile(wb, fn || ('SheetJSTableExport.' + (type || 'xlsx')));
}
</script>
<div>Default Header</div>
<div id="container"></div>
<br/>
<div>Custom Header</div>
<div id="container-2"></div>
<br/>
<table id="xport"></table>
<button type="button" id="dl-def">Download Default Header Config</button>
<button type="button" id="dl-cus">Download Custom Header Config</button>
<script src="https://cdnjs.cloudflare.com/ajax/libs/FileSaver.js/1.3.8/FileSaver.min.js"></script>
I wrote a function to iterate all object in the results array and create new flattened objects recursively. The flattenObject function shown here is similar to the previous answer and I took additional inspiration from this related answer.
The '_id' properties are specifically excluded from being added to the flattened object, since ObjectIds are still being returned as bson types even though I have the lean() option set.
I still need to figure out how to sort the objects such that they are in the order given e.g. RecordID, RecordType, Items.Title. I believe that might be easiest to achieve by creating a separate function to iterate the flattened results, although not necessarily the most performant. Let me know if anyone has any suggestions on how to achieve the object sorting by a given order or has any improvements to the solution.
const apiCtrl = {};
/**
* Async array iterator
*/
apiCtrl.asyncForEach = async (array, callback) => {
for (let index = 0; index < array.length; index++) {
await callback(array[index], index, array)
}
}
// Check if a value is an object
const isObject = (val) => {
return typeof val == 'object' && val instanceof Object && !(val instanceof Array);
}
// Check if a value is a date object
const isDateObject = (val) => {
return Object.prototype.toString.call(val) === '[object Date]';
}
/**
* Iterate object properties recursively and flatten all values to top level properties
* #param {object} obj Object to flatten
* #param {string} prefix A string to hold the property name
* #param {string} res A temp object to store the current iteration
* Return a new object with all properties on the top level only
*
*/
const flattenObject = (obj, prefix = '', res = {}) =>
Object.entries(obj).reduce((acc, [key, val]) => {
const k = `${prefix}${key}`
// Skip _ids since they are returned as bson values
if (k.indexOf('_id') === -1) {
// Check if value is an object
if (isObject(val) && !isDateObject(val)) {
flattenObject(val, `${k}.`, acc)
// Check if value is an array
} else if (Array.isArray(val)) {
// Iterate each array value and call function recursively
val.map(element => {
flattenObject(element, `${k}.`, acc);
});
// If value is not an object or an array
} else if (val !== null & val !== 'undefined') {
// Check if property has a value already
if (res[k]) {
// Check for duplicate values
if (typeof res[k] === 'string' && res[k].indexOf(val) === -1) {
// Append value with a separator character at the beginning
res[k] += '; ' + val;
}
} else {
// Set value
res[k] = val;
}
}
}
return acc;
}, res);
/**
* Convert DB query results to an array of flattened objects
* Required to build a format that is exportable to csv, xlsx, etc.
* #param {array} results Results of DB query
* Return a new array of objects with all properties on the top level only
*/
apiCtrl.buildExportColumns = async (results) => {
const data = results.data;
let exportColumns = [];
if (data && data.length > 0) {
try {
// Iterate all records in results data array
await apiCtrl.asyncForEach(data, async (record) => {
// Convert the multi-level object to a flattened object
const flattenedObject = flattenObject(record);
// Push flattened object to array
exportColumns.push(flattenedObject);
});
} catch (e) {
console.error(e);
}
}
return exportColumns;
}

Resources