I want to store sensor data in MongoDB, say at a frequency of 1hz. To do this efficiently I want to do it the way this article recommends, namely storing multiple data points in the same document, using the number of seconds elapsed as object properties.
This is how I want a typical document to look:
const record = {
startingHour: 'Sun Sep 11 2016 03:00:00', // Date object
values: {
0: { // minute
0: { lat: 52.0001, lon: 13.0001 },
// ... all other seconds
59: { lat: 52.9999, lon: 13.9999 }
},
// ... all other minutes
59: {
0: { lat: 53.0001, lon: 14.0001 },
// ...
59: { lat: 53.9999, lon: 14.9999 }
}
}
}
How could I specify in Mongoose that I want each document t represent an hour with 60 properties, each key being a number from 1 to 59 - one for each minute - and each of those properties to have 60 properties in turn, and those records to represent the actual location values?
So to save a new record I would do record.values.59.59 = { lat: 53.9999, lon: 14.9999 }
How could I represent this kind of schema in Mongoose?
I managed to do it by creating functions to generate the objects I want. I then use those generated objects as schemas, embed one within the other and put those in the final schema:
/**
* Functions to generate objects for Minutes and Seconds
* Each object containing 60 keys, going from 0 to 59
*/
function second() {
let obj = {};
for (var i = 0; i < 60; i++) {
obj[i] = Location;
}
return obj;
}
function minute() {
let obj = {};
for (var i = 0; i < 60; i++) {
obj[i] = Second;
}
return obj;
}
const Location = new Schema({
latitude: { type: Number, required: true },
longitude: { type: Number, required: true }
});
// Instantiates Second and Minute objects as schemas
const Second = new Schema(second());
const Minute = new Schema(minute());
// Actual schema
const RecordSchema = new Schema({
startingHour: {
type: Date,
required: true
},
values: {
type: Minute
}
});
This allows me to save objects like these:
{
startingHour: new Date(),
values: {
0: {
0: {
location: { latitude: 0.03, longitude: 51.7 }
},
1: {
location: { latitude: 0.03, longitude: 51.7 }
}
// ...
},
// ...
59: {
// ...
59: {
location: { latitude: 0.03, longitude: 51.7 }
}
}
}
}
Related
Let say that I have these two nested objects:
const sourceKeys = {
school: {
name: "Elisa Lemonnier",
students: 250
},
house: {
room : 2,
kids: true,
assets: "elevator",
}
}
const targetKeys = {
school: {
name: "Lucie Faure",
students: 150,
address: "123 Main St.",
phone: "555-555-5555"
},
house: {
room : 4,
kids: false,
assets: "Garden",
shop: "Auchan"
}
}
And I want the targetKeys keep ONLY the keys that are in sourceKeys. So I will get that :
const targetKeysMatchingSourceKeys = {
school: {
name: "Lucie Faure",
students: 150,
},
house: {
room : 4,
kids: false,
assets: "Garden",
}
}
I don't know how to proceed given that is a nested object. So, I will appreciate any help.
thanks you
I have find the solution, here is
const filteredJSON = Object.assign({}, TargetJsonToObject)
// Recursive function to filter the properties of the object
function filterObject(SourceJsonToObject, filteredObj) {
for (const key of Object.keys(filteredObj)) {
// If the key is not present in the source JSON, delete it from filtered JSON
if (!SourceJsonToObject.hasOwnProperty(key)) {
delete filteredObj[key]
} else if (typeof filteredObj[key] === "object") {
// If the key is present in the source JSON and the value is an object, recursively call the function on the nested object
filterObject(SourceJsonToObject[key], filteredObj[key])
}
}
}
filterObject(SourceJsonToObject, TargetJsonToObject)
I have a model of the following schema:
const mongoose = require('mongoose');
const livenessLogSchema = new mongoose.Schema({
probability: {
type: Number,
required: false,
},
timestamp: {
type: Date,
required: true,
}
}, {
timestamps: true
});
const LivenessLog = mongoose.model('LivenessLog', livenessLogSchema);
module.exports = LivenessLog;
I want to split all documents to 2 groups: those whose probability value is less than 0.001 and those whose value is greater than 0.001. Also, in each group, I want to count for each probabilty value - how many documents has the same value.
So basically if I had the following probabilities data: [0.00001, 0.000003, 0.000025, 0.000003, 0.9, 0.6, 0.6], I'd like to get as a result: { less: { 0.00001: 1, 0.000003: 2, 0.000025:1 }, greater: { 0.9: 1, 0.6: 2 }.
This is my current aggregate method:
const livenessProbilitiesData = await LivenessLog.aggregate([
{
$match: {
timestamp: {
$gte: moment(new Date(startDate)).tz('Asia/Jerusalem').startOf('day').toDate(),
$lte: moment(new Date(endDate)).tz('Asia/Jerusalem').endOf('day').toDate(),
}
}
},
{
$group: {
}
}
]);
Note that I use undeclared variables startDate, endDate. These are input I get to filter out unrelevant documents (by timestamp).
Context:
I am trying to upsert in bulk an array of data, with an additional computed field: 'status'.
Status should be either :
- 'New' for newly inserted docs;
- 'Removed' for docs present in DB, but inexistent in incoming dataset;
- a percentage explaining the evolution for the field price, comparing the value in DB to the one in incoming dataset.
Implementations:
data.model.ts
import { Document, model, Model, models, Schema } from 'mongoose';
import { IPertinentData } from './site.model';
const dataSchema: Schema = new Schema({
sourceId: { type: String, required: true },
name: { type: String, required: true },
price: { type: Number, required: true },
reference: { type: String, required: true },
lastModified: { type: Date, required: true },
status: { type: Schema.Types.Mixed, required: true }
});
export interface IData extends IPertinentData, Document {}
export const Data: Model<IData> = models.Data || model<IData>('Data', dataSchema);
data.service.ts
import { Data, IPertinentData } from '../models';
export class DataService {
static async test() {
// await Data.deleteMany({});
const data = [
{
sourceId: 'Y',
reference: `y0`,
name: 'y0',
price: 30
},
{
sourceId: 'Y',
reference: 'y1',
name: 'y1',
price: 30
}
];
return Data.bulkWrite(
data.map(function(d) {
let status = '';
// #ts-ignore
console.log('price', this);
// #ts-ignore
if (!this.price) status = 'New';
// #ts-ignore
else if (this.price !== d.price) {
// #ts-ignore
status = (d.price - this.price) / this.price;
}
return {
updateOne: {
filter: { sourceId: d.sourceId, reference: d.reference },
update: {
$set: {
// Set percentage value when current price is greater/lower than new price
// Set status to nothing when new and current prices match
status,
name: d.name,
price: d.price
},
$currentDate: {
lastModified: true
}
},
upsert: true
}
};
}
)
);
}
}
... then in my backend controller, i just call it with some route :
try {
const results = await DataService.test();
return new HttpResponseOK(results);
} catch (error) {
return new HttpResponseInternalServerError(error);
}
Problem:
I've tried lot of implementation syntaxes, but all failed either because of type casting, and unsupported syntax like the $ symbol, and restrictions due to the aggregation...
I feel like the above solution might be closest to a working scenario but i'm missing a way to grab the value of the price field BEFORE the actual computation of status and the replacement with updated value.
Here the value of this is undefined while it is supposed to point to current document.
Questions:
Am i using correct Mongoose way for a bulk update ?
if yes, how to get the field value ?
Environment:
NodeJS 13.x
Mongoose 5.8.1
MongoDB 4.2.1
EUREKA !
Finally found a working syntax, pfeeeew...
...
return Data.bulkWrite(
data.map(d => ({
updateOne: {
filter: { sourceId: d.sourceId, reference: d.reference },
update: [
{
$set: {
lastModified: Date.now(),
name: d.name,
status: {
$switch: {
branches: [
// Set status to 'New' for newly inserted docs
{
case: { $eq: [{ $type: '$price' }, 'missing'] },
then: 'New'
},
// Set percentage value when current price is greater/lower than new price
{
case: { $ne: ['$price', d.price] },
then: {
$divide: [{ $subtract: [d.price, '$price'] }, '$price']
}
}
],
// Set status to nothing when new and current prices match
default: ''
}
}
}
},
{
$set: { price: d.price }
}
],
upsert: true
}
}))
);
...
Explanations:
Several problems were blocking me :
the '$field_value_to_check' instead of this.field with undefined 'this' ...
the syntax with $ symbol seems to work only within an aggregation update, using update: [] even if there is only one single $set inside ...
the first condition used for the inserted docs in the upsert process needs to check for the existence of the field price. Only the syntax with BSON $type worked...
Hope it helps other devs in same scenario.
I am making a barcode scanner for my school project but i am stuck. I dont know how to scan through this object. I have this object with objects inside, and I need to scan through each object inside storage variable to check its barcode.
var storage = {
bolts: {
barcode: 57263144,
price: 0.5,
name: 'Plain Brackets',
stock: 25,
},
brackets: {
barcode: 13245627,
price: 0.2,
name: '100mm Bolts',
stock: 2,
},
}
I have a variable called barcode, and I need to test this variable if its the same like one of these. I tried using
for (var key in storage){
if (storage[key].barcode === barcode){
}
}
I would like the most simple way to do that.
Use Object.keys:
Object.keys(obj).forEach(function(key) {
console.log(key, obj[key]);
});
Below is the example:
var storage = {
"bolts": {
barcode: 57263144,
price: 0.5,
name: 'Plain Brackets',
stock: 25,
},
"brackets": {
barcode: 13245627,
price: 0.2,
name: '100mm Bolts',
stock: 2,
}
}
var barcode = 57263144;
Object.keys(storage).forEach(function(key) {
if(storage[key].barcode === barcode) { console.log("do something")}
});
A Fiddle:
https://jsfiddle.net/spechackers/34bhthza/
Use the recursive function to verify if exist more nodes in the objects, example:
const complexObj = {
name: "nobody",
address: { number: 22, moreNumbers: [1,2,3,4,5] },
colors: ["green", "red"],
numbersAgain: { first: 1, second: 4 }
};
function scanObj(obj){
for (let i in obj) {
/*
*Do some verificatio, example:
*I'd like to verify all numbers and if the numbers is greater than 3:
*/
if(typeof obj[i] == "number" && obj[i] > 3){ console.log(obj[i]); }
if (typeof obj[i] === "object") {
scanObj(obj[i])
}
}
}
//call the method
scanObj(complexObj);
Output: 22 4 5 4
I'm using Express 4 and Mongoose for my REST API. So I have multiple documents of the type "shop". Each shop holds (besides other information) an array called "inventory" that holds again multiple items. Each item itself has properties like name and price.
Now I would like to have an API call where I can get all the shops but only with their "cheapest" product item in the json response. But I'm totally stuck in creating this query that returns all my shops but instead of including all items of the inventoryjust includes the inventory item with the lowest price as the only item in the inventory array.
I found some hints on how to exclude fields using something like the following query but there the whole array will be excluded:
Shop.find({}, {inventory: 0},function(err, shops) {
if (err) {
res.send(err);
} else {
res.json(shops);
}
});
Update 1: My Schemas
// Shop
var ShopSchema = new Schema({
name: { type: String, required: true},
address: {
street: String,
zipCode: Number,
city: String
},
inventory: [InventoryItemSchema]
});
// InventoryItem
var InventoryItemSchema = new Schema({
name: { type: String, required: true},
currentPrice: {
amount: { type: Number, required: true },
added: Date
},
pastPrices: []
});
Update 2: This is what I came up
Shop.find(function(err, shops) {
if (err) {
res.send(err);
} else {
shops.forEach(function(shop) {
// Only keep the currently cheapest inventory Item in the array
var cheapestInventoryItem;
shop.inventory.reduce(function (previousItem, currentItem) {
if (currentItem.currentPrice.amount < previousItem.currentPrice.amount) {
cheapestInventoryItem = currentItem;
return currentItem;
} else {
cheapestInventoryItem = previousItem;
return previousItem;
}
});
shop.inventory = [cheapestInventoryItem];
});
res.json(shops);
}
});
Mongodb find method returns a document. So in your case it would be array of shops with their fold fields anyway.
You can filter items with js:
var cheapestItem;
var cheapestPrice = shop.inventory.items.reduce(function (lowest, item) {
if (item.price < lowest) {
cheapestItem = item;
return item.price;
}
}, Infinity);
Or you can normalize your schema and create collection Items:
[{itemId: 1, shopId: 1, price: 100}]
So the query is:
db.Item.group(
{
key: { shopId: 1, itemId: 1, price: 1 },
reduce: function ( lowest, res ) { if (result.price < lowest) return result.price },
initial: Infinity
})
So you must get shopId and itemId with lowest price