How can I merge two queries in mongodb? - node.js

To sort the documents inside Ads Collection I am using the below query which takes parameters from the URL and its working perfectly.
router.get("/", auth, async (req, res) => {
let query;
let queryStr = JSON.stringify(req.query);
queryStr = queryStr.replace(
/\b(gt|gte|lt|lte|in)\b/g,
(match) => `$${match}`
);
console.log(queryStr);
query = Ads.find(JSON.parse(queryStr));
const ads = await query;
res.status(200).json({ data: ads });
});
I am using the text operator in the Ads Collection for searching with the below route .
router.get("/find/:query", (req, res) => {
let query = req.params.query;
Ads.find(
{
$text: { $search: query },
},
function (err, result) {
if (err) throw err;
if (result) {
res.json(result);
} else {
res.send(
JSON.stringify({
error: "Error",
})
);
}
}
);
});
Both the routes are working perfectly but How can I merge the above two in one?
For e.g, I want to do a text search on the first route after getting a response, and similarly for the second route, after getting a response I want to apply the query parameters and get a response .
How can I merge the above two to get the desired output?

From what I can infer, are you looking for a pipeline where the parallel running of the two is possible:
Please have a look at MongoDB Aggregate which works as a pipeline. Here you can have two pipelines for the same input and have different output, or output of one can be transferred to next level to process.
Mongodb Aggregate - Facet Command Link
[1]:https://docs.mongodb.com/manual/reference/operator/aggregation/facet/#pipe._S_facet
Mongodb Aggregate links
[2]: https://docs.mongodb.com/manual/reference/operator/query/

Related

Why is my response empty from the second mongoose query? MERN-Stack

Goal: Get the students based on the given library name. The student model has the library name linked in the database.
What's happening:
1: Retrieving the name that is linked to the given ID (library ID aquired with useParams().id).
2: Looking for all students based on that found library name.
Result: Empty response. I feel like the problem has to do with the line library_name = library.name;. Simply that value hasn't set yet when the second query starts to execute? Because when I log the result right after this line, with res.send(library_name); the name is showing correctly.
app.get("/students/:id", (req, res) => {
const id = req.params.id;
let library_name = "";
LibraryModel.findById(id, (err, library) => {
library_name = library.name;
});
StudentModel.find({library: library_name}, (err, students) => {
if (err) {
res.send(err);
} else {
res.send(students);
}
});
});
You are right. library_name gehts only set in the callback function you passed, which happens after the StudentModel.find(...) gets called. Basically you are currently performing these 2 calls in parallel.
There are three ways to resolve this issue.
Moving second call to callback function
app.get("/students/:id", (req, res) => {
const id = req.params.id;
let library_name = "";
LibraryModel.findById(id, (err, library) => {
library_name = library.name;
StudentModel.find({library: library_name}, (err, students) => {
if (err) {
res.send(err);
} else {
res.send(students);
}
});
});
});
Use promises
To avoid what's known as "callback hell" can you also use promises instead of callback functions and await them:
app.get("/students/:id", async (req, res) => {
const id = req.params.id;
try {
const library = await LibraryModel.findById(id);
const students = await StudentModel.find({library: library.name});
res.send(students);
} catch (err) {
res.send(err);
}
});
Use a single aggregation pipeline
You can also merge these two separate databases queries into a single aggregation pipeline. You would need to first use $lookup and afterwards use $match to filter for the specific entries. Nevertheless there is additional information on the schemas needed in order to build this query.
Another Hint
I assume you are trying to create a RESTful API. You might want to review your path structure, since a RESTful approach would expect the ':id' to be the ID of a student, not the ID of a library.
It looks like GET '/libraries/:id/students' makes more sense in your case.

NodeJs SQLite query returns other values than direct SQL on database

I'm a bit puzzled by the situation I have now.
I've a simple SQL statement I execute from NodeJs on a SQLite database. The SQL statement returns values with a lot of decimals; although my data only contain two decimals.
When I run the exact same query in DB Browser for SQLite, I have a correct result.
My NodeJs code
app.get('/payerComparison/', (req, res) => {
// Returns labels and values within response
var response = {};
let db = new sqlite3.Database('./Spending.db', sqlite3.OPEN_READONLY, (err) => {
if(err){console.log(err.message); return}
});
response['labels'] = [];
response['data'] = [];
db.each("SELECT payer, sum(amount) AS sum FROM tickets GROUP BY payer", (err, row) => {
if(err){console.log(err.message); return}
response['labels'].push(row.payer);
response['data'].push(row.sum);
});
db.close((err) => {
if(err){console.log(err.message); return}
// Send data
console.log(response);
res.send(JSON.stringify(response));
});
})
What I have in the command line
{
labels: [ 'Aurélien', 'Commun', 'GFIS', 'Pauline' ],
data: [ 124128.26, 136426.43000000008, 5512.180000000001, 39666.93 ]
}
The result in DB Browser
I hope you can help me clarify this mystery!
Thank you
Round the values up to 2 decimals :).
SELECT payer, round(sum(amount),2) AS sum FROM tickets GROUP BY payer

get latest records from mongodb ,if those two records are same but the time is different

{"permit":"1905403","otp":"22","acct":"12556""timestamp":"06/12/2020 14:37:50"}
{"permit":"1905403","otp":"22","acct":"12556","timestamp":"06/12/2020 17:30:50"}
router.post('/getPermitNumberDetails', async (req, res) => {
try {
// console.log('req', req.body)
const permitNumber = req.body.permit
const permitNumberRelatedInfo = await dukePdf_Model.findOne({ "permit": permitNumber });
res.status(200).send({ permitNumberRelatedInfo: permitNumberRelatedInfo });
} catch (e) {
console.log(e);
res.status(400).send(e);
}});
In MongoDB collection, I have these two documents,the content is the same but the timestamp is different.i will be having a search bar in UI, when I type permit number as 1905403, there are two records for that particular permit. but I need to show the latest record i.e, with the latest timestamp.i have mentioned the router above..can anyone please help me

Convert Oracle query output to json (Oracle / NodeJS)

I'm doing an app with React / Express (NodeJS) / Oracle,
I have an Express route that gets datas from an Oracle table,
Here's a part of the code in the route :
let conn;
try {
conn = await oracledb.getConnection(config);
const result = await conn.execute("select JSON_OBJECT ('departement' VALUE departement, 'ufh' VALUE ufh, 'libelle' VALUE libelle, 'nomhopital' VALUE nomhopital, 'typeservice' VALUE typeservice, 'actif' VALUE actif) from Z_SOUPAP2CARTESITE where actif=1");
res.send(result.rows);
}
But when I go on the route in the browser, the datas have this shape :
[["92","028X362","ABC ACCUEIL URG MEDECINE","ANTOINE BECLERE","ADULTE",1],["92","028X472","ABC URGENCES PEDIATRIQUE","ANTOINE BECLERE","PEDIATRIE",1],["92","014X545","APR ACCEUIL URGENCES ADU","AMBROISE PARE","ADULTE",1]]
and I want this :
[
{"departement":"92","ufh":"028X362","libelle":"ABC ACCUEIL URG MEDECINE","nomhopital":"ANTOINE BECLERE","typeservice":"ADULTE","actif":1},
{"departement":"92","ufh":"028X472","libelle":"ABC URGENCES PEDIATRIQUE","nomhopital":"ANTOINE BECLERE","typeservice":"PEDIATRIE","actif":1}
]
Why are you using JSON_VALUE? The driver returns native JavaScript objects. You could write the query as:
select department "department",
ufh "ufh",
libelle "libelle",
nomhopital "nomhopital",
typeservice "typeservice"
from Z_SOUPAP2CARTESITE
where actif=1
In the query above, the double-quoted column aliases are used to control the case of the keys.
By default, the driver returns an array of arrays (no keys). If you want an array of objects, you need to pass an options object to execute that changes the outFormat. See this part of the doc: https://oracle.github.io/node-oracledb/doc/api.html#queryoutputformats
Here's an example from the doc:
const result = await connection.execute(
`SELECT department_id, department_name
FROM departments
WHERE manager_id < :id`,
[110], // bind value for :id
{ outFormat: oracledb.OUT_FORMAT_OBJECT }
);
console.log(result.rows);
If you want to use the JSON generation functions in Oracle, such as JSON_VALUE, you have to avoid a double parse - just access the string as JSON.
See this series for more info on building a REST API with Node.js and Oracle Database: https://jsao.io/2018/03/creating-a-rest-api-with-node-js-and-oracle-database/
The below solution works for me:
app.post('/getData', async function (req, res) {
try {
const result = await connection.execute("SELECT * FORM USER", [], { outFormat: oracledb.OUT_FORMAT_OBJECT });
// print response in json file: import => const fs = require("fs");
fs.writeFile("response.json", JSON.stringify(result.rows), err => {
if (err) throw err;
console.log('File successfully written to disk');
});
res.send({
data: result.rows,
});
} catch (err) {
console.log(err);
}});

How to minimize the response time to fetch the data from DB

I need to fetch 1000 records from mongodb , how can i do this in node js?
i have tried using pagination by sending the limit to 1000 even thought it takes 25 seconds to return the response, so is there any other way to minimize the response time ?
exports.getAll = function(filters, sort, skip, limit, callback) {
let query = {
removed: false
};
let filter = {};
let options = {
sort: []
};
req.skip = 0;
options.limit = 10000
options.sort = [
['_id', 'desc']
];
collections.findItems(query, filter, options, (err, result) => {
if (err) {
logger.error('db error', err);
return callback('Something went wrong');
}
callback(null,result);
});
};
exports.getAll = function getAll(req, res, next) {
service.getAll(req.body.filters, req.body.sort, req.body.skip, req.body.limit, (err, data) => {
if (err) {
return next({
status: 400,
error: err
});
}
res.json(data);
});
};
I guess you are using mongoose find in findItems method. find() returns mongoose documents with quite lot of stuff in it.
What you can do is use find().lean().
With lean(), simple plain javascript object of data from mongodb will be returned. It will optimize the response time a lot.
But I also suggest you to read this for more query optimizations at mongodb level: https://docs.mongodb.com/manual/core/query-optimization/
Are you using any ORM ? Like sequelize ?
What's your structure of document?
Check your mongdb logs:
tail -f /var/log/mongodb/mongod.log
If your document is straight forward then you should use indexing.
If you have index then try to rebuild them.
Sorting works efficiently when you have applied indexing. Avoid sorting on server if you can,Sort data at client side.
Because database imposes a 32MB memory limit on sorting operations

Resources