I am trying to compute the skip and limit with the total number of documents (count).
The issue is that the query object returns the count value when I try to get the items.
Follows an example:
const query = MyModel.find().or([AbilityRule1, AbilityRule2, ...]);
const count = await query.countDocuments(); // count = 3
// Some logic to compute the values of `skip` and `limit` with `count`
// const skip = ...
// const limit = ...
const items = await query.skip(skip).limit(limit); // items = 3 instead of [Model, Model, Model]
I found myself with a similar question when I was trying to implement pagination. The answer I came up with was to use the merge function on the Query object.
const query = MyModel.find().or([AbilityRule1, AbilityRule2, ...]);
const count = await MyModel.find().merge(query).countDocuments();
const items = await query.skip(skip).limit(limit);
Source: https://mongoosejs.com/docs/api/query.html#query_Query-merge
countDocuments() looks to be a method of Model, not Query. I guess in the way you're using it here by calling it on an existing query object, you may be just overwriting it.
Why not just:
const query = MyModel.find();
const count = await MyModel.countDocuments();
// ...
const items = await query.skip(skip).limit(limit);
Inspired by Mathew's answer:
I am adding this answer because I find it important that the second and third instructions do not depend on MyModel, they just depend on the query object.
const query = MyModel.find().or([AbilityRule1, AbilityRule2, ...]);
const count = await query.model.find().merge(query).countDocuments();
const items = await query.skip(skip).limit(limit);
Related
I have a simple BigQuery table with a few columns. One of the columns is named my_id (of type STRING). I'm querying my BigQuery datasets like this:
import * as bq from "#google-cloud/bigquery";
const bqdb = new bq.BigQuery();
// ...
const projectId = 'my_project';
const datasetId = "my_dataset";
const tableId = "my_table";
const dbId = [projectId, datasetId, tableId].join('.');
// myIds is an array of strings
const stringedArray = myIds.map((id) => '\'' + id + '\'');
const sql_select_query = `
SELECT my_id
FROM \`${dbId}\`
WHERE my_id IN (${String(stringedArray)})
LIMIT 1
;
`;
const dataset = bqdb.dataset(datasetId);
const destinationTable = dataset.table(tableId);
console.log("Querying database...");
const queryOptions = {
query: sql_select_query,
destination: destinationTable,
write_disposition: "WRITE_APPEND",
priority: 'BATCH',
};
// Run the query as a job
const [job] = await bqdb.createQueryJob(queryOptions);
// Wait for the job to finish.
const results = await job.getQueryResults({maxResults: 500});
const resutsArray = results[0]
This query brings back the ENTIRE table (all rows, all columns). In other words, the result of this query is the same as if I'd wrote:
const sql_select_query = `
SELECT *
FROM \`${dbId}\`
;
`;
The output is formatted like a successful query: there's no error messages or warnings. But all my conditionals are being ignored, even LIMIT.
Why is BigQuery dumping the entire table into the response?
If your query settings are set to writing results to a destination table and using write_disposition: "WRITE_APPEND", job.getQueryResults() will return the data of the destination table along with the newly appended data which is an expected behavior of BigQuery.
job.getQueryResults() will only return the initially selected result if a destination table is configured and write disposition is either 'write if empty' or 'overwrite table'.
As a workaround you can query two times, first using temporary table to return results with condition and run the query again appending to the destination table.
Using your code, you can create two query options. First query option does not have destination and second query option has the destination along with write_disposition. Then create two jobs that uses the first and second query option.
Code snippet:
const queryOptions = {
query: sql_select_query,
priority: 'BATCH',
};
const queryOptionsWrite = {
query: sql_select_query,
destination: destinationTable,
write_disposition: "WRITE_APPEND",
priority: 'BATCH',
};
const [queryJob] = await bqdb.createQueryJob(queryOptions);
const queryResults = await queryJob.getQueryResults();
console.log("Query result:");
console.log(queryResults[0]);
const [writeJob] = await bqdb.createQueryJob(queryOptionsWrite);
const writeResults = await writeJob.getQueryResults();
console.log("\nUpdated table values:");
console.log(writeResults[0]);
Test done:
I have a document where i have two fields;
category
duration in micromilliseconds
Category could have multiple values like category1, category2 etc.
I want to calculate sum of duration of all the documents where category == category1. Please help with the query.
import * as admin from 'firebase-admin'
import * as functions from 'firebase-functions'
admin.initializeApp()
export const updateCharts =
functions.firestore.document('users/{UserId}/count/{uid}')
.onWrite(async(change, _) => await updateStats(change))
async function updateStats (change:
functions.Change<functions.firestore.DocumentSnapshot>){
const chartRating = change.after.ref.parent
let title = (await chartRating.where('Title', '==', 'Game').get()).size;
//duration
const restaurantRef = chartRating.parent!
console.log('{restaurantRef.path} now has ${Title}')
await restaurantRef.update({
Title: Title,
})
Once you fetch all the documents using a query it's you can use reduce() to find total duration:
let title = await chartRating.where('Title', '==', 'Game').get();
const totalDuration = title.docs.reduce((a, b) => a + b.data()["duration"], 0)
// Adds 'duration' field of all documents
A cheaper alternative would be to read previous total from restaurantRef, subtracting old value of that's document's duration and adding the updated value. This will cost only 1 read and 1 write operation whereas reading all documents will cost N reads where N is number of documents matched.
I am working on a project with sequelize and i have a problem:
I need to find all the emails in a custom array like:
const array = ['1234#gmail.com', '14521#gmail.com', '13245#gmail.com']
and findAll users where those array elements with only one action,
Can i do that?
I think some thing like this:
const array = ['1234#gmail.com', '14521#gmail.com', '13245#gmail.com']
const users = await db.Users.findAll({ where: {
[Op.contains]: array
}}).then(user => res.send(user));
Thanks for your Help!
I have got the answer use:
const array = ['123#gmail.com', '456#gmail.com'];
const users = await db.Users.findAll({where: {
Email: array
}}).then(room => res.send(room));
I am trying to make a scraper that scrapes Post ID and Poster's ID from a Facebook public post link, using puppeteer and nodejs.
(async() => {
let url = 'https://m.facebook.com/photo/?fbid=1168301430177531&set=gm.1386874671702414'; //demo link
let brw = await puppeteer.launch();
let page = await brw.newPage();
await page.goto(url,{ waitUntil:'networkidle2'});
let data = await page.evaluate(()=>{
let ids = document.querySelector('div[class="_57-p"] > a[class="_57-s touchable"]').search; // for
image post
return{
ids
}
});
console.log(data);
and I get output like:
{
ids: '?fbid=1168301430177531&id=100009930549147&set=gm.1386874671702414&refid=13&__tn__=%2B%3E'
}
how can I filter out the unnecessary portions?(I just want fbid and id values)
Thanks in advance
It seems this is the most reliable and simple way:
const href = document.querySelector('div[class="_57-p"] > a[class="_57-s touchable"]').href;
const searchParams = new URL(href).searchParams;
return {
fbid: searchParams.get('fbid'),
id: searchParams.get('id'),
};
Try use query-string.
it will help you to parse the query strings
let search = '?foo=bar'
const parsed = queryString.parse(search);
console.log(parsed);
//=> {foo: 'bar'}
it is just a simple example of what you should do
You can use a match() method with a regex like /\Wfbid=(\w+)(\W|$)/, the search result under index 1 of the capturing groups will contain the desired parameter value.
let ids = '?fbid=1168301430177531&id=100009930549147&set=gm.1386874671702414&refid=13&__tn__=%2B%3E'
const fbid = ids.match(/\Wfbid=(\w+)(\W|$)/)[1] // 1168301430177531
const id = ids.match(/\Wid=(\w+)(\W|$)/)[1] // 100009930549147
Without [1] you'd get all matches, e.g.:
ids.match(/\Wid=(\w+)(\W|$)/)
=>
["&id=100009930549147&", "100009930549147", "&", index: 22, input: "?fbid=1168301430177531&id=100009930549147&set=gm.1386874671702414&refid=13&__tn__=%2B%3E", groups: undefined]
And you need the string between the capturing & characters, the 2nd element of the array (so: [1]).
So I am fetching values from chuck norris api. I am able to fetch and present values when given one value such as when using random event from the api. My question is, how can I present a value given to me when there is more than one value displayed in a list?
let topic = args.join(" "); //defines topic set as varliable to use in query search
fetch(`https://api.chucknorris.io/jokes/search?query=${topic}`).then(result => result.json());
const { sub } = await fetch(`https://api.chucknorris.io/jokes/search?query=${topic}`).then(result => result.json());
if(topic !== " ") { return message.channel.send(sub)};
It sounds like you're asking how to pick one item at random from an Array. Math.random gives you a random number between 0 and 1. How can you convert this to a random index in an Array? Multiply by the size of your Array first, then round down with Math.floor:
const response = await fetch(`https://api.chucknorris.io/jokes/search?query=${topic}`);
const jokes = await response.json();
const randomIndex = Math.floor(Math.random() * jokes.length);
const randomJoke = jokes[randomIndex];
return message.channel.send(randomJoke)
Easiest way to choose a random value from an array of items is to do the following:
const randomItem = allItems[Math.floor(Math.random() * allItems.length)];