How to disable pagination in feathers service? - node.js

Hi I try populate my datatable using feathers service in this way:
app.service('pupils').find({}, (error, result) => {
$('#pupils > table').DataTable({
"pageLength": view.short,
"lengthChange": false,
"info" : false,
"responsive": true,
"data": result.data,
"deferRender": true,
"columns": [ ... ]
});
});
I have more than 100 testing records but in callback I receive only 10 records.
I receive more records after added below code in feathers service:
paginate: {
default: 100,
max: 200
}
but I would like to disable pagination for received all records from mongo.
How can I do that?

To disable pagination, remove the paginate option. Not recommended for production however since it might bring down both, the client and the server if you try to send many thousands of records.
Note: the response object changes depending on whether you are using pagination:
Response with pagination: object with data array property
{
total: 572,
limit: 50,
skip: 4,
data: [/* the data is here */]
}
Response without pagination: the data array
[/* the data is here */]

As #daff already pointed out in the answer above, you can disable Pagination in your configuration, although this is really not recommended.
Also you can disable pagination for a service call once in the service call.
service.find({ paginate:false})
This was discussed on Github here
The documentation for this feature can be found here

Related

List All Docs with #ibm-cloud/cloudant

So recently IBM deprecated #cloudant/cloudant and have moved on to #ibm-cloud/cloudant. I was in the process of doing a todo app to learn Cloudant along with some other things and so had to migrate to the new package.
As you can see from my Cloudant dashboard, I have a database with name "todo-vue-hapi" and there are 3 docs in the db. According to the migration docs to list all the documents before you would use db.list now you should use service.postDesignDocs so I connected everything and used the following code:
const response = await db.postDesignDocs({
db: 'todo-vue-hapi',
})
This gives back a response object with the following result property:
{result: { total_rows: 0, offset: 0, rows: []}
The 3 docs are not there! I'm wondering how I can query everything in the database "todo-vue-hapi". Help with this issue would be greatly appreciated.
The postDesignDocs function only returns design documents, which are special Cloudant documents containing index definitions.
To return all documents in a database you need to use the _all_docs API endpoint which is implemented as the postAllDocs function:
const response = await db.postAllDocs({
db: 'todo-vue-hapi',
includeDocs: true,
limit: 10
})
This should return you all documents in the database, including regular and design documents.

Does loopback 3 supports where filter with skip + limit?

I am working on Loopback with version 3,
Does It supports where filter with skip and Limit, I am trying to apply for pagination.
For pagination, I have always used "offset" but in docs there is an option for skip also. Please checkout at skip in loopback3
If you want apply skip and limit with pagination. This code is help full for you.
example is now :
Users.getList = function (filter, cb) {
Users.find({
where: {
condition
},
limit: filter.limit,
skip: filter.skip
})
}
You can use skip and limit in the loopback where condition. I assume that you are trying to get the answer for something like the query SELECT * FROM schemaName.tableName limit 10,5 using loopback, for that you can use the following peace of code
modelName.find({
order: 'id DESC',
skip: 10,
limit: 5,
});

How to save websocket responses to mongodb

I am connected to a websocket using node.js ws module.
Each time i get response with event called "add_items" i want to save it to mLab cloud database.
Websocket response looks like this
{"event":"add_items","data":[
{"id":["13635936204"],
"m":"name1",
"p": 14.00}
{"id":["13635936220"],
"m":"name2",
"p": 50.00}
]}
What would be the best way to connect to database and save new data ?
Currently i use .forEach method to loop through array , than create objects for each new item and save it to databse using db.collection.insert() method. But the problem is that i might get over 50 items per sec and sometimes i get error "mongodb connection timed out". Is it possible to use mongoose module to simplify this process ?
You can use MongoDB's insertMany:
db.collection.insertMany(websocketEvent.data);
Where websocketEvent is the response received via Websocket. Check insertMany syntax and info here.
UPDATE:
Maybe you can loop through the array calling updateOne and decide to update or insert using the arrayFilters field, something like this:
websocketEvent.data.forEach(event => {
db.collection.updateOne(
{ _id: ObjectID(event.id[0])},
{ $set: { m: event.m, p: event.p }},
{
upsert: true,
arrayFilters: [ <filterdocument1>, ... ]
}
)
}
Check the docs for info on the arrayFilters field for updateOne.

Buildfire - How to load sorted pages dynamically from publicData/datastore

I am trying to lazy load data from publicData using the options that BuildFire describes in the wiki. I have set up some code to test that it works and it seems that is does not any way that I configure the request options. Here is the code that I am using:
var loadSortedPages = function(page) {
var skip = page*50;
var options = {
"filter": {},
"sort": {"points": 1},
"pageSize": "50",
"skip": skip.toString()
}
buildfire.publicData.search(options, 'users', function(err, records) {
console.log("RECORDS SORTED ASCENDING BY POINTS FOR PAGE " + page, records);
});
}
loadSortedPages(0);
loadSortedPages(1);
loadSortedPages(2);
I have tried, it seems, every thinkable combination of "page" and "skip" both as different combinations of string and number values. Nothing works and I always get back the first 50 sorted records for each of the loadSortedPages calls even though I am passing in different page numbers. If this something on BuildFire's end?
Here is the documentation on how to use Datastore search https://github.com/BuildFire/sdk/wiki/How-to-use-Datastore#buildfiredatastoresearchoptions-tag-optional-callback
It seems like you are mixing to pagination methods:
and for pagination you can either use:
page : is number that determine the page that need to retrieve.
pageSize: is a number of record per page , the max value is 20.
Or use:
skip : is number of record that you need to skip.
limit: is a number
of record for this call, the max value is 20.

Pagination for search indexes on cloudant

I have a cloudant db which contains documents for access logs of users. Example:
{
"name": "John Doe",
"url": "somepage.html",
"dateaccessed": "2016-08-23T21:20:25.502Z"
}
I created a search index with a function:
function (doc) {
if(doc.dateaccessed) {
var d = new Date(doc.dateaccessed);
index("dateaccessed", d.getTime(), {store: true});
}
}
Now this setup is working as expected with just a normal query. Example
{
q: 'dateaccessed:[1420041600000 TO 1471987625266]',
include_docs: true,
sort: '-dateaccessed<number>',
}
However, I wish to limit the results - let's say 5 at a time (which can be done with the "limit: 5" argument), and I want to somehow make a pagination - be able to move to the next 5 results or the previous 5 results.
I checked the cloudant documentation and there's an argument there called "bookmark" (https://cloudant.com/for-developers/search/) but I'm not sure how to use it.
May I request for any insights on this?
The Cloudant documentation shows examples on how to use bookmarks, but the gist is that you get a bookmark from the server in your search response, when requesting the next page, you use the bookmark received in your request with the bookmark parameter in either your JSON object for POST, or in the query params for GET.

Resources