This is more a design question than a lot of code.
I have my angular client query a mongodb collection via http and nodejs backend.
I want to paginate the result, so what I do is on the angular side, keep track of the page I'm on and the # of results I would like on each page, and I pass that on to my backend via an http call.
Then my backend has code that looks something like this:
schema.find({name: query, _userId: req.body._userId}).sort({ name: 'asc' }).limit(req.body.num_results).skip(req.body.page * req.body.num_results).then(response => {
Now I'd like to put "<" and ">" arrows on my client's html where every time left or right is clicked it traverses pages (via a new http call), but then make it so that "<" is disabled on page 0, and ">" is disabled on the last page of results. I've figured out how to disable the "<" - simply make the button go away when you're on page 0. But having trouble figuring out the best way to discover that I'm on the last page. One method could be for my backend to return the total number of results and then my front-end tracks where I am relative to the total #, but that involved another db call, move variables passed over http, more management on the front-end. Any ideas on how to implement a 'cleaner' solution?
Thank you.
You do not need to make any additional api call. You can return any additional information in the form of Http header when making api call. so, in this case return X-TOTAL-COUNT
Related
I have prepared a simple demo with react-router-dom 6 and react query.
I have a couple of routes and a fetch call that takes place on the first route (Home).
What I want to achieve is navigating to the About page or any other page and do not perform another request for a certain time period (maybe never again) but if I refresh the page I want to be able to re trigger the request to get the data.
I have tried using staleTime when but if I refresh the page I get no results, just a blank page. refreshInterval works on refresh but does not keep the data when I change routes.
I have also tried this pattern in this article but still I don't get the job done.
Probably it may be that something I don't understand but the question is how do I avoid making the same request over and over again, perfrm it only once but still being able to get the data if I refresh the page when navigating between different routes.
Demo
The solution to the problem came from one of the maintainers on the official github repo eventually and is related to adding placeholderData as an empty array instead of initialData and setting staleTime to Infinity in my case as I only want to perform the request once.
Setting placeholderData gives you the opportunity to show some dummy data normally until you fetch the real but in my case it seems to do the job. More to read regarding this at this source
const { isFetching, data: catImages } = useQuery({
queryKey: ["catImages"],
queryFn: getCatImages,
placeholderData: [],
staleTime: Infinity
});
I am stuck on implementing Pagination and I just need a bit of help, either some example code or even just a hint to help me proceed in the right direction.
I'm looking for some guidance on how to integrate Google Datastore database cursors with front-end UI pagination controls. I know how to build an angular pagination service, but that's with retrieving all the data at once and due to performance issues (5,000 records+) I want to use cursors to retrieve data in subsets.
NOTE: There's a similar question here, but I need more detail than this accepted answer provides. Node pagination in google datastore
QUESTION: How can I integrate the paginated datastore cursor queries with the front-end UI controls to allow the user to select the current page and control number of results displayed on each page?
I need to build a page that displays a large number of records with dynamic pagination. The user must be able to select the number of records display on each page.
Since there are several thousand records that might be returned at one time, I want to use cursors to retrieve subsets of data.
There is an example of how to paginate in the docs, but it is a pretty basic example and does not demonstrate how to integrate with front-end UI controls.
Can anyone provide a more detailed example and/or point me in the right direction on where to begin with this requirement? Unfortunately I haven't been able to find any detailed examples online.
https://googlecloudplatform.github.io/google-cloud-node/#/docs/datastore/1.1.0/datastore
Paginating Records
var express = require('express');
var app = express();
var NUM_RESULTS_PER_PAGE = 15;
app.get('/contacts', function(req, res) {
var query = datastore.createQuery('Contacts')
.limit(NUM_RESULTS_PER_PAGE);
if (req.query.nextPageCursor) {
query.start(req.query.nextPageCursor);
}
datastore.runQuery(query, function(err, entities, info) {
if (err) {
// Error handling omitted.
return;
}
// Respond to the front end with the contacts and the cursoring token
// from the query we just ran.
var frontEndResponse = {
contacts: entities
};
// Check if more results may exist.
if (info.moreResults !== datastore.NO_MORE_RESULTS) {
frontEndResponse.nextPageCursor = info.endCursor;
}
res.render('contacts', frontEndResponse);
});
});
One thing to keep in mind is the first item on the Limitations of cursors list:
A cursor can be used only by the same project that performed the original query, and only to continue the same query. It is not
possible to retrieve results using a cursor without setting up the
same query from which it was originally generated.
So you need to always be able to re-create the original query inside your handler, which means you need to pass around the equivalent of your NUM_RESULTS_PER_PAGE value from one request to another. You also need to reset the query every time that value changes - meaning you can't continue browsing from where you were after changing the number of results displayed per page.
Then, to be able to use the pagination, you also need to pass around the current cursor value from one request to another, updated at every request.
Now I'm not a NodeJS user, so I can tell exactly how this passing values around from one request to another would typically be implemented. In your code req.query.nextPageCursor and frontEndResponse.nextPageCursor appear to be intended for this, but I can't tell if that's OK or not. Maybe this is a less specific question easier to find an answer for.
In python webapp2, for example, I can store such values server-side in the user's session in one request and read them from the session in a subsequent request. Donno if this is of any help.
I'm trying to create RestAPI. There are a lot of documents in DB, let's call those - goods. Each good have some property.
Client will get those, and show to user. Something like: ...api/goods?filter value here.
It looks pretty easy, but it is a wrong way to send all goods which satisfied the filter. I need to separate those, and send only needed part of data.
The first idea was create a separate route, that will obtain filter and amount documents on page, and return page count. So, front-end can build pagination and then, using handler on each pagination tab, makes requests to server, and get needed data.
I've created something like basic-example(code is not good but the main idea is):
https://github.com/Gordienko-RU/Tiny-pagination
But I thought there are another, better implementation of it. In 'best practice' there was some note about sending pagination in header, but I couldn't figure it out.
So, I want to know, what is the best way to send data by parts, but also give to client information needed to build pagination pannel(amount of pages).
I've found some handy method. There will be one route. Client do a request for data on first page, but answer contains not only needed data, it also will be an object with information about pagination. Something like:
data: [...],
pagination: {
pageCount: ...
etc.
}
Maybe not a 'best practice' but good enough)
My goal is to server-side render my blog built with Node and React/Redux. I am following this tutorial, and using this project as an example.
I have loaded my components, created an empty store, successfully used this stuff to render and return html pages.
My remaining challenge is to figure out how to fetch the inital data so that I could put it into the store. I am assuming, I somehow need to tell the store(on the backend) to dispatch the actions that will fill it with data.
This example is using this function, that seems to be telling the store to dispatch actions, and returns the store filled with data(?). But it flies way over my head, I don't get how it works, and when I copy the code, the store still returns empty.
Can you help me to understand how this is supposed to work? What do I need to do to tell the store to execute actions, that will go to my API, fetch data, and put it into the store?
basically fetchComponentData will dispatch actions while server rendering, like the document says
fetchComponentData collects all the needs (need is an array of actions that are required to be dispatched before rendering the component) of components in the current route.
where need is an array of functions that return a redux action, which defined inside each smart components (connected with redux store and react-router)
for example, in mern-starter, when you visit index route, it'll render this component and there is the need method which will be executed by fetchComponentData when server rendering.
In my express app, a user can create a profile, and in doing so can choose a 'CustomUrl' that will be used to generate the url for their profile (example: http://www.myapp.com/profile/customUrl). A profile can also be reviewed, the end points for which might look like:
router.get('/reviews', ...) and router.post('/reviews/new',...)
The API endpoint for fetching a profile is below:
router.get('/:customUrl',...)
While I don't think a user would pick a customUrl like 'reviews', it is possible, and as such, my API endpoint to fetch their profile would be intercepted by the route to get '/reviews'.
I have a few ideas, but I'm generally new to Express, and wasn't sure if one way was better than another. Here are some of them:
1) I can put any end-point with a customUrl path above other end points. This doesn't sound acceptable as then the customUrl path would intercept any requests that are meant for '/reviews' etc.
2) I can validate a customUrl that it is not only unique, but is not an existing end point. This does not seem satisfactory as it does not scale well (would have to update the blacklist every time I add a new end point). It would also be problematic if a user has selected a customUrl that I later want to use for another end point.
3) Separate review out of profile and into its own route. This is probably the best solution, although it would present a lot of work that I'm hoping to avoid (there are other sub-routes name-spaced under '/profiles').
4) Put router.get('/reviews'...) above customUrl, but call next() if there is no req.query ('?sortBy=dateCreate' etc).
5) I didn't mention this earlier because it feels like I'd only be hiding the problem instead of fixing it, but my client-side is in React on a separate port, and so I could configure the router to display the url as .com/profile/:customUrl, but change the API end point to be something like: router.get('/',...) with a queryString of '?customUrl=...' or something like that. The url would then be 'www.myApp.com/profile/:customUrl' but the get request would be to 'api.myApp.com/profile/?customUrl=:customUrl', and the get request for reviews would be 'api.myApp.com/profile/reviews' (or something like that).
Anyway, I have a feeling that I'm missing something pretty fundamental. Any help would be great!
You probably just need to be RESTful with your route naming, for example:
GETing profile:
router.get('/profile/:customUrl', ...)
GETting reviews for a profile
router.get('/profile/:customUrl/reviews', ...)
POSTing a new review (note you don't need a route that says "new" as this can be determined by it being a POST request)
router.post('/profile/:customUrl/reviews',...)
You should use the following verbs for the reasons listed if you are being truly RESTful:
GET - Fetching records
POST - Adding records
PUT - Editing records
PATCH - Performing partial edits (e.g. not sending the whole document)
DELETE - Removing records