I'm trying to create RestAPI. There are a lot of documents in DB, let's call those - goods. Each good have some property.
Client will get those, and show to user. Something like: ...api/goods?filter value here.
It looks pretty easy, but it is a wrong way to send all goods which satisfied the filter. I need to separate those, and send only needed part of data.
The first idea was create a separate route, that will obtain filter and amount documents on page, and return page count. So, front-end can build pagination and then, using handler on each pagination tab, makes requests to server, and get needed data.
I've created something like basic-example(code is not good but the main idea is):
https://github.com/Gordienko-RU/Tiny-pagination
But I thought there are another, better implementation of it. In 'best practice' there was some note about sending pagination in header, but I couldn't figure it out.
So, I want to know, what is the best way to send data by parts, but also give to client information needed to build pagination pannel(amount of pages).
I've found some handy method. There will be one route. Client do a request for data on first page, but answer contains not only needed data, it also will be an object with information about pagination. Something like:
data: [...],
pagination: {
pageCount: ...
etc.
}
Maybe not a 'best practice' but good enough)
Related
This is more a design question than a lot of code.
I have my angular client query a mongodb collection via http and nodejs backend.
I want to paginate the result, so what I do is on the angular side, keep track of the page I'm on and the # of results I would like on each page, and I pass that on to my backend via an http call.
Then my backend has code that looks something like this:
schema.find({name: query, _userId: req.body._userId}).sort({ name: 'asc' }).limit(req.body.num_results).skip(req.body.page * req.body.num_results).then(response => {
Now I'd like to put "<" and ">" arrows on my client's html where every time left or right is clicked it traverses pages (via a new http call), but then make it so that "<" is disabled on page 0, and ">" is disabled on the last page of results. I've figured out how to disable the "<" - simply make the button go away when you're on page 0. But having trouble figuring out the best way to discover that I'm on the last page. One method could be for my backend to return the total number of results and then my front-end tracks where I am relative to the total #, but that involved another db call, move variables passed over http, more management on the front-end. Any ideas on how to implement a 'cleaner' solution?
Thank you.
You do not need to make any additional api call. You can return any additional information in the form of Http header when making api call. so, in this case return X-TOTAL-COUNT
I am stuck on implementing Pagination and I just need a bit of help, either some example code or even just a hint to help me proceed in the right direction.
I'm looking for some guidance on how to integrate Google Datastore database cursors with front-end UI pagination controls. I know how to build an angular pagination service, but that's with retrieving all the data at once and due to performance issues (5,000 records+) I want to use cursors to retrieve data in subsets.
NOTE: There's a similar question here, but I need more detail than this accepted answer provides. Node pagination in google datastore
QUESTION: How can I integrate the paginated datastore cursor queries with the front-end UI controls to allow the user to select the current page and control number of results displayed on each page?
I need to build a page that displays a large number of records with dynamic pagination. The user must be able to select the number of records display on each page.
Since there are several thousand records that might be returned at one time, I want to use cursors to retrieve subsets of data.
There is an example of how to paginate in the docs, but it is a pretty basic example and does not demonstrate how to integrate with front-end UI controls.
Can anyone provide a more detailed example and/or point me in the right direction on where to begin with this requirement? Unfortunately I haven't been able to find any detailed examples online.
https://googlecloudplatform.github.io/google-cloud-node/#/docs/datastore/1.1.0/datastore
Paginating Records
var express = require('express');
var app = express();
var NUM_RESULTS_PER_PAGE = 15;
app.get('/contacts', function(req, res) {
var query = datastore.createQuery('Contacts')
.limit(NUM_RESULTS_PER_PAGE);
if (req.query.nextPageCursor) {
query.start(req.query.nextPageCursor);
}
datastore.runQuery(query, function(err, entities, info) {
if (err) {
// Error handling omitted.
return;
}
// Respond to the front end with the contacts and the cursoring token
// from the query we just ran.
var frontEndResponse = {
contacts: entities
};
// Check if more results may exist.
if (info.moreResults !== datastore.NO_MORE_RESULTS) {
frontEndResponse.nextPageCursor = info.endCursor;
}
res.render('contacts', frontEndResponse);
});
});
One thing to keep in mind is the first item on the Limitations of cursors list:
A cursor can be used only by the same project that performed the original query, and only to continue the same query. It is not
possible to retrieve results using a cursor without setting up the
same query from which it was originally generated.
So you need to always be able to re-create the original query inside your handler, which means you need to pass around the equivalent of your NUM_RESULTS_PER_PAGE value from one request to another. You also need to reset the query every time that value changes - meaning you can't continue browsing from where you were after changing the number of results displayed per page.
Then, to be able to use the pagination, you also need to pass around the current cursor value from one request to another, updated at every request.
Now I'm not a NodeJS user, so I can tell exactly how this passing values around from one request to another would typically be implemented. In your code req.query.nextPageCursor and frontEndResponse.nextPageCursor appear to be intended for this, but I can't tell if that's OK or not. Maybe this is a less specific question easier to find an answer for.
In python webapp2, for example, I can store such values server-side in the user's session in one request and read them from the session in a subsequent request. Donno if this is of any help.
In my express app, a user can create a profile, and in doing so can choose a 'CustomUrl' that will be used to generate the url for their profile (example: http://www.myapp.com/profile/customUrl). A profile can also be reviewed, the end points for which might look like:
router.get('/reviews', ...) and router.post('/reviews/new',...)
The API endpoint for fetching a profile is below:
router.get('/:customUrl',...)
While I don't think a user would pick a customUrl like 'reviews', it is possible, and as such, my API endpoint to fetch their profile would be intercepted by the route to get '/reviews'.
I have a few ideas, but I'm generally new to Express, and wasn't sure if one way was better than another. Here are some of them:
1) I can put any end-point with a customUrl path above other end points. This doesn't sound acceptable as then the customUrl path would intercept any requests that are meant for '/reviews' etc.
2) I can validate a customUrl that it is not only unique, but is not an existing end point. This does not seem satisfactory as it does not scale well (would have to update the blacklist every time I add a new end point). It would also be problematic if a user has selected a customUrl that I later want to use for another end point.
3) Separate review out of profile and into its own route. This is probably the best solution, although it would present a lot of work that I'm hoping to avoid (there are other sub-routes name-spaced under '/profiles').
4) Put router.get('/reviews'...) above customUrl, but call next() if there is no req.query ('?sortBy=dateCreate' etc).
5) I didn't mention this earlier because it feels like I'd only be hiding the problem instead of fixing it, but my client-side is in React on a separate port, and so I could configure the router to display the url as .com/profile/:customUrl, but change the API end point to be something like: router.get('/',...) with a queryString of '?customUrl=...' or something like that. The url would then be 'www.myApp.com/profile/:customUrl' but the get request would be to 'api.myApp.com/profile/?customUrl=:customUrl', and the get request for reviews would be 'api.myApp.com/profile/reviews' (or something like that).
Anyway, I have a feeling that I'm missing something pretty fundamental. Any help would be great!
You probably just need to be RESTful with your route naming, for example:
GETing profile:
router.get('/profile/:customUrl', ...)
GETting reviews for a profile
router.get('/profile/:customUrl/reviews', ...)
POSTing a new review (note you don't need a route that says "new" as this can be determined by it being a POST request)
router.post('/profile/:customUrl/reviews',...)
You should use the following verbs for the reasons listed if you are being truly RESTful:
GET - Fetching records
POST - Adding records
PUT - Editing records
PATCH - Performing partial edits (e.g. not sending the whole document)
DELETE - Removing records
If I am creating a REST api with express for an application, as far as I'm aware, the order of the route definitions matter. For example, if I wrote my routes in the following order:
app.get('/users/:username', user.get);
app.get('/users/list', user.listAll);
then user.listAll could never be reached and the server would assign 'list' to the username parameter when a request was made to '/users/list'. So it's obvious that you want to reverse the order of the declaration of these routes. But here, we'll run into the same problem if a user creates an account with the username 'list', right?
So how do you prevent users from creating names with 'keywords' where keywords are defined here as strings that would conflict with the api routes.
I can think of two ways, but neither of them seem satisfying. First, you could keep a blacklist of strings that you compare against each username creation. But maintaining that would be a nightmare. And two, just pre-create those users in the database (as long as usernames are unique). That kind of seems a bit hacky but I can't think of many arguments against it.
Is there a simple solution to this problem?
Good question. I've faced this a while ago. You must be coming from a web background like me, because you are trying to use the 'slug' method to make user readable calls to the API. Using readable slugs is great and actually preferred in websites! There is actually not 1 way to do it in API's I've seen many ways that work, but I only know 1 way of making it scalable and descriptive in a way that works for almost every usecase.
Always use ID's
One user: /users/:id
List: /users/
Friends: /users/:id/friends
1 friend of 1 user: /users/:id/friends/:id
Above is extendable and still descriptive! In addition I use versioning. (ex: /v1/users). This allows me to upgrade the API to a v2 while still supporting older clients :)
Now how do I lookup a specific user? Use filters!.. How?
/v1/users?username=:username
The above will always return a list of users. If username is unique this will still be a list, but only 1 record or just an empty list.
Use HTTP methods POST PUT GET DELETE
POST on /users (201 created user)
PUT/DELETE on /users (400 method not allowed) (optionally remove all??? Not recommended)
PUT / DELETE on /users/:id (200 success)
POST on /users/:id (400 user already exists)
Hope this will solve your case :)
Some web design questions.
Combine POST with GET?
When user clicks a button, I need to send one POST to submit a form, then need to GET a json object to replace some DOM fields. Should I combine them into one request so save one round trip time?
Multiple GET json request?
When user clicks a button, I need to GET 3 or 4 json object. Should I send 3 or 4 GET request, or just one and combine the json into one large json at back-end?
So basically, I'm not sure which is heavier: round trip time VS a little complexed back-end and front-end logic.
Any comment is welcome!
if i understood your question right...you have a dependency on your get requests...that is to say the second get depends on the first...if that is the case, obviously you need to take consecutive get operations...however, if that is not the case that is if you know the order of get request and the response won't be affected by local conditions...then i suggest you do post/get operations on server side...trigger the first one an let the server handle the rest and get the result...
of course you would not want users do multiple get requests for one simple operation...