I implemented pagination system by using pagination container. It works great with one problem.
When i call loadMore function to get new nodes relay fetches nodes from server and adds to olds ones. But i want to delete old nodes and just want to show new ones.
For example $first = 10
When the page first loads i got 10 nodes, after loadMore i get 20 nodes. But i want to show just new fetched 10 nodes.
I used this pagination example
I could not find a way to remove old nodes after loadMore action.
I know relay pagination intended to use only infinite scroll, i just wonder maybe there is a way to solve this problem.
The Hacky/easy solution is to alway map through the last 10 items.
Another solution is to check the Relay store using ConnectionHandler and delete the edges that you don't want from the store manually.
Look at deleteNode in https://relay.dev/docs/en/relay-store for more info how to work with relay store.
Better solution is to create a re-fetch-able container and use the cursor pagination instead so every time you get the next 10 items by setting the cursor and after arguments on the connection. This way you have to query for pageInfo and manually check if hasNextPage.
Also this way you will be able to have a bi-directional pagination (Forward and backward pagination).
More info about cursor based pagination:
https://relay.dev/graphql/connections.htm
https://graphql.org/learn/pagination/
Related
I have a document library in SharePoint online. I keep on dumping the records into it. As SharePoint have a 5000 record view limitation the moment it reaches that limit, still I will be able to upload documents but it doesn't show up any where.
Eventually I end up creating a new view and apply a filter and then the document starts showing up under the new view.
My question here is: Is there a way to automatically create a view when it reaches the 5000 limitation and put the newly uploaded documents to the new view.
Yes, you can do this via MS Flow/Workflow & server side apps/scripts of course but it's not a good approach to the issue IMO.
Have you indexed the columns? I just tested this now on a document library with 20k documents and I'm able to filter. There are limitations which you should look into (complex filtering), that's where compound indexes come in.
If you still have issues then I recommend you give the highlighted content web-part a try. You can create custom search queries & it looks similar to a document library if u set the settings correctly. The only meh thing about this approach is there is a delay for search to update, from 15 mins to 6 hours depending on how much data you have
I am building a typical pagination that allows the user to click on a particular page number and view the results (similar to the google search result view). I am using the cloudant search API for this. The cloudant search API provides the limit option but no skip option. How can I skip n number of results if the user is on page 1 and clicks on page 4 ?
I can see that the pagination is implemented using bookmarks. Does it mean that I need to first get the bookmark for page 4 by sending 3 additional requests one after another to the search api ?
There are a couple of different ways of handling this - one is the one you already suggested, which is just to fetch the pages as needed to get the bookmarks. I'm not sure there are many alternatives for search results where we can't pre-calculate the results.
Another alternative, and this depends a bit on the details of what you are trying to do, is to create a view containing the data and use the keys to narrow down the view to the results you need. View outputs support use of limit and skip which would enable you to implement pagination.
There's also a good example of pagination in the docs: http://docs.couchdb.org/en/2.1.0/ddocs/views/pagination.html
I have to build a search textbox in a web page similar to facebook search box. Client side there will be ajax calls. The user need to search into around 300.000 elements that have a description of a few words or an alphanumeric code. When user enters the beginning of a word, a call is made to the server which return best match based on the starting of any word or code but also suggest first the elements most recently by the user, then by the group the user belongs to and finally from the entire set. Result can be limited to 10-20 items.
How can I build a fast search by key with the value just the description of the element? We use SQL server but any other DB could be OK.
The implementation at the time was very complex to summarise here but I came across recently to UI-select that solve the front end problem nicely and it's very good component if you are using Angular
https://github.com/angular-ui/ui-select
then backend you can put whatever you have (I did with Redis)
This seems like a pretty simple thing but I can't find any discussions that really explain how to do it.
I'm building a scraper with MongoDB and Node.js. It runs once daily and scrapes several hundred urls and records to the database. Example:
Scraper goes to this google image search page for "stack overflow"
Scraper gets the top 100 links from this page
A record of the link's url, img src, page title and domain name are saved to MongoDB.
Here's what I'm trying to achieve:
If the image is no longer in the 100 scraped links, I want to delete it from the databqse
If the image is still in the 100 scraped links, but details have changed (e.g. new page title) I want to find the mongodb record and update it.
If the image doesn't exist already, I want to create a new record
The bit I'm having trouble with is deleting entries that haven't been scraped. What's the best way to achieve this?
So far my code successfully checks whether entries exist, updates them. It's deleting records that are no longer relevant that I'm having trouble with. Pastebin link is here:
http://pastebin.com/35cXcXzk
You either need to timestamp items (and update them on every scrape) and periodically delete items which haven't been updated in a while, or you need to associate items with a particular query. In the latter case, you would gather all of the items previously associated with the query, and mark them off as the new results come in. Any items not marked off the list at the end, need to be deleted.
another possibility is to use the new TTL index option in mongodb 2.4 allowing you to set time to live on documents
http://docs.mongodb.org/manual/tutorial/expire-data/
This will let the server expire them over time instead of having to perform big expensive remove executions.
Another optimization is to use the power of 2 option for collections to avoid the high fragmentation of memory that write, remove cycles create
http://docs.mongodb.org/manual/reference/command/collMod/
I have around 53,00,000 documents in MarkLogic server and I am building a simple search application. User enters a search term and MarkLogic server searches that term in all the nodes in all the documents and returns the matching documents as the result. I have implemented a custom paging to show results per page. I am showing 10 results per page.
I am using search api for this as:-
import module namespace search="http://marklogic.com/appservices/search" at "/Marklogic/appservices/search/search.xqy";
declare variable $options:=
<options xmlns="http://marklogic.com/appservices/search">
<transform-results apply="raw"/>
</options>;
search:search($p, $options, $noRecFrom, 10)/search:result
where $p is the input from the user $noRecFrom is the number which indicates from where we have to show records. For example for page 1 $noRecFrom will be 1, for page 2 $noRecFrom will be 11, for page 3 $noRecFrom will be 21 and so on. For paging there are hyperlinks to go to First, Next, Prev and Last pages.
To calculate the total number of records returned I am using:-
for $x in search:search($p, $options)
return $x//#total;
While First, Next and Prev hyperlink works perfectly but if someone clicks Last the application stops responding and the query does not show any output. Is it due to the large number of documents in the database or I am implementing it wrongly.
Is there any efficient way for pagination in MarkLogic (for search:search) so that the user can go the Last page without delay in query result for such a large database ?
The way you've implemented it, you're running the search repeatedly in your for loop. And that would indeed be slow.
Instead, you should be calculating a $start parameter based on the #total and number of documents per page, and passing that in as an argument (I think it's the third one) to search:search.
I would also recommend making sure you can run in unfiltered mode. There is good information about optimizing for fast pagination (indexes, etc) on the developer site; the idea is to resolve queries out of indexes to give very good, accurate unfiltered performance.
If you do it that
There is a tutorial on paginated search at http://developer.marklogic.com/learn/2006-09-paginated-search
Once you have resolved the issues mentioned by cwhit above, if you still want to get to the last page of data in a faster manner, you could make your code smart enough to reverse the sort order and pull the correct offset of records.
Here's another tip:
To get better insight to what MarkLogic is doing with search:search, call
search:get-default-options()
to see the starting point for common search applications.