I created two lists in my SharePoint 2010 site. one has 10 items other has 10000 items.
On the small list I can say
http://mysharepoint.com/sites/testsite/_vti_bin/listdata.svc/SmallList?$top=1&$skip=1
no problem it works perfect.
when I say
http://mysharepoint.com/sites/testsite/_vti_bin/listdata.svc/LargeList?$top=1&$skip=1
it throws an error "An error occurred while processing this request"
The internet is full of confustion because most blogs/articles etc talk about "SP2013" where I guess there is a __next in the body of XML which you get from SP.
However I am using SP2010 and this is a PURE client side solution with NO server side object model at all.
Can anyone tell me how to paginate through large lists "specifically" on SP2010.
Please post some working URLs with rest commands.... which you have tested against a large list (please let me know if you need me to provide code which will create a large list of 10K items for you).
You could do it using JavaScript and the WebServices. With the JavaScript API library I created, called SharepointPlus, you'll have to do that (with version 3.0.7):
$SP().list("Name of your list").get({fields:"ID,Title",rowlimit:5000,paging:true},function(data) {
console.log(data.length)
})
You'll have all the items into the 'data' array. You can go thru the array and stop when you reach X elements in a JavaScript loop for example. Like that you should not have an error message due to the large size of your list.
$skip query option is not supported, but $top is.
Workaround, use a mix of $orderby on ID, $filter on ID and $top x items, and loop.
Your query becomes
http://mysharepoint.com/sites/testsite/_vti_bin/listdata.svc/LargeList()?$orderby=Id&$filter=Id gt {lastId}&$top=100
{lastId}=0 on the first run.
Loop the items found, keep track of the "Last Id" and pass it to the next query.
http://mysharepoint.com/sites/testsite/_vti_bin/listdata.svc/LargeList()?$orderby=Id&$filter=Id gt 0&$top=100
http://mysharepoint.com/sites/testsite/_vti_bin/listdata.svc/LargeList()?$orderby=Id&$filter=Id gt 123&$top=100
http://mysharepoint.com/sites/testsite/_vti_bin/listdata.svc/LargeList()?$orderby=Id&$filter=Id gt 345&$top=100
repeat.
Note 1: ID is a column that is always indexed, other
Note 2: This pattern also works with the REST API (LINQ query)
Related
I need to get all the items of a SharePoint list and send as JSON
But the default is 100
If I set a larger number like 10000 it gives this error on the flow:
The attempted operation is prohibited because it exceeds the list view threshold.
So I am trying to get it in batches of 350 items.
But how can I use the Filter Query to start at a position x and return y items?
I tried
$skip eq 350
But this error occurs:
Column 'skip' does not exist
I think your question has 2 layers to it.
To increase the number of items returned by the Get Items Actions from Sharepoint you need click on the three dots on the left side of the action, and then go to Settings and you will see the image below, then set the size to 5000, that means your Get Items action will now return a maximum of 5000 items.
$top and $skip - I am not too confident what you are trying to do with this but perhaps share a bit more context, it sounds like you want to build a dynamic pagination ? Please explain this part unless the first point already achieve what you were aiming for
I'm conducting a lucky draw to encourage colleagues to post on Yammer, and I want to do the following:
Colleague posts message on Yammer
Pull a Lucky Draw Participation code from SharePoint List
Send colleague an automated message with this Lucky Draw Participation code.
There are only 100 codes, and for obvious reasons, are single-use and cannot be easily reverse-engineered. So I have crafted them in advance by hand.
My question now is... what are the functions I can use can to set up a Flow, that plucks a code from a Sharepoint list (never re-uses it as well), and that this flow would cease after 100 times (i.e. all codes used up)
Thanks!
(Following is the screenshot where I was stuck at...)
Currently stuck here in MS Flow
One example:
First, add a flag to your SharePoint list that indicates the code (row) has been used, e.g., Yes/No field named "Used."
Use Get Items action that uses the ODATA filter query "Used ne 1" without quotes. That will only pull codes that have not been marked as used.
Set Top Count to 1. Only the next unused item will be retrieved.
Use Update Item action to mark that code as Used.
Use the output of the Get Items action however you see fit.
Note: You can also add a condition to send you a message that all codes have been used if the length of the output for the Get Items action is 0.
I'm not sure I have a syntax error here in my CAML Query or I am misunderstanding how the API works but I have a very large document library in which I am trying to identify items that have had no compliance / retention label set.
My query is trying to return any item in the library that does not have the compliance retention label set to "Test". This is the query but regardless of rowlimit, the console app returns the threshold error... "The attempted operation is prohibited because it exceeds the list view threshold enforced by the administrator."
<View Scope='RecursiveAll'><RowLimit>20</RowLimit><Query><Where><Neq><FieldRef Name='_ComplianceTag'/><Value Type='String'>Test</Value></Neq></Where></Query></View>
I have been trying the most basic query and I can confirm this works...
<View Scope='RecursiveAll'><RowLimit>20</RowLimit></View>
But as soon as I add element to the CAML, I get the error. It seems once the query is added it ignores my row limit. The plan is to bring the items back in small batches and apply the label. I've already built a console app which uses the SetComplianceTag to library method. This applies Tags to everything in the library but unfortunately my library is so large we found it missed a few.
Any insights would be much appreciated.
Having read further I now understand the query is executed on the entire list so limits apply regarless of the rowLimit in your CAML. I have now copied and adapted what Piyush K Singh has done in the tutorial linked below. There is a ListItemCollectionPosition object which can handle breaking your API request into batches the size specified in your row limit. I then loop through each batch looking for the items that match my specified conditions...
https://piyushksingh.com/2016/12/04/query-listitems-in-batches-sharepoint-online/
Thanks Piyush!
Currently we are getting all lists in all sub sites using below REST query.
'/_api/Web/Webs?$expand=Lists&select=Title,id,ParentWebUrl'
now we would like to get the all content types that are used in those lists as well. Tried expanding ContentTypes, but could not get it to work.
Can anyone help to get content types belong to those lists? Even if filtering by content type in the REST query it self is fine.
or is there any way to do this in search results of lists?
After searching everywhere and getting no answers here, I've come to the conclusion there is no possibility of doing this way. So, I've used SharePoint Search services to get my results.
querytext = 'path:"your site collection path" AND (ContentClass:"STS_List_GenericList" OR ContentClass:"STS_List_DocumentLibrary")';
Later I decided to add listIDs to the query as I was getting ListIDs from previous Item search.
querytext = '(ListID:"Your list1 ID" OR ListID:"Your list2 ID" OR ListID:"Your listn ID")" AND (ContentClass:"STS_List_GenericList" OR ContentClass:"STS_List_DocumentLibrary")';
The issue was now if the list IDs exceed more than 75, the querytext max character exceeds and no results was received.
After searching a way to increase the maximum limit and finding out there is no way to do it, I divided the list IDs into a set 75 IDs and send separate requests for each set, then combined all together.
You can read it how it was done here Batch Search requests
This is the REST call for a SharePoint Online tenant (you did not specify version). If you iterate your through your list and call this for each list, you can summarize what content types are on your site.
https://YOURTENNANT.sharepoint.com/sites/YOURSITE/_api/lists/getbytitle('YOURLISTNAME')/contenttypes
I have around 53,00,000 documents in MarkLogic server and I am building a simple search application. User enters a search term and MarkLogic server searches that term in all the nodes in all the documents and returns the matching documents as the result. I have implemented a custom paging to show results per page. I am showing 10 results per page.
I am using search api for this as:-
import module namespace search="http://marklogic.com/appservices/search" at "/Marklogic/appservices/search/search.xqy";
declare variable $options:=
<options xmlns="http://marklogic.com/appservices/search">
<transform-results apply="raw"/>
</options>;
search:search($p, $options, $noRecFrom, 10)/search:result
where $p is the input from the user $noRecFrom is the number which indicates from where we have to show records. For example for page 1 $noRecFrom will be 1, for page 2 $noRecFrom will be 11, for page 3 $noRecFrom will be 21 and so on. For paging there are hyperlinks to go to First, Next, Prev and Last pages.
To calculate the total number of records returned I am using:-
for $x in search:search($p, $options)
return $x//#total;
While First, Next and Prev hyperlink works perfectly but if someone clicks Last the application stops responding and the query does not show any output. Is it due to the large number of documents in the database or I am implementing it wrongly.
Is there any efficient way for pagination in MarkLogic (for search:search) so that the user can go the Last page without delay in query result for such a large database ?
The way you've implemented it, you're running the search repeatedly in your for loop. And that would indeed be slow.
Instead, you should be calculating a $start parameter based on the #total and number of documents per page, and passing that in as an argument (I think it's the third one) to search:search.
I would also recommend making sure you can run in unfiltered mode. There is good information about optimizing for fast pagination (indexes, etc) on the developer site; the idea is to resolve queries out of indexes to give very good, accurate unfiltered performance.
If you do it that
There is a tutorial on paginated search at http://developer.marklogic.com/learn/2006-09-paginated-search
Once you have resolved the issues mentioned by cwhit above, if you still want to get to the last page of data in a faster manner, you could make your code smart enough to reverse the sort order and pull the correct offset of records.
Here's another tip:
To get better insight to what MarkLogic is doing with search:search, call
search:get-default-options()
to see the starting point for common search applications.