I use MS Bot Framework for creating bot for MS Teams.
I need to figure out how to implement a lookup field so that it fetches information through an odata feed from a sharepoint list
I can give little help on the SharePoint side. If possible, you might consider using a filter to limit the amount of values returned:
http://www.andrewconnell.com/blog/Applying-Filters-to-Lookup-Fields-with-the-SP2013-REST-API
SharePoint 2013 REST How to select a look up field and also filter based on look up field?
https://sharepoint.stackexchange.com/questions/118633/how-to-select-and-filter-list-items-lookup-column-with-sharepoint-2013-rest-feat/118659#118659
Depending on the size that a filtered set returns (if you do so), you might also/instead want to use paging:
https://platinumdogs.me/2013/05/14/client-and-server-driven-paging-with-the-sharepoint-rest-api/
https://sharepoint.stackexchange.com/questions/45719/paging-using-rest-odata-with-sp-2013
On the botframework side; how do you want to present to the users? Do you want to supply a limited amount of data (say 20 values, for example), then have them give feedback on whether or not that contains the data they need? If so, you can split the data returned from SharePoint into chunks and use a waterfall dialog to accomplish that. Optionally, if you paged data from SharePoint you could; get one page, query user, get another page, query user again, etc. until whatever goal was achieved.
Unfortunately, you're not giving enough information on the actual goal or what you are expecting this functionality to look like in the end.
Related
TLDR of the question:
Is it possible to use Graph to query a SharePoint list, which contains lookups that would need to be fetched from a different SharePoint list?
The "old" SharePoint API can do that in one request.
Follow up question as a result of my attempts to work around that limitation:
Why does Graph not allow me to ask for multiple list entries by ID?
This makes literally no sense to me.
Background for the question:
I've been given the task to move a small SharePoint app from the normal SharePoint API over to the Graph API, so the features could be expanded into incorporating Exchange too. I've never worked with either prior to this, so I didn't really have any idea what I was getting into.
And while I did succeed in finding equivalent queries to Graph for everything that was needed so far, do I also start to doubt that Graph is seriously intended to be used for SharePoint access.
Lists are the best example. The SharePoint API offers resolving LoopupId values when requesting multiple items.
Graph doesn't even offer that when requesting an item directly, let alone multiple.
To make things worse, after I wrote my own lookup routine that picks the columns that are lookups, and having to manually tell it where to find the values for that, I discovered that Graph won't even let me request multiple items by ID...
At first I tried to chain id eq '<id>' requests, because even $batch requests are limited to 20 individual requests, limiting the amount of items I could lookup to that at most.
But filtering 'id' is apparently unintended.
https://graph.microsoft.com/v1.0/sites/{site}/lists/{list}/items?$filter=id+eq+'67'
results in "General exception while processing", which I've never even seen as a response until this.
I then tried the in keyword:
https://graph.microsoft.com/v1.0/sites/{site}/lists/{list}/items?$filter=id+in+('67')
which results in "Invalid request".
After that I thought I could be smart an add a calculated column which copies the item's id and index on that, but guess what: can't set an index on that column in the first AND it also refuses filtering on that on top.
Not even offering the header fix for indexing on unindexed columns, nope. Outright complains that the field isn't usable.
With all this, I feel like I will have to settle for a hybrid approach, unless I'm seriously missing something here.
I thought that having to write my own LookupId resolver was bad, but being unable to even optimize the requests to return all matching items from a list in one request at least, and instead having to request every single item EACH, because filtering by id is forbidden and the ONLY access by id is singular, just gives me the feeling that Graph was never meant to be used for SharePoint lists at all.
Is there an actual question here?
Microsoft has been recommending Graph for working with SharePoint Online for a while now. Though it is true there are shortcomings at present, Microsoft is constantly investing in improving Graph whereas older methods like CSOM are deprecated and are no longer updated.
If there is any sufficient reason why you have to filter by id instead of using GET .../items/{id} as many times as needed, you may try to filter by fields/ID, but don't use $ before filter parameter for that case - it won't work, so try
GET https://graph.microsoft.com/v1.0/sites/{site}/lists/{list}/items?filter=fields/ID eq 67
with setting header Prefer=HonorNonIndexedQueriesWarningMayFailRandomly
Also, getting multiple items from large lists may cause issues, so for that case I'd add &top=5000
I am setting up a large (2000+ records) "task tracking register" using a SharePoint List, and intend to use Powerapps as the UI.
As you would imagine there numerous drop drown fields in the list which I would like to use as a filter within the Powerapp, but being that these are "Complex" fields, they are non-delegatable.
I'm lead to believe that I can avoid this by creating additional Columns in the SharePoint list that use a Flow that populates them with plain text based on the Drop-down selected.
This is a bit of pain, so I'd like to limit the quantity of these helper columns as much as possible.
Can anyone advise if a Powerapps Gallery will initially filter the results being returned using the delegateable functions first, and then perform the non-delegatable search functions on those items, or whether the inclusion of a non-delgatable search criteria means that the whole query is performed in a non-delegatable manner?
i.e.
Filter 3000 records down to 800 using delegatable search, then perform the additional filtering of those 800 on the app for the non-delegatable search criteria.
I understand that it may be possible to do this via loading the initial filtered results into a collection within the app and potentially filtering that list, but have read some conflicting information as to the efficacy of this method, so not such if this is the route I should take.
Delegation can be a challenge. Here are some methods for handling it:
Users rarely need more than a few dozen records at any time in a mobile app. Try to use delegable queries to create a Collection locally. From there, its lightning fast.
If you MUST pull in all 3k+ of your records, here's my favorite hack. Collect chunks of your data source then combine into a single collection.
If you want the function to scale (and the user's wait time) you can determine the first and last ID to dynamically build a function.
Good luck!
I have Express running in a Node.js server, which serves as a backen for my React frontend application.
The frontend application fetches data from the backend (which is stored in Mongo) through a REST call, and display this data in a table.
The amount of data is growing by the day, so I though I should look into reducing the abount of data transferred to the frontend application, so avoid unnecessary strain on the backend.
I'm not sure if this is the right way to approach this, but I've been thinking I would look into having the backen fetch a limited amount of entries, so that only these data will be displayed in the frontend table.
The problem arises with searching - when the user wants to search the data in the table, I'll need to be able to search through all entries, not just the data loaded into the table.
I guess one option would be to have the search function actually query the REST API, instead of searching the table itself.
If I'm on the right track, I guess I could implement REST API pagination, somewhere along the example found in https://refactoringfactory.wordpress.com/2012/09/08/pagination-in-node-js-and-express/. Other suggestions on how to implement pagination are welcome.
I'd very much like some input on the approach I described, and suggestions for smarter ways implement this.
EDIT: I changed the title somewhat to include "Infinite scroll pagination". This is what I'm looking to implement. At the moment I have a click on pages pagination setup, but would like to replace this for the infinite scroll pagination.
I've been thinking I would look into having the backen fetch a limited amount of entries, so that only these data will be displayed in the frontend table.
This is common practice in my experience. The term for it is "pagination." Have a look at this SO question regarding best practices for pagination in REST API's: API pagination best practices.
The problem arises with searching - when the user wants to search the data in the table, I'll need to be able to search through all entries, not just the data loaded into the table.
I guess one option would be to have the search function actually query the REST API, instead of searching the table itself.
Again, you got it. Doing small filters/searches on the client is fine for a limited number of entries, but if you need to only retrieve items matching search criteria in the first place, then adding that functionality to your REST API is the right choice.
Right, you should do
pagination: you might implement it by exposing 2 arguments in the rest endpoint for the listing
?p=<number>: page number, defaults to 1
?l=<number>: number of items per page / page length, defaults to a number maybe from 10 to 100
search: implement it by exposing 1 argument in the rest endpoint for the listing
/?q=<string>: you can define to be what you want, maybe a string that matches with one or multiple fields of the data
If you want to minimize the network traffic, you might also add one more parameter to explicitly select the fields you want to be returned, like this
/?f=<string>: string could be something like id,name,age, and so the api should return only those three fields per record.
All this parameters should be accepted by a list endpoint in your RESTful API
Example:
http://example.com/api/cars/?p=2&l=15&q=toyota&f=id,brand,model,color
I have a view in my Xpage application that contains a lot of elements. From this view I need to build a table with custom rows (I can't just display the view, I need to build the rows to display myself because I need to compute data from other database, things that you can't do directly in a view).
In order to do so I know that I can use Dataview, Datatable or repeat control (other ideas maybe?). For sure I can't bring all the data on the client, it's way too much.
I am looking for a solution that will allow me to do paging (easy to do with the pager component) but more important sorting on header click. To be clear, I need sorting for all the entries of the view and not only for the current displayed page on the client.
What can be the more efficient way to do so ? I really have a lot of data to compute so I need the fastest way to do it.
(I can create several views with different sorting criteria if needed).
Any repeating control can have pagers applied to it. Also View Panels can include data not in the current view - just set the columnName property to blank and compute the value property. bear in mind you will not be able to sort on those column though - they're not columns, they're values computed at display time.
Any computed data is only computed for the entries currently shown. So if you have 5000 entries in the view but are only displaying 30 at a time, the computed data will only be computed for the current 30.
If your users need to be able to sort on all columns and you have a lot of data, basically they have to accept that they're requirements mean all that data needs computing when they enter the view...and whenever it's updated, by themselves or any other users. That's never going to be quick, and the requirements are the issue there, not the architecture. RDBMS may be better as a back-end, if that's a requirement, as long as the data doesn't go across multiple tables. Otherwise graph database structure may be a better alternative.
The bigger question is why the users need to sort on any column. Do the users really want to sort on the fifth column and then scroll down to entries beginning with a "H"? Do they want to sort on the fourth column and scroll down to entries for May 2014? On a Notes Client, that's a traditional approach, because it's easier than filtering. But usually users know what they're looking for - they don't want entries beginning "H", they want entries where the department is HR. If that's the case, sorting on all columns and paging is not the most efficient method either from a database design or a usability point of view.
To keep the processing faster and lightweight, I use JSON with JQuery DataTables.
Depending on the Data-size and usage, JSON could be generated on the fly or scheduled basis and saved in Lotus Notes Documents or ApplicationScope variables.
$.each(data, function(i, item) {
dataTable.row.add( [data[i].something1,data[i].something2,data[i].something3])
});
You can compute a viewColumn but if you have a lot going on I wouldn't go that route.
This is where Java in XPages SHINE!
Build a Java object to represent your row. So in java use backend logic to get all the data you need. Let's say you have a report of Sales Orders for a a company. And sales orders is pulling data from different places. Your company object would have a method like:
List<salesOrder> getOrders() {}
so in the repeat you call company.getOrders() and it returns all the rows that you worked out in java and populated. So your "rowData" collection name in the repeat can access all the data you want. Just build it into a table.
But now the sorting... We've been using jQuery DataTables to do just this.. It's all client side... your repeat comes down and then the DataTables kicks in and can make everything sortable... no need to rely on views.. works great...
Now it's all client side but supports paging and works pretty decent. If you're just pumping out LOTS of records - 6,000+ then you might want to look at outputting the data as json and taking advatange of some server cacheing... We're starting to use it with some really big output.. LOTS of rows and it's working well so far. Hopefully I'll have some examples on NotesIn9.com in the near future.
I have a requirement to retrieve data from share point (I guess it is 2010, but will check with admin if relevant) and generate an excel report/chart. Say we have a bug tracking system in share point. Currently, I could create a view and see some statistics, but I need to plot a graph to see historically (every week) how the number of bugs changed. For example,
get the number of bugs filed in a specific week
do some grouping based on type/severity
based on classification get number of bugs solved that week etc.
If I can get the numbers based on date range, I may use excel to plot the graph.
After some reading, SharePoint object model come close to what I used to work with (Oracle DB). I understand it may be entirely different from tradition db and querying.
Please help me with
What is the best method to approach this?
Is there a good book/resource.
Thanks a lot,
bsr
The easiest apprach would be to LINK to the sharepoint lists using Access 2007 or 2010 and then export the data to Excel for further processling. Of course, you could also write a program that uses CAML query to access the data. Your requirement sound straightforward, unless you need to automate the reporting process, the simplest approach would be to access the lists via an access database.
You could also create a web service via REST that pulls the data directly into Excel.
SharePoint has it's own query language: CAML query, and in theory that could be used to retrieve the list you seek.
And you should be prepared for "some" trial and error.
Tools I used:
http://www.u2u.be/res/tools/camlquerybuilder.aspx
http://spud.codeplex.com/
what I understand from this question is that you have the need to put the SharePoint data to an excel file and this from within the SharePoint site? So it looks to me that you could just create a simple SharePoint web part that consists of one button "generate excel file". So when the user clicks on the button you would just query your SPList object(SharePoint object model) and you would get all the necessary data from the list (SPListItems).
This is the way that I would take. Mind you that this is offcourse custom SharePoint Development (.NET c#). There are lots of books or blogs that described how to create your own web part in SharePoint.