Pulling Data from SharePoint Programmatically - sharepoint

I want to pull data from a SharePoint List on a daily basis into a csv file. However, I do not want to use server side PowerShell to acheive this, as I am not a site administrator.

If you have a SQL Server at hand, you could use SSIS it's explicitly designed for this kind of task.
http://dataqueen.unlimitedviz.com/2011/06/how-to-use-a-sharepoint-list-as-a-data-source-in-your-ssis-package/

Then you should use the Client-side object model. There are a lot of samples/examples around SO. So you use CSOM to read the SharePoint lists and use Office interop or XML office language to create Excel files.

Related

Using ExcelServices in Office 365?

I use Office 365 plan E3 and I want to copy and modify an excel file within a document library on the server. Goal is to read some values from a listitem and update the excel with these values. In the excel file there are several formulas that do the reporting the customer wants.
I already tried developing a sandboxed solution with excelservices.dll referenced, but SharePoint doesn't allow to execute this solution because of partial trust problems.
I also tried to generate a proxy for ExcelServices.asmx, but WCF needs a configuration file and in O365 I can't change web.config to add the config-sections - and it's not so much fun to generate and parse soap messages manually.
Are there any suggestions how to solve this or is it still impossible?
SharePoint have some embedded function of Excel services - maybe You can use webpart like Excel lists or something similar?

Does SharePoint Support VBA?

I have read very little content regarding Sharepoint (SP), and most of my reading has been sales pitch oriented overview material. I utilitze VBA with Office apps - especially Access - on a regular basis, and I am wondering if there is any translatable way to retain the custom functionality of writing my own VBA within Sharepoint, especially with MS Access.
I have read that Access databases can be run on SP, with tbales to list and forms to InfoPath, but I am assuming they are primarily talking about Access database apps that were built with wizards, which consist mainly of bound objects without explicitly-defined code.
Most of my app are primarily code driven with VBA because of my automation requirements, which I rely on to perform my tasks. Am I going to be able to accomplish the same thing within SP, and could anyone please provide any references on the subject, specifically?
You can use Access to distribute your front end to users, regardless of how much VBA it has, but an app with VBA code in it will not convert to run in the browser as a Web Database within Sharepoint 2010's Access Services. For that to work, you have to use the new, more powerful macros and limit yourself to the features supported by web objects. For an existing app, this means rebuilding every object from scratch.
Do you need to run your Access app in a web browser? If not, then you're barking up the wrong tree here.
AFAIK Sharepoint does not support VBA.
If you publish an Access database to SharePoint as a web database it cannot use VBA, however you can create a hybrid with the tables in SharePoint and the frontend in Access, that way you can have as much VBA etc as you want and still have the advantages of your data being stored in the SharePoint SQL server. You can store the frontend on SharePoint and have users download it through SharePoint .
The alternative is to keep a traditional Access database on the SharePoint share and access it via webDAV rather than the SharePoint web interface. You could map the SharePoint library as a local drive to make it easy.
Note that drive mapping is considered a legacy technology and will no longer be supported by Windows 11 due to the demise of IE11.

comparing sharepoint list and sqlserver table

I have a list in sharepoint which maintains particular month OnCall list,and we are maintaining employee directory in sql server. My requirement is to get complete data from sql server and show it in sharepoint and compare with sharepoint list and show small icon for the employees who are On Call for that particular Month. Can anyone please suggest me the waus of implementing this.
Thanks in advance.
Update: I have finished the part where I have to connect to the sqlserver database and get the employees information. For this we are using 3rd party web part to connect to the sql server and pull the data from the table. Now I have to show some kind of image on the employee name to show that he is on-call for that week. We are going to cretae custom list for maintaing the list of people who are on-Call. Can anyone please advise me on how to accomplish this.
Write a custom webpart which will pull the data from the list using sharepoint object model and SQL server using ADO.NET and do the said comparison.
If you were looking for out of the box, I am afraid there i too little information given here to analyze if its feasible out of the box or not.
If you have the SharePoint Enterprise version, you can look at using the Business Data Catalog. This will let you bind columns to external data sources. This might provide you with the functionality you're looking for.
If you do not have the Enterprise features, do you have access to deploy WSP packages and custom code?
You will have to write your own data access to your external data source. Your options would be to have a job that pulls data from the external data source and populates SharePoint list(s) or create a custom view that pulls the external data on-demand.
You'll have to come up with synchronization strategies. Meaning, is the data in the external SQL data source static, reference information that does not need to be updated depending on what a user does in SharePoint? This seems to be the case based on your question. If you do need to update the external data source, you'll have to hook into the on save event (so probably a custom event handler that listens for ItemAdding) to update the data, validate, and optionally cancel the operation with an error message.
If you can't deploy WSP packages / DLLs, you could take a look at the jQuery SharePoint library. This will let you interact with lists using jQuery. If you also write a WCF or Web Service wrapper around the data you need access to from your external data source that is accessible from the SharePoint environment, you can use hack together a solution.
To accomplish this you'd need to place a Content Editor Web Part on the page you need custom data access. In there you will write the code to reference the jQuery javascript library and jQuery SharePoint library. The code will have to make the calls to your external data service and make any updates you need.
This is the least reliable method to accomplish what you want since it's entirely page-based and can be broken by simply disabling script or someone editing the CEWP or removing it altogether.
If you don't have access to place a CEWP or any of the other solutions, then you have no options at all.
it relatively easy now to pull all the data using the third party webpart and saving it into a custom list. I would recommend you not only creating custom list but also creating the content types for this list. take a look at SharPoint MVP's post about creating a Custom List with Content Types

Importing bulk data into sharepoint

I have an issue with a new sharepoint install that we've recently deployed to replace an ageing content management system that I implemented a few years ago.
What I'd really like is to save my colleagues as much effort as possible by transferring the content from my CMS into sharepoint.
I'm not very good with sharepoint yet, and my development platform of choice is PHP MySQL, so basically I'm wondering if sharepoint has any facility to import sites, I can easily built filters to reformat the content in my CMS into whatever (please let it be XML) format sharepoint will accept but I have no idea if sharepoint will even let me do this.
I have limited access to the sharepoint server, although in this case I can probably negotiate more if that's the only way.
Mostly I just need some pointers - does sharepoint have any facility to do this, and where do I start doing it?
Thanks
SharePoint has the ability to import data from an Excel spreadsheet (Site Actions > Create > Import Spreadsheet).
The only problem you may run into with this method is that you don't necessarily have full control over what column types the importer uses for your data--if that's important, then it will take some trial and error.
If you're familiar with .NET and you can get access to run a program on the server, you can write a program to import data into existing lists using the SharePoint object model.
the fastest way to bulk import data into SharePoint is through the batchdata method
http://msdn.microsoft.com/en-us/library/microsoft.sharepoint.spweb.processbatchdata.aspx
it is aimed ad importing list data, but it seems there are some workarounds to make it work with publishing pages
http://social.msdn.microsoft.com/Forums/sharepoint/en-US/f8fe190d-c1ed-4e15-bda2-7792211973cc/bulk-publishing-page-creation-using-processbatchdata?forum=sharepointdevelopmentlegacy

Easiest way to extract SharePoint list data to a separate SQL Server table?

Edited:
What is the easiest way to scrape extract SharePoint list data to a separate SQL Server table? One condition: you're in a work environment where you don't control the SQL Server behind the SharePoint Server, so you can't just pull from the UserData table.
Is there there any utilities that you can use to schedule a nightly extract?
Is Microsoft planning any improvement here for "SharePoint 4"?
Update Jan 06, 2009:
http://connectionstrings.com/sharepoint
For servers where office is not installed you will need:
this download
There is a SSIS SharePoint task you can use to grab the data info a regular dataflow:
http://www.codeplex.com/SQLSrvIntegrationSrv
Scraping? As in screen scraping? Are you serious? ;)
2 Options
SharePoint Object Model - http://msdn.microsoft.com/en-us/library/ms441339.aspx
SharePoint Web Services - http://msdn.microsoft.com/en-us/library/ms479390.aspx
specifically the Lists web service
The web services is how Excel/Access communicate with SharePoint to integrate with its lists.
In fact a bit of Google foo gives these two results :-
Connecting SQL Reporting Services to a SharePoint List
Accessing SharePoint List Items with SQL Server 2005 Reporting Services
The 2 minute answer is to use Data Synchronisation Studio from Simego ( http://www.simego.com ) just point it at your List and database and it will sync all the changes.
There is an ADO.NET adapter for MOSS 2007/2010 and WSS 3.0/4.0 available which goes under the name Camelot .NET Connector for Microsoft SharePoint. It enables you to query lists in SharePoint through standard SQL language, using SharePoint as a data layer.
Besides from the connector, there will be a large number of open source tools and utilities available, such as webparts for exporting data to various formats (XML, MySQL, ..), Joomla plugins, synchronization services, etc.
See http://www.bendsoft.com for more details and to watch webcasts. BendSoft is currently looking for beta-testers and encourage all feedback from the community.
Example:
SELECT * FROM My Custom SharePoint List
INSERT INTO Calendar (EventDate,EndDate,Title,Location) VALUES ('2010-11-04 08:00:00','2010-11-04 10:00:00','Morning meeting with Leia','Starbucks')
DELETE FROM Corp Images WHERE Image Name = 'marketing.jpg'
I had written a full article about this with step by step screenshot procedures. It does not use any third party components only SQL BI Tools and Sharepoint. Have a look here
http://macaalay.com/2013/11/01/how-to-archive-sharepoint-list-items-to-sql-server/
As Ryan said I would also suggest using object model / web services to store data to separate SQL database. I think that the best approach is to write an event handler that will trigger on your least and copy the data user inserted/updated.
Regarding your query about "SharePoint 4", Bill Gates made some remarks at SharePoint Conference 2008. He suggests enriching SQL tables with SharePoint data, and goes on to mention several other potentially cool things. What exactly he means and whether it will help solve your problem in the future is hard to say until we start seeing betas of WSS4 / MOSS 14.
I would go with the simego software, but i dont have the money, maybe a 15 days trial is enough!
If you have MOSS installed, the Business Data Catalog can be setup from the Sharepoint Central Administration to automagically synchronize data for you. This is a very powerful product and is included with MOSS. I love it when a client has it enabled so I can take advantage of it.
But some don't and for myself, I've found that if they don't have BDC running and available, inevitably they don't give developers many rights to SQL Server so SSIS is generally out of the question (but maybe that's just me). No problem; for those I'll pull together a lightweight EXE that runs on a scheduled task that queries Lists.asmx and pushes changes to a SQL Server table. Fairly trivial stuff for a simple list where nothing is deleted. Get yourself Visual Studio 2008, CAML Builder, and prepare for a good time. The Lists.asmx results is a little funny in that a list's row's fields are each a single node with a lot of attributes, with no child nodes ... something like this off the top of my head ... just remember that when coding ...
<z:row ows_Id="1" ows_Field1="A1" ows_Field2="B1"/>
<z:row ows_Id="1" ows_Field1="A2" ows_Field2="B2"/>
Complications in code occur with copying lists where items are deleted, or where there is a parent/child relationship between SP lists. You'd think I'd have some code to send you, but I haven't bothered putting together something I could reuse.
I'm sure there's other ways of handling it, but the scheduled task EXE so far has been reliable for me for multiple apps for multiple years.
i wrote some code to achieve it, you can find it over here
extract data from moss 2007
Depending on the exact nature of the data you need to insert, it may be possible to just use the auto generated RSS feed to get the information you want, a process will need to read the rss and formulate a query.
Otherwise a consoleapp/service could use the object model to do the same thing, but with more control over field information.
I wish something like this was much easier to do. Something that didn't need SSIS and was boiled down to a console tool that reads a xml config file for source/target/map info.
http://blogs.officezealot.com/mtblog/archive/2008/06/03/importing-list-data-into-sql.aspx

Resources