We currently have some c# code that runs and imports data from a number of opml feeds and stores it in several sql server tables.
We are working in a moss environment and I am thinking the Business Data Catalog may be able to be utilised to make this process more robust/efficient.
Can anyone suggest if it can?
Yes it can, since the BDC is there spcifically to be able to use external data within sharepoint. You will need to have a SharePoint Enterprise edition license to user the BDC though...
Related
I have Sharepoint 2007 site and have to implement filter (comboboxes) for the list of employees stored in external database.
I can develop web part with asp:DropDownList(s), data access library and asp:Repeater stuff, but don't wanna mess with paging and sorting. May be it's better to populate standard sharepoint list that will be present under my filters via my DAL ?
How would you implement such a task?
Here are my suggestions:
SharePoint List Source and Destination - this is a CodePlex project for an SSIS SharePoint adapter. We used it successfully earlier this year for transferring data between SQL Server and SharePoint.
Custom Timer Job - you could create a SPJobDefinition class that is your own mini-homespun ETL comparing the SharePoint list to the database tables and then making any necessary transfers.
Business Data Catalog (BDC) - I'm not a fan, but you might have better luck with it.
SharePoint 2010 - I'm not sure if this is an option, but I'll mention it. The BDC of SharePoint 2007 has grown into Business Connectivity Services (BCS) in SharePoint 2010. I haven't had a chance to play with it yet, but it is supposed to be much improved for accessing external data.
Asked this first on serverfault, and someone recommended that I ask here.
I'm looking for advice from anyone out there who has experience integrating SharePoint with a business intelligence application like Cognos.
Our BI team wants to be able to report on data stored in SharePoint. Their tool of choice is Cognos. What's the best way to get the data they're looking for OUT of SharePoint and into Cognos BI for analysis?
To clarify I'm NOT looking for a way to display Cognos reports in SharePoint. We want to take the list data from SharePoint and use Cognos to report on it.
Since the SharePoint database itself is extremely complex it is not recommended to access it directly. You do however have to alternatives to pulling the data out.
List RSS Feed
The simplest and easiest way would be to enable RSS on the lists you want exported and then pulling the RSS feeds into a seperate database using an external tool.
List WebService
The second option is to use the SharePoint List Web Services. These are standard ASMX webservices that expose the data inside any list to an external source. You can access any list as a Web Service as follows:
[Sharepoint Site Url] + _vti_bin/Lists.asmx.
The details on using the List Web Service is on MSDN here
Diago is right, never touch the DB. In answer to your BI question I recently responded to a similar one here Combining data from Project Server and SharePoint into a single report
I have been tasked with developing some large ERP applications (some legacy apps being rewritten and some new apps) in Sharepoint. As I've come up to speed in Sharepoint, I see the value and ease of creating team sites, and the examples I've found online and in books are all tailored to intranet department portals, or simple line-of-business apps for companies that don't have large legacy ERP systems. I have begun to believe that if one is going to create a large application that interfaces to several different legacy systems and spans several departments, that building custom webparts in Sharepoint just isn't the way to go.
Is Sharepoint a viable application framework for creating and hosting large ERP applications?
If so, can anyone please point me to references describing the architecture of such a system?
If not, can anyone please point me to references that I can cite as arguments for not using it?
As someone who has spent the last 15 years writing ERP applications I would say Sharepoint would be an extremely bad choice upon which to build an ERP product.
The Sharepoint table structures would be very ineffective for producing reports.
There is very limited capability for validating data.
There is no native support that I am aware of for maintaining the integrity of relation ships between documents.
Sharepoint works well as a portal into existing LOB applications, not as a platform to build a data-rich application on top of.
I'm currently deploying a ERP system written in Sharepoint for a client. I've been working on this for about a year now and from what I've seen, Sharepoint introduced as many obstacles as it made things more complicated.
Things made simpler:
Credentials automatically synced to AD.
Document handling and versioning out of the box.
Great Office integration.
Things that sucked:
You need to learn the Sharepoint way of doing things.
You have another dependency in your ERP.
It's pretty clunky.
I would recommend reading Real World SharePoint 2007: Indispensable Experiences From 16 MOSS and WSS MVPs before making a decision
I use SharePoint 2007 (MOSS) at work as part of a larger ERP installation. We heavily use the Business Data Catalog to interface with external systems and make their data visible and searchable within our MOSS portal.
In our architecture, CRUD operations on the ERP data are handled in our ERP line of business systems. MOSS and the BDC then pull the data out of the ERP database and display it as datagrids embedded in various portal pages. For example, the HR site has a MOSS page for tracking the current status of pending performance reports.
Another compelling feature of MOSS and the BDC is the ability to expose BDC datasources to the MOSS search service. For example, when a user searches for John Smith using MOSS search, the public ERP record for John Smith is inlined with the search results. Clicking on the link in the search results redirects the user to the right page in our ERP system, rather than taking them to a MOSS page.
We don't use MOSS exclusively as an ERP system, but we do use it as a presentation and reporting layer on top of our ERP system.
The MS Dynamics AX system utilizes Sharepoint extensively, in fact there are tool kits that allow users to build web parts and so forth that extract data directly out of the base AX objects. Here is a link that talks a little about the Sharepoint integration AX+SP. Currently there is still a significant portion of AX that does not reside within Sharepoint, but their direction appears to Sharepoint enable a significant portion of the application. I do not think that an entire ERP system could reside within the Sharepoint infrastructure, but from an MVC perspective it could certainly become your View infrastructure. It appears to me that this is the very direction MS is heading, but I'm just making hypothesis on their future plans.
Building custom webparts in SharePoint would not be the way to go for complex legacy systems.
SharePoint would still give you a lot of mileage if you created a custom navigation provider for the main navigation (based on an xml file for example). This would allow custom asp.net pages to display "like" they were still part of SharePoint, giving the illusion of one all encompassing app.
I think the only reason people want one huge application has more to do with Information Architecture, look and feel and findability than any technical architecture reasons.
A solution that creates separate websites for appropriate applications that are still skinned and linked into the SharePoint information architecture and still searchable from the main interface could solve the "need" for the "Enterprise" part of ERP, while still creating appropriate solutions for what are really separate applications.
The mental model I use is not so much building the apps "in" SharePoint, but creating a 'window" in SharePoint to expose the app.
There's Windows SharePoint Services (WSS) and then there is Microsoft Office SharePoint Server (MOSS). MOSS considerably more expensive than WSS (which ships as part of Microsoft Server licensing).
My question is: what does MOSS do that makes it worth the extra cost?
..and does Microsoft Search Server not compete with the Business Data Cache (BDC)?
Edit: The feature comparision page is helpful in illustrating the numerous features that MOSS has and WSS does not. By the looks of it, most of MOSS's feature set is Enterprise oriented.
How would you describe the differences (or additional benefits) of MOSS over WSS in a couple of sentences? In essence, what are the "big ticket" items in MOSS (and not in WSS)?
Don't assume that WSS is free in all deployment scenarios. We got a nice wake up call when we deployed WSS in a client-facing extranet configuration. One "main site" w/ a bunch of segregated, client sub-sites. Turns out we needed to buy an "intranet license" (can't remember the exact name) for the OS. This is different from the SharePoint internet connector - it actually lets you use Win 2003 w/ an unlimited number of internet users. Not hugely expensive, but it was a couple thousand dollars we weren't expecting on paying...
About WSS vs MOSS:
WSS in not a portal, it's only a collaborative plateform (there are no publishing features in WSS)
MOSS allows you to use user profils, not WSS
Search functionalities are cheap in WSS compare to MOSS (but you can extend them using Search Server Express)
Many others: Infopath, BDC, Additional WebParts, Additional site and list templates
About Search Server and BDC: They do not compete.
Search Server is the MOSS search engine striped out. So you have only search functionalities (you can index SharePoint, WebSite, FileSystem).
The BDC (Business Data Catalog) allows you to view an external business data source, such as a SQL database (not necessarily SQL Server, it can be Oracle, MySQL....) or webservices. You'll be able to view data in your portal, and integrate this data to any of your list.
The BDC also allows you to index this content source if you have SharePoint Enterprise Edition.
Whether it's worth the extra cost really depends on how many of the added features MOSS brings to the table that you're actually going to use.
The following comparison page by Microsoft will definitely help to answer your question.
Microsoft Office SharePoint Server 2007 Edition Comparison
There is a lot built in to WSS but MOSS has a ton of extra stuff as referenced in the other answer.
On the second part of your question.. Search server and Business Data Connector are quite different.. Search server is about finding things... BDC is about merging datasources to be able to use them easily in sharepoint or in connected excel sheets etc.. The focus is on what is being delivered-- search results or data.
I would say if you just need a few collaboration sites for a few internal groups, then wss is just fine. It is when you start using SharePoint for enterprise level applications and as a primary platform for development that you should consider MOSS.
Edited:
What is the easiest way to scrape extract SharePoint list data to a separate SQL Server table? One condition: you're in a work environment where you don't control the SQL Server behind the SharePoint Server, so you can't just pull from the UserData table.
Is there there any utilities that you can use to schedule a nightly extract?
Is Microsoft planning any improvement here for "SharePoint 4"?
Update Jan 06, 2009:
http://connectionstrings.com/sharepoint
For servers where office is not installed you will need:
this download
There is a SSIS SharePoint task you can use to grab the data info a regular dataflow:
http://www.codeplex.com/SQLSrvIntegrationSrv
Scraping? As in screen scraping? Are you serious? ;)
2 Options
SharePoint Object Model - http://msdn.microsoft.com/en-us/library/ms441339.aspx
SharePoint Web Services - http://msdn.microsoft.com/en-us/library/ms479390.aspx
specifically the Lists web service
The web services is how Excel/Access communicate with SharePoint to integrate with its lists.
In fact a bit of Google foo gives these two results :-
Connecting SQL Reporting Services to a SharePoint List
Accessing SharePoint List Items with SQL Server 2005 Reporting Services
The 2 minute answer is to use Data Synchronisation Studio from Simego ( http://www.simego.com ) just point it at your List and database and it will sync all the changes.
There is an ADO.NET adapter for MOSS 2007/2010 and WSS 3.0/4.0 available which goes under the name Camelot .NET Connector for Microsoft SharePoint. It enables you to query lists in SharePoint through standard SQL language, using SharePoint as a data layer.
Besides from the connector, there will be a large number of open source tools and utilities available, such as webparts for exporting data to various formats (XML, MySQL, ..), Joomla plugins, synchronization services, etc.
See http://www.bendsoft.com for more details and to watch webcasts. BendSoft is currently looking for beta-testers and encourage all feedback from the community.
Example:
SELECT * FROM My Custom SharePoint List
INSERT INTO Calendar (EventDate,EndDate,Title,Location) VALUES ('2010-11-04 08:00:00','2010-11-04 10:00:00','Morning meeting with Leia','Starbucks')
DELETE FROM Corp Images WHERE Image Name = 'marketing.jpg'
I had written a full article about this with step by step screenshot procedures. It does not use any third party components only SQL BI Tools and Sharepoint. Have a look here
http://macaalay.com/2013/11/01/how-to-archive-sharepoint-list-items-to-sql-server/
As Ryan said I would also suggest using object model / web services to store data to separate SQL database. I think that the best approach is to write an event handler that will trigger on your least and copy the data user inserted/updated.
Regarding your query about "SharePoint 4", Bill Gates made some remarks at SharePoint Conference 2008. He suggests enriching SQL tables with SharePoint data, and goes on to mention several other potentially cool things. What exactly he means and whether it will help solve your problem in the future is hard to say until we start seeing betas of WSS4 / MOSS 14.
I would go with the simego software, but i dont have the money, maybe a 15 days trial is enough!
If you have MOSS installed, the Business Data Catalog can be setup from the Sharepoint Central Administration to automagically synchronize data for you. This is a very powerful product and is included with MOSS. I love it when a client has it enabled so I can take advantage of it.
But some don't and for myself, I've found that if they don't have BDC running and available, inevitably they don't give developers many rights to SQL Server so SSIS is generally out of the question (but maybe that's just me). No problem; for those I'll pull together a lightweight EXE that runs on a scheduled task that queries Lists.asmx and pushes changes to a SQL Server table. Fairly trivial stuff for a simple list where nothing is deleted. Get yourself Visual Studio 2008, CAML Builder, and prepare for a good time. The Lists.asmx results is a little funny in that a list's row's fields are each a single node with a lot of attributes, with no child nodes ... something like this off the top of my head ... just remember that when coding ...
<z:row ows_Id="1" ows_Field1="A1" ows_Field2="B1"/>
<z:row ows_Id="1" ows_Field1="A2" ows_Field2="B2"/>
Complications in code occur with copying lists where items are deleted, or where there is a parent/child relationship between SP lists. You'd think I'd have some code to send you, but I haven't bothered putting together something I could reuse.
I'm sure there's other ways of handling it, but the scheduled task EXE so far has been reliable for me for multiple apps for multiple years.
i wrote some code to achieve it, you can find it over here
extract data from moss 2007
Depending on the exact nature of the data you need to insert, it may be possible to just use the auto generated RSS feed to get the information you want, a process will need to read the rss and formulate a query.
Otherwise a consoleapp/service could use the object model to do the same thing, but with more control over field information.
I wish something like this was much easier to do. Something that didn't need SSIS and was boiled down to a console tool that reads a xml config file for source/target/map info.
http://blogs.officezealot.com/mtblog/archive/2008/06/03/importing-list-data-into-sql.aspx