I'm looking at the architecture for a DW project and there will be the need for some manual collection of [structured] data eg the monthly accounting results from a country manager where they need to complete a form and fill in half a dozen values etc.
I really like the idea of using SP and InfoPath for this as it gives the security, the workflow and the customisability etc that mean it can be easily deployed as the client already has SP rolled out. The bit I am less clear on is how, technically, we might interface to the SP workflows and the forms themselves. Ideally the data would end up dropped into a database and we would use our [their!] standard ETL (DataStage, possibly sat on a linux server) via ODBC and pick it up like any other datasource but I am not sure what this requires on the SP side. The alternative would be to get at the XML of the individual forms and pull the info from there.
Are these appaoches feasible? What would need to be set up on the SP side in order to make this integration as robust and seamless as possible? Can anyone point me at docs/reading matter that might give me some more background info?
Thanks,
Dex
First up, accessing sharepoint's databases is never the answer to any integration question. You should treat it as a black box.
So, how should you get the data? Web Services + HTTP. SharePoint offers a large amount of Web services to get at the data you need. If you're working with IP forms, then ultimately you will need to grab the resultant XML file from the document library and parse it to get the data you need. The Web services can be used to enumerate the IP forms, and you can use straight HTTP to grab to xml file. This is probably the approach that would be offered by most experienced sharpepoint people.
Related
I'm looking into upgrading a .net 2.0 app. The app is used by the public authorities of a certain city to keep track of expenses and generate reports and forms.
The reports and forms were generated in VS2005 using Crystal report. They follow a well defined layout, like official documents usually do.
I am looking at options to upgrade the application and the main problem I have is in determining how to deal with the crystal report files.
I have successfully upgraded to VS2008, but any version after that doesn't have CR anymore, so my company would have to pruchase CR separately and because the client and my company are both tight, I'm looking at alternatives...
The obvious one is using SSRS. I have never touched it before in my life, but after playing around with it for a bit, I get the impression that it is not very well suited to generating forms with lots of non-tabular content and lots of formatting. Or am I wrong?
It seems that every line has to be drawn separately. There is no (that I can see) accurate way of positioning lines for formatting...
But I'm just a beginner, so I might be getting this all wrong?
If that is the case, are there any other alternatives to CR and SSRS?
I was thinking of maybe having a separate MVC web site project in the solution. Have that generate the layout in html and css with data from my entity model, then view the result in a (built-in or not) web browser. Am I overcomplicating on this?
I really need advice from somebody who's done that kind of thing before.
What SSRS is good for:
Talking to SQL Server, much faster than other products as it in many cases retains the database better when in other programs IMHO they repeat query at times.
Designing collapsable grids and chart objects from datasets. You can have 'groups' that can nest aggregates of collapsed values and can be un collapsed or collapsed on demand based on expressions, parameters, or a recusive parent set.
A web service for deployment ease where you can deploy one or many objects. You can also write add ons for this service with C# and the ReportingService.asmx web service.
You can talk to the web service directly in a 'form' object in HTML and manipulate it's output.
You can schedule reports to send out via email and file saves automatically to clients or internal users.
What SSRS IS NOT GOOD FOR:
It is not event driven hardly at all except for parameters. You cannot click on many things and get other parts on the form itself to update. You may do an 'action' that goes to another location, report, or site. But in essence you are calling a seperate object, not the same instance again.
Multiple layers of reporting. Beyond tweaking tool tips you cannot do 'hover over' reporting without hacking SSRS. You can make javascript windows show other reports but it is not baked in to SSRS. So you are either clicking into new reports or tab stops in a report but not getting hover over quick objects beyond text and expressions that are in tool tips.
What do you want before considering what you need to impement?
I want to input and export things while talking to my database - ASP.NET with potentially HTML 5 or MVC4 if you want to be very new. ASP.NET is made for actively talking to a server and taking commands IN as well as OUT.
I want a form to auto update periodically on a page as a landing site and dashboard - AJAX and Javascript on top of HTML, Java or ASP.NET.
I want to create reports that exist on a Server and can be hosted on a wide variety of platforms in .NET via web service calls - SSRS.
SSRS's biggest selling point to me is it's reusability once you dial a report in. They are pretty easy to create, easy to configure, easy to deploy, and if you get a little advanced in calling the webservice you can get SSRS report objects in other technologies if you want.
There is Crystal reports for VS2010 and VS2012. It is just not shipped with them. You can download the installation from here: http://scn.sap.com/docs/DOC-7824
I am running through the same decision process at this time. There is a .NET product from a company called "Windward" that will allow you to design your reports in Microsoft Office. If you are in the MS ecosystem already or want your users to design reports instead of always calling on you, this might help.
Their template design tool is called AutoTag and you can deploy these template to their .NET based engine in a few lines of code.
I know the question is regarding SSRS vs. Crystal comparison but thought you should know there are other alternatives and some can make life easier
Ryan
I don't know about SharePoint that much, so hope that someone can give me some good explanations for me.
For SharePoint application, we don't need to consider SharePoint DB design? Everything is done through List and SharePoint will take care of the Database? Don't need to consider it at all unless we are connecting to the external Database?
For Parent Child relationship, I have to use lookup field?
If we are developing using Web parts, we need to deploy it using Features. So eventually, there can be so many features in the production? Is it recommended in this way?
1)
For SharePoint application, we don't need to consider SharePoint DB
design ? Everything is done through List and SharePoint will take care
of the Database ? Don't need to consider it at all unless we are
connecting to the external Database ?
You do need to consider the database... In term of administration. back ups, restore, size and so on.
however you never directly read or write to it. You don't need to pay attention to the schema because you won't ever be directly using it. If you want to update something in SharePoint you do it pragmatically with c# / vb.net using the SharePoint API's. You have 2 API's one that works on the server called the SharePoint object model and one that works on a remote machine called the client side object model.
From a users perspective, you are correct lists contain data in SharePoint and are somewhat smiler to DB tables in some ways.
2)
For Parent Child relationship, I have to use lookup field ?
Yip
3)
If we are developing using Web parts, we need to deploy it using
Features. So eventually, there can be so many features in the
production ? Is it recommended in this way?
This can happen if you create a new solution for every web part but you can bundle multiple web parts in the same solution in the same project in the same feature
Right click on your project --> add new item. select a web part. By default the web parts will all be part of the same feature but you can separate them out and have multiple features in the same solution if you want to
I have a list in sharepoint which maintains particular month OnCall list,and we are maintaining employee directory in sql server. My requirement is to get complete data from sql server and show it in sharepoint and compare with sharepoint list and show small icon for the employees who are On Call for that particular Month. Can anyone please suggest me the waus of implementing this.
Thanks in advance.
Update: I have finished the part where I have to connect to the sqlserver database and get the employees information. For this we are using 3rd party web part to connect to the sql server and pull the data from the table. Now I have to show some kind of image on the employee name to show that he is on-call for that week. We are going to cretae custom list for maintaing the list of people who are on-Call. Can anyone please advise me on how to accomplish this.
Write a custom webpart which will pull the data from the list using sharepoint object model and SQL server using ADO.NET and do the said comparison.
If you were looking for out of the box, I am afraid there i too little information given here to analyze if its feasible out of the box or not.
If you have the SharePoint Enterprise version, you can look at using the Business Data Catalog. This will let you bind columns to external data sources. This might provide you with the functionality you're looking for.
If you do not have the Enterprise features, do you have access to deploy WSP packages and custom code?
You will have to write your own data access to your external data source. Your options would be to have a job that pulls data from the external data source and populates SharePoint list(s) or create a custom view that pulls the external data on-demand.
You'll have to come up with synchronization strategies. Meaning, is the data in the external SQL data source static, reference information that does not need to be updated depending on what a user does in SharePoint? This seems to be the case based on your question. If you do need to update the external data source, you'll have to hook into the on save event (so probably a custom event handler that listens for ItemAdding) to update the data, validate, and optionally cancel the operation with an error message.
If you can't deploy WSP packages / DLLs, you could take a look at the jQuery SharePoint library. This will let you interact with lists using jQuery. If you also write a WCF or Web Service wrapper around the data you need access to from your external data source that is accessible from the SharePoint environment, you can use hack together a solution.
To accomplish this you'd need to place a Content Editor Web Part on the page you need custom data access. In there you will write the code to reference the jQuery javascript library and jQuery SharePoint library. The code will have to make the calls to your external data service and make any updates you need.
This is the least reliable method to accomplish what you want since it's entirely page-based and can be broken by simply disabling script or someone editing the CEWP or removing it altogether.
If you don't have access to place a CEWP or any of the other solutions, then you have no options at all.
it relatively easy now to pull all the data using the third party webpart and saving it into a custom list. I would recommend you not only creating custom list but also creating the content types for this list. take a look at SharPoint MVP's post about creating a Custom List with Content Types
I need to create reports on SQL Server Reporting Services 2005 (SSRS) on data coming from SharePoint Lists,
i was searching for ways to do that and i found,
connecting to SharePoint web service
out put XML, the problem here is it
sometimes work and sometimes don't,
and also i couldn't do inner join
between different lists may be it is
there but it will be trouble to do
it i think (if there is easy way
tell me please)
Creating a link table in MS Access
2007 that links to sharepoint lists,
and then connect to the MS ACCESS
link tables from SSRS 2005 : this
will work but i am kind of scared of
having ACCESS in the middle i always
want to avoid using access
connecting to the views provided in
the sharepoints backend database
(there is a view for all lists and
another view for all list items) :
here column names will be troublsome
to create the query the clounm names
are like
(float1,float2,...,date1,date2,date3,....)
, also i am not sure if it is good
thing to make SSRS talk directly to
sharepoints backend database views
can you tell me which of these ways is best (or if there is a better way plz tell me)
i kind of like the third way but not sure if i should use it
I have summarized this problem on our wiki, but it is not complete.
http://wiki.threewill.com/display/enterprise/Reporting+on+List+Data+in+SharePoint.
Note that we have since tried out the CorasWorks DIT and it looks like a promising approach.
We are exporting all the data to a SQL database everynight ( we have also started to move items away from Sharepoint because of performance). Then we use standard Reporting Services to create the reports....
(+) reporting doesn't hit the performance of day to day activities in Sharepoint
(-) the data is "one day" old
Check out SharePoint List Assocation Manager, SLAM.
Allows you to do exactly what you're looking to do in real time and it's free and very well supported.
http://slam.codeplex.com
Edited:
What is the easiest way to scrape extract SharePoint list data to a separate SQL Server table? One condition: you're in a work environment where you don't control the SQL Server behind the SharePoint Server, so you can't just pull from the UserData table.
Is there there any utilities that you can use to schedule a nightly extract?
Is Microsoft planning any improvement here for "SharePoint 4"?
Update Jan 06, 2009:
http://connectionstrings.com/sharepoint
For servers where office is not installed you will need:
this download
There is a SSIS SharePoint task you can use to grab the data info a regular dataflow:
http://www.codeplex.com/SQLSrvIntegrationSrv
Scraping? As in screen scraping? Are you serious? ;)
2 Options
SharePoint Object Model - http://msdn.microsoft.com/en-us/library/ms441339.aspx
SharePoint Web Services - http://msdn.microsoft.com/en-us/library/ms479390.aspx
specifically the Lists web service
The web services is how Excel/Access communicate with SharePoint to integrate with its lists.
In fact a bit of Google foo gives these two results :-
Connecting SQL Reporting Services to a SharePoint List
Accessing SharePoint List Items with SQL Server 2005 Reporting Services
The 2 minute answer is to use Data Synchronisation Studio from Simego ( http://www.simego.com ) just point it at your List and database and it will sync all the changes.
There is an ADO.NET adapter for MOSS 2007/2010 and WSS 3.0/4.0 available which goes under the name Camelot .NET Connector for Microsoft SharePoint. It enables you to query lists in SharePoint through standard SQL language, using SharePoint as a data layer.
Besides from the connector, there will be a large number of open source tools and utilities available, such as webparts for exporting data to various formats (XML, MySQL, ..), Joomla plugins, synchronization services, etc.
See http://www.bendsoft.com for more details and to watch webcasts. BendSoft is currently looking for beta-testers and encourage all feedback from the community.
Example:
SELECT * FROM My Custom SharePoint List
INSERT INTO Calendar (EventDate,EndDate,Title,Location) VALUES ('2010-11-04 08:00:00','2010-11-04 10:00:00','Morning meeting with Leia','Starbucks')
DELETE FROM Corp Images WHERE Image Name = 'marketing.jpg'
I had written a full article about this with step by step screenshot procedures. It does not use any third party components only SQL BI Tools and Sharepoint. Have a look here
http://macaalay.com/2013/11/01/how-to-archive-sharepoint-list-items-to-sql-server/
As Ryan said I would also suggest using object model / web services to store data to separate SQL database. I think that the best approach is to write an event handler that will trigger on your least and copy the data user inserted/updated.
Regarding your query about "SharePoint 4", Bill Gates made some remarks at SharePoint Conference 2008. He suggests enriching SQL tables with SharePoint data, and goes on to mention several other potentially cool things. What exactly he means and whether it will help solve your problem in the future is hard to say until we start seeing betas of WSS4 / MOSS 14.
I would go with the simego software, but i dont have the money, maybe a 15 days trial is enough!
If you have MOSS installed, the Business Data Catalog can be setup from the Sharepoint Central Administration to automagically synchronize data for you. This is a very powerful product and is included with MOSS. I love it when a client has it enabled so I can take advantage of it.
But some don't and for myself, I've found that if they don't have BDC running and available, inevitably they don't give developers many rights to SQL Server so SSIS is generally out of the question (but maybe that's just me). No problem; for those I'll pull together a lightweight EXE that runs on a scheduled task that queries Lists.asmx and pushes changes to a SQL Server table. Fairly trivial stuff for a simple list where nothing is deleted. Get yourself Visual Studio 2008, CAML Builder, and prepare for a good time. The Lists.asmx results is a little funny in that a list's row's fields are each a single node with a lot of attributes, with no child nodes ... something like this off the top of my head ... just remember that when coding ...
<z:row ows_Id="1" ows_Field1="A1" ows_Field2="B1"/>
<z:row ows_Id="1" ows_Field1="A2" ows_Field2="B2"/>
Complications in code occur with copying lists where items are deleted, or where there is a parent/child relationship between SP lists. You'd think I'd have some code to send you, but I haven't bothered putting together something I could reuse.
I'm sure there's other ways of handling it, but the scheduled task EXE so far has been reliable for me for multiple apps for multiple years.
i wrote some code to achieve it, you can find it over here
extract data from moss 2007
Depending on the exact nature of the data you need to insert, it may be possible to just use the auto generated RSS feed to get the information you want, a process will need to read the rss and formulate a query.
Otherwise a consoleapp/service could use the object model to do the same thing, but with more control over field information.
I wish something like this was much easier to do. Something that didn't need SSIS and was boiled down to a console tool that reads a xml config file for source/target/map info.
http://blogs.officezealot.com/mtblog/archive/2008/06/03/importing-list-data-into-sql.aspx