Importing bulk data into sharepoint - sharepoint

I have an issue with a new sharepoint install that we've recently deployed to replace an ageing content management system that I implemented a few years ago.
What I'd really like is to save my colleagues as much effort as possible by transferring the content from my CMS into sharepoint.
I'm not very good with sharepoint yet, and my development platform of choice is PHP MySQL, so basically I'm wondering if sharepoint has any facility to import sites, I can easily built filters to reformat the content in my CMS into whatever (please let it be XML) format sharepoint will accept but I have no idea if sharepoint will even let me do this.
I have limited access to the sharepoint server, although in this case I can probably negotiate more if that's the only way.
Mostly I just need some pointers - does sharepoint have any facility to do this, and where do I start doing it?
Thanks

SharePoint has the ability to import data from an Excel spreadsheet (Site Actions > Create > Import Spreadsheet).
The only problem you may run into with this method is that you don't necessarily have full control over what column types the importer uses for your data--if that's important, then it will take some trial and error.
If you're familiar with .NET and you can get access to run a program on the server, you can write a program to import data into existing lists using the SharePoint object model.

the fastest way to bulk import data into SharePoint is through the batchdata method
http://msdn.microsoft.com/en-us/library/microsoft.sharepoint.spweb.processbatchdata.aspx
it is aimed ad importing list data, but it seems there are some workarounds to make it work with publishing pages
http://social.msdn.microsoft.com/Forums/sharepoint/en-US/f8fe190d-c1ed-4e15-bda2-7792211973cc/bulk-publishing-page-creation-using-processbatchdata?forum=sharepointdevelopmentlegacy

Related

what's the correct way to get all lists of a site

I need to get all the lists used in my site and render theire items. Since i'm not allowed to use code i have to rely on something like xlst. However after several days of searching i haven't come up with a solution yet. Any idea on how to do this?
Alex depending on what you are trying to do using the client object model may work. This allows you to access SharePoint objects without writing code on the server itself. This does require writing some code but usually a farm admin won't mind it since it doesn't affect SharePoint in any way.
The javascript object model may provide you with what you want.
Here is a link that gets all lists in a SharePoint site
http://msdn.microsoft.com/en-us/library/hh185009
Then to get items from a list
http://msdn.microsoft.com/en-us/library/hh185007
You can using *.asmx Sharepoint services SharePoint 2010 Web Services. Example from Visual Studio: Connecting to SharePoint Online Web Services
and
from Sharepoint Designer: Connect to another library in SharePoint Designer 2010
Good luck!

Does SharePoint Support VBA?

I have read very little content regarding Sharepoint (SP), and most of my reading has been sales pitch oriented overview material. I utilitze VBA with Office apps - especially Access - on a regular basis, and I am wondering if there is any translatable way to retain the custom functionality of writing my own VBA within Sharepoint, especially with MS Access.
I have read that Access databases can be run on SP, with tbales to list and forms to InfoPath, but I am assuming they are primarily talking about Access database apps that were built with wizards, which consist mainly of bound objects without explicitly-defined code.
Most of my app are primarily code driven with VBA because of my automation requirements, which I rely on to perform my tasks. Am I going to be able to accomplish the same thing within SP, and could anyone please provide any references on the subject, specifically?
You can use Access to distribute your front end to users, regardless of how much VBA it has, but an app with VBA code in it will not convert to run in the browser as a Web Database within Sharepoint 2010's Access Services. For that to work, you have to use the new, more powerful macros and limit yourself to the features supported by web objects. For an existing app, this means rebuilding every object from scratch.
Do you need to run your Access app in a web browser? If not, then you're barking up the wrong tree here.
AFAIK Sharepoint does not support VBA.
If you publish an Access database to SharePoint as a web database it cannot use VBA, however you can create a hybrid with the tables in SharePoint and the frontend in Access, that way you can have as much VBA etc as you want and still have the advantages of your data being stored in the SharePoint SQL server. You can store the frontend on SharePoint and have users download it through SharePoint .
The alternative is to keep a traditional Access database on the SharePoint share and access it via webDAV rather than the SharePoint web interface. You could map the SharePoint library as a local drive to make it easy.
Note that drive mapping is considered a legacy technology and will no longer be supported by Windows 11 due to the demise of IE11.

Using ETL (non-MS) to get data from Infopath forms stored in Sharepoint 2007

I'm looking at the architecture for a DW project and there will be the need for some manual collection of [structured] data eg the monthly accounting results from a country manager where they need to complete a form and fill in half a dozen values etc.
I really like the idea of using SP and InfoPath for this as it gives the security, the workflow and the customisability etc that mean it can be easily deployed as the client already has SP rolled out. The bit I am less clear on is how, technically, we might interface to the SP workflows and the forms themselves. Ideally the data would end up dropped into a database and we would use our [their!] standard ETL (DataStage, possibly sat on a linux server) via ODBC and pick it up like any other datasource but I am not sure what this requires on the SP side. The alternative would be to get at the XML of the individual forms and pull the info from there.
Are these appaoches feasible? What would need to be set up on the SP side in order to make this integration as robust and seamless as possible? Can anyone point me at docs/reading matter that might give me some more background info?
Thanks,
Dex
First up, accessing sharepoint's databases is never the answer to any integration question. You should treat it as a black box.
So, how should you get the data? Web Services + HTTP. SharePoint offers a large amount of Web services to get at the data you need. If you're working with IP forms, then ultimately you will need to grab the resultant XML file from the document library and parse it to get the data you need. The Web services can be used to enumerate the IP forms, and you can use straight HTTP to grab to xml file. This is probably the approach that would be offered by most experienced sharpepoint people.

Creating a new CMS, importing content from ASP - Umbraco or Sharepoint?

I'm working on a project to create a CMS, which will entail importing a lot of existing content, most of which is static, but in ASP (so they're not all just pure HTML, there are includes and sometimes other server-side code).
We're considering using Umbraco or Sharepoint (2010) for managing the external content, which currently comprises a few thousand pages. I've read this and I think there are good cases to be made for both sides. However, while I've read about the features of adding and managing content, I have not seen anything regarding the importing of existing content into either. And since we have a lot of content that will have to be imported, the ability of either CMS to facilitate this will be a major factor in the decision.
I'd like to know if anyone has any experience trying to import a lot of content into either Umbraco or Sharepoint, or if you have any idea how I might go about doing that. Is it easy for either? Are there plug-ins I can find, or scripts I can write? Or will I pretty much have to import each existing file manually with either CMS?
If you have experience with Umbraco or Sharepoint and have any ideas about this, I would value your input and/or recommendations.
Are you just using SharePoint as a CMS? IMHO whilst SharePoint can be used as a CMS that is not where its real strengths lie - its more suited to Intranet/Portals/Collaboration tools.
I am sure someone will be on in a minute with links to SharePoint showcase sites but the disadvantages :-
Its expensive (even with 'free' WSS
version you need Internet connector
license) + windows licenses.
The
markup can be fairly 'heavy' and
difficult to customise (tables galore in 2007
and javascript files measured in
hundreds of kb)
Questionable cross
browser functionality in 2007
Relatively poor 'website' features e.g. blogging engine as compared to some dedicated CMS's
Basically - if all you are after is a CMS then perhaps there are better options?
(I should say that I think that in an Intranet/Portal setting SharePoint is brilliant, frustrating sometimes for sure, but brilliant).
I cannot speak for SharePoint but I have had to import content from a MS Content Management Server 2002 database into Umbraco.
Umbraco is very extensible and I was able to build a dashboard component that allowed me to do this.
It effectively examined the MSCMS channels and postings and recreated the structure using Umbraco document types. It was very much working at the API level but I would say the learning curve wasn't too steep and Umbraco documentation has come on leaps and bounds over the last two years.
There is also the possiblity that someone has already written a package to do what you need to do so it is worth checking out the community at http://our.umbraco.org.
Hi I don't know Sharepoint but I build a package for Umbraco which can help you importing data from other systems into Umbraco. In the way you dewscribe it you could export your site to a file format using the HTML Agility pack and then use my tool www.cmsimport.com to import the data into Umbraco.
Hope this helps,
Richard
I haven't done any Sharepoint but I've imported content into Umbraco and found it very flexible.
I imported data from a database and created doctypes and custom datatypes in Umbraco then created and populated umbraco documents with code like this:
using umbraco.cms.businesslogic.web;
...
DocumentType dt = DocumentType.GetByAlias("myDoc");
Document doc = Document.MakeNew(name, dt, user, parentId);
doc.getProperty("whatever").Value = getWhateverXML();
doc.Save();

Easiest way to extract SharePoint list data to a separate SQL Server table?

Edited:
What is the easiest way to scrape extract SharePoint list data to a separate SQL Server table? One condition: you're in a work environment where you don't control the SQL Server behind the SharePoint Server, so you can't just pull from the UserData table.
Is there there any utilities that you can use to schedule a nightly extract?
Is Microsoft planning any improvement here for "SharePoint 4"?
Update Jan 06, 2009:
http://connectionstrings.com/sharepoint
For servers where office is not installed you will need:
this download
There is a SSIS SharePoint task you can use to grab the data info a regular dataflow:
http://www.codeplex.com/SQLSrvIntegrationSrv
Scraping? As in screen scraping? Are you serious? ;)
2 Options
SharePoint Object Model - http://msdn.microsoft.com/en-us/library/ms441339.aspx
SharePoint Web Services - http://msdn.microsoft.com/en-us/library/ms479390.aspx
specifically the Lists web service
The web services is how Excel/Access communicate with SharePoint to integrate with its lists.
In fact a bit of Google foo gives these two results :-
Connecting SQL Reporting Services to a SharePoint List
Accessing SharePoint List Items with SQL Server 2005 Reporting Services
The 2 minute answer is to use Data Synchronisation Studio from Simego ( http://www.simego.com ) just point it at your List and database and it will sync all the changes.
There is an ADO.NET adapter for MOSS 2007/2010 and WSS 3.0/4.0 available which goes under the name Camelot .NET Connector for Microsoft SharePoint. It enables you to query lists in SharePoint through standard SQL language, using SharePoint as a data layer.
Besides from the connector, there will be a large number of open source tools and utilities available, such as webparts for exporting data to various formats (XML, MySQL, ..), Joomla plugins, synchronization services, etc.
See http://www.bendsoft.com for more details and to watch webcasts. BendSoft is currently looking for beta-testers and encourage all feedback from the community.
Example:
SELECT * FROM My Custom SharePoint List
INSERT INTO Calendar (EventDate,EndDate,Title,Location) VALUES ('2010-11-04 08:00:00','2010-11-04 10:00:00','Morning meeting with Leia','Starbucks')
DELETE FROM Corp Images WHERE Image Name = 'marketing.jpg'
I had written a full article about this with step by step screenshot procedures. It does not use any third party components only SQL BI Tools and Sharepoint. Have a look here
http://macaalay.com/2013/11/01/how-to-archive-sharepoint-list-items-to-sql-server/
As Ryan said I would also suggest using object model / web services to store data to separate SQL database. I think that the best approach is to write an event handler that will trigger on your least and copy the data user inserted/updated.
Regarding your query about "SharePoint 4", Bill Gates made some remarks at SharePoint Conference 2008. He suggests enriching SQL tables with SharePoint data, and goes on to mention several other potentially cool things. What exactly he means and whether it will help solve your problem in the future is hard to say until we start seeing betas of WSS4 / MOSS 14.
I would go with the simego software, but i dont have the money, maybe a 15 days trial is enough!
If you have MOSS installed, the Business Data Catalog can be setup from the Sharepoint Central Administration to automagically synchronize data for you. This is a very powerful product and is included with MOSS. I love it when a client has it enabled so I can take advantage of it.
But some don't and for myself, I've found that if they don't have BDC running and available, inevitably they don't give developers many rights to SQL Server so SSIS is generally out of the question (but maybe that's just me). No problem; for those I'll pull together a lightweight EXE that runs on a scheduled task that queries Lists.asmx and pushes changes to a SQL Server table. Fairly trivial stuff for a simple list where nothing is deleted. Get yourself Visual Studio 2008, CAML Builder, and prepare for a good time. The Lists.asmx results is a little funny in that a list's row's fields are each a single node with a lot of attributes, with no child nodes ... something like this off the top of my head ... just remember that when coding ...
<z:row ows_Id="1" ows_Field1="A1" ows_Field2="B1"/>
<z:row ows_Id="1" ows_Field1="A2" ows_Field2="B2"/>
Complications in code occur with copying lists where items are deleted, or where there is a parent/child relationship between SP lists. You'd think I'd have some code to send you, but I haven't bothered putting together something I could reuse.
I'm sure there's other ways of handling it, but the scheduled task EXE so far has been reliable for me for multiple apps for multiple years.
i wrote some code to achieve it, you can find it over here
extract data from moss 2007
Depending on the exact nature of the data you need to insert, it may be possible to just use the auto generated RSS feed to get the information you want, a process will need to read the rss and formulate a query.
Otherwise a consoleapp/service could use the object model to do the same thing, but with more control over field information.
I wish something like this was much easier to do. Something that didn't need SSIS and was boiled down to a console tool that reads a xml config file for source/target/map info.
http://blogs.officezealot.com/mtblog/archive/2008/06/03/importing-list-data-into-sql.aspx

Resources