SharePoint: Script to automatically import documents from a network share on a regular schedule - sharepoint

Question

I am new to SharePoint development.
I am trying automate the task of:
Uploading documents from a network share to a SharePoint document library.
The related metadata for each of the files also need to be brought into the SharePoint columns. (will taken from a single CSV file in the same location with filenames mapped in it)
This process needs to automatically run on a predefined schedule (eg: daily/once an hour etc)
I want to know:
What is the best way to achieve this? (PowerShell script running on Windows task scheduler, creating a service etc.)
Starting point for coding or any code already available since it looks like this might be a very common requirement.

Answer 1

Typically StackOverflow is the wrong place to ask this kind of question. But, since I just finished creating a script for uploading files to SharePoint from a CSV, I thought I can share it.
Disclaimer: I offer no guarantees that this will work with your environment, or that it will not mess things up. Review. Understand it. Use at own risk.
The script can be easily changed to add/change columns, (under the #Add Column Information section)
CSV File format:
SubFolder,Name,Column1,Column2
Where SubFolder can be blank if your source file is on the root of the Source Folder. Name is the file name, and Column1 and Column2 are SharePoint columns.
Save the below code to the two PowerShell files. ImportSPfromCSV.ps1 contains the logic, Run_ImportSPfromCSV.ps1 contains the code to run it.
To run, set up a scheduled task on a SharePoint server (or server that has the SharePoint PowerShell module on it) and run with the command:
powershell.exe -file "C:\scripts\Run_ImportSPfromCSV.ps1"
Run_ImportSPfromCSV.ps1
# Load Script
. .\ImportSPfromCSV.ps1
# Run Function
Import-SPData -CSVFilePath "CSVFile.csv" -SourceFolder "C:\temp" -SiteURL "http://SharePointWebApp/sites/SiteCollection" -Library "Shared Documents" -Approve $true -CheckIn $true -Overwrite $false -OverwriteFields $true
ImportSPfromCSV.ps1
<#
.SYNOPSIS
Imports items into SharePoint library from a CSV File
.DESCRIPTION
Imports items that are saved to disk into SharePoint library
using and tagging the information from a CSV File
.NOTES
NAME: ImportSPfromCSV.ps1
AUTHOR: Kristoph Minchau
CREATED: May 2, 2013
LASTEDIT: 2014-03-06
VERSION: 1.10
KEYWORDS: SharePoint
REVISION LOG:
[2013/05/01] - km - v1.00 ** Created **
[2014-03-06] - km - v1.10 * Minor edits
.EXAMPLE
# Load Script
. .\ImportSPfromCSV.ps1
# Run Function
Import-SPData -CSVFilePath "CSVFile.csv" -SourceFolder "C:\temp" -SiteURL "http://SharePointWebApp/sites/SiteCollection" -Library "Shared Documents" -Approve $true -CheckIn $true -Overwrite $false -OverwriteFields $true
.LINK
Import-SPData
#>
Add-PSSnapin Microsoft.SharePoint.Powershell -ErrorAction SilentlyContinue
<#
.SYNOPSIS
Get SharePoint file
.DESCRIPTION
Gets a SharePoint File
.PARAMETER SPFileName
The file name to Upload to the SharePoint library if different than original File name
ex. "NewName.docx"
[String]
.PARAMETER Web
The SharePoint web site
[Microsoft.SharePoint.SPWeb]
.PARAMETER Library
The Destination Library on SharePoint
Default value = "Retail Banking"
.INPUTS
System.String SPFileName
.OUTPUTS
System.String Return SPFile
.EXAMPLE
$Web = Get-SPWeb "http://SharePointWebApp/sites/SiteCollection"
Get-SPFile -Web $Web -Library "Shared Documents" -SPFileName "File1.doc"
.LINK
Get-SPFile
#>
Function Get-SPFile {
[CmdletBinding()]
Param(
[Parameter(Position=0, Mandatory=$true, ValueFromPipeline=$true, ValueFromPipelineByPropertyName=$true)]
[string]$SPFileName,
[Parameter(Position=1, Mandatory=$true)]
[Microsoft.SharePoint.SPWeb]$Web,
[Parameter(Position=2, Mandatory=$true)]
[string]$Library = "Shared Documents"
) #end param
Process
{
Try
{
# Get Library
$docLibrary = $web.Lists[$Library]
$folder = $docLibrary.RootFolder
# Assign the new document name, and build the destination SharePoint path
$SPFilePath = ($folder.URL + "/" + $SPFileName)
#Check if the file exists and the overwrite option is selected before adding the file
$spFile = $web.GetFile($SPFilePath)
#Return SP File
Write-Output $spFile
} #end Try
Catch
{
Write-Error $Error[0]
}
} #end Process
} #end Get-SPFile
<#
.SYNOPSIS
Adds a file to SharePoint Document Library.
.DESCRIPTION
Adds a file to SharePoint Document Library, optionally checks it in, approve it, or overwrite it.
.PARAMETER FilePath
The file to Upload
ex. "c:\Test.docx"
[String]
.PARAMETER SPFileName
The file name to Upload to the SharePoint library.
Rename the File to something that is SharePoint/URL friendly (e.g. remove special characters "/?=<>" etc.)
ex. "NewName.docx"
[String]
.PARAMETER Web
The SharePoint web site
[Microsoft.SharePoint.SPWeb]
.PARAMETER Library
The Destination Library on SharePoint
[String]
.PARAMETER Approve
If approval workflow is enabled, when uploading the document, approve the uploaded document
[bool] Default value = $true
.PARAMETER CheckIn
If workflow is enabled, when uploading the document, CheckIn the uploaded document
[bool] Default value = $true
.PARAMETER Overwrite
When uploading the document, if a document with the same name is encountered, overwrite the document?
[bool] Default value = $false
.INPUTS
N/A - Pipeline Inputs
.OUTPUTS
SP File
.EXAMPLE
$Web = Get-SPWeb "http://SharePointWebApp/sites/SiteCollection"
Add-SPFileToLibrary -File "Test.docx" -SPFileName "NewName.docx" -Web $Web -Library "Shared Documents" -Approve $true -CheckIn $true -Overwrite $false
.LINK
Add-SPFileToLibrary
#>
Function Add-SPFileToLibrary {
[CmdletBinding()]
Param(
[Parameter(Position=0, Mandatory=$true, ValueFromPipeline=$true, ValueFromPipelineByPropertyName=$true)]
[string]$FilePath,
[Parameter(Position=1, Mandatory=$true)]
[string]$SPFileName,
[Parameter(Position=2, Mandatory=$true)]
[Microsoft.SharePoint.SPWeb]$Web,
[Parameter(Position=3, Mandatory=$true)]
[string]$Library,
[Parameter(Position=4, Mandatory=$true)]
[bool]$Approve = $true,
[Parameter(Position=5, Mandatory=$true)]
[bool]$CheckIn = $true,
[Parameter(Position=6, Mandatory=$true)]
[bool]$Overwrite = $false
) #end param
Process
{
Try
{
# Get Library
$docLibrary = $web.Lists[$Library]
$folder = $docLibrary.RootFolder
# Assign the new document name, and build the destination SharePoint path
$SPFilePath = ($folder.URL + "/" + $SPFileName)
#Check if the file exists and the overwrite option is selected before adding the file
if ((!$web.GetFile($SPFilePath).Exists) -or ($Overwrite))
{
$Message = "Uploading... $FilePath ...To... $SPFilePath"
Write-Debug $Message
Write-Verbose $Message
#Support for -WhatIf parameter
if($PsCmdlet.ShouldProcess($Message))
{
Write-Host $Message
#Upload file
$File = Get-ChildItem $FilePath
$spFile = $folder.Files.Add($SPFilePath, $File.OpenRead(), $true) #Upload with overwrite = $true (SP file path, File stream, Overwrite?)
$spItem = $spFile.Item
#Check in file to document library (if required)
#MinorCheckIn=0, MajorCheckIn=1, OverwriteCheckIn=2
If ($CheckIn) {
If ($spFile.CheckOutStatus -ne "None") {
If ($Overwrite){
#$spFile.CheckIn("File copied from " + $filePath, 2)
$spFile.CheckIn("Version 1.0", 2)
}
Else
{
#$spFile.CheckIn("File copied from " + $filePath, 1)
$spFile.CheckIn("Version 1.0", 1)
}
Write-Host "$spfile.Name Checked In"
}
} #end CheckIn
#Approve file (if required)
If ($Approve)
{
If ($spItem.ListItems.List.EnableModeration -eq $true)
{
#$spFile.Approve("File automatically approved after copying from " + $filePath)
$spFile.Approve("Approved")
If ($spFile.Item["Approval Status"] -eq 0)
{
Write-Host "$spfile.Name Approved"
}
Else
{
Write-Host -Background Yellow -Foreground Black "Warning: $spfile.Name Not Approved"
}
}
} #end Approve
#Return SP File
Write-Output $spFile
}# end upload file
}
Else
{
If($web.GetFile($SPFilePath).Exists)
{
Write-Verbose "File Exists and Overwrite = $false returning file"
# File Exists and Overwrite = $false, Try finding and returning the file
$spFile = Get-SPFile -Web $web -Library $Library -SPFileName $SPFileName
#Return SP File
Write-Output $spFile
}
else
{
# File does not exist, and Overwrite = $false, we can't return anything, Throw Warning
Write-Host -Background Yellow -Foreground Black "Warning: $SPFilePath does not Exist and Overwrite is set to False. No SP File handle returned"
Write-Output $null
}
}# end File Exists and Overwrite Check
} #end Try
Catch
{
Write-Error $Error[0]
}
} #end Process
} #end Add-SPFileToLibrary
<#
.SYNOPSIS
Imports files into SharePoint library from a CSV File
.DESCRIPTION
Imports files that are saved to disk into SharePoint library
using and tagging the information from a CSV File
.PARAMETER CSVFilePath
The CSV import file to use to import the data from
Default value = "CSVFile.csv"
.PARAMETER SourceFolder
The Source Folder for locating the Public Folder files
Default value = "."
.PARAMETER SiteURL
The Site URL on SharePoint
Default value = "http://SharePointWebApp/sites/SiteCollection"
.PARAMETER Library
The Destination Library on SharePoint
Default value = "Shared Documents"
.PARAMETER Approve
If approval workflow is enabled, when uploading the document, approve the uploaded document
[bool] Default value = $true
.PARAMETER CheckIn
If workflow is enabled, when uploading the document, CheckIn the uploaded document
[bool] Default value = $true
.PARAMETER Overwrite
When uploading the document, if a document with the same name is encountered, overwrite the document?
[bool] Default value = $false
.PARAMETER OverwriteFields
If the document is already uploaded, overwrite the fields with the fields from the CSV file
[bool] Default value = $true
.INPUTS
N/A - Pipeline Inputs
.OUTPUTS
System.String Returns
.EXAMPLE
#Default Initial Import of all data
Import-SPData -CSVFilePath "CSVFile.csv" -SourceFolder "C:\temp" -SiteURL "http://SharePointWebApp/sites/SiteCollection" -Library "Shared Documents" -Approve $true -CheckIn $true -Overwrite $false -OverwriteFields $true
.LINK
Import-SPData
#>
Function Import-SPData {
[CmdletBinding()]
Param(
[Parameter(Position=0, Mandatory=$true, ValueFromPipeline=$true, ValueFromPipelineByPropertyName=$true)]
[string]$CSVFilePath = "CSVFile.csv",
[Parameter(Position=1, Mandatory=$true)]
[string]$SourceFolder = ".",
[Parameter(Position=2, Mandatory=$true)]
[string]$SiteURL = "http://SharePointWebApp/sites/SiteCollection",
[Parameter(Position=3, Mandatory=$true)]
[string]$Library = "Shared Documents",
[Parameter(Position=4, Mandatory=$true)]
[bool]$Approve = $true,
[Parameter(Position=5, Mandatory=$true)]
[bool]$CheckIn = $true,
[Parameter(Position=6, Mandatory=$true)]
[bool]$Overwrite = $false,
[Parameter(Position=7, Mandatory=$true)]
[bool]$OverwriteFields = $true
) #end param
Process
{
Try
{
#Get Web
$web = Get-SPWeb $SiteUrl
Write-Host -Background Black -Foreground Green "Uploading Documents..."
# Loop through the CSV file entries
Import-Csv $CSVFilePath | ForEach-Object{
# File Path
if($_.Subfolder)
{
#Subfolder exists
$FilePath = ($SourceFolder + "\" + $_.Subfolder + "\" + $_.Name)
}
else
{
#Subfolder does not exist
$FilePath = ($SourceFolder + "\" + $_.Name)
}
# Assign the new document name, and build the destination SharePoint path
$SPFileName = $_.Name
# Add the file to the SP Library. Don't approve or check in yet because we have to change the column values.
$spFile = Add-SPFileToLibrary -FilePath $FilePath -SPFileName $SPFileName -Web $web -Library $Library -Approve $Approve -CheckIn $CheckIn -Overwrite $Overwrite
if($spFile -And $OverwriteFields)
{
$spItem = $spFile.Item
$Message = "Changing column values for $($_.Name) ..."
Write-Debug $Message
Write-Verbose $Message
#Support for -WhatIf parameter
if($PsCmdlet.ShouldProcess($Message))
{
Write-Host $Message
#Add Column Information
Try
{
# Column1
Write-Verbose "Setting Column1 to $($_.Column1)"
$spItem["Column1"] = $($_.Column1)
# Column2
Write-Verbose "Setting Column1 to $($_.Column2)"
$spItem["Column2"] = $($_.Column2)
#Update document with new column values
$spItem.SystemUpdate($false)
Write-Host "...Complete"
}
Catch
{
Write-Error "Error Changing Column information"
Write-Error $Error[0]
}
}#end Change Column values
}
else
{
if($OverwriteFields)
{
# No file handle returned
Write-Host -Background DarkRed -ForeGround Yellow "Error: File not uploaded or no File handle returned: $_.Name"
}
else
{
# File handle returned, but OverwriteFields = $false
Write-Verbose "Not modifying any columns for $_.Name"
}
}#end if $spFile is set to something and OverwriteFields = $true
} #end CSV loop
} #end Try
Catch
{
Write-Error $Error[0]
}
} #end Process
} #end function Import-SPData