Export version history event log of SharePoint List - sharepoint

I'm looking for an event log or full version history which holds all changes ever made to the items of a SharePoint List.
Version history is enabled. When I right-mouse click on an item and select Version History, it actually shows me all the data I need:
Timestamp of modification
(Column, value) pair of the new modified column and it's new value
Modified by who
Now I "only" need to get this data structured in a table or a file (xlsx, xml, json, csv). And for all the List's items.
I can see that the data is there. But I haven't found a way to export it. So far I've tried customizing an .iqy file, using PowerShell and using Power Automate, without success.
I hope there is some way for me to get the full history. I need it to calculate average run times.

For SharePoint Online:
#Load SharePoint CSOM Assemblies
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll"
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll"
Function Export-VersionHistory()
{
param
(
[Parameter(Mandatory=$true)] [string] $SiteURL,
[Parameter(Mandatory=$true)] [string] $ListName,
[Parameter(Mandatory=$true)] [string] $CSVFile
)
Try {
#Delete the Output report file if exists
if (Test-Path $CSVFile) { Remove-Item $CSVFile }
#Get Credentials to connect
$Cred= Get-Credential
$Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($Cred.Username, $Cred.Password)
#Setup the context
$Ctx = New-Object Microsoft.SharePoint.Client.ClientContext($SiteURL)
$Ctx.Credentials = $Credentials
#Get the List
$List = $Ctx.Web.Lists.GetByTitle($ListName)
$Ctx.Load($List)
$Ctx.ExecuteQuery()
#Get all items
$Query = [Microsoft.SharePoint.Client.CamlQuery]::CreateAllItemsQuery()
$ListItems = $List.GetItems($Query)
$Ctx.Load($ListItems)
$Ctx.ExecuteQuery()
#Array to hold result
$VersionHistoryData = #()
#Iterate throgh each item
Foreach ($Item in $ListItems)
{
write-host "Processing Item:" $item.id -f Yellow
#Get all versions of the list item
$Versions = $Item.versions
$ctx.Load($Versions)
$Ctx.ExecuteQuery()
If($Versions.count -gt 0)
{
#Iterate each version
Foreach($Version in $Versions)
{
#Get the Creator object of the version
$CreatedBy = $Version.createdby
$Ctx.Load($CreatedBy)
$Ctx.ExecuteQuery()
#Send Data to object array
$VersionHistoryData += New-Object PSObject -Property #{
'Item ID' = $Item.ID
'Title' = $Version.FieldValues["Title"]
'Version Label' = $Version.VersionLabel
'Version ID' = ($Version.VersionId/512)
'Created On' = (Get-Date ($Version.Created) -Format "yyyy-MM-dd/HH:mm:ss")
'Created By' = $CreatedBy.Email
}
}
}
}
#Export the data to CSV
$VersionHistoryData | Export-Csv $CSVFile -Append -NoTypeInformation
write-host -f Green "Version History Exported Successfully to:" $CSVFile
}
Catch {
write-host -f Red "Error Exporting version History to CSV!" $_.Exception.Message
}
}
#Set parameter values
$SiteURL="https://tenant.sharepoint.com"
$ListName="list name"
$CSVFile="C:\VersionHistory.csv"
#Call the function to generate version History Report
Export-VersionHistory -SiteURL $SiteURL -ListName $ListName -CSVFile $CSVFile
For SharePoint Server:
# ******* Variables Section ******************
#Define these variables
$WebURL="site collection URL"
$ListName ="list name"
$ReportFile = "C:\VersionHistory.csv"
# *********************************************
#delete the file if exists
If (Test-Path $ReportFile)
{
Remove-Item $ReportFile
}
#Get the Web and List
$Web = Get-SPWeb $WebURL
$List = $web.Lists.TryGetList($ListName)
#Check if list exists
if($List -ne $null)
{
#Get all list items
$ItemsColl = $List.Items
#Write Report Header
Add-Content -Path $ReportFile -Value "Item ID, Version Lable, Created by, Created at, Title"
#Loop through each item
foreach ($item in $ItemsColl)
{
#Iterate each version
ForEach($version in $item.Versions)
{
#Get the version content
$VersionData = "$($item.id), $($version.VersionLabel), $($version.CreatedBy.User.DisplayName), $($version.Created), $($version['Title'])"
#Write to report
Add-Content -Path $ReportFile -Value $VersionData
}
}
}
Write-Host "Version history has been exported successfully!"

Related

How do I find the set of Site Pages to which I have assigned unique permissions?

I am the owner of a SharePoint Online site, and I have a handful of pages that I wanted to hide from Visitors, so I went to the Site Pages folder and clicked Manage Access for each file I wanted to hide, and I clicked the name of the SiteVisitors group and clicked Stop Sharing. The result is that Visitors cannot navigate or even see those pages in a Search. Now I want to restore the access for Visitors.
BUT, let's say I forgot the names of the pages that I hid. I can still see the pages because I am the Owner, but I forgot which pages the Visitors cannot see. So how do I generate a list of all pages that have non-standard permissions?
You can get all pages with unqiue permissions in your Site Pages library using below CSOM powershell.
#Load SharePoint CSOM Assemblies
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server
Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll"
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server
Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll"
#Function to call a non-generic method Load
Function Invoke-LoadMethod() {
param([Microsoft.SharePoint.Client.ClientObject]$Object = $(throw "Please provide a Client Object"),[string]$PropertyName)
$ctx = $Object.Context
$load = [Microsoft.SharePoint.Client.ClientContext].GetMethod("Load")
$type = $Object.GetType()
$clientLoad = $load.MakeGenericMethod($type)
$Parameter = [System.Linq.Expressions.Expression]::Parameter(($type), $type.Name)
$Expression = [System.Linq.Expressions.Expression]::Lambda([System.Linq.Expressions.Expression]::Convert([System.Linq.Expressions.Expression]::PropertyOrField($Parameter,$PropertyName),[System.Object] ), $($Parameter))
$ExpressionArray = [System.Array]::CreateInstance($Expression.GetType(), 1)
$ExpressionArray.SetValue($Expression, 0)
$clientLoad.Invoke($ctx,#($Object,$ExpressionArray))
}
#Define Parameter values
$SiteURL="https://crescent.sharepoint.com/sites/PMO"
$ListName="Site Pages"
Try {
#Setup Credentials to connect
$Cred= Get-Credential
$Credentials = New-Object
Microsoft.SharePoint.Client.SharePointOnlineCredentials($Cred.Username, $Cred.Password)
#Setup the context
$Ctx = New-Object Microsoft.SharePoint.Client.ClientContext($SiteURL)
$Ctx.Credentials = $Credentials
#Get All Lists of the web
$List = $Ctx.Web.Lists.GetByTitle($ListName)
$Ctx.Load($List)
$Ctx.ExecuteQuery()
Write-host "Total List Items Found:"$List.ItemCount
#Query to Get 2000 items from the list
$Query = New-Object Microsoft.SharePoint.Client.CamlQuery
$Query.ViewXml = "<View Scope='RecursiveAll'><RowLimit>2000</RowLimit></View>"
#Batch process list items - to mitigate list threshold issue on larger lists
Do {
$ListItems = $List.GetItems($Query)
$Ctx.Load($ListItems)
$Ctx.ExecuteQuery()
$Query.ListItemCollectionPosition = $ListItems.ListItemCollectionPosition
#Loop through each List item
ForEach($ListItem in $ListItems)
{
Invoke-LoadMethod -Object $ListItem -PropertyName "HasUniqueRoleAssignments"
$Ctx.ExecuteQuery()
if ($ListItem.HasUniqueRoleAssignments -eq $true)
{
Write-Host -f Green "List Item '$($ListItem["Title"])' with ID '$($ListItem.ID)' has Unique Permissions"
}
else
{
Write-Host -f Yellow "List Item '$($ListItem["Title"])' with ID '$($ListItem.ID)' is inhering Permissions from the Parent"
}
}
} While ($Query.ListItemCollectionPosition -ne $null)
}
Catch {
write-host -f Red "Error Checking Unique Permissions!" $_.Exception.Message
}
Or use PnP PowerShell.
#Set Variables
$SiteURL = "https://crescent.sharepoint.com/sites/PMO"
$ListName = "Site Pages"
#Connect to PnP Online
Connect-PnPOnline -Url $SiteURL -UseWebLogin
#Get all list items in batches
$ListItems = Get-PnPListItem -List $ListName -PageSize 2000
#Iterate through each list item
ForEach($ListItem in $ListItems)
{
#Check if the Item has unique permissions
$HasUniquePermissions = Get-PnPProperty -ClientObject $ListItem -Property
"HasUniqueRoleAssignments"
If($HasUniquePermissions)
{
Write-Host -f Green "List Item '$($ListItem["Title"])' with ID '$($ListItem.ID)' has Unique Permissions"
}
Else
{
Write-Host -f Yellow "List Item '$($ListItem["Title"])' with ID '$($ListItem.ID)' is inhering Permissions from its Parent"
}
}
Reference: SharePoint Online: Get All List Items with Unique Permissions using PowerShell
You can also delete all unique permissions in a library using PowerShell. Reference: SharePoint Online: Delete Unique Permissions for All Items in a List using PowerShell

Error: Cannot find an overload for "restore" and the argument count: "1"

I am getting this error from the following code. It's coming from $Context.Load($RecycleBinItems). Any idea what's wrong with the code? I am attempting to restore all recyclebin items.
Add-Type -Path "C:\Program Files\WindowsPowerShell\Modules\SharePointPnPPowerShellOnline\3.17.2001.2\Microsoft.SharePoint.Client.dll"
Add-Type -Path "C:\Program Files\WindowsPowerShell\Modules\SharePointPnPPowerShellOnline\3.17.2001.2\Microsoft.SharePoint.Client.Runtime.dll"
Import-Module 'Microsoft.PowerShell.Security'
#Get the Site Owners Credentials to connect the SharePoint
$SiteUrl = "https://phaselinknet.sharepoint.com"
$UserName = Read-host "Enter the Email ID"
$Password = Read-host - assecurestring "Enter Password for $AdminUserName"
$Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($UserName, $Password)
# Once Connected, get the Site information using current Context objects
Try {
$Context = New-Object Microsoft.SharePoint.Client.ClientContext($SiteUrl)
$Context.Credentials = $Credentials
$Site = $Context.Site
$RecycleBinItems = $Site.RecycleBin
$Context.Load($Site)
$Context.Load($RecycleBinItems)
$Context.ExecuteQuery()
Write-Host "Total Number of Files found in Recycle Bin:" $RecycleBinItems.Count
}
catch {
write - host "Error: $($_.Exception.Message)" - foregroundcolor Red
}
# using for loop to restore the item one by one
Try {
if($RecycleBinItems)
{
foreach($Item in $RecycleBinItems)
{
$Site.RecycleBin.restore($Item.ID)
#Write-Host "Item restored:"$Item.Title
}
}
}
catch {
write-host "Error: $($_.Exception.Message)" -foregroundcolor Red
}
The error message is giving you you answer. There is not a version of the method Restore that takes 1 parameter.
You need to load up a list of items simular to this
$Item = $RecycleBin | Where{$_.Title -eq $ItemName}
Then call restore for the items.
if($Item -ne $null)
{
$Item.Restore()
}
Thanks for the tip. So I load up the first 10 items in the recyclebin, and Write-Host does write out the correct files, but the $Item.Restore() does noting as the files are still not restored:
$itemsToRestore = #()
for ($i = 0; $i -lt 10; $i++)
{
$Item = $RecycleBinItems[$i]
$itemsToRestore += $Item
}
Write-Host "Total Number of Files to Restore:" $itemsToRestore.Count
foreach($item in $itemsToRestore)
{
Write-Host "Item:" $Item.Title
$item.Restore()
}
I found the problem. I missed $Context.ExecuteQuery() after $Item.Restore(). It works now.

do we have CSOM code to retrieve all lists under a site collection?

I am trying hard to get details of all the lists from a SharePoint site collection and export to CSV.
I am trying to run remotely from my machine and it doesn't work. I am not an export CSOM guy so asking for help, please.
I heard from someone that CSOM can run from anywhere and able to get info from SHarePoint 2013 server.
import-module C:\powershell\SharePointPnPPowerShell2013\3.16.1912.0\sharepointpnppowershell2013.psd1 -DisableNameChecking
$username = "testing#domain.com"
$password="Password"
$secureStringPwd = $Password | ConvertTo-SecureString -AsPlainText -Force
$Credentials = New-Object System.Management.Automation.PSCredential -ArgumentList $Username, $secureStringPwd
$connection=Connect-PnPOnline -Url https://cys.test.domain.com -Credentials $Credentials
$siteurl = "https://cys.test.domain.com/sites/testsite"
function Get-DocInventory([string]$siteUrl) {
$site = New-Object Microsoft.SharePoint.SPSite "https://cys.test.domain.com"
$web = Get-SPWeb "https://cys.test.domain.com/sites/testsite"
foreach ($list in $web.Lists) {
foreach ($item in $list.Items) {
#foreach($version in $item.Versions){
$data = #{
"List Name" = $list.Title
"Created By" = $item["Author"]
"Created Date" = $item["Created"]
"Modified By" = $item["Editor"]
"Modified Date" = $item["Modified"]
"Item Name" = $item.File.Name
"URL"=$web.Site.MakeFullUrl("$($web.ServerRelativeUrl.TrimEnd('/'))/$($item.Url)");
}
New-Object PSObject -Property $data | Select "List Name", "Item Name", "Created By", "Created Date", "Modified By", "Modified Date", "URL"
}
#}
$web.Dispose();
}
}
Get-DocInventory | Export-Csv -NoTypeInformation -Path "d:\test\test.csv"
If you are using SharePoint 2013 On-Premise, try to install SharePoint 2013 CSOM firstly:
SharePoint Server 2013 Client Components SDK
Then use this script to get list details in a site collection includes sub sites:
#Load SharePoint CSOM Assemblies
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\ISAPI\Microsoft.SharePoint.Client.dll"
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\ISAPI\Microsoft.SharePoint.Client.Runtime.dll"
#Variables for Processing
$SiteUrl= "http://sp/sites/devtest"
$username="administrator"
$password="******"
$domain="Contoso"
#Setup Credentials to connect
$Credentials = New-Object System.Net.NetworkCredential($username, $password, $domain)
Try {
#Function to Get all lists from the web
Function Get-SPOList($Web)
{
#Get All Lists from the web
$Lists = $Web.Lists
$Context.Load($Lists)
$Context.ExecuteQuery()
#Get all lists from the web
ForEach($List in $Lists)
{
#Get the List Name
Write-host $List.Title
Write-host $List.Created
}
}
#Function to get all webs from given URL
Function Get-SPOWeb($WebURL)
{
#Set up the context
$Context = New-Object Microsoft.SharePoint.Client.ClientContext($WebURL)
$Context.Credentials = $Credentials
$Web = $context.Web
$Context.Load($web)
#Get all immediate subsites of the site
$Context.Load($web.Webs)
$Context.executeQuery()
#Call the function to Get Lists of the web
Write-host "Processing Web :"$Web.URL
Get-SPOList $Web
#Iterate through each subsite in the current web
foreach ($Subweb in $web.Webs)
{
#Call the function recursively to process all subsites underneaththe current web
Get-SPOWeb($SubWeb.URL)
}
}
#Call the function to get all sites
Get-SPOWeb $SiteUrl
}
catch {
write-host "Error: $($_.Exception.Message)" -foregroundcolor Red
}

SharePoint: Script to automatically import documents from a network share on a regular schedule

I am new to SharePoint development.
I am trying automate the task of:
Uploading documents from a network share to a SharePoint document library.
The related metadata for each of the files also need to be brought into the SharePoint columns. (will taken from a single CSV file in the same location with filenames mapped in it)
This process needs to automatically run on a predefined schedule (eg: daily/once an hour etc)
I want to know:
What is the best way to achieve this? (PowerShell script running on Windows task scheduler, creating a service etc.)
Starting point for coding or any code already available since it looks like this might be a very common requirement.
Typically StackOverflow is the wrong place to ask this kind of question. But, since I just finished creating a script for uploading files to SharePoint from a CSV, I thought I can share it.
Disclaimer: I offer no guarantees that this will work with your environment, or that it will not mess things up. Review. Understand it. Use at own risk.
The script can be easily changed to add/change columns, (under the #Add Column Information section)
CSV File format:
SubFolder,Name,Column1,Column2
Where SubFolder can be blank if your source file is on the root of the Source Folder. Name is the file name, and Column1 and Column2 are SharePoint columns.
Save the below code to the two PowerShell files. ImportSPfromCSV.ps1 contains the logic, Run_ImportSPfromCSV.ps1 contains the code to run it.
To run, set up a scheduled task on a SharePoint server (or server that has the SharePoint PowerShell module on it) and run with the command:
powershell.exe -file "C:\scripts\Run_ImportSPfromCSV.ps1"
Run_ImportSPfromCSV.ps1
# Load Script
. .\ImportSPfromCSV.ps1
# Run Function
Import-SPData -CSVFilePath "CSVFile.csv" -SourceFolder "C:\temp" -SiteURL "http://SharePointWebApp/sites/SiteCollection" -Library "Shared Documents" -Approve $true -CheckIn $true -Overwrite $false -OverwriteFields $true
ImportSPfromCSV.ps1
<#
.SYNOPSIS
Imports items into SharePoint library from a CSV File
.DESCRIPTION
Imports items that are saved to disk into SharePoint library
using and tagging the information from a CSV File
.NOTES
NAME: ImportSPfromCSV.ps1
AUTHOR: Kristoph Minchau
CREATED: May 2, 2013
LASTEDIT: 2014-03-06
VERSION: 1.10
KEYWORDS: SharePoint
REVISION LOG:
[2013/05/01] - km - v1.00 ** Created **
[2014-03-06] - km - v1.10 * Minor edits
.EXAMPLE
# Load Script
. .\ImportSPfromCSV.ps1
# Run Function
Import-SPData -CSVFilePath "CSVFile.csv" -SourceFolder "C:\temp" -SiteURL "http://SharePointWebApp/sites/SiteCollection" -Library "Shared Documents" -Approve $true -CheckIn $true -Overwrite $false -OverwriteFields $true
.LINK
Import-SPData
#>
Add-PSSnapin Microsoft.SharePoint.Powershell -ErrorAction SilentlyContinue
<#
.SYNOPSIS
Get SharePoint file
.DESCRIPTION
Gets a SharePoint File
.PARAMETER SPFileName
The file name to Upload to the SharePoint library if different than original File name
ex. "NewName.docx"
[String]
.PARAMETER Web
The SharePoint web site
[Microsoft.SharePoint.SPWeb]
.PARAMETER Library
The Destination Library on SharePoint
Default value = "Retail Banking"
.INPUTS
System.String SPFileName
.OUTPUTS
System.String Return SPFile
.EXAMPLE
$Web = Get-SPWeb "http://SharePointWebApp/sites/SiteCollection"
Get-SPFile -Web $Web -Library "Shared Documents" -SPFileName "File1.doc"
.LINK
Get-SPFile
#>
Function Get-SPFile {
[CmdletBinding()]
Param(
[Parameter(Position=0, Mandatory=$true, ValueFromPipeline=$true, ValueFromPipelineByPropertyName=$true)]
[string]$SPFileName,
[Parameter(Position=1, Mandatory=$true)]
[Microsoft.SharePoint.SPWeb]$Web,
[Parameter(Position=2, Mandatory=$true)]
[string]$Library = "Shared Documents"
) #end param
Process
{
Try
{
# Get Library
$docLibrary = $web.Lists[$Library]
$folder = $docLibrary.RootFolder
# Assign the new document name, and build the destination SharePoint path
$SPFilePath = ($folder.URL + "/" + $SPFileName)
#Check if the file exists and the overwrite option is selected before adding the file
$spFile = $web.GetFile($SPFilePath)
#Return SP File
Write-Output $spFile
} #end Try
Catch
{
Write-Error $Error[0]
}
} #end Process
} #end Get-SPFile
<#
.SYNOPSIS
Adds a file to SharePoint Document Library.
.DESCRIPTION
Adds a file to SharePoint Document Library, optionally checks it in, approve it, or overwrite it.
.PARAMETER FilePath
The file to Upload
ex. "c:\Test.docx"
[String]
.PARAMETER SPFileName
The file name to Upload to the SharePoint library.
Rename the File to something that is SharePoint/URL friendly (e.g. remove special characters "/?=<>" etc.)
ex. "NewName.docx"
[String]
.PARAMETER Web
The SharePoint web site
[Microsoft.SharePoint.SPWeb]
.PARAMETER Library
The Destination Library on SharePoint
[String]
.PARAMETER Approve
If approval workflow is enabled, when uploading the document, approve the uploaded document
[bool] Default value = $true
.PARAMETER CheckIn
If workflow is enabled, when uploading the document, CheckIn the uploaded document
[bool] Default value = $true
.PARAMETER Overwrite
When uploading the document, if a document with the same name is encountered, overwrite the document?
[bool] Default value = $false
.INPUTS
N/A - Pipeline Inputs
.OUTPUTS
SP File
.EXAMPLE
$Web = Get-SPWeb "http://SharePointWebApp/sites/SiteCollection"
Add-SPFileToLibrary -File "Test.docx" -SPFileName "NewName.docx" -Web $Web -Library "Shared Documents" -Approve $true -CheckIn $true -Overwrite $false
.LINK
Add-SPFileToLibrary
#>
Function Add-SPFileToLibrary {
[CmdletBinding()]
Param(
[Parameter(Position=0, Mandatory=$true, ValueFromPipeline=$true, ValueFromPipelineByPropertyName=$true)]
[string]$FilePath,
[Parameter(Position=1, Mandatory=$true)]
[string]$SPFileName,
[Parameter(Position=2, Mandatory=$true)]
[Microsoft.SharePoint.SPWeb]$Web,
[Parameter(Position=3, Mandatory=$true)]
[string]$Library,
[Parameter(Position=4, Mandatory=$true)]
[bool]$Approve = $true,
[Parameter(Position=5, Mandatory=$true)]
[bool]$CheckIn = $true,
[Parameter(Position=6, Mandatory=$true)]
[bool]$Overwrite = $false
) #end param
Process
{
Try
{
# Get Library
$docLibrary = $web.Lists[$Library]
$folder = $docLibrary.RootFolder
# Assign the new document name, and build the destination SharePoint path
$SPFilePath = ($folder.URL + "/" + $SPFileName)
#Check if the file exists and the overwrite option is selected before adding the file
if ((!$web.GetFile($SPFilePath).Exists) -or ($Overwrite))
{
$Message = "Uploading... $FilePath ...To... $SPFilePath"
Write-Debug $Message
Write-Verbose $Message
#Support for -WhatIf parameter
if($PsCmdlet.ShouldProcess($Message))
{
Write-Host $Message
#Upload file
$File = Get-ChildItem $FilePath
$spFile = $folder.Files.Add($SPFilePath, $File.OpenRead(), $true) #Upload with overwrite = $true (SP file path, File stream, Overwrite?)
$spItem = $spFile.Item
#Check in file to document library (if required)
#MinorCheckIn=0, MajorCheckIn=1, OverwriteCheckIn=2
If ($CheckIn) {
If ($spFile.CheckOutStatus -ne "None") {
If ($Overwrite){
#$spFile.CheckIn("File copied from " + $filePath, 2)
$spFile.CheckIn("Version 1.0", 2)
}
Else
{
#$spFile.CheckIn("File copied from " + $filePath, 1)
$spFile.CheckIn("Version 1.0", 1)
}
Write-Host "$spfile.Name Checked In"
}
} #end CheckIn
#Approve file (if required)
If ($Approve)
{
If ($spItem.ListItems.List.EnableModeration -eq $true)
{
#$spFile.Approve("File automatically approved after copying from " + $filePath)
$spFile.Approve("Approved")
If ($spFile.Item["Approval Status"] -eq 0)
{
Write-Host "$spfile.Name Approved"
}
Else
{
Write-Host -Background Yellow -Foreground Black "Warning: $spfile.Name Not Approved"
}
}
} #end Approve
#Return SP File
Write-Output $spFile
}# end upload file
}
Else
{
If($web.GetFile($SPFilePath).Exists)
{
Write-Verbose "File Exists and Overwrite = $false returning file"
# File Exists and Overwrite = $false, Try finding and returning the file
$spFile = Get-SPFile -Web $web -Library $Library -SPFileName $SPFileName
#Return SP File
Write-Output $spFile
}
else
{
# File does not exist, and Overwrite = $false, we can't return anything, Throw Warning
Write-Host -Background Yellow -Foreground Black "Warning: $SPFilePath does not Exist and Overwrite is set to False. No SP File handle returned"
Write-Output $null
}
}# end File Exists and Overwrite Check
} #end Try
Catch
{
Write-Error $Error[0]
}
} #end Process
} #end Add-SPFileToLibrary
<#
.SYNOPSIS
Imports files into SharePoint library from a CSV File
.DESCRIPTION
Imports files that are saved to disk into SharePoint library
using and tagging the information from a CSV File
.PARAMETER CSVFilePath
The CSV import file to use to import the data from
Default value = "CSVFile.csv"
.PARAMETER SourceFolder
The Source Folder for locating the Public Folder files
Default value = "."
.PARAMETER SiteURL
The Site URL on SharePoint
Default value = "http://SharePointWebApp/sites/SiteCollection"
.PARAMETER Library
The Destination Library on SharePoint
Default value = "Shared Documents"
.PARAMETER Approve
If approval workflow is enabled, when uploading the document, approve the uploaded document
[bool] Default value = $true
.PARAMETER CheckIn
If workflow is enabled, when uploading the document, CheckIn the uploaded document
[bool] Default value = $true
.PARAMETER Overwrite
When uploading the document, if a document with the same name is encountered, overwrite the document?
[bool] Default value = $false
.PARAMETER OverwriteFields
If the document is already uploaded, overwrite the fields with the fields from the CSV file
[bool] Default value = $true
.INPUTS
N/A - Pipeline Inputs
.OUTPUTS
System.String Returns
.EXAMPLE
#Default Initial Import of all data
Import-SPData -CSVFilePath "CSVFile.csv" -SourceFolder "C:\temp" -SiteURL "http://SharePointWebApp/sites/SiteCollection" -Library "Shared Documents" -Approve $true -CheckIn $true -Overwrite $false -OverwriteFields $true
.LINK
Import-SPData
#>
Function Import-SPData {
[CmdletBinding()]
Param(
[Parameter(Position=0, Mandatory=$true, ValueFromPipeline=$true, ValueFromPipelineByPropertyName=$true)]
[string]$CSVFilePath = "CSVFile.csv",
[Parameter(Position=1, Mandatory=$true)]
[string]$SourceFolder = ".",
[Parameter(Position=2, Mandatory=$true)]
[string]$SiteURL = "http://SharePointWebApp/sites/SiteCollection",
[Parameter(Position=3, Mandatory=$true)]
[string]$Library = "Shared Documents",
[Parameter(Position=4, Mandatory=$true)]
[bool]$Approve = $true,
[Parameter(Position=5, Mandatory=$true)]
[bool]$CheckIn = $true,
[Parameter(Position=6, Mandatory=$true)]
[bool]$Overwrite = $false,
[Parameter(Position=7, Mandatory=$true)]
[bool]$OverwriteFields = $true
) #end param
Process
{
Try
{
#Get Web
$web = Get-SPWeb $SiteUrl
Write-Host -Background Black -Foreground Green "Uploading Documents..."
# Loop through the CSV file entries
Import-Csv $CSVFilePath | ForEach-Object{
# File Path
if($_.Subfolder)
{
#Subfolder exists
$FilePath = ($SourceFolder + "\" + $_.Subfolder + "\" + $_.Name)
}
else
{
#Subfolder does not exist
$FilePath = ($SourceFolder + "\" + $_.Name)
}
# Assign the new document name, and build the destination SharePoint path
$SPFileName = $_.Name
# Add the file to the SP Library. Don't approve or check in yet because we have to change the column values.
$spFile = Add-SPFileToLibrary -FilePath $FilePath -SPFileName $SPFileName -Web $web -Library $Library -Approve $Approve -CheckIn $CheckIn -Overwrite $Overwrite
if($spFile -And $OverwriteFields)
{
$spItem = $spFile.Item
$Message = "Changing column values for $($_.Name) ..."
Write-Debug $Message
Write-Verbose $Message
#Support for -WhatIf parameter
if($PsCmdlet.ShouldProcess($Message))
{
Write-Host $Message
#Add Column Information
Try
{
# Column1
Write-Verbose "Setting Column1 to $($_.Column1)"
$spItem["Column1"] = $($_.Column1)
# Column2
Write-Verbose "Setting Column1 to $($_.Column2)"
$spItem["Column2"] = $($_.Column2)
#Update document with new column values
$spItem.SystemUpdate($false)
Write-Host "...Complete"
}
Catch
{
Write-Error "Error Changing Column information"
Write-Error $Error[0]
}
}#end Change Column values
}
else
{
if($OverwriteFields)
{
# No file handle returned
Write-Host -Background DarkRed -ForeGround Yellow "Error: File not uploaded or no File handle returned: $_.Name"
}
else
{
# File handle returned, but OverwriteFields = $false
Write-Verbose "Not modifying any columns for $_.Name"
}
}#end if $spFile is set to something and OverwriteFields = $true
} #end CSV loop
} #end Try
Catch
{
Write-Error $Error[0]
}
} #end Process
} #end function Import-SPData

downloading a file from SharePoint Online with PowerShell

I have a requirement to download files from a sharepoint online document library using powershell
I've managed to get to the point where the download should happen but no luck.
I know its something to do with how I am using the stream/writer
any hints would be greatly appreciated
*Edit
No error messages are thrown just 0 length files in my local Directory
$SPClient = [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client")
$SPRuntime = [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client.Runtime")
$webUrl = Read-Host -Prompt "HTTPS URL for your SP Online 2013 site"
$username = Read-Host -Prompt "Email address for logging into that site"
$password = Read-Host -Prompt "Password for $username" -AsSecureString
$folder = "PoSHTest"
$destination = "C:\\test"
$ctx = New-Object Microsoft.SharePoint.Client.ClientContext($webUrl)
$ctx.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($username, $password)
$web = $ctx.Web
$lists = $web.Lists.GetByTitle($folder)
$query = [Microsoft.SharePoint.Client.CamlQuery]::CreateAllItemsQuery(10000)
$result = $lists.GetItems($query)
$ctx.Load($Lists)
$ctx.Load($result)
$ctx.ExecuteQuery()
#Edited the foreach as per #JNK
foreach ($File in $result) {
Write-host "Url: $($File["FileRef"]), title: $($File["FileLeafRef"]) "
$binary = [Microsoft.SharePoint.Client.File]::OpenBinaryDirect($ctx,$File["FileRef"])
$Action = [System.IO.FileMode]::Create
$new = $destination + "\\" + $File["FileLeafRef"]
$stream = New-Object System.IO.FileStream $new, $Action
$writer = New-Object System.IO.BinaryWriter($stream)
$writer.write($binary)
$writer.Close()
}
You could also utilize WebClient.DownloadFile Method by providing SharePoint Online credentials to download the resource from SharePoint Online as demonstrated below.
Prerequisites
SharePoint Online Client Components SDK have to be installed on the machine running the script.
How to download a file in SharePoint Online/O365 in PowerShell
Download-File.ps1 function:
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client")
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client.Runtime")
Function Download-File([string]$UserName, [string]$Password,[string]$FileUrl,[string]$DownloadPath)
{
if([string]::IsNullOrEmpty($Password)) {
$SecurePassword = Read-Host -Prompt "Enter the password" -AsSecureString
}
else {
$SecurePassword = $Password | ConvertTo-SecureString -AsPlainText -Force
}
$fileName = [System.IO.Path]::GetFileName($FileUrl)
$downloadFilePath = [System.IO.Path]::Combine($DownloadPath,$fileName)
$client = New-Object System.Net.WebClient
$client.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($UserName, $SecurePassword)
$client.Headers.Add("X-FORMS_BASED_AUTH_ACCEPTED", "f")
$client.DownloadFile($FileUrl, $downloadFilePath)
$client.Dispose()
}
Usage
Download-File -UserName "username#contoso.onmicrosoft.com" -Password "passowrd" -FileUrl https://consoto.sharepoint.com/Shared Documents/SharePoint User Guide.docx -DownloadPath "c:\downloads"
I was able to download the file successfully with the following relevant code snippet. You should be able to extend it for your situation.
Add-Type –Path "C:\Program Files\Common Files\microsoft shared\Web Server Extensions\15\ISAPI\Microsoft.SharePoint.Client.dll"
Add-Type –Path "C:\Program Files\Common Files\microsoft shared\Web Server Extensions\15\ISAPI\Microsoft.SharePoint.Client.Runtime.dll"
$siteUrl = Read-Host -Prompt "Enter web URL"
$username = Read-Host -Prompt "Enter your username"
$password = Read-Host -Prompt "Enter password" -AsSecureString
$source = "/filepath/sourcefilename.dat" #server relative URL here
$target = "C:/detinationfilename.dat" #URI of the file locally stored
$ctx = New-Object Microsoft.SharePoint.Client.ClientContext($siteUrl)
$credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($username, $password)
$ctx.Credentials = $credentials
[Microsoft.SharePoint.Client.FileInformation] $fileInfo = [Microsoft.SharePoint.Client.File]::OpenBinaryDirect($ctx,$source);
[System.IO.FileStream] $writeStream = [System.IO.File]::Open($target,[System.IO.FileMode]::Create);
$fileInfo.Stream.CopyTo($writeStream);
$writeStream.Close();
While the CSOM code above likely can be made to work I find it easier to use the web client method.
(from http://soerennielsen.wordpress.com/2013/08/25/use-csom-from-powershell/)
I've used the code below, to retrieve a bunch of files (metadata from CSOM queries) to a folder (using your $result collection, other params should be adjusted a bit):
#$siteUrlString site collection url
#$outPath path to export directory
$siteUri = [Uri]$siteUrlString
$client = new-object System.Net.WebClient
$client.UseDefaultCredentials=$true
if ( -not (Test-Path $outPath) ) {
New-Item $outPath -Type Directory | Out-Null
}
$result |% {
$url = new-object Uri($siteUri, $_["FileRef"])
$fileName = $_["FileLeafRef"]
$outFile = Join-Path $outPath $fileName
Write-Host "Downloading $url to $outFile"
try{
$client.DownloadFile( $url, $outFile )
}
catch{
#one simple retry...
try{
$client.DownloadFile( $url, $outFile )
}
catch{
write-error "Failed to download $url, $_"
}
}
}
The trick here is the
$client.UseDefaultCredentials=$true
which will authenticate the webclient for you (as the current user).
The direct and almost shortest answer to the question is simply:
$url = 'https://the.server/path/to/the/file.txt'
$outfile = "$env:userprofile\file.txt"
Invoke-WebRequest -Uri $url -OutFile $outfile -Credential (Get-Credential)
This works at least in Powershell 5.1...
So I gave up on this. it turned out to be much easier to write an SSIS script component to do the job.
I have awarded Soeren as he posted some code that will work for regular websites but not sodding SharePoint Online.
Thanks Sorean!
Short an easy approach to download a file from sharepoint online, using just powershell and sharepoint online url ( no pnp powershell )
This approach can also be used to perform Sharepoint REST queries, with just powershell and sharepoint REST api
# required MS dependencies
# feel free to download them from here https://www.microsoft.com/en-us/download/details.aspx?id=42038
Add-Type -Path 'C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll' -ErrorAction Stop
Add-Type -Path 'C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll' -ErrorAction Stop
# prepare passwords
$spCredential = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($user, $(ConvertTo-SecureString -AsPlainText $pass -Force))
# prepare and perform rest api query
$Context = New-Object Microsoft.SharePoint.Client.ClientContext($targetSiteUrl)
$Context.Credentials = $spCredential
try {
#this may return an error, but still will finish context setup
$Context.ExecuteQuery()
}
catch {
write-host "TODO: fix executeQuery() err 400 bug" -ForegroundColor Yellow
}
$AuthenticationCookie = $Context.Credentials.GetAuthenticationCookie($targetSiteUrl, $true)
$WebSession = New-Object Microsoft.PowerShell.Commands.WebRequestSession
$WebSession.Credentials = $Context.Credentials
$WebSession.Cookies.SetCookies($targetSiteUrl, $AuthenticationCookie)
$WebSession.Headers.Add("Accept", "application/json;odata=verbose")
Invoke-WebRequest -Uri $spFileUrl -OutFile $outputFilePath -WebSession $WebSession -errorAction Stop
Where
$outputFilePath is the target output file in which you want to save the remote file.
$targetSiteUrl is the target sp site url.
$spFileUrl is the "[sharepoint file full url]"
$user plain text sp user email
$pass plain text sp user pass

Resources