I have written a code that have to upload multiple files in azure web app using powershell.
I want to upload folder saved in $appdirectory variable.
$appdirectory="C:\scriptfolder\*"
$webappname="myapitestapp1"
$xml = [xml](Get-AzureRmWebAppPublishingProfile -Name $webappname -ResourceGroupName sibs -OutputFile null)
$xml = [xml]$xml
username = $xml.SelectNodes("//publishProfile[#publishMethod=`"FTP`"]/#userName").value
$password = $xml.SelectNodes("//publishProfile[#publishMethod=`"FTP`"]/#userPWD").value
$url = $xml.SelectNodes("//publishProfile[#publishMethod=`"FTP`"]/#publishUrl").value
Set-Location $appdirectory
$webclient = New-Object -TypeName System.Net.WebClient
$webclient.Credentials = New-Object System.Net.NetworkCredential($username,$password)
$files = Get-ChildItem -Path $appdirectory -Recurse
foreach ($file in $files)
{
$relativepath = (Resolve-Path -Path $file.FullName -Relative).Replace(".\", "").Replace('\', '/')
$uri = New-Object System.Uri("$url/$relativepath")
if($file.PSIsContainer)
{
#$uri.AbsolutePath + "is Directory"
$ftprequest = [System.Net.FtpWebRequest]::Create($uri);
$ftprequest.Method = [System.Net.WebRequestMethods+Ftp]::MakeDirectory
$ftprequest.UseBinary = $true
$ftprequest.Credentials = New-Object System.Net.NetworkCredential($username,$password)
$response = $ftprequest.GetResponse();
$response.StatusDescription
continue
}
"Uploading to " + $uri.AbsoluteUri
$webclient.UploadFile($uri, $file.FullName)
}
$webclient.Dispose()
its copying files that is from subfolder and uploading but it is not uploading the root directory. I want to upload all folder and files using powershell.
$relativepath = (Resolve-Path -Path $file.FullName -Relative).Replace(".\", "").Replace('\', '/')
was wrong.
$relativepath = (Resolve-Path -Path $file.FullName -Relative).Replace('\', '/')
This worked and stopped hiding files
Related
I have this Powershell script I am trying to run, to do me a 'bacpac' file from an Azure tenancy database, to on-prem (local folder).
# Load SMO Assembly
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SMO") | Out-Null
# Define the source database
$sourceServer = "myserver-sql-server"
$sourceDB = "mydb-sql-db"
# Define the target file
$targetFile = "c:\temp\mydb.bacpac"
# Connect to the source database
$sourceServer = New-Object Microsoft.SqlServer.Management.Smo.Server $sourceServer
$sourceDB = $sourceServer.Databases[$sourceDB]
# Export the database to the target file
$sourceDB.ExportBacpac($targetFile)
The error I am getting is on the last line...
You cannot call a method on a null-valued expression. At line:2 char:1
$sourceDB.ExportBacpac($targetFile)
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull
The variables have values. Am I missing a parameter calling 'ExportBacPac'?
This script worked...
Function Get-Bacpacs {
Param(
[string]$location
, [string]$server
, [string]$smolibrary
, [string]$daclibrary
, [string]$username
, [string]$password
)
Process
{
$dt = Get-Date -uFormat "%Y%m%d"
Add-Type -Path $smolibrary
$scon = "Data Source=$server.database.windows.net;Initial Catalog=master;User ID=$username;Password=$password;"
$servercon = New-Object Microsoft.SqlServer.Management.Common.ServerConnection
$servercon.ConnectionString = $scon
$srv = New-Object Microsoft.SqlServer.Management.SMO.Server($servercon)
foreach ($db in $srv.Databases | Where-Object {$_.Name -ne "master"})
{
$database = $db.Name
$bak_scon = "Data Source=$server.database.windows.net;Initial Catalog=$database;Connection Timeout=0;User ID=$username;Password=$password;"
if (!(Test-Path $location))
{
New-Item $location -ItemType Directory
}
$outfile = $location + $database + "_" + $dt + ".bacpac"
Add-Type -Path $daclibrary
$d_exbac = New-Object Microsoft.SqlServer.Dac.DacServices $bak_scon
try
{
$d_exbac.ExportBacpac($outfile, $database)
}
catch
{
Write-Warning $_
### Other alerting can go here
}
}
}
}
### This makes it easier for us who don't have really long screens! Location may vary.
$smo = "C:\Program Files (x86)\Microsoft SQL Server\110\SDK\Assemblies\Microsoft.SqlServer.Smo.dll"
$dac = "C:\Program Files (x86)\Microsoft SQL Server\110\DAC\bin\Microsoft.SqlServer.Dac.dll"
Get-Bacpacs -location "" -server "" -smolibrary $smo -daclibrary $dac -username "" -password ""
Hi I am trying to upload published files from Azure Git artifacts to FTP. But randomly I am getting the below error.
This is only happening for files not for any folder or subfolders.
“UploadFile” with “2” argument(s): “The Content-Type header cannot be set to a multipart type for this request.
All the files are available in the artifacts.
Observation & tried:
All files are present in artifacts
Getting the error only for files. (Folder & subfolders are creating successfully)
Stopped the Web App and then tried to upload
Tried Sleep also between uploading two files
After all of these, the result is the same.
Can anyone please help me out?
Get-ChildItem -Path $target_directory
# Upload files recursively
write-host "target_directory - $target_directory"
Set-Location $target_directory
$webclient = New-Object -TypeName System.Net.WebClient
$webclient.Credentials = New-Object System.Net.NetworkCredential($username,$password)
$files = Get-ChildItem -Path $target_directory -Recurse
write-host 'Uploading started............................'
foreach ($file in $files)
{
write-host "file -" + $file.FullName
if($file.FullName -match "web.config" -or $file.FullName -match ".pdb" -or $file.FullName -match "roslyn" -or $file.FullName -match "obj\Debug"){
write-host "ignoring " + $file.FullName
continue
}
$relativepath = (Resolve-Path -Path $file.FullName -Relative).Replace(".\", "").Replace('\', '/')
$uri = New-Object System.Uri("$url/$relativepath")
write-host "uri -" + $uri.AbsoluteUri
write-host '--------'
if($file.PSIsContainer)
{
write-host 'PSIsContainer - All dir/files within' $file
get-childitem -path $file -Recurse
try
{
$makeDirectory = [System.Net.WebRequest]::Create($uri);
$makeDirectory.Credentials = New-Object System.Net.NetworkCredential($username,$password);
$makeDirectory.Method = [System.Net.WebRequestMethods+FTP]::MakeDirectory;
$makeDirectory.GetResponse();
#folder created successfully
}
catch [Net.WebException]
{
try
{
#if there was an error returned, check if folder already existed on server
$checkDirectory = [System.Net.WebRequest]::Create($uri);
$checkDirectory.Credentials = New-Object System.Net.NetworkCredential($username,$password);
$checkDirectory.Method = [System.Net.WebRequestMethods+FTP]::PrintWorkingDirectory;
$response = $checkDirectory.GetResponse();
$response.StatusDescription
#folder already exists!
}
catch [Net.WebException]
{
#if the folder didn't exist, then it's probably a file perms issue, incorrect credentials, dodgy server name etc
}
}
continue
}
try{
write-host "Uploading to " $uri.AbsoluteUri " from " $file.FullName
$webclient.UploadFile($uri, $file.FullName)
start-sleep -Seconds 1
}
catch{
##[error]Error message
Write-Host "##[error]Error in uploading " $file -ForegroundColor red
Write-Host $_
$disputedFiles = $disputedFiles + $file.FullName
$_file = #{
FileName = $file.FullName
FTPPath = $uri.AbsoluteUri
}
$_o = New-Object psobject -Property $_file;
$disputedFilesList = $disputedFilesList + $_o
}
}
Write-Host "##[debug] Starting uploading the disputed files....."
foreach($file in $disputedFilesList){
try{
write-host "Uploading to " $file.FTPPath " from " $file.FileName
$webclient.UploadFile($file.FTPPath, $file.FileName)
start-sleep -Seconds 1
}
catch{
write-host "##[error]Error(2) in uploading to " $file.FTPPath " from " $file.FileName
}
}
Write-Host "##[debug] Ending uploading the disputed files....."
remove-item -path $target_directory\* -Recurse
get-childitem -path $target_directory
write-host "Directory Empty after cleanup"
$webclient.Dispose()
A few weeks ago, I had to remove password protection from excel files which were created by an application. I had no passwords. Can this task be done with powershell, using xml transformation?
This is my solution I want to share with you. The powershell script removes passwords and sheet protections from Excel files using powershell. No Excel application and no passwords are needed. The script is working only for .xlsx file types, not for .xls. If you have ideas for improvement, let me know. Thank you.
Add-Type -AssemblyName System.IO.Compression
Add-Type -AssemblyName System.IO.Compression.FileSystem
#-----------------------------------------------------------------------
function Remove-Excel-WriteProtection {
<#
// Removes all password and write protection from existing excel file
// (workbook and worksheets).
// No password needed.
//
// Input: Path to Excel file (must newer xlsx format)
// Output: true if successfull
#>
#-----------------------------------------------------------------------
param(
[Parameter(Mandatory=$true)]
[string]$filePathExcel
)
if( !(Test-Path -Path $filePathExcel) -or
!(Split-Path -Path $filePathExcel -Leaf).EndsWith('xlsx') ) {
return $false
}
$fileItem = Get-Item $filePathExcel
$filePathZip = $fileItem.DirectoryName + '\' + $fileItem.BaseName + '.zip'
$filePathTemp = $fileItem.DirectoryName + '\' + ([System.Guid]::NewGuid()).Guid
Rename-Item -Path $filePathExcel -NewName $filePathZip -Force
Expand-Archive -Path $filePathZip -DestinationPath $filePathTemp -Force
$xml = New-Object System.Xml.XmlDocument
$xml.PreserveWhitespace = $true
$workbookCollection = (Get-ChildItem -Path $filePathTemp -Filter 'workbook.xml' -Recurse -Force)
foreach( $workbook in $workbookCollection ) {
[void]$xml.RemoveAll()
[void]$xml.Load($workbook.FullName)
if( $xml.workbook.fileSharing.readOnlyRecommended -or $xml.workbook.fileSharing.reservationPassword ) {
if( $xml.workbook.fileSharing.readOnlyRecommended ) {
$xml.workbook.fileSharing.readOnlyRecommended = '0'
}
if( $xml.workbook.fileSharing.reservationPassword ) {
$xml.workbook.fileSharing.reservationPassword = ''
}
[void]$xml.Save($workbook.FullName)
}
}
$worksheetCollection = (Get-ChildItem -Path $filePathTemp -Filter 'sheet*.xml' -Recurse -Force)
foreach( $worksheet in $worksheetCollection ) {
[void]$xml.RemoveAll()
[void]$xml.Load($worksheet.FullName)
if( $xml.worksheet.sheetProtection ) {
[void]$xml.worksheet.RemoveChild($xml.worksheet.sheetProtection)
[void]$xml.Save($worksheet.FullName)
}
}
Remove-Item -Path $filePathZip -Force
Compress-Archive -Path ($filePathTemp + '\*') -DestinationPath $filePathZip -Force -CompressionLevel Optimal
Remove-Item -Path $filePathTemp -Recurse -Force
Rename-Item -Path $filePathZip -NewName $filePathExcel -Force
return $true
}
# Remove all passwords for test.xlsx
$result = Remove-Excel-WriteProtection -filePathExcel 'C:\Users\YourName\Desktop\test.xlsx'
I'm working on a script that uses get-childitem on the other server, but need to change it so it uses credentials of the local account on the other server to do that. When I was just using Active Directory to do that, I was saving the task in our scheduler with my AD login, and it was good on the other server, using the UNC path. But we decided to change it to the local login there recently and I'm getting an error message, trying to use net use. Does anyone know of a good way to do this with the UNC path instead? Or, any idea why the following is giving an error message?
function GetSecureLogin(){
$global:username = "stuff"
$global:password = get-content C:\filename.txt | convertto-securestring
}
function Cleanup([string]$Drive) {
try {
$deleteTime = -42
$now = Get-Date
**#this is saying cannot find path '\\name.na.xxx.net\20xServerBackup\V' name truncated**
Get-ChildItem -Path $Drive -Recurse -Force |Where-Object {$_.LastWriteTime -lt $limit} | Remove-Item -Force
}
Catch{
Write-Host "Failed"
}
}
#####################start of script####################
$share = '\\name.na.xxx.net\20xServerBackup\'
$TheDrive = '\\name.na.xxx.net\20xServerBackup\VMs\'
$global:password = ""
$global:username = ""
GetSecureLogin
net use $share $global:password /USER:$global:username
[array]$DriveArray = #(TheDrive)
try{
$i=0
for ($i = $DriveArray.GetLowerBound(0); $i -le $DriveArray.GetUpperBound(); $i++) {
$tempDrv = $DriveArray[$i]
Cleanup $tempDrv
}
}
catch [Exception] {
Write-Host $_.Exception.Message
}
As you can see, I started using the example at this link with net use, but it's not doing the trick to use credentials to access the other server. powershell unc path cred
I got it to work this way, with New-PSDrive as #robert.westerlund suggests above:
$DestPath = split-path "$Drive" -Parent #this gives format without slash at and and makes powerShell *very happy*
New-PSDrive -Name target -PSProvider FileSystem -Credential $global:cred -Root "$DestPath" | Out-Null
$temp1 = Get-ChildItem -Path target:\VMs\ -Recurse -Force | Where-Object { $_.LastWriteTime -lt $limit}
Get-ChildItem -Path $Drive -Recurse -Force | Where-Object { $_.LastWriteTime -lt $limit} | Remove-Item -Force
Remove-PSDrive target
I had to add the cred part like this too:
$global:cred = new-object -typename System.Management.Automation.PSCredential -argumentlist $global:username, $global:password
I have a requirement to download files from a sharepoint online document library using powershell
I've managed to get to the point where the download should happen but no luck.
I know its something to do with how I am using the stream/writer
any hints would be greatly appreciated
*Edit
No error messages are thrown just 0 length files in my local Directory
$SPClient = [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client")
$SPRuntime = [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client.Runtime")
$webUrl = Read-Host -Prompt "HTTPS URL for your SP Online 2013 site"
$username = Read-Host -Prompt "Email address for logging into that site"
$password = Read-Host -Prompt "Password for $username" -AsSecureString
$folder = "PoSHTest"
$destination = "C:\\test"
$ctx = New-Object Microsoft.SharePoint.Client.ClientContext($webUrl)
$ctx.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($username, $password)
$web = $ctx.Web
$lists = $web.Lists.GetByTitle($folder)
$query = [Microsoft.SharePoint.Client.CamlQuery]::CreateAllItemsQuery(10000)
$result = $lists.GetItems($query)
$ctx.Load($Lists)
$ctx.Load($result)
$ctx.ExecuteQuery()
#Edited the foreach as per #JNK
foreach ($File in $result) {
Write-host "Url: $($File["FileRef"]), title: $($File["FileLeafRef"]) "
$binary = [Microsoft.SharePoint.Client.File]::OpenBinaryDirect($ctx,$File["FileRef"])
$Action = [System.IO.FileMode]::Create
$new = $destination + "\\" + $File["FileLeafRef"]
$stream = New-Object System.IO.FileStream $new, $Action
$writer = New-Object System.IO.BinaryWriter($stream)
$writer.write($binary)
$writer.Close()
}
You could also utilize WebClient.DownloadFile Method by providing SharePoint Online credentials to download the resource from SharePoint Online as demonstrated below.
Prerequisites
SharePoint Online Client Components SDK have to be installed on the machine running the script.
How to download a file in SharePoint Online/O365 in PowerShell
Download-File.ps1 function:
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client")
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client.Runtime")
Function Download-File([string]$UserName, [string]$Password,[string]$FileUrl,[string]$DownloadPath)
{
if([string]::IsNullOrEmpty($Password)) {
$SecurePassword = Read-Host -Prompt "Enter the password" -AsSecureString
}
else {
$SecurePassword = $Password | ConvertTo-SecureString -AsPlainText -Force
}
$fileName = [System.IO.Path]::GetFileName($FileUrl)
$downloadFilePath = [System.IO.Path]::Combine($DownloadPath,$fileName)
$client = New-Object System.Net.WebClient
$client.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($UserName, $SecurePassword)
$client.Headers.Add("X-FORMS_BASED_AUTH_ACCEPTED", "f")
$client.DownloadFile($FileUrl, $downloadFilePath)
$client.Dispose()
}
Usage
Download-File -UserName "username#contoso.onmicrosoft.com" -Password "passowrd" -FileUrl https://consoto.sharepoint.com/Shared Documents/SharePoint User Guide.docx -DownloadPath "c:\downloads"
I was able to download the file successfully with the following relevant code snippet. You should be able to extend it for your situation.
Add-Type –Path "C:\Program Files\Common Files\microsoft shared\Web Server Extensions\15\ISAPI\Microsoft.SharePoint.Client.dll"
Add-Type –Path "C:\Program Files\Common Files\microsoft shared\Web Server Extensions\15\ISAPI\Microsoft.SharePoint.Client.Runtime.dll"
$siteUrl = Read-Host -Prompt "Enter web URL"
$username = Read-Host -Prompt "Enter your username"
$password = Read-Host -Prompt "Enter password" -AsSecureString
$source = "/filepath/sourcefilename.dat" #server relative URL here
$target = "C:/detinationfilename.dat" #URI of the file locally stored
$ctx = New-Object Microsoft.SharePoint.Client.ClientContext($siteUrl)
$credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($username, $password)
$ctx.Credentials = $credentials
[Microsoft.SharePoint.Client.FileInformation] $fileInfo = [Microsoft.SharePoint.Client.File]::OpenBinaryDirect($ctx,$source);
[System.IO.FileStream] $writeStream = [System.IO.File]::Open($target,[System.IO.FileMode]::Create);
$fileInfo.Stream.CopyTo($writeStream);
$writeStream.Close();
While the CSOM code above likely can be made to work I find it easier to use the web client method.
(from http://soerennielsen.wordpress.com/2013/08/25/use-csom-from-powershell/)
I've used the code below, to retrieve a bunch of files (metadata from CSOM queries) to a folder (using your $result collection, other params should be adjusted a bit):
#$siteUrlString site collection url
#$outPath path to export directory
$siteUri = [Uri]$siteUrlString
$client = new-object System.Net.WebClient
$client.UseDefaultCredentials=$true
if ( -not (Test-Path $outPath) ) {
New-Item $outPath -Type Directory | Out-Null
}
$result |% {
$url = new-object Uri($siteUri, $_["FileRef"])
$fileName = $_["FileLeafRef"]
$outFile = Join-Path $outPath $fileName
Write-Host "Downloading $url to $outFile"
try{
$client.DownloadFile( $url, $outFile )
}
catch{
#one simple retry...
try{
$client.DownloadFile( $url, $outFile )
}
catch{
write-error "Failed to download $url, $_"
}
}
}
The trick here is the
$client.UseDefaultCredentials=$true
which will authenticate the webclient for you (as the current user).
The direct and almost shortest answer to the question is simply:
$url = 'https://the.server/path/to/the/file.txt'
$outfile = "$env:userprofile\file.txt"
Invoke-WebRequest -Uri $url -OutFile $outfile -Credential (Get-Credential)
This works at least in Powershell 5.1...
So I gave up on this. it turned out to be much easier to write an SSIS script component to do the job.
I have awarded Soeren as he posted some code that will work for regular websites but not sodding SharePoint Online.
Thanks Sorean!
Short an easy approach to download a file from sharepoint online, using just powershell and sharepoint online url ( no pnp powershell )
This approach can also be used to perform Sharepoint REST queries, with just powershell and sharepoint REST api
# required MS dependencies
# feel free to download them from here https://www.microsoft.com/en-us/download/details.aspx?id=42038
Add-Type -Path 'C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll' -ErrorAction Stop
Add-Type -Path 'C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll' -ErrorAction Stop
# prepare passwords
$spCredential = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($user, $(ConvertTo-SecureString -AsPlainText $pass -Force))
# prepare and perform rest api query
$Context = New-Object Microsoft.SharePoint.Client.ClientContext($targetSiteUrl)
$Context.Credentials = $spCredential
try {
#this may return an error, but still will finish context setup
$Context.ExecuteQuery()
}
catch {
write-host "TODO: fix executeQuery() err 400 bug" -ForegroundColor Yellow
}
$AuthenticationCookie = $Context.Credentials.GetAuthenticationCookie($targetSiteUrl, $true)
$WebSession = New-Object Microsoft.PowerShell.Commands.WebRequestSession
$WebSession.Credentials = $Context.Credentials
$WebSession.Cookies.SetCookies($targetSiteUrl, $AuthenticationCookie)
$WebSession.Headers.Add("Accept", "application/json;odata=verbose")
Invoke-WebRequest -Uri $spFileUrl -OutFile $outputFilePath -WebSession $WebSession -errorAction Stop
Where
$outputFilePath is the target output file in which you want to save the remote file.
$targetSiteUrl is the target sp site url.
$spFileUrl is the "[sharepoint file full url]"
$user plain text sp user email
$pass plain text sp user pass