I have a list of devices that I would like to ping against to see if they are active or not. If they are active (even if not), I would like to log to a file. So, the active devices go to one file, and the "unpingable" devices are written to another file. The script I have below works okay for a smaller sampling of computers, but when I add the complete list, the "test-connection" section of the script takes hours to complete (about 8,000 devices). Is there a way to improve the performance by running the command in parallel? In my search, i came across this wrapper function by David Wyatt => TestConnectionAsync but I'm unsure how to make it work with separating results in two files. Any help would be appreciated.
Code:
ForEach ($PC in $Computer_List) {
If (Test-Connection -ComputerName $PC -Quiet -Count 1) {
Add-Content -value $PC -Path "$UPpath"
} Else {
Add-Content -Value $PC -Path "$DOWNpath"
}
}
The output from Test-ConnectionAsync makes this quite easy:
When you specify the -Quiet switch of Test-ConnectionAsync, it returns a collection of PSCustomObjects with the properties "ComputerName" and "Success". Success is the boolean value you'd have received from Test-Connection -Quiet ; the PSCustomObject just allows you to associate that result with the target address.
So all you've got to do is:
Ping all computers and capture the output
Look at the Success attribute to see if you should put it in one file or another
# 1. Gather ping results
$TestResults = $ComputerList | Test-ConnectionAsync -Quiet
# 2. Loop through results and look at Success
foreach($Result in $TestResults){
$FilePath = if($Result.Success){
$UPpath
} else {
$DOWNpath
}
Add-Content -Path $FilePath -Value $Result.ComputerName
}
Related
I have created a script in the Azure PowerShell.
If I use the "echo" command, it displays output to the console.
However, if I use Write-Output and Write-Error, I don't see the output.
I have uploaded the script "change-to-static.ps1" to a storage account. Then I open the "Cloud Shell" with a button at the top bar. Then I type "./change-ip-to-static.ps1" in the PowerShell console.
Therefore, the script does not produce any output unless I replace "Write-Output" and "Write-Error" with "echo" or "print".
Please help me. What should I do to see the output?
The script is below.
There is a similar question at How to output something in PowerShell. I have read it, but there are no concrete examples on how to achieve my goal, i.e. how to modify my script to see the output. And in my case, it does not output even if I redirect to a text file. However, commands like "echo" and "print" in my case work but they are not covered in the above example. See the script below.
$IPs = Get-AzPublicIpAddress;
$Static = "Static";
foreach ($PublicIP in $IPs) {
$Method = $PublicIP.PublicIpAllocationMethod;
$Name = $PublicIP.Name;
if ($Method -eq $Static) {
$message = "The method of " + $Name + " is already " + $Static;
Write-Progress -Activity $message;
}
else {
Write-Progress -Activity "Changing the method of "+$Name+" from "+$Method+" to "+$Static+"...";
$PublicIP.PublicIpAllocationMethod = $Static;
Set-AzPublicIpAddress -PublicIpAddress $PublicIP;
Write-Progress -Activity "Querying the method of "+$Name+"...";
$ModifiedAddress = Get-AzPublicIpAddress -Name $Name -ResourceGroupName $PublicIP.ResourceGroupName -Location $PublicIP.Location
$NewMethod = $ModifiedAddress.PublicIpAllocationMethod;
if ($NewMethod -eq $Static) {
Write-Output "The method for "+$Name+" has successfully changed to "+$Static;
}
else {
Write-Error -Message "Cannot change the method for "+$Name+" to "+$Static+", it is still "+$NewMethod+"!!!";
}
}
}
P.S. I have updated the script (use this URL) according to the suggestions, but there is still no output. Only "echo" or "print" gives the output.
P.P.S. The Write-Progress does not even show a temporary message in the status line during Set-AzPublicIpAddress which takes a couple of seconds to complete, or if I add the Start-Sleep cmdlet. It does only set during Get-AzPublicIpAddress.
After reading your last edit to my answer, I believe you made a bit of confusion in using Write-* commandlets, and in your script logic, so I provided a more detailed answer with context.
echo in the Powershell Azure Cloud Shell is an alias of Write-Output, as executing echo without parameters clearly shows (docs here ).
PS /home/mikelangelo> echo
cmdlet Write-Output at command pipeline position 1
Supply values for the following parameters:
InputObject:
Moreover: the unix echo can also be run in the Powershell Azure Cloud Shell.
PS /home/mikelangelo> which echo
/usr/bin/echo
PS /home/mikelangelo> /usr/bin/echo ciao mondo
ciao mondo
print, on the other hand, is not a Powershell alias, so the unix counterpart is the one which always get executed when using the print keyword (presently a symlink to run-mailcap - but it's not clear to me how it comes into play into your use case.)
PS /home/mikelangelo> which print
/usr/bin/print
So, basically, echo and Write-Output will both work, because they call the same commandlet, unless you execute /usr/bin/echo directly, mixing up technologies and effectively impairing portability.
Back to the question:
Write-Output works as expected. The logic is faulty: You use = as a comparison operator, but you need to use -eq instead.
Write-Progress needs to be used differently, replace it with Write-Host or Write-Output. Refer to the docs for an explanation.
Note that Write-Output sends an object down the pipeline, which can eventually be represented as a console output.
Write-Progress and Write-Host, on the other hand, do not generate output - the latter sends an object to the host for displaying, though, so Write-Host is the recommended way to display something in the console. Refer to this question for more details on Write-Host, Write-Output and the Powershell pipeline.
Like other commenters before me I can also confirm that the code from your gist works just fine in Azure Cloud Shell.
I noticed that there is only an output if you have at least one dynamic public ip that the script can change to static. The reason is, that only in this case you use Write-Output to return a string to the console.
If there is no dynamic public ip left, your script only writes a progress message, but you never get to see it, as the script execution ends too quickly after you write the message and progress messages don't linger.
Put the command Start-Sleep -Seconds 2 under the line with Write-Progress and you will see what I mean:
$IPs = Get-AzPublicIpAddress;
$Static = "Static";
foreach ($PublicIP in $IPs) {
$Method = $PublicIP.PublicIpAllocationMethod;
$Name = $PublicIP.Name;
if ($Method -eq $Static) {
$message = "The method of $Name is already $Static";
Write-Progress -Activity $message;
Start-Sleep -Seconds 2 # This will keep the script running 2 seconds longer and the message visible.
}
else {
Write-Progress -Activity "Changing the method of $Name from $Method to $Static ...";
$PublicIP.PublicIpAllocationMethod = $Static;
Set-AzPublicIpAddress -PublicIpAddress $PublicIP;
Write-Progress -Activity "Querying the method of $Name ...";
$ModifiedAddress = Get-AzPublicIpAddress -Name $Name -ResourceGroupName $PublicIP.ResourceGroupName
$NewMethod = $ModifiedAddress.PublicIpAllocationMethod;
if ($NewMethod -eq $Static) {
Write-Output "The method for $Name has successfully changed to $Static";
}
else {
Write-Error -Message "Cannot change the method for $Name to $Static, it is still $NewMethod!!!";
}
}
}
Write-Progress is probably not the cmdlet that you want to use to write out the progress of your script (despite it's name). As I do not see how you would need to further process the output of your script, you might as well replace it with Write-Host.
I have multiple server log files. In total they contain around 500.000 lines of log text. I only want to keep the lines that contain "Downloaded" and "Log". Lines I want to exclude are focussing on error logs and basic system operations like "client startup", "client restart" and so on.
An example of the lines we are looking for is this one:
[22:29:05]: Downloaded 39 /SYSTEM/SAP logs from System-4, customer (000;838) from 21:28:51,705 to 21:29:04,671
The lines that are to be kept should be complemented by the date string, which is part of the log-file name. ($date)
Further, as the received logs are rather unstructured, the filtered files should be transformed into one csv-file (columns: timestamp, log downloads, system directory, system type, customer, start time, end time, date [to be added to every line from file name]. The replace operation of turning spaced into comma is just a first try to bring in some structure to the data. This file is supposed to be loaded into a python dashboard program.
At the moment it takes 2,5 mins to preprocess 3 Txt-Files, while the target is 5-10 seconds maximum, if even possible.
Thank you really much for your support, as I'm struggeling with this since Monday last week. Maybe powershell is not the best way to go? I'm open for any help!
At the moment I'm running this powershell script:
$files = Get-ChildItem "C:\Users\AnonUser\RestLogs\*" -Include *.log
New-Item C:\Users\AnonUser\RestLogs\CleanedLogs.txt -ItemType file
foreach ($f in $files){
$date = $f.BaseName.Substring(22,8)
(Get-Content $f) | Where-Object { ($_ -match 'Downloaded' -and $_ -match 'SAP')} | ForEach-Object {$_ -replace " ", ","}{$_+ ','+ $date} | Add-Content CleanedLogs.txt
}
This is about the fastest I could manage. I didn't test using -split vs -replace or special .NET methods:
$files = Get-ChildItem "C:\Users\AnonUser\RestLogs\*" -Include *.log
New-Item C:\Users\AnonUser\RestLogs\CleanedLogs.txt -ItemType file
foreach ($f in $files) {
$date = $f.BaseName.Substring(22,8)
(((Get-Content $f) -match "Downloaded.*?SAP") -replace " ",",") -replace "$","$date" | add-content CleanedLogs.txt
}
In general, speed is gained by removing loops and Where-Object "filtering."
foreach ($Target in $TargetUSBs)
{
$LogPath= #"
$SourceUSB\$(((Get-CimInstance -ClassName Win32_volume)|where {$_.DriveType -eq "2" -and $_.DriveLetter -eq $Target}).SerialNumber)_
$(((Get-CimInstance -ClassName Win32_OperatingSystem).LocalDateTime).Year)$(((Get-CimInstance -ClassName Win32_OperatingSystem).LocalDateTime).Month)
$(((Get-CimInstance -ClassName Win32_OperatingSystem).LocalDateTime).Day)_$(((Get-CimInstance -ClassName Win32_OperatingSystem).LocalDateTime).Hour)
$(((Get-CimInstance -ClassName Win32_OperatingSystem).LocalDateTime).Minute)$(((Get-CimInstance -ClassName Win32_OperatingSystem).LocalDateTime).Second).txt
"#
$LogPath = $LogPath.Replace("`n","").Trim()
New-item -Path "$LogPath"
}
The Irony is when I copy and paste the contents of my variable and manually create a new-item -path and paste said contents it works but when I use the variable it does not...
Brief summary of my goal I am taking a USB labelled ORIGINAL and obtaining the S/N of every USB plugged in at the time and creating separate log files for each with the title consisting of SERIALNUMBER_DATE_TIME.txt and these files are created in ORIGINAL USB
$LogPath contains for example the following: E:\Mattel\1949721369_2018912_93427.txt
Yet when I use the Variable in New-item it indicates "Illegal characters in Path"
FYI $LogPath is a System.String not an object
$TargetUSBs is filled with all USB drives plugged into the system
this method of using a variable for a path usually works fine for me only difference is the here-string I used this time around does this cause my problem? I hope not because I really don't want to fill that variable all on one line. New-Item's helpfiles shows <String[]> for -path parameter does this mean I have to use a string array? and if so how do I convert this to make this work?
Your problem is that Windows uses CRLF line endings (Unix only LF), so you still have CR chars in your path.
To fix this just use:
.Replace("`r`n","")
However you can easily simplify your code so you do not require the messy here-string or replace/trim...
By using a single Get-Date call you can format it to your desired output. This means you can just build the Path as a simple string and involves much less code:
foreach ($Target in $TargetUSBs)
{
$SerialNumber = Get-CimInstance -ClassName Win32_volume | where {$_.DriveType -eq "2" -and $_.DriveLetter -eq $Target} | Select-Object -ExpandProperty SerialNumber
$DateTime = Get-Date -Format "yyyyMd_Hms"
New-item -Path "$SourceUSB\$SerialNumber_$DateTime.txt"
}
I am looking for a bit of help, hope nobody will bash me for being an ignorant.
Not that long ago I became something of an AD admin, organisation is big so the tasks vary. I can easily complete what I require via Powershell or snap-ins in most cases.
However I have a task on my hands that exceed my "creativity". I have a list of over 10 000 users in .csv which I need to look up in on-premises AD if they exist. My two problems are:
-I am very new to scripting and getting increasingly frustrated that I can't comprehend it and make my scripts work as I need them to
-Deadline for this task and other responsibilities give me little time to read more on scripting basics and learn. As such I am in most cases forced to look for script snippets on the web and modify them a bit to meet my needs. This worked up until now as the script I have on my hands is a bit too complex for me.
Biggest problem I was facing so far is creating a forest-wide search. My organization have a single root domain and 4 child domains. When running a simple foreach loop a like the one below:
ForEach ($User in (Import-Csv c:\users\public\users.csv))
{ If (Get-ADUser $User.mail -server GLOBALCATALOGADDRESS:xxxx)
{ Write-Host "User found: $($User.mail)"
}
Else
{ Write-Host "User not found: $($User.mail)"
}
}
It searches only domain to which my computer is connected.
So I managed to find and modify a forest-wide search script and came up with following:
#Get Domain List
$objForest = [System.DirectoryServices.ActiveDirectory.Forest]::GetCurrentForest()
$DomainList = #($objForest.Domains | Select-Object Name)
$Domains = $DomainList | foreach {$_.Name}
$User = Import-CSV c:\users\public\users.csv
#Act on each domain
foreach($Domain in ($Domains))
{
Write-Host "Checking $Domain" -fore red
$ADsPath = [ADSI]"LDAP://$Domain"
$objSearcher = New-Object System.DirectoryServices.DirectorySearcher($ADsPath)
#The filter
Foreach($mail in($User))
{
$objSearcher.Filter = "(&(objectCategory=user)(mail=$User.mail))"
$objSearcher.SearchScope = "Subtree"
$colResults = $objSearcher.FindAll()
foreach ($objResult in $colResults)
{
$objArray = $objResult.GetDirectoryEntry()
write-host $objArray.mail
}
}
}
The script seems to be good in its original form (found here: http://powershell.nicoh.me/powershell-1/active-directory/forest-wide-object-searches) and searches well with wildcard and single parameter as filter.
However I have no idea what am I missing to make it search for every email address I have in .csv and to make it return information whether or not user with such mail was found.
Script itself runs but given the time it takes and blank output it feels like it searches for only one user. I am 100% sure that at least one user from the list exists in on-prem AD.
Any suggestions are very welcome.
Thanks for your attention.
[EDIT]
Final script:
#Get Domain List and load user e-mails from file
$objForest = [System.DirectoryServices.ActiveDirectory.Forest]::GetCurrentForest()
$DomainList = #($objForest.Domains | Select-Object Name)
$Domains = $DomainList | foreach {$_.Name}
$Users = Import-CSV c:\users\public\users.csv
#Act on each domain
foreach($Domain in ($Domains))
{
Write-Host "Checking $Domain" -fore red
Foreach($mail in ($Users.mail))
{
Get-ADUser -filter {mail -eq $mail} -Server $domain -properties mail | select mail
}
}
Do yourself a favour and download AD Powershell module: http://blogs.msdn.com/b/rkramesh/archive/2012/01/17/how-to-add-active-directory-module-in-powershell-in-windows-7.aspx
You will then be able to simplify your code and run things like this, making your task much clearer:
...
foreach($Domain in ($Domains))
{
Write-Host "Checking $Domain" -fore red
Foreach($mail in ($User.mail))
{
Get-ADUser -filter {mail -eq $mail} -Server $domain -Properties mail |
select-object -ExpandProperty mail
}
}
...
More on AD PS cmdlets: http://technet.microsoft.com/en-us/library/ee617195.aspx
Use -LDAPfilter & point the -Server to GC.
Get-ADUser -Server DC01.Contoso.com:3268
-Ldapfilter "(ObjectClass=user)(mailnickname=David)"
The above command will search the GC DC01.contoso.com for all the users that their Alias/mailnickname is David.
Is is enough to contact the Domain itself instead of a DC of the domain. Thus this shoud also work
get-aduser -Filter {mailnickname -eq "David") -Server contoso.com:3268
What's the DOS FINDSTR equivalent for PowerShell? I need to search a bunch of log files for "ERROR".
Here's the quick answer
Get-ChildItem -Recurse -Include *.log | select-string ERROR
I found it here which has a great indepth answer!
For example, find all instances of "#include" in the c files in this directory and all sub-directories.
gci -r -i *.c | select-string "#include"
gci is an alias for get-childitem
Just to expand on Monroecheeseman's answer. gci is an alias for Get-ChildItem (which is the equivalent to dir or ls), the -r switch does a recursive search and -i means include.
Piping the result of that query to select-string has it read each file and look for lines matching a regular expression (the provided one in this case is ERROR, but it can be any .NET regular expression).
The result will be a collection of match objects, showing the line matching, the file, and and other related information.
if ($entry.EntryType -eq "Error")
Being Object Oriented, you want to test the property in question with one of the standard comparison operators you can find here.
I have a PS script watching logs remotely for me right now - some simple modification should make it work for you.
edit: I suppose I should also add that is a cmdlet built for this already if you don't want to unroll the way I did. Check out:
man Get-EventLog
Get-EventLog -newest 5 -logname System -EntryType Error
On a related note, here's a search that will list all the files containing a particular regex search or string. It could use some improvement so feel free to work on it. Also if someone wanted to encapsulate it in a function that would be welcome.
I'm new here so if this should go in it's own topic just let me know. I figured I'd put it her since this looks mostly related.
# Search in Files Script
# ---- Set these before you begin ----
$FolderToSearch="C:\" # UNC paths are ok, but remember you're mass reading file contents over the network
$Search="Looking For This" # accepts regex format
$IncludeSubfolders=$True #BUG: if this is set $False then $FileIncludeFilter must be "*" or you will always get 0 results
$AllMatches=$False
$FileIncludeFilter="*".split(",") # Restricting to specific file types is faster than excluding everything else
$FileExcludeFilter="*.exe,*.dll,*.wav,*.mp3,*.gif,*.jpg,*.png,*.ghs,*.rar,*.iso,*.zip,*.vmdk,*.dat,*.pst,*.gho".split(",")
# ---- Initialize ----
if ($AllMatches -eq $True) {$SelectParam=#{AllMatches=$True}}
else {$SelectParam=#{List=$True}}
if ($IncludeSubfolders -eq $True) {$RecurseParam=#{Recurse=$True}}
else {$RecurseParam=#{Recurse=$False}}
# ---- Build File List ----
#$Files=Get-Content -Path="$env:userprofile\Desktop\FileList.txt" # For searching a manual list of files
Write-Host "Building file list..." -NoNewline
$Files=Get-ChildItem -Include $FileIncludeFilter -Exclude $FileExcludeFilter -Path $FolderToSearch -ErrorAction silentlycontinue #RecurseParam|Where-Object{-not $_.psIsContainer} # #RecurseParam is basically -Recurse=[$True|$False]
#$Files=$Files|Out-GridView -PassThru -Title 'Select the Files to Search' # Manually choose files to search, requires powershell 3.0
Write-Host "Done"
# ---- Begin Search ----
Write-Host "Searching Files..."
$Files|
Select-String $Search #SelectParam| #The # instead of $ lets me pass the hastable as a list of parameters. #SelectParam is either -List or -AllMatches
Tee-Object -Variable Results|
Select-Object Path
Write-Host "Search Complete"
#$Results|Group-Object path|ForEach-Object{$path=$_.name; $matches=$_.group|%{[string]::join("`t", $_.Matches)}; "$path`t$matches"} # Show results including the matches separated by tabs (useful if using regex search)
<# Other Stuff
#-- Saving and restoring results
$Results|Export-Csv "$env:appdata\SearchResults.txt" # $env:appdata can be replaced with any UNC path, this just seemed like a logical place to default to
$Results=Import-Csv "$env:appdata\SearchResults.txt"
#-- alternate search patterns
$Search="(\d[-|]{0,}){15,19}" #Rough CC Match
#>
This is not the best way to do this:
gci <the_directory_path> -filter *.csv | where { $_.OpenText().ReadToEnd().Contains("|") -eq $true }
This helped me find all csv files which had the | character in them.
PowerShell has basically precluded the need for findstr.exe as the previous answers demonstrate. Any of these answers should work fine.
However, if you actually need to use findstr.exe (as was my case) here is a PowerShell wrapper for it:
Use the -Verbose option to output the findstr command line.
function Find-String
{
[CmdletBinding(DefaultParameterSetName='Path')]
param
(
[Parameter(Mandatory=$true, Position=0)]
[string]
$Pattern,
[Parameter(ParameterSetName='Path', Mandatory=$false, Position=1, ValueFromPipeline=$true)]
[string[]]
$Path,
[Parameter(ParameterSetName='LiteralPath', Mandatory=$true, ValueFromPipelineByPropertyName=$true)]
[Alias('PSPath')]
[string[]]
$LiteralPath,
[Parameter(Mandatory=$false)]
[switch]
$IgnoreCase,
[Parameter(Mandatory=$false)]
[switch]
$UseLiteral,
[Parameter(Mandatory=$false)]
[switch]
$Recurse,
[Parameter(Mandatory=$false)]
[switch]
$Force,
[Parameter(Mandatory=$false)]
[switch]
$AsCustomObject
)
begin
{
$value = $Pattern.Replace('\', '\\\\').Replace('"', '\"')
$findStrArgs = #(
'/N'
'/O'
#('/R', '/L')[[bool]$UseLiteral]
"/c:$value"
)
if ($IgnoreCase)
{
$findStrArgs += '/I'
}
function GetCmdLine([array]$argList)
{
($argList | foreach { #($_, "`"$_`"")[($_.Trim() -match '\s')] }) -join ' '
}
}
process
{
$PSBoundParameters[$PSCmdlet.ParameterSetName] | foreach {
try
{
$_ | Get-ChildItem -Recurse:$Recurse -Force:$Force -ErrorAction Stop | foreach {
try
{
$file = $_
$argList = $findStrArgs + $file.FullName
Write-Verbose "findstr.exe $(GetCmdLine $argList)"
findstr.exe $argList | foreach {
if (-not $AsCustomObject)
{
return "${file}:$_"
}
$split = $_.Split(':', 3)
[pscustomobject] #{
File = $file
Line = $split[0]
Column = $split[1]
Value = $split[2]
}
}
}
catch
{
Write-Error -ErrorRecord $_
}
}
}
catch
{
Write-Error -ErrorRecord $_
}
}
}
}
FYI:
If you update to Powershell version 7 you can use grep...
I know egrep is in powershell on Azure CLI...
But SS is there!
An old article here: [https://devblogs.microsoft.com/powershell/select-string-and-grep/]