I need to find a way so that I can read values from an excel file and then replace all the corresponding values in another file accordingly. Basically, I found some discrepancy in one of the automated task we run and I need to convert some values within the file before I send it to the automated task. I have an excel file that list the "wrong" values and their corresponding "correct" values and I need to how Power shell can help me in this.
$docID = $args[0] $docid #Read Z ticker file
$Zfile = 'I:\IS\Rishabh\Z tickers Active.xls' # Find the .rps file imported automatically from schwab trust
$RPSFile= 'L:\Trading\Schwab Trust\Import\CS<%dmmdd-01yy>.RPS'
While (Get-Content $ZFile)
{
$_-cmatch 'A$','B$'| Set-Variable X-ticker # End Loop
}
(Get-Content $RPSfile) | ForEach-Object { $_-replace '%, ' ,'X-ticker' #End Loop }
Set-Content $RPSFile
You don't need to use Powershell. Excel itself has built in mechanisms for doing what you want. For example you could use the LOOKUP function in Excel.
Related
In input :
i have an \users\myself\desktop\test\file.xslx containing multiples column like this :
ColumnA ColumnB ... ColumnQ (for a total of 17 columns)
each column have some data.
In output :
I would like to have a cmd like this :
New-ADUser -Name $(columnAdata) -GivenName "$(columnBdata)" -Surname "$(columnCdata)" -DisplayName "$(columnDdata)" -SamAccountName "$(columnEdata)" ... etc until -blabla "$(ColumnQdata)"
Is that possible to store de columndata in variables to insert them in a command ?
Thanks a lot.
I would suggest to first change the column headers to be the same as the parameters you intend to use with the New-ADUser cmdlet.
Having matching headers would help greatly in not making mistakes.
Next, save your Excel file as CSV, let's say a file called NewUsers.csv
The code then can be quite simple and easy to maintain:
# import the CSV file using the same separator character as Excel uses on your system
Import-Csv -Path 'X:\NewUsers.csv' -UseCulture | ForEach-Object {
# use Splatting: create a Hashtable with all properties needed taken from the CSV
# see: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_splatting
$userProperties = #{
Name = $_.Name # as opposed to $_.columnAdata
GivenName = $_.GivenName # as opposed to $_.columnBdata
Surname = $_.Surname
DisplayName = $_.DisplayName
SamAccountName = $_.SamAccountName
# etcetera
}
New-ADUser #userProperties
}
I need help comparing two Excel files in Powershell.
I have an Excel-file which contains 6 000 rows and 4-5 columns with headers:
"Number" "Name" "Mobile data".
Let's call it: $Services
Now, I want to compare that file with other Excel-files. For example:
one file containing 50 rows with header columns: "Number", "Name", etc.
Let's call it $Department
The important thing is that in $Services, it contains more important columns like "Mobile data",
so my mission is to compare column: "Number" from $Services with column "Number" from each other Excel file.
Then if they match, write "the whole row" from $Services
I'm not that familiar with Excel, so I thought, this should be possible to do in Powershell.
I'm novice in Powershell, so I only know basic stuff. I'm not that familiar with pscustomobject and param.
Anyway, what I tried to do was to first declare them in variables with ImportExcel:
$Services = Import-Excel -Path 'C:\Users\*.xlsx'
$Department = Import-Excel -Path 'C:\Users\*.xlsx'
Then I made a foreach statement:
foreach ($Service in $Services) {
if (($Service).Number -like ($Department).Number)
{Write-Output "$Service"}
}
The problem with this is that it is collecting all empty columns from ($Services).Number and writing the output of each row in $Services.
I tried to add a nullorEmpty to $Department, if the .Number is empty, but it didn't make any difference. I also tried to add that if the row is empty in .Number, add "1234", but still it collects all .Number that is empty in $Services.
I also tried to do a: $Services | ForEach-Object -Process {if (($_).Number -match ($Department).Number)
{Write-Output $_}} But it didn't match any. When I tried -notmatch it took all.
I don't know but it seems that I have to convert the files to objects, like the columns to object so each string becomes an object. But right now my head is just spinning and I need some hints on where I can start with this.
I would recommend downloading the Module ImportExcel from the PSGallery.
Import-Excel can easily import your Excel sheet(s) to rows of objects, especially if your sheets are 'clean', i.e., only contain (optional) headers and data rows.
Simply import the cells to PowerShell objects and use Compare-Object to discover differences.
EDIT (after reading the additional questions by poster in the comments):
To compare using specific properties you'll need to add these to the Compare-Object parameters.
Using a trivial "PSCustomObject" to create a simple set of objects to show this idea it might look like this:
$l = 1..4 | ForEach-Object { [pscustomobject]#{a=$_;b=$_+1} }
$r = 1,2,4,5 | ForEach-Object { [pscustomobject]#{a=$_;b=$_+1} }
compare-object $l $r -Property B
B SideIndicator
- -------------
6 =>
4 <=
You may also compare multiple properties this way:
compare-object $l $r -Property A,B
A B SideIndicator
- - -------------
5 6 =>
3 4 <=
FYI: I find myself typing "Get-Command -Syntax SomeCommand" so often every day that I just made a function "Get-Syntax" (which also expands aliases) and then aliased this to simply "syn".
90% of the time once you understand the structure of PowerShell cmdlets (at least well-written ones) there is no need to even look at the full help -- the "syntax" blocks are sufficient.
Until then, type HELP (Get-Help) a lot -- 100+ times per day. :)
So the solution for my whole problem was to add -PassThru.
Because my mission was to compare the numbers of the two Excel-files, select the numbers that equals and then take all the properties from one file. So my script became like this:
$Compare = Compare-Object $Services $Department -Property Numbers -IncludeEqual -ExcludeDifferent -PassThru
$Compare | Export-Excel -Path 'C:\Users\*
But I wonder, -PassThru sends all the objects from ReferenceObject, how can I send all the objects from DifferenceObject?
I need your help regarding PowerShell programming on CSV file.
I've made some searches but cannot find what I'm looking for (or perhaps I don't know the technical terms). Basically, I have an Excel workbook with large amount of data (more or less 38 columns x 350.000 rows), and there are a couple of formulas that take hours to calculate.
I was first wondering if PowerShell could speed up a bit the calculation compared to Excel. The calculations taking most of my time are in fact not that complex (at least at first glance). My data is more or less constructed like this:
Ref Title
----- --------------------------
A/001 "free_text"
A/002 "free_text A/001 free_text"
... ...
A/005 "free_text A/004 free_text"
A/006 "free_text"
B/001 "free_text"
B/002 "free_text"
C/001 "free_text"
C/002 "free_text"
...
C/050 "free_text C/047 free_text"
... ...
C/103 "free_text"
D/001 "free_text"
D/002 "free_text D/001 free_text"
... ....
Basically the data is as follows:
the Ref field contains unique values, in {letter}/{incremental value} format.
In some rows, the Title field may call up one of the Ref data. For example, in line 2, the Title calls for the A/001 Ref. In the last row, the Title calls for the D/001 Ref, etc.
There is no logic pattern defining when this ref could be called up in a title. This is random.
However, what I'm 100% sure of is the following:
The Ref called in the Title is always belonging to the same {letter} block. For example: the string 'C/047' in the Title field can only be found in the block where the Ref {letter} is C.
The Ref called in the Title will always be located 'after' (or in a lower row) than the Ref it refers to. In other words, I cannot have a line with following pattern:
Ref Title
------------ -----------------------------------------
{letter/i} {free_text {letter/j} free_text} with j<i
→ This is not possible.
→ j is always > i
I've used these characteristics in Excel to minimize my lookup arrays. But it still takes an hour to calculate everything.
I've therefore looked into PowerShell, and started to 'play' a bit with the CSV, and looping with the ForEach-Object hoping I would have quicker results. Up to now I basically ended-up looping twice on my CSV file.
$CSV1 = myfile.csv
$CSV2 = myfile.csv
$CSV1 | ForEach-Object {
# find Title
$TitSearch = $_.$Ref
$CSV2 | ForEach-Object {
if ($_.$Title -eq $TitSearch) {
myinstructions
}
}
}
It works but it's really really really long. So I then tried the following instead of using the $CSV2 | ForEach...:
$CSV | where {$_.$Title -eq $TitleSearch} | % $Ref
In either case, it's too long and not efficient at all. Additionally with these 2 solutions, I'm not using above characteristics which could reduce the lookup array and as already stated, it seems I end up looping twice on the CSV file from its beginning up to the end.
Questions:
Is there a leaner way to do this?
Am I wasting my time with PowerShell?
I though about creating 1 file per Ref {letter} block (1 file for block A, 1 for B, etc...). However I have about 50.000 blocks to create. Or create them one by one, carry out the analysis, put the results in a new file, and delete them. Would that be quicker?
Note: this is for work, to be used by other colleagues, and Excel and PowerShell are really the only softwares we may use. I know VBA but ok... At the end I'm curious about how and if this can be solved in a simple manner using PowerShell.
As far as I can see your base algorithm do N^2 iteration (~120 billion). There is a standard way to make it efficient - you need to build a hashtable first. Hashtable is a key/value storage, and look up is pretty much instantaneous, so algorithm's time complexity will become ~N.
Powershell has built-in data type for that. In your case the key would be ref, and the value an array of cell data (assuming your table is smth like: ref, title, col1, ..., colN)
$hash = #{}
foreach($row in $table} {$hash.Add($row.ref, #($row.title, $row.col1, ...)}
#it will take 350K steps to generate it
#then you can iterate over it again
foreach($key in $hash.Keys) {
$key # access current ref
$rowData = $hash.$key # access to current row elements (by index)
$refRowData = $hash[$rowData[$j]] # lookup from other rows, assuming lookup reference is in some column
}
So it's a general idea how to solve the time issue. To be honest I don't believe you need to recreate a wheel and code it yourself. What you need is a relational database. Since you have excel, you should have MS ACCESS too. Just import your data in there, make ref and title an index, then all you need to do is self join. MS Access suck, but I'm sure it will handle 350K row just fine.
Ideally you'd need to get a database on some corporate MSSQL server (open a ticket, talk to your manger, etc). It will calculate all that in seconds, and then you can link the output to a spreadsheet as well.
I've found myriad methods to retrieve data FROM a string with substrings, but what I want to do is create a new string that contains substrings. The reason for this is that I want to pass that string to a CSV through the Export-CSV cmdlet. This is in a PowerShell Forms created app.
So the plan would be to
1). Read the contents of each text box:
(e.g. $endusername.text $endusernumber.text $locationname.text)
2). Store those into a new string with substrings
($formoutput.endusername $formoutput.endusernumber $formoutput.locationname)
3). Output the string to a .CSV
Export-CSV -InputObject $formoutput "c:\output\formoutput.csv"
Basically, if I take any existing cmdlet (say, Get-Mailbox), store its output as a string, and then pass that string through the Export-CSV in the way explained above, it performs exactly the way I like - creating a .CSV with each of the substrings as a column, and the contents of that substring in the appropriately headed column. I just want to be able to do that with a string containing substrings that I define.
I think you are confusing nomenclature a little bit. It sounds like what you want is a custom object not a string. Here is some pseudo-code to get you going in the right direction:
$formOutput = New-Object PsCustomObject -Property #{'EndUserName' = $endUserName.Text;
'EndUserNumber' = $endUserNumber.Text;
'LocationName' = $locatioName.Text}
$formOutput | Export-CSV .\FileName.csv -NoTypeHeader
I have multiple folders. There are multiple txt files inside these folder. I need to extract data (just a single value: value --->554) from a particular type of txt file in this folder.(individual_values.txt)
No 100 Value 555 level match 0.443 top level 0.443 bottom 4343
There will be many folders with same txt file names but diff value. Can all these values be copyed to excel one below the other.
I have to extract a value from a txt file which i mentioned above. Its a same text file with same name located inside different folders. All i want to do is extract this value from all the text file and paste it in excel or txt one below the other in each row.
Eg: The above is a text file here I have to get the value of 555 and similarly from other diff values.
555
666
666
776
Yes.
(you might want to clarify your question )
Your question isn't very clear, I imagine you want to know how this can be done.
You probably need to write a script that traverses the folders, reads the individual files, parses them for the value you want, and generates a Comma Separated Values (CSV) file. CSV files can easily be imported to Excel.
There are two or three basic methods you can use to get stuff into a Excel Spreadsheet.
You can use OLE wrappers to manipulate Excel.
You can write the file in a binary form
You can use Excel's import methods to take delimited text in as a spreadsheet.
I chose the latter way, because 1) it is the simplest, and 2) your problem is so poorly stated as it does not require a more complex way. The solution below outputs a tab-delimited text file that Excel can easily support.
In Perl:
use IO::File;
my #field_names = split m|/|, 'No/Value/level match/top level/bottom';
#' # <-- catch runaway quote
my $input = IO::File->new( '<data.txt' );
die 'Could not open data.txt for input!' unless $input;
my #data_rows;
while ( my $line = <$input> ) {
my %fields = $line =~ /(level match|top level|bottom|Value|No)\s+(\d+\S*)/g;
push #data_rows, \%fields if exists $fields{Value};
}
$input->close();
my $tab_file = IO::File->new( '>data.tab' );
die 'Could not open data.tab for output!' unless $tab_file;
$tab_file->print( join( "\t", #field_names ), "\n" );
foreach my $data_ref ( #data ) {
$tab_file->print( join( "\t", #$data_ref{#field_names} ), "\n" );
}
$tab_file->close();
NOTE: Excel's text processing is really quite neat. Try opening the text below (replacing the \t with actual tabs) -- or even copying and pasting it:
1\t2\t3\t=SUM(A1:C1)
I chose c#, because i thought it would be fun to use a recursive lambda. This will create the csv file containing matches to the regex pattern.
string root_path = #"c:\Temp\test";
string match_filename = "test.txt";
Func<string,string,StringBuilder, StringBuilder> getdata = null;
getdata = (path,filename,content) => {
Directory.GetFiles(path)
.Where(f=>
Path.GetFileName(f)
.Equals(filename,StringComparison.OrdinalIgnoreCase))
.Select(f=>File.ReadAllText(f))
.Select(c=> Regex.Match(c, #"value[\s\t]*(\d+)",
RegexOptions.IgnoreCase))
.Where(m=>m.Success)
.Select(m=>m.Groups[1].Value)
.ToList()
.ForEach(m=>content.AppendLine(m));
Directory.GetDirectories(path)
.ToList()
.ForEach(d=>getdata(d,filename,content));
return content;
};
File.WriteAllText(
Path.Combine(root_path, "data.csv"),
getdata(root_path, match_filename, new StringBuilder()).ToString());
No.
just making sure you have a 50/50 chance of getting the right answer
(assuming it was a question answerable by Yes and No) hehehe
File_not_found
Gotta have all three binary states for the response.