Decoding base64 with powershell - base64

I'm attempting to decode an attachment in an email file from base64 and save it to disk.
For testing purposes, this is my code. Where input.txt contains just the base64 encoded data, which is an HTML file.
$file = "C:\input.txt"
$data = Get-Content $file
[System.Text.Encoding]::ASCII.GetString([System.Convert]::FromBase64String($data)) > out.html
The decoding works fine, and it generates a new file that contains all of the lines, and is visibly identical to the original attachment. The problem is that the output file is twice the size (actually (filesize*2)+6 bytes, in this case).
Am I decoding this improperly? I've also tried UTF8 instead of ASCII... same result.

Well I got it working. Who knew Out-File re-encoded to Unicode by default? Solved by doing the following:
$file = "C:\input.txt"
$data = Get-Content $file
[System.Text.Encoding]::ASCII.GetString([System.Convert]::FromBase64String($data)) | Out-File -Encoding "ASCII" out.html

This one-liner preserves the original encoding of the base64 encoded file, so it will work with binary files such as a PDF or ZIP. Change ".\input.txt" and output.bin as needed - this will take .\input.txt, base 64 decode it, and then write the bytes out to output.bin exactly as they were when the file was encoded.
$file = ".\input.txt"; [System.Convert]::FromBase64String((Get-Content $file)) | Set-Content output.bin -Encoding Byte

On Windows 10, using PowerShell you can do:
certutil -decode in.b64 out.txt

Old question, new answer. Solution that preserv newlines
// Generate Content
ssh-keygen -t rsa -b 2048 -f example_rsa
// Encode
[System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($(Get-Content -Path example_rsa -Encoding utf8 -Raw))) >> encoded
// Decode
[System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String($(Get-Content -Path encoded -Encoding utf8 -Raw))) >> decoded

I know it's an old question, but i found another answer.
You can use unicode at the base64 conversion, it will fit after.
$file = "C:\temp\input.txt"
$data = Get-Content $file
[System.Text.Encoding]::Unicode.GetString([System.Convert]::FromBase64String($data)) > out.html

This function should behave the same as the linux base64 command, at least in regards to encoding and decoding. It can also accept values from pipes.
function base64
{
[CmdletBinding()]
param(
[Parameter(Mandatory, ValueFromPipeline)]
[string]
$str,
[switch]
$decode = $false
)
if ($decode)
{
return [System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String($str))
}
else
{
return [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($str))
}
}
PS > base64 -decode YXNk
asd
PS > base64 YXNk
WVhOaw==
PS > echo YXNk | base64
WVhOaw==
PS > echo YXNk | base64 -decode
asd
Note that Get-Content returns an Object[] by default
PS > $z=Get-Content .\README.md; $z.GetType()
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True True Object[] System.Array
Piping this into the func above will only convert the first line.
To convert an entire file, instead use Get-Content -Raw
Get-Content -Raw README.md | base64

Related

Linux Base64 encoded value produces spaces

I have the following bash script
TOKEN=$(aws ecr get-login-password --region us-east-2)
echo $TOKEN
USERNAME="AWS"
ENCODE_ME="${USERNAME}:${TOKEN}"
echo $ENCODE_ME
BASE_64_ENCODED=$(echo -n "${USERNAME}:${TOKEN}" | base64)
# BASE_64_ENCODED=$(printf "%s" "${CONVERT_ME}" | base64)
echo $BASE_64_ENCODED
the value of $ENCODE_ME gives something like below:
AWS:eyJwYXlsb2FkIjoidS9ralRSNTJpQWdJVC9DV1RQbFJOdEduUXUrdkkxcFc2dzYzckhiR0NSdGpLbEZsK0ZFN1YzU1l3ckZjSVh0RnNlcXJmMzJQMFVMTzQ5VWFLNdmQVlKS29aSWh2Y05BUWNHb0c4d2JRSUJBREJvQmdrcWhraUc5dzBCQndFd0hnWUpZSVpJQVdVREJBRXVNQkVFREExZ1k3TEtib1A3d1JKN3lBSUJFSUE3c1NtRFNkUC9yUXhLaDhWOno1NWhIQmhCa3k5LzFnQU1GWFVzS2ZCMUVnZHZENHFOWTlzS0tSN3BjRkVLUUJYY25sa2RJZUJUS2Q0SUVaWDhFeTBWUzNueklkZFc4bmhVaDlTbklhMk1ZenU4V3dVUkR1TTYvUW5KMXkrQ3lKaDFJTGZaaWJnd1c2VHpTaU5UdHpxVkJ3TkdtS2tXOGxQL1h0VG5UVIbzQ2SU1aVFB0TE1FUkFRamJkbit0N0ppeEYzS2U5MGxEUElLUE0wMTA3OUtnV0FydDBvWmZ3RVQwUVZzRGJ4eFZraWVXNmxsampnSlVRaXR4SERKTkhHMDRqOEl4OWhka3NzUUlFd29jUlNdmQVlKS29aSWh2Y05BUWNHb0c4d2JRSUJBREJvQmdrcWhraUc5dzBCQndFd0hnWUpZSVpJQVdVREJBRXVNQkVFREExZ1k3TEtib1A3d1JKN3lBSUJFSUE3c1NtRFNkUC9yUXhLaDhWOenNrRGZiRVM4N3ZraXliTjkzLzQ1Mkk2aUwwWmY0YzVvQWZpMXVGeFpqYVZCTUlKUHJsamJuSy84bDNWbmJaWS80bHRSK0pTemJ0M0tZcldLTUhzUVhEUTRWTlVFd1h2dW9xWXBSVHQvVkJvOTB4UDVMWnFRbDI0SWxyS3kzOU9iYXNzWnduV29rbW5yaDhIMDluaG14N05vMlN4SlVKTXpBUm10K1h3b28yQjZMZWJkK1A1TDlPSk9vSWdRWWZXdlpoQzMrWDZKZ3ZKTjh5M3lzeGVZbW5mejNKTTlmQlNOMFJ1NndRbTNoMjJ5bk5XNGdLclFLWG5EazZQcGZyV1EvdnEiLCJkYXRha2V5IjoiQVFFQkFIakI3L2lnd01nNE5Qd2F1cnhTSVl4NEhmbnh1R2MvNDhiRHd2d0RwTllXWmdBQUFINHdmQVlKS29aSWh2Y05BUWNHb0c4d2JRSUJBREJvQmdrcWhraUc5dzBCQndFd0hnWUpZSVpJQVdVREJBRXVNQkVFREExZ1k3TEtib1A3d1JKN3lBSUJFSUE3c1NtRFNkUC9yUXhLaDhWOHIrc2plRDRUQTArNWRTaTVTTnVBVzJMVjIzNVNLamVQaFBROHpMYXBOdFNrWFVoUHhLd2swSkIxSkdJaWQ1ST0iLCJ2ZXJzaW9uIjoiMiIsInR5cGUiOiJEQVRBX0tFWSIsImV4cGlyYXRpb24iOjE2MTE0NDQyNzZ9
and the BASE_64_ENCODED gives this value with spaces:
QVdTOmV5SndZWGxzYjJGa0lqb2lZbmhrUVUwelNVWnRhR1p2ZEhONWNsSlJXVE54YURCdFVrRk9Z VWRhVDBSd2JFVmlprTm5CTmVUTnRObGxCY0RWNlVtcHRVRFJIYW1Ka1JsTkVRMXB5YWxoSFptTlVaV1ZUTW5B clN6SXlUR2hOV1VVaUxDSmtZWFJoYTJWNUlqb2lRVkZGUWtGSWFrSTNMMmxuZDAxbk5FNVFkMkYx Y25oVFNWbDRORWhtYm5oMVIyTXZORGhpUkhkMmQwUndUb0eVlXaFBTVEpvU1RkTUwyTTNaMjExTDFsT04yUmtWMEkw WVdadmFVMHlVSFJKVEdSNlFteENkM2t2Ykc5QmNVUk5XRXBaU2xkSGQydHZMMU5OZWt4SVNHVlVj R2gxYjNalprTm5CTmVUTnRObGxCY0RWNlVtcHRVRFJIYW1Ka1JsTkVRMXB5YWxoSFptTlVaV1ZUTW5B clN6SXlUR2hOV1VVaUxDSmtZWFJoYTJWNUlqb2lRVkZGUWtGSWFrSTNMMmxuZDAxbk5FNVFkMkYx Y25oVFNWbDRORWhtYm5oMVIyTXZORGhpUkhkMmQwUndUbxWnFSVW8zVERGcWRFTkxlVWR1T0ZaR01FaGhS MnNyVGtock5YcFhjalpFUzB4SWNHWXhObmxaY1c5TVNIWnZiV05rTTNsUlkxSjZhMngxYlhGVWFW RktSeXRoU3l0QlJXSnJaVEpMVjFKSkswZFJSVWxMWmxGUmR6ZDFUR3MzTWxCdVpXMXZjMkUwTDFs TmEzTldkMVJhVVhoc1NrSnJSMVY1VkhaaFlsaHdZbFZYZEVsSVVGRlFiMGhMY2sxWGRXUmthM0kx ZVZsdUwyTm9TbEJpU1RSNE5uUm5lWGx1Yml0a2EyMUljbWRHWVZnd2FXUjZXRFZIVUhWbmNrcExO RmczVXk5aWVWWkNjbFZZWlcxTWF6UnVjRkZyWmtRMk9XY3hNR05FTW14NVVqTTJNVGRFVTA4NVlX NTJhalpRTUdwT1FqaENUazVUTjNwTmRIcG9hSGhWWlRkeVFVZEhObVJqVTNZeWIwMVhjbXQ1V25o TlltdDFRazFKUm1aRVJHaDRRWG95VEdOd0wxa3JaWFJCWmpWMlJYZEdPSFF3YUZWd1UxZzRUV3Rx VTFGb1MycFVTazllprTm5CTmVUTnRObGxCY0RWNlVtcHRVRFJIYW1Ka1JsTkVRMXB5YWxoSFptTlVaV1ZUTW5B clN6SXlUR2hOV1VVaUxDSmtZWFJoYTJWNUlqb2lRVkZGUWtGSWFrSTNMMmxuZDAxbk5FNVFkMkYx Y25oVFNWbDRORWhtYm5oMVIyTXZORGhpUkhkMmQwUndUbuUnlUMWh3UjIxU1VFaExa RGd4TTBGaU5qbFNTVGRhVkd4cWF5OUhaa2s1TW5VM2FXWk5TbXR0TVdObWF6bDRNVEl6T1hSM1Ru aEVlbTQxVVV0T04zWm5jVWhoWjFoeGVXdFhXRU0yYVU0M1RtMWhMMXBWTlRGUlVWQk1OR2hMU0M4 dmVVMU1UM0ZVV1VsbEwyaERjVkI1Y0ZSNlRFRTBWVXREWm1SV1VVdHNUM1pRVUVKTGRWTjNhbk5W Wld4YWVHYzFOVmhuZEhOaE5sbEZVV1ZzTld4UmFYWXhNMXB6VkhGMVYwSkdNazFLUzFVemJqZ3dZ eTk2UVU1VldESk5ia0ZpVlZGWlVXVndWVFZTTkdOSGVqSjRUVVJNZEU0clNXcG5PVXN4VmxsblZt bDVUblprTm5CTmVUTnRObGxCY0RWNlVtcHRVRFJIYW1Ka1JsTkVRMXB5YWxoSFptTlVaV1ZUTW5B clN6SXlUR2hOV1VVaUxDSmtZWFJoYTJWNUlqb2lRVkZGUWtGSWFrSTNMMmxuZDAxbk5FNVFkMkYx Y25oVFNWbDRORWhtYm5oMVIyTXZORGhpUkhkMmQwUndUbGxYV21kQlFVRklOSGRtUVZsS1MyOWFT V2gyWTA1QlVXTkhiMGM0ZDJKUlNVSkJSRUp2UW1kcmNXaHJhVWM1ZHpCQ1FuZEZkMGhuV1VwWlNW cEpRVmRWUkVKQlJYVk5Ra1ZGUkUxQmFIaFlhMEUwZVdkR1dIbDJWM2xSU1VKRlNVRTNNWEZHUWts Sk0zUnNNR1E1Ymtjd2JqZDNRWGhISzBndmN6aFNWbUpDYTJkcVptOUNTVFJFZWtwemFtTjZlbGxq TnpkTVNtMXBObVZUWkZJM2FVOTFWa054TlZrelFVTTVTbGs1VWpoclNUMGlMQ0oyWlhKemFXOXVJ am9pTWlJc0luUjVjR1VpT2lKRVFWUkJYMHRGV1NJc0ltVjRjR2x5WVhScGIyNGlPakUyTVRFME5q QXpNREo5
Why does it produces spaces in-between?
Thank You!
base64(1) by default wraps lines at column 76. What you're seeing is the whitespace of those newlines.
If you add -w0 (disable wrapping altogether) to the base64 options, it will spit out its result without any wrapping.
Do man base64 for further information.

How to parse string in powershell

I have a Powershell command that outputs multiple lines.
I want to output only one line that contains the name of a .zip file.
Currently, all lines are returned when substring .zip is found:
$p.Start() | Out-Null
$p.WaitForExit()
$output = $p.StandardOutput.ReadToEnd()
$output += $p.StandardError.ReadToEnd()
foreach($line in $output)
{
if($line.Contains(".zip"))
{
$line
}
}
Since you're using .ReadToEnd(), $output receives a single, multi-line string, not an array of lines.
You must therefore split that string into individual lines yourself, using the -split operator.
You can then apply a string-comparison operator such as -match or -like directly to the array of lines to extract matching lines:
# Sample multi-line string.
$output = #'
line 1
foo.zip
another line
'#
$output -split '\r?\n' -match '\.zip' # -> 'foo.zip'
-split is regex-based, and regex \r?\n matches newlines (line breaks) of either variety (CRLF, as typical on Windows, as well as LF, as typical on Unix-like platforms).
-match is also regex-based, which is why the . in \.zip is \-escaped, given that . is a regex metacharacter (it matches any character other than LF by default).
Note that -match, like PowerShell in general, is case-insensitive by default, so both foo.zip and foo.ZIP would match, for instance;
if you do want case-sensitivity, use -cmatch.
As an aside:
I wonder why you're running your command via a [System.Diagnostics.Process] instance, given that you seem to be invoking synchronously while capturing its standard streams.
PowerShell allows you to do that much more simply by direct invocation, optionally with redirection:
$output = ... 2>&1

Accelerate Powershell script runtime

I'm using a POWERSHELL script which converts a specific log format to a tab or comma separated (CSV) format and it looks like this:
$filename = "filename.log"
foreach ($line in [System.IO.File]::ReadLines($filename)) {
$x = [regex]::Split( $line , 'regex')
$xx = $x -join ","
$xx >> Results.csv
}
And it works fine, but for a 20MB log file it takes almost 20 min to be converted! Is there a way to accelerate it?
My System: CPU: Corei7 3720QM / RAM: 8GB
Update: The log format is like this:
192.168.1.5:24652 172.16.30.8:80 http://www.example.com "useragent"
I want destination format to be:
192.168.1.5,24652,172.16.30.8,80,http://www.example.com,"useragent"
REGEX: ^([\d\.]+):(\d+)\s+([\d\.]+):(\d+)\s+([^ ]*)\s+(\".*\")$
As Lieven Keersmaekers points out, you can do a single -replace operation to do the work.
Additionally, foreach($thing in $o.GetThings()){} will initially block until GetThings() return and then store the entire result in memory, which you have no need for. You can avoid this by using the pipeline instead.
Finally, your regex can be simplified so that the engine doesn't have to parse the entire string before splitting, by matching on either : preceded by a digit or whitespace:
Get-Content filename.log |ForEach-Object {
$_ -replace '(?:(?<=\d)\:|\s+)',','
} |Out-File results.csv

Multi-Line String to Single-Line String conversion in PowerShell

I have a text file that has multiple 'chunks' of text. These chunks have multiple lines and are separated with a blank line, e.g.:
This is an example line
This is an example line
This is an example line
This is another example line
This is another example line
This is another example line
I need these chunks to be in single-line format e.g.
This is an example lineThis is an example lineThis is an example line
This is another example lineThis is another example lineThis is another example line
I have researched this thoroughly and have only found ways of making whole text files single-line. I need a way (preferably in a loop) of making an array of string chunks single-line. Is there any way of achieving this?
EDIT:
I have edited the example content to make it a little clearer.
# create a temp file that looks like your content
# add the A,B,C,etc to each line so we can see them being joined later
"Axxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Bxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Cxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Dxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Exxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Fxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Gxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Hxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Ixxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" | Set-Content -Path "$($env:TEMP)\JoinChunks.txt"
# read the file content as one big chunk of text (rather than an array of lines
$textChunk = Get-Content -Path "$($env:TEMP)\JoinChunks.txt" -Raw
# split the text into an array of lines
# the regex "(\r*\n){2,}" means 'split the whole text into an array where there are two or more linefeeds
$chunksToJoin = $textChunk -split "(\r*\n){2,}"
# remove linefeeds for each section and output the contents
$chunksToJoin -replace '\r*\n', ''
# one line equivalent of above
((Get-Content -Path "$($env:TEMP)\JoinChunks.txt" -Raw) -split "(\r*\n){2,}") -replace '\r*\n', ''
A bit of a fudge:
[String] $strText = [System.IO.File]::ReadAllText( "c:\temp\test.txt" );
[String[]] $arrLines = ($strText -split "`r`n`r`n").replace("`r`n", "" );
This relies on the file having Windows CRLFs.
There a several ways to approach a task like that. One is to use a regular expression replacement with a negative lookahead assertion:
(Get-Content 'C:\path\to\input.txt' | Out-String) -replace "`r?`n(?!`r?`n)" |
Set-Content 'C:\path\to\output.txt'
You could also work with a StreamReader and StreamWriter:
$reader = New-Object IO.StreamReader 'C:\path\to\input.txt'
$writer = New-Object IO.StreamWriter 'C:\path\to\output.txt'
while ($reader.Peek() -gt 0) {
$line = $reader.ReadLine()
if ($line.Trim() -ne '') {
$writer.Write($line)
} else {
$writer.WriteLine()
}
}

Base64 decoding has "%" at the end of the result sometimes. Is it the result supposed to be? Any solution to that?

I am just studying base64 encoding and decoding algorithms and try some programs. I found some example code online, but the result looks a little weird for me.
Here is the link: http://knol2share.blogspot.com/2011/07/base64-encoding-and-decoding-in-c.html
I tried to use it to encode and decode a string.
Enter a string:
02613
Base64 Encoded value: MDI2MTM=
Base64 Decoded value: 02613% -- I do not know why there is a "%", is there a way to get the correct result
I even tried the Base64 program in linux and got the same result after removing the newline in encoding.
Here is the result:
%echo -n 02613 |base64
MDI2MTM=
%echo -n MDI2MTM= | base64 --decode
02613%
Does anyone know how I can get the exact same result with the input string? Thanks.
It is printed if the decoded text does not end with a newline.
$ printf "foobar\n" | base64 | base64 --decode
foobar
$ printf "foobar" | base64 | base64 --decode
foobar%
Isn't the % sign a command prompt ?
Add new line after decoded b64 and check.
Here's the example with echo command, -n option will remove newline character, that's why in first case we don't have the symbol % and in the second case with have one appended.
➜ ~ echo "HELLO" | base64 | base64 --decode
HELLO
➜ ~ echo -n "HELLO" | base64 | base64 --decode
HELLO%

Resources