Essentially, I want to read from file.txt with contents
apple
banana
carrot
and write to newfile.txt so that it will have contents
apple banana carrot
I need to do this on a Windows computer on which I do not have install permissions.
I tried
set row=
for /f %%x in (file.txt) do set row=%row% %%x
echo row > newfile.txt
and I tried using PowerShell statements (I cannot run PowerShell scripts) instead of the CMD-style for loop.
powershell -Command "(Gc file.txt) | Foreach-Object -Process {set row=%row% _$} | Out-File newFile.txt"
but both produce an empty file.
Is there a way to do this?
Get-Content returns the content of a file as an array of lines with the line breaks already removed, so all you need to do (in PowerShell) is to join the lines and write the result back to a file:
(Get-Content 'input.txt') -join ' ' | Set-Content 'output.txt'
Not recommended, but if you must do this in batch you need something like this:
#echo off
setlocal EnableDelayedExpansion
set row=
for /f %%x in (file.txt) do set "row=!row! %%x"
>newfile.txt echo %row%
Note that delayed expansion is required for this to work. Without it %row% in the loop body would be expanded at parse time (when the variable is still empty), so you'll end up with just the last line from the input file in the variable after the loop completes. With delayed expansion enabled (and using !row! instead of %row%) the variable is expanded at run time, i.e. during the loop iterations as one would normally expect.
For further information on delayed expansion see Raymond Chen's blog.
To complement Ansgar Wiechers' helpful answer:
Executing the following command from a batch file / a cmd.exe console window should do what you want:
powershell -command "\"$(Get-Content file.txt)\" > newFile.txt"
Note the escaping of embedded " as \", which PowerShell requires when called from the outside (by contrast, PowerShell-internally, ` is the escape character).
Enclosing the Get-Content file.txt call - which outputs an array of lines - in a double-quoted string, using subexpression operator $(...), means that the array elements are implicitly joined with a space each.
Note, however, that PowerShell's output-redirection operator, >, creates UTF16-LE ("Unicode") encoded files by default, as does Out-File (at least in Windows PowerShell; the cross-platform PowerShell Core defaults to (BOM-less) UTF-8).
To control the output encoding, use the -Encoding parameter, which you can apply to Out-File or, preferably - knowing that strings are being output - Set-Content.
In Windows PowerShell, note that Set-Content - in contrast with > / Out-File - defaults to the encoding implied by the legacy "ANSI" code page, typically Windows-1252.
Related
I have a piece of code that is meant to send the following to the linux command line:
wc -l C:/inputdirectory/P*
However, I need to run this script in Windows, and am trying to find the equivalent command. I have tried
find /c /v C:/inputdirectory/P*
But this throws an error, that /v is not a valid command. Can you please tell me why this isn't working?
*note, the command itself doesn't say "inputdirectory", it has the correct directory, it's just too tedious and private to type out
Courtesy of Eryk Sun:
Try searching for "", i.e. an empty string; use only backslash as the path separator; and quote the path if it has spaces in it:
find /c /v "" "C:\inputdirectory\P*"
From cmd.exe (the Command Prompt / a batch file), which is obsolescent:
Use the accepted answer.
From PowerShell, you have two options:
Option A (suboptimal): Use the accepted answer too, with a small tweak:
find --% /c /v "" "C:\inputdirectory\P*"
Note: --%, the stop-parsing symbol, tells PowerShell to pass subsequent arguments through as-is to the target program (after expanding cmd-style environment-variable references such as %USERNAME%, if any).
In the case at hand, this prevents PowerShell from parsing "" and - mistakenly - neglecting to pass it through to the external target program (find.exe).
For a summary of PowerShell's highly problematic handling of quotes when calling external programs, see this answer.
Output from the above find.exe command - and, indeed, any external program, is just text, and in this case it looks something like this:
---------- PFILE1.TXT: 42
---------- PFILE2.TXT: 666
...
While this output is easy to grasp for a human observer, it makes subsequent programmatic processing cumbersome, because text parsing is required.
Using a PowerShell-native command (cmdlet), as described below, offers more flexibility, because PowerShell commands typically emit objects with typed properties, which greatly facilitates subsequent processing.
Option B (preferred): Use PowerShell's own Measure-Object cmdlet with the -Line switch:
Note: While this command is more verbose than the find solution, it ultimately offers more flexibility due to outputting objects with typed properties, which greatly facilitates subsequent processing; additionally, PowerShell's sophisticated output-formatting system offers user-friendly default representations.
Get-Item -Path "C:\inputdirectory\P*" -PipelineVariable file | ForEach-Object {
Get-Content -LiteralPath $file |
Measure-Object -Line |
Select-Object #{ Name='File'; Expression={ $file } }, Lines
}
The above outputs objects that have a .File and .Lines property each, which PowerShell prints as follows by default:
File Lines
---- -----
C:\inputdirectory\Pfile1.txt 42
C:\inputdirectory\Pfile2.txt 666
...
In addition to a nicer presentation of the output, the object-oriented nature of the output makes it easy to programmatically process the results.
For instance, if you wanted to limit the output to those files whose line count is 100 or greater, pipe to the following Where-Object call to the above command:
... | Where-Object Lines -ge 100
If you (additionally) wanted to sort by highest line count first, pipe to the Sort-Object cmdlet:
... | Sort-Object -Descending Lines
How can I count the lines in a set of files?
Use the following batch file (CountLines.cmd):
#echo off
Setlocal EnableDelayedExpansion
for /f "usebackq" %%a in (`dir /b %1`) do (
for /f "usebackq" %%b in (`type %%a ^| find "" /v /c`) do (
set /a lines += %%b
)
)
echo %lines%
endlocal
Usage:
CountLines C:/inputdirectory/P*
Further Reading
An A-Z Index of the Windows CMD command line - An excellent reference for all things Windows cmd line related.
dir - Display a list of files and subfolders.
find - Search for a text string in a file & display all the lines where it is found.
for /f - Loop command against the results of another command.
I've created bat file named vk.bat . The code is as following :-
SET "tcs="
FOR %%A IN (%*) DO (
Set "tcs=%tcs% -t %%A"
)
Echo %tcs%
I am executing this bat from cmd as following :-
c:\vk.bat Apple Cat Play
I want the final string as " -t Apple -t Cat -t Play"
But I am getting final string as "-t Play" . I am not able to find out that why and how it's overwriting the previous contents of string tcs.
DelayedExpansion-free Solution
Using the call command, we can emulate the delayed expansion. Take a look at this example
call command %%foo%% = setlocal enableDelayedExpansion
command !foo!
According to your case, the code should be changed to this:
SET "tcs="
FOR %%G IN (%*) DO (
CALL SET "tcs=%%tcs%% -t %%G"
)
ECHO %tcs%
Some Buggy Behaviour With CALL
Call doesn't handle redirection characters and some special characters properly:
&
|
<
>
"foo^bar"
The redirection of the first 4 examples won't function as intended, while in the last example, the caret(^) will be doubled. Source: SS64
You have fallen into the delayed expansion trap, like Squashman mentioned.
What Is Delayed Expansion?
In earlier batch files, variables are expanded(changed to it's value) when each line is phrased.
The command processor treats the entire for loop as one command, so the variable %tcs% is treated as nothing. (Because tcs was previously set to nothing.
How To Make Batch Variables Get Expanded At Run-Time?
To preserve compatibility with older batch files, delayed expansion feature is added. To allow the processor to do so, add this line to the start of the batch file:
setlocal enableDelayedExpansion
and we also need to tell the processor which variables to be expanded at run-time, to do so, change the following:
From -> To
%var% -> !var!
Note only %n% variables can be changed to !n!. Metavariables like %%G/%G and %0 cannot be changed.
Here's is the fixed code using delayed expansion.:
SETLOCAL EnableDelayedExpansion
SET "tcs="
FOR %%G IN (%*) DO (
SET "tcs=!tcs! -t %%G"
)
ECHO %tcs%
I need to create ONE integrated script that sets some environment variables, downloads a file using wget and runs it.
The challenge is that it needs to be the SAME script that can run on both Windows PowerShell and also bash / shell.
This is the shell script:
#!/bin/bash
# download a script
wget http://www.example.org/my.script -O my.script
# set a couple of environment variables
export script_source=http://www.example.org
export some_value=floob
# now execute the downloaded script
bash ./my.script
This is the same thing in PowerShell:
wget http://www.example.org/my.script -O my.script.ps1
$env:script_source="http://www.example.org"
$env:some_value="floob"
PowerShell -File ./my.script.ps1
So I wonder if somehow these two scripts can be merged and run successfully on either platform?
I've been trying to find a way to put them in the same script and get bash and PowerShell.exe to ignore errors but have had no success doing so.
Any guesses?
It is possible; I don't know how compatible this is, but PowerShell treats strings as text and they end up on screen, Bash treats them as commands and tries to run them, and both support the same function definition syntax. So, put a function name in quotes and only Bash will run it, put "exit" in quotes and only Bash will exit. Then write PowerShell code after.
NB. this works because the syntax in both shells overlaps, and your script is simple - run commands and deal with variables. If you try to use more advanced script (if/then, for, switch, case, etc.) for either language, the other one will probably complain.
Save this as dual.ps1 so PowerShell is happy with it, chmod +x dual.ps1 so Bash will run it
#!/bin/bash
function DoBashThings {
wget http://www.example.org/my.script -O my.script
# set a couple of environment variables
export script_source=http://www.example.org
export some_value=floob
# now execute the downloaded script
bash ./my.script
}
"DoBashThings" # This runs the bash script, in PS it's just a string
"exit" # This quits the bash version, in PS it's just a string
# PowerShell code here
# --------------------
Invoke-WebRequest "http://www.example.org/my.script.ps1" -OutFile my.script.ps1
$env:script_source="http://www.example.org"
$env:some_value="floob"
PowerShell -File ./my.script.ps1
then
./dual.ps1
on either system.
Edit: You can include more complex code by commenting the code blocks with a distinct prefix, then having each language filter out its own code and eval it (usual security caveats apply with eval), e.g. with this approach (incorporating suggestion from Harry Johnston ):
#!/bin/bash
#posh $num = 200
#posh if (150 -lt $num) {
#posh write-host "PowerShell here"
#posh }
#bash thing="xyz"
#bash if [ "$thing" = "xyz" ]
#bash then
#bash echo "Bash here"
#bash fi
function RunBashStuff {
eval "$(grep '^#bash' $0 | sed -e 's/^#bash //')"
}
"RunBashStuff"
"exit"
((Get-Content $MyInvocation.MyCommand.Source) -match '^#posh' -replace '^#posh ') -join "`n" | Invoke-Expression
While the other answer is great (thank you TessellatingHeckler and Harry Johnston)
(and also thank you j-p-hutchins for fixing the error with true)
We can actually do way better
Work with more shells (e.g. work for Ubuntu's dash)
Less likely to break in future situations
No need to waste processing time re-reading/evaling the script
Waste less characters/lines on confusing syntax(we can get away with a mere 41 chars, and mere 3 lines)
Even Keep syntax highlighting functional
Copy Paste Code
Save this as your_thing.ps1 for it to run as powershell on Windows and run as shell on all other operating systems.
#!/usr/bin/env sh
echo --% >/dev/null;: ' | out-null
<#'
#
# sh part
#
echo "hello from bash/dash/zsh"
echo "do whatver you want just dont use #＞ directly"
echo "e.g. do #""> or something similar"
# end bash part
exit #>
#
# powershell part
#
echo "hello from powershell"
echo "you literally don't have to escape anything here"
How? (its actually simple)
We want to start a multi-line comment in powershell without causing an error in bash/shell.
Powershell has multi-line comments <# but as-is they would cause problems in bash/shell languages. We need to use a string like "<#" for bash, but we need it to NOT be a string in powershell.
Powershell has a stop-parsing arg --% lets write single quote without starting a string, e.g. echo --% ' blah ' will print out ' blah '. This is great because in shell/bash the single quotes do start a string, e.g. echo --% ' blah ' will print out blah
We need a command in order to use powershell's stop-parsing-args, lucky for us both powershell and bash have an echo command
So, in bash we can echo a string with <#, but powershell the same code finishes the echo command then starts a multi-line comment
Finally we add >/dev/null to bash so that it doesn't print out --% every time, and we add | out-null so that powershell doesn't print out >/dev/null;: ' every time.
The syntax highlighting tells the story more visually
Powershell Highlighting
All the green stuff is ignored by powershell (comments)
The gray --% is special
The | out-null is special
The white parts are just string-arguments without quotes
(even the single quote is equivlent to "'")
The <# is the start of a multi-line comment
Bash Highlighting
For bash its totally different.
Lime green + underline are the commands.
The --% isn't special, its just an argument
But the ; is special
The purple is output-redirection
Then : is just the standard "do nothing" shell command
Then the ' starts a string argument that ends on the next line
Caveats?
Almost almost none. Powershell legitimately has no downside. The Bash caveats are easy to fix, and are exceedingly rare
If you need #> in a bash string, you'll need to escape it somehow.
changing "#>" to "#"">"or from ' blah #> ' to ' blah #''> '.
If you have a comment #> and for some reason you CANNOT change that comment (this is what I mean by exceedingly rare), you can actually just use #>, you just have to add re-add those first two lines (eg true --% etc) right after your #> comment
One even more exceedingly rare case is where you are using the # to remove parts of a string (I bet most don't even know this is a bash feature). Example code below
https://man7.org/linux/man-pages/man1/bash.1.html#EXPANSION
var1=">blah"
echo ${var1#＞}
# ^ removes the > from var1
To fix this one, well there are alternative ways of removeing chars from the begining of a string, use them instead.
Following up on Jeff Hykin's answer, I have found that the first line, while it is happy in bash, produces this output in PowerShell. Note that it is still fully functional, just noisy.
true : The term 'true' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling
of the name, or if a path was included, verify that the path is correct and try again.
At C:\Users\jp\scratch\envar.ps1:4 char:1
+ true --% ; : '
+ ~~~~
+ CategoryInfo : ObjectNotFound: (true:String) [], CommandNotFoundException
hello from powershell
I am experimenting with changing the first lines from:
true --% ; : '
<#'
to:
echo --% > /dev/null ; : ' | out-null
<#'
In very limited testing this seems to be working in bash and powershell. For reference, I am "sourcing" the scripts not "calling" them, e.g. . env.ps1 in bash and . ./env.ps1 in powershell.
This question has been asked a lot on stackoverflow, but I can't seem to be able to make it work. Any hints appreciated. Here is a text file (extension .mpl) containing offending text that needs to be removed:
plotsetup('ps', 'plotoutput = "plotfile.eps"', 'plotoptions' = "color=rgb,landscape,noborder");
print(PLOT3D(MESH(Array(1..60, 1..60, 1..3, [[[.85840734641021,0.,-0.],
[HFloat(undefined),HFloat(undefined),HFloat(undefined)],[.857971665313419,.0917163905694189,-.16720239349226],
... more like that ...
[.858407346410207,-3.25992468340355e-015,5.96532373555817e-015]]], datatype = float[8], order = C_order)),SHADING(ZHUE),STYLE(PATCHNOGRID),TRANSPARENCY(.3),LIGHTMODEL(LIGHT_4),ORIENTATION(35.,135.),SCALING(CONSTRAINED),AXESSTYLE(NORMAL)));
I want to remove every instance of:
[HFloat(undefined),HFloat(undefined),HFloat(undefined)],
and there are thousands such instances!. Note: the square brackets and the comma are to be removed. There is no space, so I have pages and pages of:
[HFloat(undefined),HFloat(undefined),HFloat(undefined)],
[HFloat(undefined),HFloat(undefined),HFloat(undefined)],
[HFloat(undefined),HFloat(undefined),HFloat(undefined)],
I won't list here all my failed attempts. Below is the closest I've come:
#echo off
SetLocal
cd /d %~dp0
if exist testCleaned.mpl del testCleaned.mpl
SetLocal EnableDelayedExpansion
Set OldString=[HFloat(undefined),HFloat(undefined),HFloat(undefined)],
Set NewString=
pause
FOR /F "tokens=* delims= " %%I IN (test.mpl) DO (
set str=%%I
set str=!str:OldString=NewString!
echo !str! >> testCleaned.mpl
endlocal
)
EndLocal
The above was strung together, as it were, from pieces of code found on the web, especially at stackoverflow, e.g. Problem with search and replace batch file
What it does is produce a truncated file, as follows:
plotsetup('ps', 'plotoutput = "plotfile.eps"', 'plotoptions' = "color=rgb,landscape,noborder");
!str!
Please don't hesitate to request clarifications. Apologies if you feel that this question has already been answered. I would very much appreciate if you would copy-paste the relevant code for me, as I have tried for several hours.
Bonus: can this automatic naming be made to work? "%%~nICleaned.mpl"
The biggest problem with your existing code is the SetLocal enableDelayedExpansion is missplaced - it should be within the loop after set str=%%I.
Other problems:
will strip lines beginning with ;
will strip leading spaces from each line
will strip blank (empty) lines
will print ECHO is off if any lines becomes empty or contains only spaces after substitution
will add extra space at end of each line (didn't notice this until I read jeb's answer)
Optimization issue - using >> can be relatively slow. It is faster to enclose the whole loop in () and then use >
Below is about the best you can do with Windows batch. I auto named the output as requested, doing one better - It automatically preserves the extension of the original name.
#echo off
SetLocal
cd /d %~dp0
Set "OldString=[HFloat(undefined),HFloat(undefined),HFloat(undefined)],"
Set "NewString="
set file="test.mpl"
for %%F in (%file%) do set outFile="%%~nFCleaned%%~xF"
pause
(
for /f "skip=2 delims=" %%a in ('find /n /v "" %file%') do (
set "ln=%%a"
setlocal enableDelayedExpansion
set "ln=!ln:*]=!"
if defined ln set "ln=!ln:%OldString%=%NewString%!"
echo(!ln!
endlocal
)
)>%outFile%
Known limitations
limited to slightly under 8k per line, both before and after substitution
search string cannot include = or !, nor can it start with * or ~
replacement string cannot include !
search part of search and replace is case insensitive
last line will always end with newline <CR><LF> even if original did not
All but the first limitation could be eliminated, but it would require a lot of code, and would be horrifically slow. The solution would require a character by character search of each line. The last limitation would require some awkward test to determine if the last line was newline terminated, and then last line would have to be printed using <nul SET /P "ln=!ln!" trick if no newline wanted.
Interesting feature (or limitation, depending on perspective)
Unix style files ending lines with <LF> will be converted to Windows style with lines ending with <CR><LF>
There are other solutions using batch that are significantly faster, but they all have more limitations.
Update - I've posted a new pure batch solution that is able to do case sensitive searches and has no restrictions on search or replacement string content. It does have more restrictions on line length, trailing control characters, and line format. Performance is not bad, especially if the number of replacements is low. http://www.dostips.com/forum/viewtopic.php?f=3&t=2710
Addendum
Based on comments below, a batch solution will not work for this particular problem because of line length limitation.
But this code is a good basis for a batch based search and replace utility, as long as you are willing to put up with the limitations and relatively poor performance of batch.
There are much better text processing tools available, though they are not standard with Windows. My favorite is sed within the GNU Utilities for Win32 package. The utilities are free, and do not require any installation.
Here is a sed solution for Windows using GNU utilities
#echo off
setlocal
cd /d %~dp0
Set "OldString=\[HFloat(undefined),HFloat(undefined),HFloat(undefined)\],"
Set "NewString="
set file="test.mpl"
for %%F in (%file%) do set outFile="%%~nFCleaned%%~xF"
pause
sed -e"s/%OldString%/%NewString%/g" <%file% >%outfile%
Update 2013-02-19
sed may not be an option if you work at a site that has rules forbidding the installation of executables downloaded from the web.
JScript has good regular expression handling, and it is standard on all modern Windows platforms, including XP. It is a good choice for performing search and replace operations on Windows platforms.
I have written a hybrid JScript/Batch search and replace script (REPL.BAT) that is easy to call from a batch script. A small amount of code gives a lot of powerful features; not as powerful as sed, but more than enough to handle this task, as well as many others. It is also quite fast, much faster than any pure batch solution. It also does not have any inherent line length limitations.
Here is a batch script that uses my REPL.BAT utility to accomplish the task.
#echo off
setlocal
cd /d %~dp0
Set "OldString=[HFloat(undefined),HFloat(undefined),HFloat(undefined)],"
Set "NewString="
set file="test.txt"
for %%F in (%file%) do set outFile="%%~nFCleaned%%~xF"
pause
call repl OldString NewString le <%file% >%outfile%
I use the L option to specify a literal search string instead of a regular expression, and the E option to pass the search and replace strings via environment variables by name, instead of using string literals on the command line.
Here is the REPL.BAT utility script that the above code calls. Full documentation is encluded within the script.
#if (#X)==(#Y) #end /* Harmless hybrid line that begins a JScript comment
::************ Documentation ***********
:::
:::REPL Search Replace [Options [SourceVar]]
:::REPL /?
:::
::: Performs a global search and replace operation on each line of input from
::: stdin and prints the result to stdout.
:::
::: Each parameter may be optionally enclosed by double quotes. The double
::: quotes are not considered part of the argument. The quotes are required
::: if the parameter contains a batch token delimiter like space, tab, comma,
::: semicolon. The quotes should also be used if the argument contains a
::: batch special character like &, |, etc. so that the special character
::: does not need to be escaped with ^.
:::
::: If called with a single argument of /? then prints help documentation
::: to stdout.
:::
::: Search - By default this is a case sensitive JScript (ECMA) regular
::: expression expressed as a string.
:::
::: JScript syntax documentation is available at
::: http://msdn.microsoft.com/en-us/library/ae5bf541(v=vs.80).aspx
:::
::: Replace - By default this is the string to be used as a replacement for
::: each found search expression. Full support is provided for
::: substituion patterns available to the JScript replace method.
::: A $ literal can be escaped as $$. An empty replacement string
::: must be represented as "".
:::
::: Replace substitution pattern syntax is documented at
::: http://msdn.microsoft.com/en-US/library/efy6s3e6(v=vs.80).aspx
:::
::: Options - An optional string of characters used to alter the behavior
::: of REPL. The option characters are case insensitive, and may
::: appear in any order.
:::
::: I - Makes the search case-insensitive.
:::
::: L - The Search is treated as a string literal instead of a
::: regular expression. Also, all $ found in Replace are
::: treated as $ literals.
:::
::: E - Search and Replace represent the name of environment
::: variables that contain the respective values. An undefined
::: variable is treated as an empty string.
:::
::: M - Multi-line mode. The entire contents of stdin is read and
::: processed in one pass instead of line by line. ^ anchors
::: the beginning of a line and $ anchors the end of a line.
:::
::: X - Enables extended substitution pattern syntax with support
::: for the following escape sequences:
:::
::: \\ - Backslash
::: \b - Backspace
::: \f - Formfeed
::: \n - Newline
::: \r - Carriage Return
::: \t - Horizontal Tab
::: \v - Vertical Tab
::: \xnn - Ascii (Latin 1) character expressed as 2 hex digits
::: \unnnn - Unicode character expressed as 4 hex digits
:::
::: Escape sequences are supported even when the L option is used.
:::
::: S - The source is read from an environment variable instead of
::: from stdin. The name of the source environment variable is
::: specified in the next argument after the option string.
:::
::************ Batch portion ***********
#echo off
if .%2 equ . (
if "%~1" equ "/?" (
findstr "^:::" "%~f0" | cscript //E:JScript //nologo "%~f0" "^:::" ""
exit /b 0
) else (
call :err "Insufficient arguments"
exit /b 1
)
)
echo(%~3|findstr /i "[^SMILEX]" >nul && (
call :err "Invalid option(s)"
exit /b 1
)
cscript //E:JScript //nologo "%~f0" %*
exit /b 0
:err
>&2 echo ERROR: %~1. Use REPL /? to get help.
exit /b
************* JScript portion **********/
var env=WScript.CreateObject("WScript.Shell").Environment("Process");
var args=WScript.Arguments;
var search=args.Item(0);
var replace=args.Item(1);
var options="g";
if (args.length>2) {
options+=args.Item(2).toLowerCase();
}
var multi=(options.indexOf("m")>=0);
var srcVar=(options.indexOf("s")>=0);
if (srcVar) {
options=options.replace(/s/g,"");
}
if (options.indexOf("e")>=0) {
options=options.replace(/e/g,"");
search=env(search);
replace=env(replace);
}
if (options.indexOf("l")>=0) {
options=options.replace(/l/g,"");
search=search.replace(/([.^$*+?()[{\\|])/g,"\\$1");
replace=replace.replace(/\$/g,"$$$$");
}
if (options.indexOf("x")>=0) {
options=options.replace(/x/g,"");
replace=replace.replace(/\\\\/g,"\\B");
replace=replace.replace(/\\b/g,"\b");
replace=replace.replace(/\\f/g,"\f");
replace=replace.replace(/\\n/g,"\n");
replace=replace.replace(/\\r/g,"\r");
replace=replace.replace(/\\t/g,"\t");
replace=replace.replace(/\\v/g,"\v");
replace=replace.replace(/\\x[0-9a-fA-F]{2}|\\u[0-9a-fA-F]{4}/g,
function($0,$1,$2){
return String.fromCharCode(parseInt("0x"+$0.substring(2)));
}
);
replace=replace.replace(/\\B/g,"\\");
}
var search=new RegExp(search,options);
if (srcVar) {
WScript.Stdout.Write(env(args.Item(3)).replace(search,replace));
} else {
while (!WScript.StdIn.AtEndOfStream) {
if (multi) {
WScript.Stdout.Write(WScript.StdIn.ReadAll().replace(search,replace));
} else {
WScript.Stdout.WriteLine(WScript.StdIn.ReadLine().replace(search,replace));
}
}
}
The Batch file below have the same restrictions of previous solutions on characters that can be processed; these restrictions are inherent to all Batch language programs. However, this program should run faster if the file is large and the lines to replace are not too much. Lines with no replacement string are not processed, but directly copied to the output file.
#echo off
setlocal EnableDelayedExpansion
set "oldString=[HFloat(undefined),HFloat(undefined),HFloat(undefined)],"
set "newString="
findstr /N ^^ inFile.mpl > numberedFile.tmp
find /C ":" < numberedFile.tmp > lastLine.tmp
set /P lastLine=<lastLine.tmp
del lastLine.tmp
call :ProcessLines < numberedFile.tmp > outFile.mpl
del numberedFile.tmp
goto :EOF
:ProcessLines
set lastProcessedLine=0
for /F "delims=:" %%a in ('findstr /N /C:"%oldString%" inFile.mpl') do (
call :copyUpToLine %%a
echo(!line:%oldString%=%newString%!
)
set /A linesToCopy=lastLine-lastProcessedLine
for /L %%i in (1,1,%linesToCopy%) do (
set /P line=
echo(!line:*:=!
)
exit /B
:copyUpToLine number
set /A linesToCopy=%1-lastProcessedLine-1
for /L %%i in (1,1,%linesToCopy%) do (
set /P line=
echo(!line:*:=!
)
set /P line=
set line=!line:*:=!
set lastProcessedLine=%1
exit /B
I would appreciate if you may run a timing test on this an other solutions and post the results.
EDIT: I changed the set /A lastProcessedLine+=linesToCopy+1 line for the equivalent, but faster set lastProcessedLine=%1.
I'm no expert on batch files, so I can't offer a direct solution to your problem.
However, to solve your problem, it might be simpler to use an alternative to batch files.
For example, I'd recommend using http://www.csscript.net/ (if you know C#). This tool will allow you to run C# files like batch files, but giving you the power to write your script using C#, instead of horrible batch file syntax :)
Another alternative would be python, if you know python.
But I guess the point is, that this kind of task may be easier in another programming language.
You defined delims=<space>, that's a bad idea if you want to preserve your lines, as it splits after the first space.
You should change this to FOR /F "tokens=* delims=" ....
Your echo !str! >> testCleaned.mpl will always append one extra space to each line, better use echo(!str!>>testCleaned.mpl.
You will also lose all empty lines, and all exclamation marks in all lines.
You could also try the code of Improved BatchSubstitute.bat
What language should I use for file and string manipulation?
This might seem objective, but really isn't I think. There's lot to say about this. For example I can see clearly that for most usages Perl would be a more obvious candidate than Java. I need to do this quite often and at this time I use C# for it, but I would like a more scriptlike language to do this.
I can imagine Perl would be a candidate for it, but I would like to do it in PowerShell since PowerShell can access the .NET library (easy). Or is Python a better candidate for it? If I have to learn a new language, Python is certainly one on my list, rather than Perl.
What I want to do for example, is to read a file, make some changes and save it again. E.g.: open it, number all lines (say with 3 digits) and close it.
Any example, in any language, would be welcome, but the shorter the better. It is utility scripting I'm after here, not OO, TDDeveloped, unit-tested stuff of course.
What I would very much like to see is something as (pseudocode here):
open foobar.as f
foreach line in f.lines
line.addBefore(currenIteratorCounter.format('ddd') + '. ')
close f
So:
bar.txt
Frank Zappa
Cowboy Henk
Tom Waits
numberLines bar.txt
bar.txt
001. Frank Zappa
002. Cowboy Henk
003. Tom Waits
UPDATE:
The Perl and Python examples here are great, and definitely in the line of what I was hoping and expecting. But aren't there any PowerShell guys out there?
This is actually pretty easy in PowerShell:
function Number-Lines($name) {
Get-Content $name | ForEach-Object { $i = 1 } { "{0:000}. {1}" -f $i++,$_ }
}
What I'm doing here is getting the contents of the file, this will return a String[], over which I iterate with ForEach-Object and apply a format string using the -f operator. The result just drops out of the pipeline as another String[] which can be redirected to a file if needed.
You can shorten it a little by using aliases:
gc .\someFile.txt | %{$i=1}{ "{0:000}. {1}" -f $i++,$_ }
but I won't recommend that for a function definition.
You way want to consider using two passes, though and constructing the format string on the fly to accommodate for larger numbers of lines. If there are 1500 lines {0:000} it won't be sufficient anymore to get neatly aligned output.
As for which language is best for such tasks, you might look at factors such as
conciseness of code (Perl will be hard to beat there, especially that one-liner in another answer)
readability and maintainability of code
availability of the tools (Perl and Python aren't installed on Windows by default (PowerShell only since Windows 7), so deployment might be hindered.)
In the light of the last point you might even be better off using cmd for this task. The code is similarly pretty simple:
#echo off
setlocal
set line=1
for /f "delims=" %%l in (%1) do call :process %%l
endlocal
goto :eof
:process
call :lz %line%
echo %lz%. %*
set /a line+=1
goto :eof
:lz
if %1 LSS 10 set lz=00%1&goto :eof
if %1 LSS 100 set lz=0%1&goto :eof
set lz=%1&goto :eof
goto :eof
That assumes, of course, that it has to run somewhere else than your own machine. If not, then use whatever fits your needs :-)
perl -i -ne 'printf("00%d. %s",$.,$_)' your-filename-here
You may want %03d instead.
It isn't what you wanted, but please recall findstr.exe(and find.exe) at times...
findstr /n ".*" filename
find "" /v /n filename
Python
target = open( "bar_with_numbers.txt", "w" )
source = open( "bar.txt", "r" )
for count, line in enumerate( source ):
target.write( "%3d. %s\n" % ( count+1, line ) )
source.close()
target.close()
First, it's a bad policy to "update" files in place. In the long run, this becomes a regrettable decision because debugging is made harder by the loss of history.
If you use OS redirection features, this program can be simplified.
import sys
for count, line in enumerate( sys.stdin ):
sys.stdout.write( "%3d. %s\n" % ( count+1, line ) )
Then you can run this enumerate.py as follows
python enumerate.py <bar.txt >bar_with_numbers.txt
More importantly, you can also do this.
python enumerate.py <bar.txt | the_next_step
On a Debian system (and probably other linux distros) you could do this:
$ nl -w 3 -n rz -s ". " [filename] > [newfilename]
Definitely Perl. It supports inline replacement (on Windows you have to start the script with perl .i.bak (because Windows cannot do this inline and creates a .bak file with the same name.)
open(IN,'+>'.$yourfile) || die "Can not open file $yourfile: $!";
my $line_no = 1;
while(<IN>){
print "$line_no. $_";
$line_no++;
}
close IN;
Code just typed from memory without testing. But that should work.
You probably want to add some logic for formatting $line_no (e.g. first count lines and then add as much zero digits as you need.)