I would like to rename multiple files, but not just appending my string to the end or the beginning of the file. I would like to place it in a specific location. This is the command I am working with right now, but it can only add things at the beginning and the end of the file name.
for f in `ls ~/tmp/*`; do FILE=`basename $f`; echo "Rename:"$f;echo $FILE; mv "$f" "/home/tmp/JC_"${FILE%.*}"_hg19."${FILE#*.}""; done
Lets say the file names are as follows hell_1.txt (and lets say there is a ton of them each with a different number for simplicity) I would like to add an o into the file name so the resulting name would be hello_1.txt it would be nice if you had a general solution not just for this example.
this should work:
for x in ~/tmp/*.txt; do mv $x `echo $x | sed -e 's#hell#hello#'`; done
if i understand you wish to change any "hell.*_NNN.txt" to "hel.*o_NNN.txt" (keeping the .* between "hell" and "_NNN.txt" (NNN being any number).
then:
for x in ~/tmp/*.txt; do
mv "$x" "$(echo "$x" | LC_COLLATE=C sed -e 's#\(hell.*\)\(_[0-9]*\.txt$\)#\1o\2#')"
done
I added the LC_COLLATE=C during sed invocation so you can rely on the "[0-9]" matching only digits '0' or '1' or ... or '9'
(If you wonder why adding the LC_COLLATE: with some locales [A-Z] could match every letters A-Z or a-y (except 'z'!) as in such locales letters appears in this order: 'A' 'a' 'B' 'b' ... 'Z' 'z'. And with other locales, who knows?)
(note: you could also replace "[0-9]" with the "[[:digit:]]" notation, but it could be less portable : "old" version of sed won't know about this notation and will try to match any of '[' or ':' or ... or 't' or ':', followed by a ']' (*, so 0,1 or more times) ... That's why I don't like using those special [[:things:]] with sed, tr, etc : i see them as less portable. Use perl instead if you prefer to use those?)
How about
rename 's/hell_/hello_/' /tmp/*.txt
Related
I need to replace one variable with another variable in a multiple strings.
For example:
string1="One,two"
string2="three.four"
string3="five:six"
y=";"
for str in string1 string2 string3; do
x="$(echo "$str" | sed 's/[a-zA-Z]//g')" # extracting a character between letters
sed 's/$x/$y/'$str # I tried this, but it does not work at all.
echo "$str"
done
Expecting output:
One;two
three;four
five;six
In my output, nothing changes:
One,two
three.four
five:six
You can use bash's substitution operator instead of sed. And simply replace anything that isn't a letter with $y.
#!/bin/bash
string1="One,two"
string2="three.four"
string3="five:six"
y=";"
for str in "$string1" "$string2" "$string3"; do
x=${str//[^a-zA-Z]+/$y}
echo "$x"
done
Output is:
One;two
three;four
five;six
Note that your general approach wouldn't work if the input string has muliple delimiters, e.g. One,two,three. When you remove all the letters you get ,,, but that doesn't appear anywhere in the string.
Addressing issues with OP's current code:
referencing variables requires a leading $, preferably a pair of {}, and (usually) double quotes (eg, to insure embedded spaces are considered as part of the variable's value)
sed can take as input a) a stream of text on stdin, b) a file, c) process substitution or d) a here-document/here-string
when building a sed script that includes variable refences the sed script must be wrapped in double quotes (not single quotes)
Pulling all of this into OP's current code we get:
string1="One,two"
string2="three.four"
string3="five:six"
y=";"
for str in "${string1}" "${string2}" "${string3}"; do # proper references of the 3x "stringX" variables
x="$(echo "$str" | sed 's/[a-zA-Z]//g')"
sed "s/$x/$y/" <<< "${str}" # feeding "str" as here-string to sed; allowing variables "x/y" to be expanded in the sed script
echo "$str"
done
This generates:
One;two # generated by the 2nd sed call
One,two # generated by the echo
;hree.four # generated by the 2nd sed call
three.four # generated by the echo
five;six # generated by the 2nd sed call
five:six # generated by the echo
OK, so we're now getting some output but there are obviously some issues:
the results of the 2nd sed call are being sent to stdout/terminal as opposed to being captured in a variable (presumably the str variable - per the follow-on echo ???)
for string2 we find that x=. which when plugged into the 2nd sed call becomes sed "s/./;/"; from here the . matches the first character it finds which in this case is the 1st t in string2, so the output becomes ;hree.four (and the . is not replaced)
dynamically building sed scripts without knowing what's in x (and y) becomes tricky without some additional coding; instead it's typically easier to use parameter substitution to perform the replacements for us
in this particular case we can replace both sed calls with a single parameter substitution (which also eliminates the expensive overhead of two subprocesses for the $(echo ... | sed ...) call)
Making a few changes to OP's current code we can try:
string1="One,two"
string2="three.four"
string3="five:six"
y=";"
for str in "${string1}" "${string2}" "${string3}"; do
x="${str//[^a-zA-Z]/${y}}" # parameter substitution; replace everything *but* a letter with the contents of variable "y"
echo "${str} => ${x}" # display old and new strings
done
This generates:
One,two => One;two
three.four => three;four
five:six => five;six
I have a directory that includes a lot of java files, and in each file I have a class variable:
String system = "x";
I want to be able to create a bash script which I execute in the same directory, which will go to only the java files in the directory, and replace this instance of x, with y. Here x and y are a word. Now this may not be the only instance of the word x in the java script, however it will definitely be the first.
I want to be able to execute my script in the command line similar to:
changesystem.sh -x -y
This way I can specify what the x should be, and the y I wish to replace it with. I found a way to find and print the line number at which the first instance of a pattern is found:
awk '$0 ~ /String system/ {print NR}' file
I then found how to replace a substring on a given line using:
awk 'NR==line_number { sub("x", "y") }'
However, I have not found a way to combine them. Maybe there is also an easier way? Or even, a better and more efficient way?
Any help/advice will be greatly appreciated
You may create a changesystem.sh file with the following GNU awk script:
#!/bin/bash
for f in *.java; do
awk -i inplace -v repl="$1" '
!x && /^\s*String\s+system\s*=\s*".*";\s*$/{
lwsp=gensub(/\S.*/, "", 1);
print lwsp"String system = \""repl"\";";
x=1;next;
}1' "$f";
done;
Or, with any awk:
#!/bin/bash
for f in *.java; do
awk -v repl="$1" '
!x && /^[[:space:]]*String[[:space:]]+system[[:space:]]*=[[:space:]]*".*";[[:space:]]*$/{
lwsp=$0; sub(/[^[:space:]].*/, "", lwsp);
print lwsp"String system = \""repl"\";";
x=1;next
}1' "$f" > tmp && mv tmp "$f";
done;
Then, make the file executable:
chmod +x changesystem.sh
Then, run it like
./changesystem.sh 'new_value'
Notes:
for f in *.java; do ... done iterates over all *.java files in the current directory
-i inplace - GNU awk feature to perform replacement inline (not available in a non-GNU awk)
-v repl="$1" passes the first argument of the script to the awk command
!x && /^\s*String\s+system\s*=\s*".*";\s*$/ - if x is false and the record starts with any amount of whitespace (\s* or [[:space:]]*), then String, any 1+ whitespaces, system, = enclosed with any zero or more whitesapces, and then a " char, then has any text and ends with "; and any zero or more whitespaces, then
lwsp=gensub(/\S.*/, "", 1); puts the leading whitespace in the lwsp variable (it removes all text starting with the first non-whitespace char from the line matched)
lwsp=$0; sub(/[^[:space:]].*/, "", lwsp); - same as above, just in a different way since gensub is not supported in non-GNU awk and sub modifies the given input string (here, lwsp)
{print "String system = \""repl"\";";x=1;next}1 - prints the String system = " + the replacement string + ";, assigns 1 to x, and moves to the next line, else, just prints the line as is.
You don't need to pre-compute the line number. The whole job can be done by one not-too-complicated sed command. You probably do want to script it, though. For example:
#!/bin/bash
[[ $# -eq 3 ]] || {
echo "usage: $0 <context regex> <target regex> <replacement text>" 1>&2
exit 1
}
sed -si -e "/$1/ { s/\\<$2\\>/$3/; t1; p; d; :1; n; b1; }" ./*.java
That assumes that the files to modify are java source files in the current working directory, and I'm sure you understand the (loose) argument check and usage message.
As for the sed command itself,
the -s option instructs sed to treat each argument as a separate stream, instead of operating as if by concatenating all the inputs into one long stream.
the -i option instructs sed to modify the designated files in-place.
the sed expression takes the default action for each line (printing it verbatim) unless the line matches the "context" pattern given by the first script argument.
for lines that do match the context pattern,
s/\\<$2\\>/$3/ - attempt to perform the wanted substitution
the \< and \> match word start and end boundaries, respectively, so that the specified pattern will not match a partial word (though it can match multiple complete words if the target pattern allows)
t1 - if a substitution was made, then branch to label 1, otherwise
p; d - print the current line and immediately start the next cycle
:1; n; b1 - label 1 (reachable only by branching): print the current line and read the next one, then loop back to label 1. This prints the remainder of the file without any more tests or substitutions.
Example usage:
/path/to/replace_first.sh 'String system' x y
It is worth noting that that does expose the user to some details of seds interpretation of regular expressions and replacement text, though that does not manifest for the example usage.
Note that that could be simplified by removing the context pattern bit if you are sure you want to modify the overall first appearance of the target in each file. You could also hard-code the context, the target pattern, and/or the replacement text. If you hard-code all three then the script would no longer need any argument handling or checking.
How would you go about removing everything after x number of characters? For example, cut everything after 15 characters and add ... to it.
This is an example sentence should turn into This is an exam...
GnuTools head can use chars rather than lines:
head -c 15 <<<'This is an example sentence'
Although consider that head -c only deals with bytes, so this is incompatible with multi-bytes characters like UTF-8 umlaut ü.
Bash built-in string indexing works:
str='This is an example sentence'
echo "${str:0:15}"
Output:
This is an exam
And finally something that works with ksh, dash, zsh…:
printf '%.15s\n' 'This is an example sentence'
Even programmatically:
n=15
printf '%.*s\n' $n 'This is an example sentence'
If you are using Bash, you can directly assign the output of printf to a variable and save a sub-shell call with:
trim_length=15
full_string='This is an example sentence'
printf -v trimmed_string '%.*s' $trim_length "$full_string"
Use sed:
echo 'some long string value' | sed 's/\(.\{15\}\).*/\1.../'
Output:
some long strin...
This solution has the advantage that short strings do not get the ... tail added:
echo 'short string' | sed 's/\(.\{15\}\).*/\1.../'
Output:
short string
So it's one solution for all sized outputs.
Use cut:
echo "This is an example sentence" | cut -c1-15
This is an exam
This includes characters (to handle multi-byte chars) 1-15, c.f. cut(1)
-b, --bytes=LIST
select only these bytes
-c, --characters=LIST
select only these characters
Awk can also accomplish this:
$ echo 'some long string value' | awk '{print substr($0, 1, 15) "..."}'
some long strin...
In awk, $0 is the current line. substr($0, 1, 15) extracts characters 1 through 15 from $0. The trailing "..." appends three dots.
Todd actually has a good answer however I chose to change it up a little to make the function better and remove unnecessary parts :p
trim() {
if (( "${#1}" > "$2" )); then
echo "${1:0:$2}$3"
else
echo "$1"
fi
}
In this version the appended text on longer string are chosen by the third argument, the max length is chosen by the second argument and the text itself is chosen by the first argument.
No need for variables :)
Using Bash Shell Expansions (No External Commands)
If you don't care about shell portability, you can do this entirely within Bash using a number of different shell expansions in the printf builtin. This avoids shelling out to external commands. For example:
trim () {
local str ellipsis_utf8
local -i maxlen
# use explaining variables; avoid magic numbers
str="$*"
maxlen="15"
ellipsis_utf8=$'\u2026'
# only truncate $str when longer than $maxlen
if (( "${#str}" > "$maxlen" )); then
printf "%s%s\n" "${str:0:$maxlen}" "${ellipsis_utf8}"
else
printf "%s\n" "$str"
fi
}
trim "This is an example sentence." # This is an exam…
trim "Short sentence." # Short sentence.
trim "-n Flag-like strings." # Flag-like strin…
trim "With interstitial -E flag." # With interstiti…
You can also loop through an entire file this way. Given a file containing the same sentences above (one per line), you can use the read builtin's default REPLY variable as follows:
while read; do
trim "$REPLY"
done < example.txt
Whether or not this approach is faster or easier to read is debatable, but it's 100% Bash and executes without forks or subshells.
I have file like this
TT;12-11-18;text;abc;def;word
AA;12-11-18;tee;abc;def;gih;word
TA;12-11-18;teet abc;def;word
TT;12-11-18;tdd;abc;def;gih;jkl;word
I want output like this
TT;12-11-18;text;abc;def;word
TA;12-11-18;teet abc;def;word
I want to get word if it occur at position 5 after date 12-11-18. I do not want this occurrence if its found after this position that is at 6th or 7th position. Count of position start from date 12-11-18
I want tried this command
cat file.txt|grep "word" -n1
This print all occurrence in which this pattern word is matched. How should I solve my problem?
Try this(GNU awk):
awk -F"[; ]" '/12-11-18/ && $6=="word"' file
Or sed one:
sed -n '/12-11-18;\([^; ]*[; ]\)\{3\}word/p' file
Or grep with basically the same regex(different escape):
grep -E "12-11-18;([^; ]*[; ]){3}word" file
[^; ] means any character that's not ; or (space).
* means match any repetition of former character/group.
-- [^; ]* means any length string that don't contain ; or space, the ^ in [^; ] is to negate.
[; ] means ; or space, either one occurance.
() is to group those above together.
{3} is to match three repetitives of former chracter/group.
As a whole ([^; ]*[; ]){3} means ;/space separated three fields included the delimiters.
As #kvantour points out, if there could be multiple spaces at one place they could be faulty.
To consider multiple spaces as one separator, then:
awk -F"(;| +)" '/12-11-18/ && $6=="word"'
and
grep -E "12-11-18;([^; ]*(;| +)){3}word"
or GNU sed (posix/bsd/osx sed does not support |):
sed -rn '/12-11-18;([^; ]*(;| +)){3}word/p'
I have a file filename with 2 lines:
2018-Feb-22 06:02:01.1234|AVC-00123HHGF|427654|Default|Name1 [1]|2334|2344444|(00:00:00.45567)|
2018-Feb-22 07:02:01.1234|BCV-00123HHGF|427654|Default|Name1 [1]|2334|2344444|(00:00:00.45567)|
I want to concat string
"Warning: Time elapsed:,3444, is smaller than Name2:44222"
At the end of the line which is equal with
Var1="2018-Feb-22 06:02:01.1234|AVC-00123HHGF|427654|Default|Name1 [1]|2334|2344444|(00:00:00.45567)|"
Or has the following pattern
Var2="2018-Feb-22 06:02:01.1234|AVC-00123HHGF|"
And then filename will contain
2018-Feb-22 06:02:01.1234|AVC-00123HHGF|427654|Default|Name1 [1]|2334|2344444|(00:00:00.45567)|"Warning: Time elapsed:,3444, is smaller than Name2:44222"
2018-Feb-22 07:02:01.1234|BCV-00123HHGF|427654|Default|Name1 [1]|2334|2344444|(00:00:00.45567)|
This is what i've tried:
Var3='2018-Feb-22 06:02:01.1234|AVC-00123HHGF|427654|Default|Name1 [1]|2334|2344444|(00:00:00.45567)|"Warning: Time elapsed:,3444, is smaller than Name2:44222"'
sed -i 's/'"$Var1"'/'"$Var3"'/' filename
sed -i "s/$Var1/$Var3/" filename
Var4='"Warning: Time elapsed:,3444, is smaller than Name2:44222"'
sed -i "/$Var1/a $Var4" filename
But nothing happens. Not even an error.
It's there any other way to do this? I need to keep the same order of the lines within filename.
UPDATE: i've gave up on using sed and tried a less optimal solution, but it works.
I have 2 files:
File_to_change
File_with_lines_to_add
While read line; do
Prkey=##calculate pk
N=0
While read linetoadd; do
Prmkey=##calculate pk
If [ "$Prkey" =="$Prmkey" ]; then
N=1
echo "$line$linetoadd">>outfile
Fi
Done < File_with_lines_to_add
If [ "$N" == "0" ]; then
echo "$line">>outfile
Fi
Done < File_to_change
suffix="Warning: Time elapsed:,3444, is smaller than Name2:44222"
pattern="AVC-"
sed -E "/$pattern/s/^(.*)$/\1$suffix/" filename
2018-Feb-22 06:02:01.1234|AVC-00123HHGF|427654|Default|Name1 [1]|2334|2344444|(00:00:00.45567)|Warning: Time elapsed:,3444, is smaller than Name2:44222
2018-Feb-22 07:02:01.1234|BCV-00123HHGF|427654|Default|Name1 [1]|2334|2344444|(00:00:00.45567)|
sed -E : -E allows later usage of () for grouping, without masking
"..." : the command. Double qoutes allow $x expressions to be evaluated by the shell, before sed gets them to read
/$pattern/ : look for this pattern and only act, if pattern is found
s/a/b/ : substitute expression a with b
/^(.*)$/ : our a-expression
^ Start of line
(.*) : an arbitrary character, and in arbitrary count, captured as a group for later reference as \1, since it's the first group.
$ : end of line
/\1$suffix/ : our b-expression
\1 : what matched above the (.*) pattern
$suffix : what was replaced by the shell
filename
Note that many keywords (better key-characters, since most of them are only 1 character long) change their meaning by context, and quotation is important, and flags like -E, -i, -r.
For example, the $ can be interpreted by the shell, but if not touched, in can mean 'end of line' or 'last line' or 'Dollar Sign'.
'+' can mean at least one, '.' can mean 'any character', a \ is used for masking in sed, to introduce back references like \1. It's a mass but very useful to learn.
Use sed with care.
The vertical bar in "34|AVC-00123HHGF|42" will be interpreted by sed als alternative, either 4 or A and either F or 4. So that would match:
"34VC-00123..."
"3AVC-00123.."
"...HHGF2"
"...HHG42"
which makes for 4 combinations of 2x2 alternatives, none of them matching "34|AVC-00123HHGF|42". How to handle that? Well - masking:
"34\|AVC-00123HHGF\|42"
which might again be done by other sed programs, but you guess where that leads to.
"34.AVC-00123HHGF.42" would match, so make reasonable paranoid decisions, and test and control. :)
Try this:
sed -i '' '/2018-Feb-22 06:02:01.1234|AVC-00123HHGF|/s/$/\"Warning: Time elapsed:,3444, is smaller than Name2:44222\"/' gilename
If that doesn't work, retreat to something simpler, tell us what happens when you try this:
sed 's/2018/XXXX/' filename