I have a file withe following data
:1:aaaaa:aaa:aaa
and i want to remove the leading colon using bash to be like
1:aaaaa:aaa:aaa
You could use sed:
sed 's/^://' filename
^ denotes the start of line, so ^: would match a colon at the beginning of a line. Replace it by nothing!
str=':1:aaaaa:aaa:aaa'
echo ${str:1} #=> 1:aaaaa:aaa:aaa
Resources: Bash string manipulation
As well as sed, you may get better performance for such a simple operation on a large file by using cut:
cut myfile -d : -f 2-
You can also extract additional fields this way with other -f values.
If you want to remove the leading colon from data in a variable, such as in a loop, you can also do
myvar=":1:aaaaa:aaa:aaa"
echo ${myvar#:}
Related
This question already has answers here:
Replace whole line when match found with sed
(4 answers)
Closed 4 years ago.
How do I find line starts with and replace complete line?
File output:
xyz
abc
/dev/linux-test1/
Code:
output=/dev/sda/windows
sed 's/^/dev/linux*/$output/g' file.txt
I am getting below Error:
sed: -e expression #1, char 9: unknown option to `s'
File Output expected after replacement:
xyz
abc
/dev/sda/windows
Let's take this in small steps.
First we try changing "dev" to "other":
sed 's/dev/other/' file.txt
/other/linux-test1/
(Omitting the other lines.) So far, so good. Now "/dev/" => "/other/":
sed 's//dev///other//' file.txt
sed: 1: "s//dev///other//": bad flag in substitute command: '/'
Ah, it's confused, we're using '/' as both a command delimiter and literal text. So we use a different delimiter, like '|':
sed 's|/dev/|/other/|' file.txt
/other/linux-test1/
Good. Now we try to replace the whole line:
sed 's|^/dev/linux*|/other/|' file.txt
/other/-test1/
It didn't replace the whole line... Ah, in sed, '*' means the previous character repeated any number of times. So we precede it with '.', which means any character:
sed 's|^/dev/linux.*|/other/|' file.txt
/other/
Now to introduce the variable:
sed 's|^/dev/linux.*|$output|' file.txt
$output
The shell didn't expand the variable, because of the single quotes. We change to double quotes:
sed "s|^/dev/linux.*|$output|" file.txt
/dev/sda/windows
This might work for you (GNU sed):
output="/dev/sda/windows"; sed -i '\#/dev/linux.*/#c'"$output" file
Set the shell variable and change the line addressed by /dev/linux.*/ to it.
N.B. The shell variable needs to interpolated hence the ; i.e. the variable may be set on a line on its own. Also the the delimiter for the sed address must be changed so as not to interfere with the address, hence \#...#, and finally the shell variable should be enclosed in double quotes to allow full interpolation.
I'd recommend not doing it this way. Here's why.
Sed is not a programming language. It's a stream editor with some constructs that look and behave like a language, but it offers very little in the way of arbitrary string manipulation, format control, etc.
Sed only takes data from a file or stdin (also a file). Embedding strings within your sed script is asking for errors -- constructs like s/re/$output/ are destined to fail at some point, almost regardless of what workarounds you build into your sed script. The best solutions for making sed commands like this work is to do your input sanitization OUTSIDE of sed.
Which brings me to ... this may be the wrong tool for this job, or might be only one component of the toolset for the job.
The error you're getting is obviously because the sed command you're using is horribly busted. The substitute command is:
s/pattern/replacement/flags
but the command you're running is:
s/^/dev/linux*/$output/g
The pattern you're searching for is ^, the null at the beginning of the line. Your replacement pattern is dev, then you have a bunch of text that might be interpreted as flags. This plainly doesn't work, when your search string contains the same character that you're using as a delimiter to the options for the substitute command.
In regular expressions and in sed, you can escape things. You while you might get some traction with s/^\/dev\/linux.*/$output/, you'd still run into difficulty if $output contained slashes. If you're feeding this script to sed from bash, you could use ${output//\//\\\/}, but you can't handle those escapes within sed itself. Sed has no variables.
In a proper programming language, you'd have better separation of variable content and the commands used for the substitution.
output="/dev/sda/windows"
awk -v output="$output" '$1~/\/dev\/linux/ { $0=output } 1' file.txt
Note that I've used $1 here because in your question, your input lines (and output) appear to have a space at the beginning of each line. Awk automatically trims leading and trailing space when assigning field (positional) variables.
Or you could even do this in pure bash, using no external tools:
output="/dev/sda/windows"
while read -r line; do
[[ "$line" =~ ^/dev/linux ]] && line="$output"
printf '%s\n' "$line"
done < file.txt
This one isn't resilient in the face of leading whitespace. Salt to taste.
So .. yes, you can do this with sed. But the way commands get put together in sed makes something like this risky, and despite the available workarounds like switching your substitution command delimiter to another character, you'd almost certainly be better off using other tools.
I have a very long string, with over 2000 occurrences, and it look like that:
Input:
1a2a3a4a5a6a7a8absoad8ryaa90thneas... and more than
I want replace mutiple occurrences at the 3rd to the 450th occurrence in string to output:
1a2a3A4A5A6A7A8AbsoAd8ryAA90thneAs... and more than
I replaced "a" to "A", it replaced from the 3rd occurrence to ending, but I only want to replace from 3rd to 450th, this is my old script:
echo "1a2a3a4a5a6a7a8absoad8ryaa90thneas..." | sed 's/a/A/3g';
Does anyone help me? Or is there any other way? Thanks!
Save the string to a variable and then use brace expansion to target the positional character in the string you want to replace using bash global replace. Consider the following example:
## Our sample string. ##
string="abcde01234
## New, changed string. ##
echo "${string//${string:0:1}/${string:5:1}}"
In this example, when run as a bash script, the first character of $string is replaced with the sixth character of $string. Knowing this little trick, I am sure you will figure out a way to do what you need without having to use sed. Or, you can use the brace expansion similarly in sed.
## Our sample string ##
string="abcde01234"
## New, changed, string. Output is the same as above example. ##
sed -e "s/${string:0:1}/${string:5:1}/g" <(echo "$string")
This should be enough to get you headed in the right direction as far as figuring out the best way for your needs.
choose a character that you know for sure is not present in your input.. for example the ASCII NUL character, then you could do
$ # replacing 3rd to 4th
$ echo "a1a23a5a62a34a235a" | sed 's/a/\x0/3g; s/\x0/a/3g; s/\x0/A/g'
a1a23A5A62a34a235a
$ # replacing 3rd to 5th
$ echo "a1a23a5a62a34a235a" | sed 's/a/\x0/3g; s/\x0/a/4g; s/\x0/A/g'
a1a23A5A62A34a235a
$ # replacing 3rd to 6th
$ echo "a1a23a5a62a34a235a" | sed 's/a/\x0/3g; s/\x0/a/5g; s/\x0/A/g'
a1a23A5A62A34A235a
This might work for you (GNU sed):
sed -r 's/a/\n&/3;s//&\n/450;h;y/a/A/;G;s/.*\n(.*)\n.*\n(.*)\n.*\n/\2\1/' file
Split the string up into pieces using a marker (newline cannot occur in an unadulterated string because it is how sed splits up files), make a copy and then make changes, append the copy and rearrange the pieces.
Alternatively use GNU sed global replace flag:
sed 's/a/\n/451g;s//A/3g;s/\n/a/g' file
sed -E ':again ; s/^(.{4,449})a/\1A/ ; t again' file
you can use t command to branch to label. This is a conditional go to :label when a replacement has happened. So for a character range [x:y], regexp .*{x-1,y-1}a will match greedy for first time, but we loop until there are no more matches.
I want to delete first sign in a file (without creating new file). That is the line (and this line isn't the first one or last one):
#$config['rrdcached'] = "unix:/var/run/rrdcached.sock";
I'm trying to do thuis with sed command but it doesn't work. That is my command:
sed -i "s/#$config\['rrdcached'\].*$/$config\['config'\]/g" text.txt
Any suggestions?
Just replace the first match of # with following command:
sed -i '1 s/#//' test.txt
The $ characters are causing two problems.
First, the shell is treating $config as a variable reference, and replacing it with the value. You need to escape the $ to prevent that.
Second, $ has special meaning in regular expressions, so you need to escape it at that level as well. So you need to escape the backslash and the $.
sed -i "s/^#\\\$config\['rrdcached'\].*\$/\$config['config']/" text.txt
There's no need for the g modifier since you only want to replace the first match on the line. And you should use the ^ anchor so it only matches this at the beginning of the line.
It's also not necessary to escape special regexp characters in the replacement string.
This command works, but i forgot that there is something after '='. At now, everything after that is deleted.
I wrote this:
sed -i "s/^#\\\$config\['rrdcached'\] = "unix:/var/run/rrdcached.sock";.*\$/\\\$config\['config'\]/ = "unix:/var/run/rrdcached.sock";" text.txt
I know, many people have had questions on how to use sed to replace a string, but I have a difficult one here.
I have a file I need to replace a string of text that prompts the user to enter content. I want to automate this so the user does not interact. By replacing this string with a static file path. but the text is a bash script and has ' and " within the string I want to replace. It does not work. Either because I have syntax errors in my formatting, or it simply is not possible to do this action with sed. Please advice!
Here is what I am attempting to do:
I want to replace this long string
read -e -p 'Enter path for Boot Partition : ' BOOTUSERFILEPATH
with a string that looks like this:
BOOTUSERFILEPATH=../board-support/prebuilt-images
My attempt:
sed -i "/read -e -p 'Enter path for Boot Partition : ' BOOTUSERFILEPATH/BOOTUSERFILEPATH="../board-support/prebuilt-images"" file_to_search.sh
Update: I fixed the syntax error, but file still is not updated with the new path information... :(
found the problem. the search was not finding the strings because of an extra space in my search command. It works now!
There are forward slashes in your string so one needs to use a different delmiter. Here I use '|' as the delimiter.
sed "s|read -e -p \'Enter path for Boot Partition : \' BOOTUSERFILEPATH|BOOTUSERFILEPATH=../board-support/prebuilt-images|g" oldfile > newfile
You may note that the -i option to sed which allows files to be edited in place is not a POSIX supported option.
However if you wish to use it:
sed -i "s|read -e -p \'Enter path for Boot Partition : \' BOOTUSERFILEPATH|BOOTUSERFILEPATH=../board-support/prebuilt-images|g" oldfile
You may find it easier to use a pattern with sed which matches part of this string and then replaces its entirety:
sed 's|read -e -p .* BOOTUSERFILEPATH|BOOTUSERFILEPATH=../board-support/prebuilt-images|g' filename > newfilename
From the POSIX specification page for sed:
s/BRE/replacement/flags
Substitute the replacement string for instances of the BRE in the pattern space. Any character other than <backslash> or <newline> can be used instead of a slash to delimit the BRE and the replacement.
You need to use an actual substitute command, and you need to avoid the slashes in the replacement text from confusing sed. Personally, I'd probably use:
sed -i.bak "s%^read .* BOOTUSERFILEPATH$%BOOTUSERFILEPATH=../board-support/prebuilt-images%" file_to_search.sh
or even more likely:
BOOTUSERFILEPATH="../board-support/prebuild-images"
sed -i.bak "s%^read .* BOOTUSERFILEPATH$%BOOTUSERFILEPATH=$BOOTUSERFILEPATH%" file_to_search.sh
The s%%% uses % instead of / to delimit the parts of the command. I cheated on the match pattern, working on the assumption that you don't have many similar lines in the file.
I've read lots of posts to understand how to correctly escape white spaces and special characters inside strings using sed, but still i can't make it, here's what i'm trying to achieve.
I have a file containing the some strings like this one:
JAVA_OPTS="$JAVA_OPTS -Dorg.apache.catalina.jsessionid=some_value"
and i'm trying to replace 'some_value' using the following:
sed -i "s/^\(JAVA_OPTS=\"\$JAVA_OPTS[ \t]*-Dorg\.apache\.catalina\.jsessionid*=\s*\).*\$/\1$DORG_APACHE_CATALINA_JSESSIONID/" $JBOSS_CONFIGURATION/jboss.configuration
$JBOSS_CONFIGURATION is a variable containing an absolute Linux path.
jboss.configuration is a file i'm pointing as the target for replace
operations.
$DORG_APACHE_CATALINA_JSESSIONID contains the value i want instead
of 'some_value'.
Please note that the pattern:
JAVA_OPTS="$JAVA_OPTS -D
Is always present, and org.apache.catalina.jsessionid is an example of a variable value i'm trying to replace with this script.
What's missing/wrong ? i tried also escaping whitespaces using \s without success,
and echoing the whole gives me the following:
echo "s/^\(JAVA_OPTS=\"\$JAVA_OPTS[ \t]*-Dorg\.apache\.catalina\.jsessionid*=\s*\).*\$/\1$DORG_APACHE_CATALINA_JSESSIONID/"
s/^\(JAVA_OPTS="$JAVA_OPTS[ \t]*-Dorg\.apache\.catalina\.jsessionid*=\s*\).*$/\1/
is echo interpreting the search pattern as sed does ?
any info/help/alternative ways of doing it are highly welcome,
thank you all
echo 'JAVA_OPTS="$JAVA_OPTS -Dorg.apache.catalina.jsessionid=some_value"' | (export DORG_APACHE_CATALINA_JSESSIONID=FOO/BAR/FOOBAR; sed "s/^\(JAVA_OPTS=\"\$JAVA_OPTS[ \t]*-Dorg\.apache\.catalina\.jsessionid*=\s*\).*\$/\1${DORG_APACHE_CATALINA_JSESSIONID////\/}\"/")
Note the bash expansion (in order to escape any / that may trip up sed) and the extra \" after $DORG_APACHE_CATALINA_JSESSIONID in order to properly close the double quote. Other than that your sed expression works for me and the above command outputs the follwoing result:
JAVA_OPTS="$JAVA_OPTS -Dorg.apache.catalina.jsessionid=FOO/BAR/FOOBAR"
You can use sed like this:
sed -r '/\$JAVA_OPTS -D/{s/^(.+=).*$/\1'"$DORG_APACHE_CATALINA_JSESSIONID"'/;}' $JBOSS_CONFIGURATION/jboss.configuration
You can specify a pattern that'll match the desired string rather than trying to specify it exactly.
The following should work for you:
sed -i 's#^\(JAVA_OPTS.*Dorg.apache.catalina.jsessionid\)=\([^"]*\)"#\1='"$DORG_APACHE_CATALINA_JSESSIONID"'"#' $JBOSS_CONFIGURATION/jboss.configuration
sed 's/=\w.*$/='"$DORG_APACHE_CATALINA_JSESSIONID"'/' $JBOSS_CONFIGURATION/jboss.configuration