How to pass argument to sed without input files. The sed is in single quote and has pipe in it already.
git filter-branch -f --index-filter \
'git ls-files -s | sed -i "s-\t\"*-&dirname/-" |
GIT_INDEX_FILE=$GIT_INDEX_FILE.new \
git update-index --index-info &&
mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"' HEAD
The above code works fine if dirname is hard-coded. But i need to pass the dirname from command line arguments to script. I am trying like below but no luck.
'set subdir=$folder;git ls-files -s | sed -i "s-\t\"*-&$subdir/-" |
The sed says input file is missing. I can not directly pass the input file to sed. I am seeing error like below.
Rewrite 9c86de42b7f3228e3d45a278c8caf7e24c8e55cf (1/2) (0 seconds passed, remaining 0 predicted)
sed: no input files
You have two choices here. You can either evaluate $subdir to build up a fixed sed command that you then set as your filter; or you can evaluate a variable in the shell fragment that git filter-branch will invoke.
To understand the latter, realize that your --index-filter string becomes an ordinary shell variable:
--index-filter)
filter_index="$OPTARG"
;;
which is then passed to eval:
eval "$filter_index" < /dev/null ||
die "index filter failed: $filter_index"
The eval means that the expression in $filter_index, set from your --index-filter argument, has access to all the shell variables and functions in the filter-branch script. Unfortunately, none of its private variables holds the expression you'd like—but you can access its environment variables, which means you can put the value into an environment variable. In other words, you could supply subdir=<whatever> as an environment to your original expression.
In any case, as bk2204 answered, you need to remove the -i option. Besides that, some versions of sed don't accept \t as a tab character (presumably yours does, just be aware of this).
To expand the variable earlier, just do that. For instance:
... --index-filter \
'git ls-files -s | sed "s-\t\"*-&'$folder'/-" | ...
(I've removed the -i here myself). Note how this exits single quotes, expands $folder, then re-enters single quotes. If $folder might contain whitespace, be sure to use double quotes while expanding it here:
... --index-filter \
'git ls-files -s | sed "s-\t\"*-&'"$folder"'/-" | ...
The nesting of quotes here is pretty tricky: the stuff inside the single quotes is all one big string, provided as the argument that sets the variable $filter_index inside the filter-branch script. The eval runs it through a second pass of evaluation, breaking up into the pipeline (git ls-files, piped to sed, piped to git update-index) and running the various multiple commands, all of which have their stdin redirected to /dev/null.
You're using sed -i, which edits its command-line arguments (which are names of files) in place. However, you're reading from standard input and haven't provided any command-line arguments. If you want to just filter the standard input like this, omit the -i from your sed command.
Related
There's a local variable in netmap/LINUX/configure named BUILDDIR and its value is BUILDDIR=$PWD.It should have to be redirected to $(#D) which is netmap package build directory, /usr/local/buildroot/output/build/netmap-master in my case;otherwise, object files will be outputed to buildroot root directory.
I created a variable named NETMAP_CURRENT_BUILD and let it be /usr/local/buildroot/output/build/netmap-master,$(#D),
and then I wanna replace BUILDDIR=$PWD to
BUILDDIR=/usr/local/buildroot/output/build/netmap-master. By using sample code as following, it can't be done.
Sample Code :(sed part worked fine at terminal console)
define NETMAP_BUILD_CMDS
NETMAP_CURRENT_DIR = $(sed -e 's/\//\\\//g' <<< "$(#D)") -- empty
echo "$$(sed -e 's/\//\\\//g' <<< "$(#D)")" -- this line works fine
...
sed -e 's/BUILDDIR=$$PWD/BUILDDIR=$(NETMAP_CURRENT_DIR)/g' -i $(#D)/LINUX/configure
(double $$PWD, it has to be like this, which means string $PWD is needed rather than its value.)
...
endef
You don't actually need this, see below. However, if you do need the result of a shell command in a make variable, use $(shell ...)
Since this is a makefile, the $ are interpreted by make, not by the shell. Therefore, make will try to evaluate sed -e 's/\//\\\//g' <<< "$(#D)" as a variable name. There is of course no variable with that name, so you get an empty string.
To let make run a shell command and store the result in a make variable, use the $(shell ...) function. So
NETMAP_CURRENT_DIR = $(shell sed -e 's/\//\\\//g' <<< "$(#D)")
Note that that should be done outside of the NETMAP_BUILD_CMDS, because the contents of NETMAP_BUILD_CMDS is in fact a shell script: make first expands all variables in that script, then passes it to the shell line-by-line. The shell will actually error out on the above statement because you have spaces around the = so NETMAP_CURRENT_DIR is interpreted as a command name.
If you want to store the result in a shell variable rather than a make variable, then the easiest solution is to use backticks `...` to delimit the command instead of $(...):
define NETMAP_BUILD_CMDS
NETMAP_CURRENT_DIR=`sed -e 's/\//\\\//g' <<< "$(#D)"`; \
sed -e 's/BUILDDIR=$$PWD/BUILDDIR=$$NETMAP_CURRENT_DIR/g' -i $(#D)/LINUX/configure
endef
However, in your case you don't need this at all.
First of all, you can do simple substitutions directly in make.
NETMAP_CURRENT_DIR = $(subst /,\/,$(#D))
Even better, you can simply use a different delimiter than / in your sed expression, then there is no need to quote it:
sed -e 's#BUILDDIR=$$PWD#BUILDDIR=$(#D)#g' -i $(#D)/LINUX/configure
One final note: modifications to the code like this are better done in NETMAP_POST_PATCH_HOOKS than in NETMAP_BUILD_CMDS. So finally, the best solution is:
define NETMAP_FIX_BUILDDIR
sed -e 's#BUILDDIR=$$PWD#BUILDDIR=$(#D)#g' -i $(#D)/LINUX/configure
endef
NETMAP_POST_PATCH_HOOKS += NETMAP_FIX_BUILDDIR
I'm trying to write a bash script that adds a new line to a continually growing paragraph of a file. For every time I run the script it should add a new line to that paragraph but instead it returns the entire content of the file.
Here's my code...
function registerServiceProvider {
# register service provider inside config/app.php
sed '/App\Providers\*::class,/a \ App\Providers\${repoName}${provider}::class,' ./config/app.php
}
registerServiceProvider
By default sed prints the resulting stream to stdout. If you have GNU sed you can use -i to modify the file in-place. With many BSD sed you can use -i but it requires an argument which will be the extension added to the backup file created.
If you want to stay more POSIXy you can redirect the output to a new file, then move that temp file over the old name (or redirect the output to make sure you don't change the permissions). Don't try to do it with redirection in one step though, because the first thing the shell will do is truncate the file, then you'll try to read it and it will be empty.
So, with GNU sed you can do:
sed -i -e '/App\Providers\*::class,/a \ App\Providers\${repoName}${provider}::class,' ./config/app.php
or with BSD (or GNU, the argument works but is optional with GNU)
sed -i .bak -e '/App\Providers\*::class,/a \ App\Providers\${repoName}${provider}::class,' ./config/app.php
or portably
sed '/App\Providers\*::class,/a \ App\Providers\${repoName}${provider}::class,' ./config/app.php > tmp_paragraph
cat tmp_paragraph > ./config/app.php
rm tmp_paragraph
If you want the repoName and provider variables to be shell variables that get expanded you need to put the sed script in double quotes. bash doesn't expand variables inside single quoted strings. So to take the GNU sed example you would change it to use double quotes like so:
sed -i -e "/App\\\\Providers\\\\*::class,/a \\ App\\\\Providers\\\\${repoName}${provider}::class," ./config/app.php
and note that we had to do plenty of extra escaping of the slashes, since bash will treat them as escape characters inside double quotes. We can use the single quotes to help with that as long as we put the variables outside it though (and note that I'm still going to double the quotes because I think you meant to do that to get actual back slashes in those spots)
sed -i -e '/App\\Providers\\*::class,/a \ App\\Providers\\'${repoName}${provider}'::class,' ./config/app.php
We have couple of scripts where we want to replace variable evaluation method from $VAR_NAME to ${VAR_NAME}
This is required so that scripts will have uniform method for variable evaluation
I am thinking of using sed for the same, I wrote sample command which looks like follows,
echo "\$VAR_NAME" | sed 's/^$[_a-zA-Z0-9]*/${&}/g'
output for the same is
${$VAR_NAME}
Now i don't want $ inside {}, how can i remove it?
Any better suggestions for accomplishing this task?
EDIT
Following command works
echo "\$VAR_NAME" | sed -r 's/\$([_a-zA-Z]+)/${\1}/g'
EDIT1
I used following command to do replacement in script file
sed -i -r 's:\$([_a-zA-Z0-9]+):${\1}:g' <ScriptName>
Since the first part of your sed command searches for the $ and VAR_NAME, the whole $VAR_NAME part will be put inside the ${} wrapper.
You could search for the $ part with a lookbehind in your regular expression, so that you end up ending the sed call with /{&}/g as the $ will be to the left of your matched expression.
http://www.regular-expressions.info/lookaround.html
http://www.perlmonks.org/?node_id=518444
I don't think sed supports this kind of regular expression, but you can make a command that begins perl -pe instead. I believe the following perl command may do what you want.
perl -p -e 's/(?<=\$)[_a-zA-Z0-9]*/{$&}/g'
PCRE Regex to SED
I am trying to change the values in a text file using sed in a Bash script with the line,
sed 's/draw($prev_number;n_)/draw($number;n_)/g' file.txt > tmp
This will be in a for loop. Why is it not working?
Variables inside ' don't get substituted in Bash. To get string substitution (or interpolation, if you're familiar with Perl) you would need to change it to use double quotes " instead of the single quotes:
# Enclose the entire expression in double quotes
$ sed "s/draw($prev_number;n_)/draw($number;n_)/g" file.txt > tmp
# Or, concatenate strings with only variables inside double quotes
# This would restrict expansion to the relevant portion
# and prevent accidental expansion for !, backticks, etc.
$ sed 's/draw('"$prev_number"';n_)/draw('"$number"';n_)/g' file.txt > tmp
# A variable cannot contain arbitrary characters
# See link in the further reading section for details
$ a='foo
bar'
$ echo 'baz' | sed 's/baz/'"$a"'/g'
sed: -e expression #1, char 9: unterminated `s' command
Further Reading:
Difference between single and double quotes in Bash
Is it possible to escape regex metacharacters reliably with sed
Using different delimiters for sed substitute command
Unless you need it in a different file you can use the -i flag to change the file in place
Variables within single quotes are not expanded, but within double quotes they are. Use double quotes in this case.
sed "s/draw($prev_number;n_)/draw($number;n_)/g" file.txt > tmp
You could also make it work with eval, but don’t do that!!
This may help:
sed "s/draw($prev_number;n_)/draw($number;n_)/g"
You can use variables like below. Like here, I wanted to replace hostname i.e., a system variable in the file. I am looking for string look.me and replacing that whole line with look.me=<system_name>
sed -i "s/.*look.me.*/look.me=`hostname`/"
You can also store your system value in another variable and can use that variable for substitution.
host_var=`hostname`
sed -i "s/.*look.me.*/look.me=$host_var/"
Input file:
look.me=demonic
Output of file (assuming my system name is prod-cfm-frontend-1-usa-central-1):
look.me=prod-cfm-frontend-1-usa-central-1
I needed to input github tags from my release within github actions. So that on release it will automatically package up and push code to artifactory.
Here is how I did it. :)
- name: Invoke build
run: |
# Gets the Tag number from the release
TAGNUMBER=$(echo $GITHUB_REF | cut -d / -f 3)
# Setups a string to be used by sed
FINDANDREPLACE='s/${GITHUBACTIONSTAG}/'$(echo $TAGNUMBER)/
# Updates the setup.cfg file within version number
sed -i $FINDANDREPLACE setup.cfg
# Installs prerequisites and pushes
pip install -r requirements-dev.txt
invoke build
Retrospectively I wish I did this in python with tests. However it was fun todo some bash.
Another variant, using printf:
SED_EXPR="$(printf -- 's/draw(%s;n_)/draw(%s;n_)/g' $prev_number $number)"
sed "${SED_EXPR}" file.txt
or in one line:
sed "$(printf -- 's/draw(%s;n_)/draw(%s;n_)/g' $prev_number $number)" file.txt
Using printf to build the replacement expression should be safe against all kinds of weird things, which is why I like this variant.
I want to define a macro that will replace some placeholders in makefiles and systemd unit files with the results of RPM variables (macros). However, I don't know if the way expansion works will make the following behave correctly:
%define repl_vars() (sed -e "s:\${LIBEXECDIR}:%{_libexecdir}:g" -e "s:\${LOCALSTATEDIR}:%{_localstatedir}:g" -e "s:\${SYSCONFIGDIR}:%{_sysconfdir}:g" %{1} > %{1}.new && mv %{1}.new %{1})
Where the capitalized ${...} are the placeholders to be replaced with the actual paths held by the standard RPM variables (macros).
Also, escaping the $ of the placeholders with \ works in Bash and stuff I put in the %install section of the SPEC file, but is that still valid in a macro? And is the %{1} valid, as I've never seen an example -- and if not, how do I concatenate .new to %1?
If this is wrong, how do I do it?
tl;dr Yes, that looks basically correct to me. See http://www.rpm.org/wiki/PackagerDocs/Macros for some (somewhat outdated but still largely relevant) documentation.
Macros that are to be expanded in a shell context in a spec file just want to expand to the literal lines that you would write in the spec file section manually.
So assuming you want something to the effect of:
%install
....
sed -e 's:${LIBEXECDIR}:%{_libexecdir}:g' -e 's:${LOCALSTATEDIR}:%{_localstatedir}:g' -e 's:${SYSCONFIGDIR}:%{_sysconfdir}:g' 'some_file' > 'some_file.new' && mv 'some_file.new' 'some_file'
....
And you want to call your macro as %repl_vars some_file then you want a macro roughly like this:
%define repl_vars() sed -e 's:${LIBEXECDIR}:%{_libexecdir}:g' -e 's:${LOCALSTATEDIR}:%{_localstatedir}:g' -e 's:${SYSCONFIGDIR}:%{_sysconfdir}:g' '%{1}' > '%{1}.new' && mv '%{1}.new' '%{1}'
Notice I switched to single quotes instead of double quotes to avoid $ from being evaluated and needing to be escaped. I also dropped the wrapping () because this didn't seem to need the forced sub-shell.