I have ruby script that receives name of config file as argument.
I need to run it in loop changing some param inside the config each iteration.
Everything ok with sed, however I have no idea how can I pass the sed's script output to ruby, so that ruby will think that it's file? Is it possible?
It might be clearer with code:
That is how it's usually launched:
ruby script.rb config.conf
What I want is:
sed 's/one_param/another_param/' config.conf | ruby script.rb ???????
What should I put so that ruby script were think that it received file with content as sed's output?
I thought about workaround with saving temporary file as sed's output and then passing the file to script.rb, but I sure there is better way to achieve it
See this answer on how to use process substitution.
In short:
cat <( echo "yo")
Or in your case:
ruby script.rb <(sed 's/one_param/another_param/' config.conf)
To create a process substitution you enclose the command with <(...) like: <(COMMAND)
Check http://mywiki.wooledge.org/ProcessSubstitution
Conventionally, UNIX programs accept - as a filename to mean "read from standard input":
echo foo | wc -
This is a convention that works basically everywhere.
However, script writers who don't know Unix may not think to implement this. This is a bug that should be fixed by them, but you can work around it using /dev/stdin instead:
echo foo | wc /dev/stdin
In your example, this would be one of
sed 's/one_param/another_param/' config.conf | ruby script.rb -
sed 's/one_param/another_param/' config.conf | ruby script.rb /dev/stdin
Related
We have couple of scripts where we want to replace variable evaluation method from $VAR_NAME to ${VAR_NAME}
This is required so that scripts will have uniform method for variable evaluation
I am thinking of using sed for the same, I wrote sample command which looks like follows,
echo "\$VAR_NAME" | sed 's/^$[_a-zA-Z0-9]*/${&}/g'
output for the same is
${$VAR_NAME}
Now i don't want $ inside {}, how can i remove it?
Any better suggestions for accomplishing this task?
EDIT
Following command works
echo "\$VAR_NAME" | sed -r 's/\$([_a-zA-Z]+)/${\1}/g'
EDIT1
I used following command to do replacement in script file
sed -i -r 's:\$([_a-zA-Z0-9]+):${\1}:g' <ScriptName>
Since the first part of your sed command searches for the $ and VAR_NAME, the whole $VAR_NAME part will be put inside the ${} wrapper.
You could search for the $ part with a lookbehind in your regular expression, so that you end up ending the sed call with /{&}/g as the $ will be to the left of your matched expression.
http://www.regular-expressions.info/lookaround.html
http://www.perlmonks.org/?node_id=518444
I don't think sed supports this kind of regular expression, but you can make a command that begins perl -pe instead. I believe the following perl command may do what you want.
perl -p -e 's/(?<=\$)[_a-zA-Z0-9]*/{$&}/g'
PCRE Regex to SED
I have this
exec 5<>/dev/tcp/twitter.ca/80
echo -e "GET / HTTP/1.0\n" >&5
cat <&5
I looked a similar script
curl http://cookpad.com 2>&1 | grep -o -E 'href="([^"#]+)"' | cut -d'"' -f2
but I need to use the sed command only.
the output i get is this
sed: -e expression #1, char 2: extra characters after command
#!/bin/bash
exec 5<>/dev/tcp/twitter.ca/80
echo -e "GET / HTTP/1.0\n" >&5
cat <&5 | sed -r -e 'href="([^"#]+)"'
Is what I currently have and I guess what im trying to do is how to use sed to strip it of all extras and keep it with just the htmls?
my output should be look something like this:
href="UnixFortune.apk"
href="UnixFortune-1.0.tgz"
href="BeagleCar.apk"
href="BeagleCar.zip"
sed is a scripting language. Your command looks like you are trying to use the h command (copy pattern to hold space) with options starting with ref=... but the h command doesn't take any options.
Anyway, the command you want is the s command, which performs substitutions. Namely, you want to substitute everything before and after the matching group with nothing (and thus print only the captured group).
sed -r -e 's/.*href="([^"#]+)".*/\1/'
However, this still doesn't do the right thing if there are multiple matches on a line (or lines without a match, although that is easy to fix with sed -n 's/.../p'). You can certainly solve that in sed, but I would suggest you go with grep -o instead, unless you specifically want to learn, write, and maintain sed script. (Or, alternatively, rewrite into an Awk or Perl script. Perl in particular has a lot more leverage for tasks like this.)
And of course, for this particular task, the proper tool is an HTML parser. There is no way to properly pick apart HTML using just regular expressions. See e.g. How to extract links from a webpage using lxml, XPath and Python?
I have to implement an application in shell programming (Unix/Linux).
I have to search a word from a text file and replace that word with my given word. I have a knowledge on shell and still learning.
I am not expecting source code. Can anybody help me or suggest me or give me some similar solution....
cat abc.txt | grep "pattern" | sed 's/"pattern"/"new pattern"/g'
The above command should work
Thanks,
Regards,
Dheeraj Rampally
Say you are looking for pattern in a file (input.txt) and want to replace it with "new pattern" in another (output.txt)
Here is the main idea, without UUOC:
<input.txt sed 's/"pattern"/"new pattern"/g' >output.txt
todo
Now you need to embed this line in your program. You may want to make it interactive, or a command that you could use with 3 parameters.
edit
I tried to avoid the use of output.txt as a temporary file with this:
<input.txt sed 's/"pattern"/"new pattern"/g' >input.txt
but it empties input.txt for a reason I can't understand. So I tried with a subshell, so:
echo $(<input.txt sed 's/pattern/"new pattern"/g')>input.txt
... but the echo command removes line breaks... still looking.
edit2
From https://unix.stackexchange.com/questions/11067/is-there-a-way-to-modify-a-file-in-place , it looks like writing to the very same file at once it not easy at all. However, I could do what I wanted with sed -i for linux only:
sed -i 's/pattern/"new pattern"/g' input.txt
From sed -i + what the same option in SOLARIS , it looks like there's no alternative, and you must use a temporary file:
sed 's/pattern/"new pattern"/g' input.txt > input.tmp && mv input.tmp input.txt
I have lots of Perl scripts that need to be updated to a consistent version required and have various versions and flags currently. All of these statements lie in the first line of the script. I have not had much luck here, obviously trying to use sed but is there something better?
Example line that needs to be edited
#!/opt/local/bin/perl5
I tried a bash script as follows, but it does not work for me.
#!/bin/bash
echo "Changing Perl headers in the following..."
for file in $(ls $1*.pl)
do
echo $file
sed 's/#!\/opt\/local/bin\/perl5/#![PERL5]/' $file > /dev/null
# Perl Version: PERL5
done
Any help from someone experienced with sed would be great! Thanks!
Use Perl to fix your Perl scripts. You can use braces for the regexes to avoid leaning toothpick syndrome.
perl -i -pe 's{^#!/opt/local/bin/perl5}{#!/some/new/path}' *.pl
-i and -p are used to edit files in place, kinda like sed. See perlrun for details.
I'm not exactly sure what it is you want. If you're wanting to edit all of the files in $DIR so that their first line is #![PERL5] instead of #!/opt/local/bin/perl5, this will do that:
sed --in-place '1s:^#!/opt/local/bin/perl5:#![PERL5]:' "$DIR"/*.pl
To edit files with perl, you'd use the -pi switches. Since you only wanted to change the first line, I was considering a way to do so without causing the regex to change anything else in the file. This little hack will allow that. Not sure if it's worth the effort, since shebang-like strings should not really be found elsewhere.
perl -pi.bak -e '
$x //= s|^#!/opt/local/bin/perl5|#![PERL5]|;
undef $x if eof;' *.pl
The //= operator will not attempt to execute the RHS if $x is already defined, so this will in effect only perform one change. The addition of undef $x at eof will reset the variable for the next file, causing each file in the argument list to be altered once.
You may not want backups, in which case you'd remove .bak from the -i switch.
Suppose if you want to change perl5 to perl6
Try this:
perl -pi -e 's/perl5/perl6/' $file
I have a program that writes to fd3 and I want to process that data with grep and sed. Here is how the code looks so far:
exec 3> >(grep "good:"|sed -u "s/.*:\(.*\)/I got: \1/")
echo "bad:data1">&3
echo "good:data2">&3
Nothing is output until I do a
exec 3>&-
Then, everything that I wanted finally arrives as I expected:
I got: data2
It seems to reply immediately if I use only a grep or only a sed, but mixing them seems to cause some sort of buffering. How can I get immediate output from fd3?
I think I found it. For some reason, grep doesn't automatically do line buffering. I added a --line-buffered option to grep and now it responds immediately.
You only need to tell grep and sed to not bufferize lines:
grep --line-buffered
and
sed -u
An alternate means to stop sed from buffering is to run it through the s2p sed-to-Perl translator and insert a directive to have it command-buffered, perhaps like
BEGIN { $| = 1 }
The other reason to do this is that it gives you the more convenient notation from EREs instead of the backslash-annoying legacy BREs. You also get the full complement of Unicode properties, which is often critical.
But you don’t need the translator for such a simple sed command. And you do not need both grep and sed, either. These all work:
perl -nle 'BEGIN{$|=1} if (/good:/) { s/.*:(.*)/I got: $1/; print }'
perl -nle 'BEGIN{$|=1} next unless /good:/; s/.*:(.*)/I got: $1/; print'
perl -nle 'BEGIN{$|=1} next unless /good:/; s/.*:/I got: /; print'
Now you also have access to the minimal quantifier, *?, +?, ??, {N,}?, and {N,M}?. These now allow things like .*? or \S+? or [\p{Pd}.]??, which may well be preferable.
You can merge the grep into the sed like so:
exec 3> >(sed -une '/^good:/s//I got: /p')
echo "bad:data1">&3
echo "good:data2">&3
Unpacking that a bit: You can put a regexp (between slashes as usual) before any sed command, which makes it only be applied to lines that match that regexp. If the first regexp argument to the s command is the empty string (s//whatever/) then it will reuse the last regexp that matched, which in this case is the prefix, so that saves having to repeat yourself. And finally, the -n option tells sed to print only what it is specifically told to print, and the /p suffix on the s command tells it to print the result of the substitution.
The -e option is not strictly necessary but is good style, it just means "the next argument is the sed script, not a filename".
Always put sed scripts in single quotes unless you need to substitute a shell variable in there, and even then I would put everything but the shell variable in single quotes (the shell variable is, of course, double-quoted). You avoid a bunch of backslash-related grief that way.
On a Mac, brew install coreutils and use gstdbuf to control buffering of grep and sed.
Turn off buffering in pipe seems to be the easiest and most generic answer. Using stdbuf (coreutils) :
exec 3> >(stdbuf -oL grep "good:" | sed -u "s/.*:\(.*\)/I got: \1/")
echo "bad:data1">&3
echo "good:data2">&3
I got: data2
Buffering has other dependencies, for example depending on mawk either gawk reading this pipe :
exec 3> >(stdbuf -oL grep "good:" | awk '{ sub(".*:", "I got: "); print }')
In that case, mawk would retain the input, gawk wouldn't.
See also How to fix stdio buffering