I would like dynamically to change some values on a file, with name waf.py.
the line is like this
pp.pprint(api.post("/sites/<SITEID>/waf", {}))
I have stored value to another file with name .adnid
I am processing with command
perl -pe 's/<SITEID>/`cat .adnid`/ge' -i waf.py
It replaces the contents but it add \n (enter) on it and the remaining content goes to a new row, like
pp.pprint(api.post("/sites/626041
/waf", {}))
Instead of excepted (1 row)
pp.pprint(api.post("/sites/626041/waf", {}))
How can I make it work without the newline?
You can use the cat, but you need to chomp off the newline it appends.
perl -pe 'BEGIN { $id=`cat .adnid`; chomp $id } s/<SITEID>/$id/ge' -i waf.py
This adds a BEGIN block that will only be executed once. The variable gets read using cat and then chomped. The substitution is applied to every line of your file. This has the additional benefit that you don't shell out to cat for every single line of input.
Note that BEGIN behaves differently than normal with -p. This is documented in perlmod somewhere close to the large block of examples.
A more Perlish way would be to use open and actually read from the file, which you would also have to do in a BEGIN block. But for a throw-away script it doesn't really matter and the cat way is shorter.
Related
I have a text file, called texto.txt in Documentos folder, with some values like the ones below:
cat ~/Documentos/texto.txt
65f8: Testado
a4a1: Testado 2
So I want to change a whole line by using a customized function which gets as parameters the new value.
The new value will always keep the first 6 characters, changing only what comes after them. Although I am testing only the first four.
Then I edited my .bashrc including my function like shown below.
muda()
{
export BUSCA="$(echo $* | cut -c 1-4)";
sed -i "/^$BUSCA/s/.*/$*/" ~/Documentos/texto.txt ;}
When I run the command below it works like a charm, but I feel it could be improved.
muda a4a1: Testado 3
Result:
cat ~/Documentos/texto.txt
65f8: Testado
a4a1: Testado 3
Is there a smarter way to do this? Maybe by getting rid of BUSCA variable?
I'd write:
muda() {
local new_line="$*"
local key=${newline:0:4}
sed -i "s/^${key//\//\\/}.*/${new_line//\//\\/}/" ~/Documentos/texto.txt
}
Notes:
using local variables, not exported environment variables
does not call out to cut, bash can extract a substring
escaping any slashes in the variable values so the sed code is not broken.
What I want:
There is a file /scripts/backup/config.cfg which contains variables. In my specific case the important ones are:
BACKUPLOCATION=""
ROOTLOCATION="/backup"
Then there is a script /scripts/backup/performBackup.sh
For a specific reason I want a part of the script do the following operations:
read the value of the variable ROOTLOCATION
add a ("/" and) timestamp (Date&Time)
safe the new created value to BACKUPLOCATION (by replacing its current value)
Example
If this is the previous state of the config.cfg:
BACKUPLOCATION="dummy"
ROOTLOCATION="/backup"
After the script ran it should be:
BACKUPLOCATION="/backup/2020-05-02-23-00"
ROOTLOCATION="/backup/"
What I tried
First of all the config file gets "loaded" using
source /scripts/backup/config.cfg
I then tried to use the sed command but the quotes are messing with me. Here is one try (which didn't work):
sed -i 's/BACKUPLOCATION\=.*/BACKUPLOCATION="'$ROOTLOCATION/$(date +%Y-%m-%d-%H-%M)'"/' /scripts/backup/config.cfg
Try this:
source /scripts/backup/config.cfg
sed -i 's|BACKUPLOCATION=.*|BACKUPLOCATION="'"$ROOTLOCATION/$(date +%Y-%m-%d-%H-%M)"'"|' /scripts/backup/config.cfg
The problem with your sed is that you use / as delimiter, which is present in $ROOTLOCATION after expansion, therefore sed fails. I used |, which is usually is not present in filenames. If you ever create a file with |, that sed will fail too! So, "know your data" :)
My Aim -->
Files Listing from a command has to be read line by line and be used as part of another command.
Description -->
A command in linux returns
archive/Crow.java
archive/Kaka.java
mypmdhook.sh
which is stored in changed_files variable. I use the following while loop to read the files line by line and use it as part of a pmd command
while read each_file
do
echo "Inside Loop -- $each_file"
done<$changed_files
I am new to writing shell script but my assumption was that the lines would've been separated in the loop and printed in each iteration but instead I get the following error --
mypmdhook.sh: 7: mypmdhook.sh: cannot open archive/Crow.java
archive/Kaka.java
mypmdhook.sh: No such file
Can you tell me how I can just get the value as a string and not as a file what is opened. By the way, the file does exist which made me feel even more confused.(and later use it inside a command). I'd be happy with any kind of answer that helps me understand and resolve this issue.
Since you have data stored in a variable, use a "here string" instead of file redirection:
changed_files="archive/Crow.java
archive/Kaka.java
mypmdhook.sh"
while read each_file
do
echo "Inside Loop -- $each_file"
done <<< "$changed_files"
Inside Loop -- archive/Crow.java
Inside Loop -- archive/Kaka.java
Inside Loop -- mypmdhook.sh
Extremely important to quote "$changed_files" in order to preserve the newlines, so the while-read loop works as you expect. A rule of thumb: always quote variables, unless you knows exactly why you want to leave the quotes off.
What happens here is that the value of your variable $changed_files is substituted into your command, and you get something like
while read each_file
do
echo "Inside Loop -- $each_file"
done < archive/Crow.java
archive/Kaka.java
mypmdhook.sh
then the shell tries to open the file for redirecting the input and obviously fails.
The point is that redirections (e.g. <, >, >>) in most cases accept filenames, but what you really need is to give the contents of the variable to the stdin. The most obvious way to do that is
echo $changed_files | while read each_file; do echo "Inside Loop -- $each_file"; done
You can also use the for loop instead of while read:
for each_file in $changed_files; do echo "inside Loop -- $each_file"; done
I prefer using while read ... if there is a chance that some filename may contain spaces, but in most cases for ... in will work for you.
Rather than storing command's output in a variable use while loop like this:
mycommand | while read -r each_file; do echo "Inside Loop -- $each_file"; done
If you're using BASH you can use process substitution:
while read -r each_file; do echo "Inside Loop -- $each_file"; done < <(mycommand)
btw your attempt of done<$changed_files will assume that changed_files represents a file.
I have a huge file (8GB), I want replace on the first 30 lines the String LATIN1 with UTF-8 what is the most efficient method? Means exist there a way to use probably sed but to quit after parsed first 30 lines.
VIM was not able to save the file in 3 hours.
The problem is that in the event of a replacement, all programs will make a copy of the file with the substitution in place in order to replace the original file ultimately -- they don't want to risk losing the original for obvious reasons.
With perl, you can do this in a one-liner, but that doesn't make it any shorter (well, it probably does compared to vim, since vim preserves history in yet another file, which perl doesn't):
perl -pi -e 's,\bLATIN1\b,UTF-8,g if $. <= 30' thefile
With sed, you can quit using q:
sed -e 's/LATIN1/UTF-8/g' -e 30q
untested, but I think ed will edit the file in-place without writing to a temp file.
ed yourBigFile << END
1,30s/LATIN1/UTF-8/g
w
q
END
I have a 2GB text file on my linux box that I'm trying to import into my database.
The problem I'm having is that the script that is processing this rdf file is choking on one line:
mismatched tag at line 25462599, column 2, byte 1455502679:
<link r:resource="http://www.epuron.de/"/>
<link r:resource="http://www.oekoworld.com/"/>
</Topic>
=^
I want to replace the </Topic> with </Line>. I can't do a search/replace on all lines but I do have the line number so I'm hoping theres some easy way to just replace that one line with the new text.
Any ideas/suggestions?
sed -i yourfile.xml -e '25462599s!</Topic>!</Line>!'
sed -i '25462599 s|</Topic>|</Line>|' nameoffile.txt
The tool for editing text files in Unix, is called ed (as opposed to sed, which as the name implies is a stream editor).
ed was once intended as an interactive editor, but it can also easily scripted. The way ed works, is that all commands take an address parameter. The way to address a specific line is just the line number, and the way to change the addressed line(s) is the s command, which takes the same regexp that sed would. So, to change the 42nd line, you would write something like 42s/old/new/.
Here's the entire command:
FILENAME=/path/to/whereever
LINENUMBER=25462599
ed -- "${FILENAME}" <<-HERE
${LINENUMBER}s!</Topic>!</Line>!
w
q
HERE
The advantage of this is that ed is standardized, while the -i flag to sed is a proprietary GNU extension that is not available on a lot of systems.
Use "head" to get the first 25462598 lines and use "tail" to get the remaining lines (starting at 25462601). Though... for a 2GB file this will likely take a while.
Also are you sure the problem is just with that line and not somewhere previous (ie. the error looks like an XML parse error which might mean the actual problem is someplace else).
My shell script:
#!/bin/bash
awk -v line=$1 -v new_content="$2" '{
if (NR == line) {
print new_content;
} else {
print $0;
}
}' $3
Arguments:
first: line number you want change
second: text you want instead original line contents
third: file name
This script prints output to stdout then you need to redirect. Example:
./script.sh 5 "New fifth line text!" file.txt
You can improve it, for example, by taking care that all your arguments has expected values.