How can I replace a specific line by line number in a text file? - linux

I have a 2GB text file on my linux box that I'm trying to import into my database.
The problem I'm having is that the script that is processing this rdf file is choking on one line:
mismatched tag at line 25462599, column 2, byte 1455502679:
<link r:resource="http://www.epuron.de/"/>
<link r:resource="http://www.oekoworld.com/"/>
</Topic>
=^
I want to replace the </Topic> with </Line>. I can't do a search/replace on all lines but I do have the line number so I'm hoping theres some easy way to just replace that one line with the new text.
Any ideas/suggestions?

sed -i yourfile.xml -e '25462599s!</Topic>!</Line>!'

sed -i '25462599 s|</Topic>|</Line>|' nameoffile.txt

The tool for editing text files in Unix, is called ed (as opposed to sed, which as the name implies is a stream editor).
ed was once intended as an interactive editor, but it can also easily scripted. The way ed works, is that all commands take an address parameter. The way to address a specific line is just the line number, and the way to change the addressed line(s) is the s command, which takes the same regexp that sed would. So, to change the 42nd line, you would write something like 42s/old/new/.
Here's the entire command:
FILENAME=/path/to/whereever
LINENUMBER=25462599
ed -- "${FILENAME}" <<-HERE
${LINENUMBER}s!</Topic>!</Line>!
w
q
HERE
The advantage of this is that ed is standardized, while the -i flag to sed is a proprietary GNU extension that is not available on a lot of systems.

Use "head" to get the first 25462598 lines and use "tail" to get the remaining lines (starting at 25462601). Though... for a 2GB file this will likely take a while.
Also are you sure the problem is just with that line and not somewhere previous (ie. the error looks like an XML parse error which might mean the actual problem is someplace else).

My shell script:
#!/bin/bash
awk -v line=$1 -v new_content="$2" '{
if (NR == line) {
print new_content;
} else {
print $0;
}
}' $3
Arguments:
first: line number you want change
second: text you want instead original line contents
third: file name
This script prints output to stdout then you need to redirect. Example:
./script.sh 5 "New fifth line text!" file.txt
You can improve it, for example, by taking care that all your arguments has expected values.

Related

How to use m with the ed function in a Bash Script [duplicate]

I just need to move a line up in sed. I can select the line with
sed -i '7s///'
I need to move line 7 up 2 lines so it will be line 5.
I can't find anything on the internet to do this without complicated scripts, I can't find a simple solution of moving a specific line a specific number of times.
seq 10|sed '5{N;h;d};7G'
when up to line 5 append next line(line 6) into pattern space then save them into hold space and delete them from pattern space; up to line 7 then append the hold space content("5\n6") behind the line 7; now, pattern space is "7\n5\n6";finally,sed will print the pattern space at the end of current cycle by default(if no "-n" parameter)
ed is better at this, since it has a "move" command that does exactly what you want. To move line 7 to be the line after line 4, just do 7m4. ed doesn't write the data back by default, so you need to explicitly issue a w command to write the data:
printf '7m4\nw\n' | ed input
Although it is perhaps better to use a more modern tool:
ex -s -c 7m4 -c w -c q input

Replace exact content into file from cat results though terminal

I would like dynamically to change some values on a file, with name waf.py.
the line is like this
pp.pprint(api.post("/sites/<SITEID>/waf", {}))
I have stored value to another file with name .adnid
I am processing with command
perl -pe 's/<SITEID>/`cat .adnid`/ge' -i waf.py
It replaces the contents but it add \n (enter) on it and the remaining content goes to a new row, like
pp.pprint(api.post("/sites/626041
/waf", {}))
Instead of excepted (1 row)
pp.pprint(api.post("/sites/626041/waf", {}))
How can I make it work without the newline?
You can use the cat, but you need to chomp off the newline it appends.
perl -pe 'BEGIN { $id=`cat .adnid`; chomp $id } s/<SITEID>/$id/ge' -i waf.py
This adds a BEGIN block that will only be executed once. The variable gets read using cat and then chomped. The substitution is applied to every line of your file. This has the additional benefit that you don't shell out to cat for every single line of input.
Note that BEGIN behaves differently than normal with -p. This is documented in perlmod somewhere close to the large block of examples.
A more Perlish way would be to use open and actually read from the file, which you would also have to do in a BEGIN block. But for a throw-away script it doesn't really matter and the cat way is shorter.

grep specific part out of a line of text

this is my first question here so please bear with me.
I have a large text file from which I need only one specific part of one line. I can grep the line but I do not know how I can get that specific part out of that line.
here is my text line (stored in output.txt)
><source src="https://download.foobar.com/content/mp4/web01/2017/05/08/24599/mp4_web01.mp4" type="video/mp4" data-label="Laag - 360p" /><source src="https://download.foobar.com/content/mp4/web02/2017/05/08/24599/mp4_web02.mp4" type="video/mp4" data-label="Hoog - 720p" /><source src="https://download.foobar.com/content/mp4/web03/2017/05/08/24599/mp4_web03.mp4" type="video/mp4" data-label="Normaal - 480p" /></video></div></div>
the part I need to extract from this line is:
https://download.foobar.com/content/mp4/web02/2017/05/08/24599/mp4_web02.mp4
Now I can do a grep like this but that gives me back three lines:
grep -Po '><source src="\K[^"]+' output.txt
gives me:
https://download.omroep.nl/nos/content/mp4/web01/2017/05/08/24599/mp4_web01.mp4
https://download.omroep.nl/nos/content/mp4/web02/2017/05/08/24599/mp4_web02.mp4
https://download.omroep.nl/nos/content/mp4/web03/2017/05/08/24599/mp4_web03.mp4
I would like to get only the line I am looking for without making the extra sed command to remove the first and third line of the results.
How can I grep the input line and only get back the intended line. I only need the link to the mp4_web02.mp4 file.
Can anyone help me get this into one grep command?

how to edit a file to keep lines of same length only by using shell command

I have a file with contents like below.
7f22cebc9330
600e98
7fff1814ff50
7f22cebc95c0
7f22cebc95b8
4002a8
7f22cebc95bc
You can see that some have 12 characters (eg:7f22cebc9330 ), and some have six (eg: 600e98).
How can I edit this file such that only lines with 12 characters are kept in the file, removing all the lines that are NOT of 12 characters length ?
So that my new file would look like this:
7f22cebc9330
7fff1814ff50
7f22cebc95c0
7f22cebc95b8
7f22cebc95bc
I mean by using shell command in linux.
Thanks.
awk 'length() == 12' input.file > output.file
There are several tools that will allow you to edit the file directly (gnu sed, perl, etc), but doing so is a mistake. Write the output to a new file, and use the shell to rename if necessary.

Linux command to replace string in HUGE file with another string

I have a huge file (8GB), I want replace on the first 30 lines the String LATIN1 with UTF-8 what is the most efficient method? Means exist there a way to use probably sed but to quit after parsed first 30 lines.
VIM was not able to save the file in 3 hours.
The problem is that in the event of a replacement, all programs will make a copy of the file with the substitution in place in order to replace the original file ultimately -- they don't want to risk losing the original for obvious reasons.
With perl, you can do this in a one-liner, but that doesn't make it any shorter (well, it probably does compared to vim, since vim preserves history in yet another file, which perl doesn't):
perl -pi -e 's,\bLATIN1\b,UTF-8,g if $. <= 30' thefile
With sed, you can quit using q:
sed -e 's/LATIN1/UTF-8/g' -e 30q
untested, but I think ed will edit the file in-place without writing to a temp file.
ed yourBigFile << END
1,30s/LATIN1/UTF-8/g
w
q
END

Resources