I am trying to insert a pipe symbol in between nth line of a file. Like the one present in the output of diff command.
Do not want to use VI editor.
For example,desired line is 2nd line of the file:
cat filename
Hi Hi
Hello Hello
This This
is Is
it it
desired output:
cat filename
Hi Hi
Hello | Hello
This This
is Is
it it
For your own sanity, just use awk:
$ awk 'NR==2{mid=length($0)/2; $0=substr($0,1,mid) "|" substr($0,mid+1)} 1' file
Hi Hi
Hello | Hello
This This
is Is
it it
To modify the original file you can add > tmp && mv tmp file or use -i inplace if you have GNU awk.
You basically cannot modify in place some textual file by inserting a character inside a line. You need to read all its content, modify that content in memory, then write the new content.
You might be interested in GNU ed, which can edit a file programmatically (inside some script).
You could use awk (or any other filter) and redirect its output to some temporary file, then rename that temporary file as the original one.
sed '
# select 2nd line
/2/ {
# keep original in memory
G;h
:divide
# cycle by taking the 2 char at the egde of string (still string end by the \n here) and copy only first to other part of the separator (\n)
s/\(.\)\(.*\)\(.\)\(\n.*\)/\2\4\1/
# still possible, do it again
t divide
# add last single char if any and remove separator
s/\(.*\)\n\(.*\)/\2\1/
# add original string (with a new line between)
G
# take first half string and end of second string, and place a pipe in the middle in place of other char
s/\(.*\)\n\1\(.*\)/\1|\2/
}' YourFile
posix sed, so --POSIXfor GNU sed
Sel explain
Related
I asked a question here to remove unwanted lines which contained strings which matched a particular pattern:
Remove lines containg string followed by x number of numbers
anubhava provided a good line of code which met my needs perfectly. This code removes any line which contains the string vol followed by a space and three or more consecutive numbers:
grep -Ev '\bvol([[:blank:]]+[[:digit:]]+){2}' file > newfile
The command will be used on a fairly large csv file and be initiated by crontab. For this reason, I would like to keep a record of the lines this command is removing, just so I can go back to check the correct data that is being removed- I guess it will be some sort of log containing the name sof the lines that did not make the final cut. How can I add this functionality?
Drop grep and use awk instead:
awk '/\<vol([[:blank:]]+[[:digit:]]+){2}/{print >> "deleted"; next} 1' file
The above uses GNU awk for word delimiters (\<) and will append every deleted line to a file named "deleted". Consider adding a timestamp too:
awk '/\<vol([[:blank:]]+[[:digit:]]+){2}/{print systime(), $0 >> "deleted"; next} 1' file
For example I have:
Hello1 :
Hello2 :
Hello3 :
How Could I delete all of these with a shell script. The number reaches up all the way to 1000.
sed -i '/^Hello[[:digit:]]\+\>/d' file.txt
Or, if you want to output to a different file:
sed '/^Hello[[:digit:]]\+\>/d' file.txt > newfile.txt
If you wish to delete all the lines that contain only Hello(number) : use below :
Sample Input in file
Hell
Hello1 :
No hello stuff here
Unjulating stuff
Hello2 :
Some sentence
Hello99 :
Script
sed -Ei '/^Hello[[:digit:]]+ :$/d' file
Sample Output in the modified file
Hell
No hello stuff here
Unjulating stuff
Some sentence
What happens above
Using the ^ in the pattern we check for the beginning of the line.
We check the pattern Hello(number) : using Hello[[:digit:]]+ :$. Note that I used -E to enable sed extended regular expressions so I need not escape the + ie (\+). Here [[:digit:]] is a class which contains
all the decimal digits and + is used to check if the pattern before it matches at least one time.
Check the end of the line using $
For a matched pattern, delete it line using the d option
I have also used the sed inplace edit option -i so that the changes
are directly saved to the file.
If you wish to change the a line the begins with Hello(number) : then use the below script
sed -Ei '/^Hello[[:digit:]]+ :/d' file
You might have notices that I just removed the $, so our pattern matches any line that starts with Hello(number) :
Hope this helps.
I want to insert word after nth line after pattern using sed.
I tied to modify this command but it inserts only in first line after pattern.
sed -i '/myPattern/a \ LineIWantToinser ' myFile
What command should I use to insert for example in third line after pattern?
Easiest way to do it with GNU sed is.. (maybe some direct solution exists!?)
sed -n '/pattern/=' file
to see line where pattern is (grep also can be used here with -n)
then if linenumber+ numoflines is for example 123
sed '123aSOME INSERTED TEXT AFTER THAT LINE' file
where little a is append command (after that line, if i is used will be pre pattern line)
ps. I'm eager to see if #neronlevelu (or other sed Lover) will find some better sed solution.
Edit: i've found it, it seems a for append or i for insert must? be on first position on line when using { with ; inside } like
sed '/pattern/{N;N;N;
a SOME TEXT FOR INSERTING
}' file
sed '/pattern/{N;N;N;i \
Line to add after 3 lines with patterne as starting counter
' YourFile
number of N to add line between pattern and inserted line.
there is no check for end of file or pattern in the 3 lines. (not specified in PO)
A version with bash and ed:
ed -s myFile <<<$'/myPattern/+3a\n LineIWantToinser \n.\nwq'
ed enables us to use the line addressing /myPattern/+3.
Say I have some arbitrary multi-line text file:
sometext
moretext
lastline
How can I remove only the last character (the e, not the newline or null) of the file without making the text file invalid?
A simpler approach (outputs to stdout, doesn't update the input file):
sed '$ s/.$//' somefile
$ is a Sed address that matches the last input line only, thus causing the following function call (s/.$//) to be executed on the last line only.
s/.$// replaces the last character on the (in this case last) line with an empty string; i.e., effectively removes the last char. (before the newline) on the line.
. matches any character on the line, and following it with $ anchors the match to the end of the line; note how the use of $ in this regular expression is conceptually related, but technically distinct from the previous use of $ as a Sed address.
Example with stdin input (assumes Bash, Ksh, or Zsh):
$ sed '$ s/.$//' <<< $'line one\nline two'
line one
line tw
To update the input file too (do not use if the input file is a symlink):
sed -i '$ s/.$//' somefile
Note:
On macOS, you'd have to use -i '' instead of just -i; for an overview of the pitfalls associated with -i, see the bottom half of this answer.
If you need to process very large input files and/or performance / disk usage are a concern and you're using GNU utilities (Linux), see ImHere's helpful answer.
truncate
truncate -s-1 file
Removes one (-1) character from the end of the same file. Exactly as a >> will append to the same file.
The problem with this approach is that it doesn't retain a trailing newline if it existed.
The solution is:
if [ -n "$(tail -c1 file)" ] # if the file has not a trailing new line.
then
truncate -s-1 file # remove one char as the question request.
else
truncate -s-2 file # remove the last two characters
echo "" >> file # add the trailing new line back
fi
This works because tail takes the last byte (not char).
It takes almost no time even with big files.
Why not sed
The problem with a sed solution like sed '$ s/.$//' file is that it reads the whole file first (taking a long time with large files), then you need a temporary file (of the same size as the original):
sed '$ s/.$//' file > tempfile
rm file; mv tempfile file
And then move the tempfile to replace the file.
Here's another using ex, which I find not as cryptic as the sed solution:
printf '%s\n' '$' 's/.$//' wq | ex somefile
The $ goes to the last line, the s deletes the last character, and wq is the well known (to vi users) write+quit.
After a whole bunch of playing around with different strategies (and avoiding sed -i or perl), the best way i found to do this was with:
sed '$! { P; D; }; s/.$//' somefile
If the goal is to remove the last character in the last line, this awk should do:
awk '{a[NR]=$0} END {for (i=1;i<NR;i++) print a[i];sub(/.$/,"",a[NR]);print a[NR]}' file
sometext
moretext
lastlin
It store all data into an array, then print it out and change last line.
Just a remark: sed will temporarily remove the file.
So if you are tailing the file, you'll get a "No such file or directory" warning until you reissue the tail command.
EDITED ANSWER
I created a script and put your text inside on my Desktop. this test file is saved as "old_file.txt"
sometext
moretext
lastline
Afterwards I wrote a small script to take the old file and eliminate the last character in the last line
#!/bin/bash
no_of_new_line_characters=`wc '/root/Desktop/old_file.txt'|cut -d ' ' -f2`
let "no_of_lines=no_of_new_line_characters+1"
sed -n 1,"$no_of_new_line_characters"p '/root/Desktop/old_file.txt' > '/root/Desktop/my_new_file'
sed -n "$no_of_lines","$no_of_lines"p '/root/Desktop/old_file.txt'|sed 's/.$//g' >> '/root/Desktop/my_new_file'
opening the new_file I created, showed the output as follows:
sometext
moretext
lastlin
I apologize for my previous answer (wasn't reading carefully)
sed 's/.$//' filename | tee newFilename
This should do your job.
A couple perl solutions, for comparison/reference:
(echo 1a; echo 2b) | perl -e '$_=join("",<>); s/.$//; print'
(echo 1a; echo 2b) | perl -e 'while(<>){ if(eof) {s/.$//}; print }'
I find the first read-whole-file-into-memory approach can be generally quite useful (less so for this particular problem). You can now do regex's which span multiple lines, for example to combine every 3 lines of a certain format into 1 summary line.
For this problem, truncate would be faster and the sed version is shorter to type. Note that truncate requires a file to operate on, not a stream. Normally I find sed to lack the power of perl and I much prefer the extended-regex / perl-regex syntax. But this problem has a nice sed solution.
I have a text file (FILE_A.txt) with this content:
Content1
Content2
Content3
And other text file (FILE_B.txt) with this content:
A[#]
B[#]
C[#]
I would like to combine FILE_A.txt and FILE_B.txt in other file (FILE_C.txt) in this way:
A[Content1]
B[Content2]
C[Content3]
How could I make this using bash shell in linux (sed, cut, grep, etc)?
Here we go.
# awk 'NR==FNR{a[NR]=$0;next;} sub(/#/,a[FNR])' FILE_A.txt FILE_B.txt
A[Content1]
B[Content2]
C[Content3]
How does this work?
NR==FNR - causes the following statements to be run if the record number matches the FILE record number - that is, we are currently reading only the firs tfile.
{a[NR]=$0;next;} - Store values from the first file in an array.
sub(/#/,a[FNR]) - Once we're in the second file, substitute # for the matching value stored from the first file. Note that this isn't inside curly brackets, so it's being evaluated as a condition. If the sub() statement succeeds, the current line is printed.
Use paste and sed as follows:
$ paste File_B.txt File_A.txt | sed 's/#]\s*\(.*$\)/\1]/g'
A[Content1]
B[Content2]
C[Content3]
The following reads both files concurrently, one line at a time, and store the lines in $value and $template. We then use bash's variable substring replacement to replace # within $template with the contents of $value.
exec 6<"FILE_B.txt" # open file for reading and assign file descriptor 6
while read -r value; do # loop through FILE_A.txt, storing each line as $value
read -r template <&6 # read a line from FILE_B.txt, store as $template
echo ${template/\#/$value} # replace value into the template in place of `#`
done <"FILE_A.txt"
exec 6<&- # close input file descriptor 6