How do I copy the first few lines of a giant file and add a line of text at the end of it, using some Linux commands?
The head command can get the first n lines. Variations are:
head -7 file
head -n 7 file
head -7l file
which will get the first 7 lines of the file called "file". The command to use depends on your version of head. Linux will work with the first one.
To append lines to the end of the same file, use:
echo 'first line to add' >> file
echo 'second line to add' >> file
echo 'third line to add' >> file
or:
echo 'first line to add
second line to add
third line to add' >> file
to do it in one hit.
So, tying these two ideas together, if you wanted to get the first 10 lines of the input.txt file to output.txt and append a line with five "=" characters, you could use something like:
( head -10 input.txt ; echo '=====' ) > output.txt
In this case, we do both operations in a sub-shell so as to consolidate the output streams into one, which is then used to create or overwrite the output file.
I am assuming what you are trying to achieve is to insert a line after the first few lines of of a textfile.
head -n10 file.txt >> newfile.txt
echo "your line >> newfile.txt
tail -n +10 file.txt >> newfile.txt
If you don't want to rest of the lines from the file, just skip the tail part.
First few lines: man head.
Append lines: use the >> operator (?) in Bash:
echo 'This goes at the end of the file' >> file
sed -n '1,10p' filename > newfile
echo 'This goes at the end of the file' >> newfile
Related
I'm sure this question has been answered somewhere, but while searching for it, I can't find my exact scenario. I have two files that I am concatenating into a single file, but I am also adding user input between the two files. The problem is that a newline is being inserted after the first file, the rest of it works as desired.
touch newFile
cat file1 > newFile
echo -n $userInput >> newFile
cat file2 >> newFile
How do I prevent or remove the newline when file1 is added to newFile? If I cat file1 there seems to be a newline added by cat but everything I see about cat says it doesn't do that. If I vim file1 there's not a blank line at the end of the file that would indicate the newline is a part of the file, so either cat is actually adding a newline, or the redirect > is doing it, or echo adds a newline at the beginning of its output, none of which would be desirable in this situation. One solution I saw was to use
cat file1 | tr -d '\n'
but that discards all the newlines in the file, also not desirable. So, to repeat my question:
How do I cat file1 into the new file and add user input without adding the newline between them?
(cat is not a requirement, but I am not familiar with printf, so if that's the solution then please elaborate on its use).
With these inputs:
userInput="Test Test Test"
echo "Line 1
Line 2
Line 3" >file1
echo "Line 4
Line 5
Line 6" >file2
I would do:
printf "%s%s%s" "$(cat file1)" "$userInput" "$(cat file2)" >newfile
The creation of >newfile is equivalent to touch and adding content in your first step. A bit easier to see intent with this.
I get:
$ cat newfile
Line 1
Line 2
Line 3Test Test TestLine 4
Line 5
Line 6
Like all other Unix tools, Vim considers \n a line terminator and not a line separator.
This means that a linefeed after the last piece of text will be considered part of the last line, and will not show an additional blank line.
If there is no trailing linefeed, Vim will instead show [noeol] in the status bar when the file is loaded:
foo
~
~
~
~
~
"file" [noeol] 1L, 3C 1,1 All
^---- Here
So no, the linefeed is definitely part of your file and not being added by bash in any way.
If you want to strip all trailing linefeeds, you can do this as a side effect of command expansion:
printf '%s' "$(<file1)" >> newfile
touch newFile
echo -n "$(cat file1)" > newFile
echo -n $userInput >> newFile
cat file2 >> newFile
That did the trick.
I have a file called flw.py and would like to write a bash script that will replace some text in the file (take out the last two lines and add two new lines). I apologize if this seems like a stupid question. A thorough explanation would be appreciated since I am still learning to script. Thanks!
head -n -2 flw.py > tmp # (1)
echo "your first new line here..." >> tmp # (2)
echo "your second new line here...." >> tmp #
mv tmp flw.py # (3)
Explanation:
head normally prints out the first ten lines of a file. The -n argument can change the number of lines printed out. So if you wanted to print out the first 15 lines you would use head -n 15. If you give negative numbers to head it means the opposite: print out all lines but the last N lines. Which happens to be what you want: head -n -2
Then we redirect the output of our head command to a temporary file named tmp. > does the redirecting magic here. tmp now contains everything of flw.py but the last two lines.
Next we add the two new lines by using the echo command. We append the output of the echo "your first new line here..." to our tmp file. >> appends to an existing file, whereas > will overwrite an existing file.
We do the same thing for the second line we want to append.
Last, we move the tmp file to flw.py and the job is done.
You can use single sed command to get you expect result
sed -n 'N;$!P;$!D;a\line\n\line2' fly.py
Example:
cat fly.py
1
2
3
4
5
sed -n 'N;$!P;$!D;a\line\n\line2' fly.py
Output :
1
2
3
line1
line2
Note :
Using -i option to update your file
I have a large text file of ~45 GB, and need to append a line both at the beginning and at the end of the file. What is the quickest way to accomplish this? I don't have access to any big data framework such as Hadoop, etc.
In addition, if I have to also do a string replace in this large file, is there a similar efficient way?
I tried with echo command and it really is taking ages.
You can use this sed to add a line in the beginning:
sed -i '1s/^/your heading line\n/' file
And to the end:
echo "my new line" >> file
Test
$ cat a
hello
bye
$ sed -i '1s/^/new line sample\n/' a
$ cat a
new line sample
hello
bye
$ echo "my new line" >> a
$ cat a
new line sample
hello
bye
my new line
Using sed, you could say:
sed -e '1ifoo' -e '$abar'
and this would insert foo at the top of the file and bar at the end of the file.
$ seq 5 | sed -e '1ifoo' -e '$abar'
foo
1
2
3
4
5
bar
I have a text file, say, text.
What I want to do is to put the text file into pipe (as in cat text |), but with added line at top and bottom - but without creating a new file (because the text file is kind of big) or modifying it.
Is that possible? I was thinking echo $(echo "line"; cat text; echo "line") but I don't like that.
From man bash:
(list) list is executed in a subshell environment (see COMMAND EXECUTION ENVIRONMENT below). Variable assignments and builtin commands that affect the shell's environment do not remain in effect after the command completes.
The return status is the exit status of list.
So this should work:
(echo first line; cat file; echo last line) | some_command
As suggested by chepner, when you don't need a subshell you can use {} instead of (). In this case, the ; is mandatory including for the last command in the group.
In the comments Pyrolistical remembers this also works if you have an incoming pipe instead of a file:
some_program | (echo first line; cat -; echo last line) | some-other-command
As in many unix command line applications, if the filename argument for cat is - it means "read from standard input".
Copied from this question Add new lines to top and bottom of every text file:
cat text |sed -e '1 i beginning_line'| sed -e '$s#$#\nending_line#' |
it seems to work.
add lines to the pipe output,
add a line to the top: (source - #Karel Bilek's comment)
cat text.txt | sed -e '1 i my first line'
add a line to the bottom: ( source - https://askubuntu.com/a/968585 )
cat text.txt | sed -e '$a my last line'
e.g: try this with single quotes
cat text.txt | sed -e '1 i my first line' | sed -e '$a my last line'
output:
my first line
this is
my
text
file
my last line
I can grap a specific line from a file using sed. Is there an easy way to take this line or paragraph and insert onto a specific line in another file?
sed -n 1,10p >> foo appends the result to foo, which places it at the bottom. Is there a standard unix tool to insert onto a specific line?
Perhaps you are looking for sed's r command?
sed '123r file.txt' main.txt
inserts the contents of file.txt at line 123 of main.txt, printing everything to standard output.
(If your sed has the -i option, you can make it modify main.txt directly; otherwise, it will not modify its input files.)
If you want to replace the nth line in file foo you can do it with
cp foo foo.tmp
head -n $((n-1)) foo.tmp > foo
echo "newline" >> foo
tail -n +$((n+1)) foo.tmp >> foo
So you take the first n-1 lines with head -n NR, append your new line and then append the rest starting from line n+1 with tail -n +NR.
This might work for you (GNU sed):
sed '123s|.*|sed '\''1,10!d'\'' insert.txt|e' main.txt