I'm using a diff command and it's printing out to a file. The file keeps getting an extra line in the end that I don't need to appear. How can I prevent it from being there?
The command is as follows:
diff -b <(grep -B 2 -A 1 'bedrock.local' /Applications/MAMP/conf/apache/httpd.conf) /Applications/MAMP/conf/apache/httpd.conf > test.txt
The file being used is here (thought I don't think it matters): http://yaharga.com/httpd.txt
Perhaps at least I'd like to know how to check the last line of the file and delete it only if it's blank.
To delete empty last line you can use sed, it will delete it only if it's blank:
sed '${/^\s*$/d;}' file
Ok i made research with your file on my MacOS.
I created file new.conf by touch new.conf and then copied data from your file to it.
btw i checked file and didn't have extra empty line at the bottom of it.
I wrote script script.sh with following:
diff -b <(grep -B 2 -A 1 'bedrock.local' new.conf) new.conf > test.txt
sed -i.bak '1d;s/^>//' test.txt
It diffed what was needed and deleted first useless row and all > saving it to a new file test.txt
I checked again and no extra empty line was presented.
Additionaly i would suggest you to try and delete the extra line you have like this: sed -i.bak '$d' test.txt
And check a number of lines before and after sed = test.txt
Probably your text editor somehow added this extra line to your file. Try something else - nano for example or vi
Related
How do I copy lines from one file to another in Linux without opening source and destination files and I need to exclude the comments when copying the lines.
I do not want to copy the comments in the first file and the files are in different locations
Assuming lines are commented with # at the very beginning of each line, the following should work:
grep -v "^#" path/to/input/file >path/to/output/file
(Note: This will either create a new output file or irreversibly overwrite the output file if it already exists)
Assuming comment lines in your file contain # at the beginning of each line, the following sed command will delete these lines:
$ sed '/^#/d' path/to/input-file > path/to/output-file
If your file can also contain lines with whitespace before the #, the following sed command will delete lines beginning with zero or more spaces or tabs (in any order), followed by a hash (#) character:
$ sed '/^[ \t]*#/d' path/to/input-file > path/to/output-file
If your file also contains lines containing code followed by a comment, the following sed command should work:
$ sed -e '/^[ \t]*#/d' -e 's/#.*$//' path/to/input-file > path/to/output-file
I have files which are very large (> 5G), and I want to remove some lines by the line numbers without moving (copy and paste) files.
I know this command works for a small size file. (my sed command do not recognize -i option)
sed "${line}d" file.txt > file.tmp && mv file.tmp file.txt
This command takes relatively long time because of the size. I just need to remove the first line and the last line, but also want to know how to remove line number n, for example.
Because of the way files are stored on standard filesystems (NTFS, EXTFS, ...), you cannot remove parts of a file in-place.
The only thing you can do in-place is
append to the end of a file (append mode)
modify data in a file (read-write mode)
Other operations must use a temporary file, or temporary memory to read the file fully and write it back modified.
EDIT: you can also "shrink" a file as read here using a C program (Linux or Windows would work) so that means that you could remove the last line (but still not the first line or any line in between)
you can use the ed command which is quite similar to sed
ed -s file.text
you can use the d command, $d will delete the last line while 1d will delete the first one, and wq will write and exit.
The following command will do everything (delete first and last line, write, and exit)
echo -e '1d\n$d\nwq' | ed -s test.txt
using sed you can use the same commands sed '1d;$d' test.txt
If you are using a recent Linux, you can remove chunks of the file in any position: https://lwn.net/Articles/415889/
There's a command to remove any part of the file: fallocate
See: https://manpages.ubuntu.com/manpages/xenial/man1/fallocate.1.html
For example: fallocate -p -o 10G -l 1G qqq
I have a file that contains a list of file names to be opened later.
After I load lines (file names) to variables, for a reason unknown to me I cannot open it as a file later.
Here is a simplified example of what i'm trying to do:
Main file's contents:
first_file.txt
second_file.txt
Bash commands:
read line < $main_file # file with the list, received as an argument
echo $line # to check that correct filename has been read
cat $line # attempt to dump "first_file.txt" contents <- FAILS
cat first_file.txt # read "first_file.txt" contents manually
Execution esult:
first_file.txt
: No such file or directory
*** this is 1st file's contents ***
*** ....
So, cat first_file.txt works, $line contains "first_file.txt", but cat $line fails...
I obviously misunderstand something here, suggestions are welcomed!
Edit:
As requested, here is cat -v $main_file's output:
first_file.txt^M
second_file.txt^M
third_file.txt^M
^M
^M
The ^M characters are carriage returns (a.k.a. \r) and are often part of a Windows line ending. They don't show up when you echo them, but they are messing up your ability to open a file with the text having it at the end.
The best solution is to remove them from your "main file." You could use the dos2unix tool if you have it, or you could use GNU sed like sed -i -e 's/\s+$//g' $main_file to edit it in place and remove the extra white space (which includes ^M) from the end of each line.
I have two files data.txt and results.txt, assuming there are 5 lines in data.txt, I want to copy all these lines and paste them in file results.txt starting from the line number 4.
Here is a sample below:
Data.txt file:
stack
ping
dns
ip
remote
Results.txt file:
# here are some text
# please do not edit these lines
# blah blah..
this is the 4th line that data should go on.
I've tried sed with various combinations but I couldn't make it work, I'm not sure if it fit for that purpose as well.
sed -n '4p' /path/to/file/data.txt > /path/to/file/results.txt
The above code copies line 4 only. That isn't what I'm trying to achieve. As I said above, I need to copy all lines from data.txt and paste them in results.txt but it has to start from line 4 without modifying or overriding the first 3 lines.
Any help is greatly appreciated.
EDIT:
I want to override the copied data starting from line number 4 in
the file results.txt. So, I want to leave the first 3 lines without
modifications and override the rest of the file with the data copied
from data.txt file.
Here's a way that works well from cron. Less chance of losing data or corrupting the file:
# preserve first lines of results
head -3 results.txt > results.TMP
# append new data
cat data.txt >> results.TMP
# rename output file atomically in case of system crash
mv results.TMP results.txt
You can use process substitution to give cat a fifo which it will be able to read from :
cat <(head -3 result.txt) data.txt > result.txt
head -n 3 /path/to/file/results.txt > /path/to/file/results.txt
cat /path/to/file/data.txt >> /path/to/file/results.txt
if you can use awk:
awk 'NR!=FNR || NR<4' Result.txt Data.txt
I want to add a row of headers to an existing CSV file, editing in place. How can I do this?
echo 'one, two, three' > testfile.csv
and I want to end up with
column1, column2, column3
one, two, three
Changing the initial CSV output is out of my hands.
Any standard command will do. The important thing is the file is edited in place, and the line is inserted at the beginning of the file.
To answer your original question, here's how you do it with sed:
sed -i '1icolumn1, column2, column3' testfile.csv
The "1i" command tells sed to go to line 1 and insert the text there.
The -i option causes the file to be edited "in place" and can also take an optional argument to create a backup file, for example
sed -i~ '1icolumn1, column2, column3' testfile.csv
would keep the original file in "testfile.csv~".
This adds custom text at the beginning of your file:
echo 'your_custom_escaped_content' > temp_file.csv
cat testfile.csv >> temp_file.csv
mv temp_file.csv testfile.csv
This doesn't use sed, but using >> will append to a file. For example:
echo 'one, two, three' >> testfile.csv
Edit: To prepend to a file, try something like this:
echo "text"|cat - yourfile > /tmp/out && mv /tmp/out yourfile
I found this through a quick Google search.
As far as I understand, you want to prepend column1, column2, column3 to your existing one, two, three.
I would use ed in place of sed, since sed write on the standard output and not in the file.
The command:
printf '0a\ncolumn1, column2, column3\n.\nw\n' | ed testfile.csv
should do the work.
perl -i is worth taking a look as well.
sed is line based, so I'm not sure why you want to do this with sed. The paradigm is more processing one line at a time( you could also programatically find the # of fields in the CSV and generate your header line with awk) Why not just
echo "c1, c2, ... " >> file
cat testfile.csv >> file
?
Use perl -i, with a command that replaces the beginning of line 1 with what you want to insert (the .bk will have the effect that your original file is backed up):
perl -i.bk -pe 's/^/column1, column2, column3\n/ if($.==1)' testfile.csv
Add a given line at the beginning of a file in two commands:
cat <(echo "blablabla") input_file.txt > tmp_file.txt
mv tmp_file.txt input_file.txt
how to add line inside a file
sed -i -e "48r../../../../folder1/lines_to_add.txt" a.txt
a.txt - the file you want to change 48r is line number of a.txt some
lines are inside lines_to_add.txt file
../../../../scripts3_2d/lines_to_add.txt
-i update a.txt, try without -i before run the code, be careful with new lines,
"keep a newline at the end of lines_to_add.txt"