I'm trying to use sed but can't get it to work. My system is Ubuntu 16.04.2 with GNU sed 4.2.2
I have a folder with numerous text files that I want to edit. From a terminal in that folder I have tried the following (separately - one command at a time).
To remove blank lines from all the files: sed -i '/^$/d' *txt
In case the lines include white spaces: sed -i '/^\s*$/d' *txt
To remove line 4: sed -i '4d' *.txt
In each case, there is no error message in the terminal but the changes do not happen. I've tried the same commands but for an individual file rather than all, so with the filename instead of *.txt, but still no changes achieved.
The only sign of sed being active at all is if I don't do the -i but ask for a new file: sed '4d' fred.txt > fred2.text
The new file fred2 was created - but with line 4 still there!
What am I missing? How can I get sed to actually carry out these commands?
Thanks for any help.
After looking at the files in more detail, and doing some experimenting, I discovered what the problem was: The text files I was trying to edit with sed had CR line-end characters. When I changed this to LF the sed commands worked fine.
Thank you to those who made suggestions.
Related
Can we do it using sed and if so how?
How to replace a entire line from a file which has a content /u02/app/oracle-1/product/12.2.0/db_1:N to /u01/app/oracle/product/12.2.0/db_1:Y using Sed
It is very easy with sed command. Use below sed command which will do it for you.
sed 's|'/u02/app/oracle-1/product/12.2.0/db_1:N'|'/u01/app/oracle/product/12.2.0/db_1:Y'|g' your_original_file > newfile
mv newfile your_original_file
In above example, first sed command will replace /u02/app/oracle-1/product/12.2.0/db_1:N with /u01/app/oracle/product/12.2.0/db_1:Y and modified text will be redirect to newfile. In the new file you can review the changes whether correct or not.
With mv command you can rename newfile to your_original_file. So above example is safe. But you can do so with a single command like below, but you should be careful here, because if anything wrong you will be in trouble :-)
sed -i.bak 's|'/u02/app/oracle-1/product/12.2.0/db_1:N'|'/u01/app/oracle/product/12.2.0/db_1:Y'|g' yourfile
After lot of trials got it done.
sudo sed -i.bak '/^G:/u01/app/oracle/product/12.2.0/db_1:/ s/N/Y/'
I have an enormous file, about 10 MB, and it has about 175,000 lines. I tried truncated it like this:
sed '500,175000d' <file-name.data>
I reopen the file, and all of the lines are still there! I tested this with other files and it works. For some reason the .data extension doesn't work? How do I delete these lines?
You need to either redirect the output to a new file like
sed '500,175000d' file-name.data >newFile
or use the edit in place option which rewrites the input file
sed -i '500,175000d' file-name.data
as pointed out by Wintermute
Edit:
A faster sed would be just
sed -i '500q' file-name.data # prints 1-500 and quits after line 500
I am trying to remove blank lines from large text files. For some reason it seems that neither
sed "/^$/d" file.txt > trimmed.txt
nor
grep -v "^$" file.txt > trimmed.txt
nor
awk /./ file.txt > trimmed.txt
do anything. Any thoughts?
UPDATE
Thanks to the great comments by #fedorqui & #Sebastian Stigler the problem was quickly identified as DOS/Windows carriage returns (^M$) at the end of each line.
While I appreciate Sebatian's suggestion to reformat the files using dos2unix I would rather have a solution using the tools generally available in most linux distributions.
The solution that worked for me was an answer given by #Jeremy Stein to this question [Can't remove empty lines with sed regex:
sed -n '/[!-~]/p' file.txt > trimmed.txt
I just tried the commandos with a toy example and they work fine as long the file.txt was a file with unix newlines. If the file contains windows newlines then none of the commands were able to remove the blank lines.
You can use the dos2unix linux tool to convert the newlines in file.txt to unix newlines. If you need the output on a windows system then you can use unix2dos to convert trimmed.txt into a file with windows newlines.
I am trying to replace a specific link which exists on many html pages with its https version. I have tried:
grep -rl "http://server.iad.liveperson.net/hc/88956865/" ./ | xargs sed -i "s/http:\/\/server.iad.liveperson.net\/hc\/88956865\//https:\/\/server.iad.liveperson.net\/hc\/88956865\//g"
When I do this, even as sudo, I am getting
sed: couldn't open temporary file ./customers/sedTR3AMu: Permission denied
customers is just the first directory in ./. So, it is hanging on the first file I reckon, but not sure what is wrong beyond that.
Any help is appreciated!
First thing you should try is to run the sed command as stand alone, for a file that you previously know that contains that string. I have the feeling that the sed command might be complaining about the / characters...
You should try changing the sed command to something like:
sed -i 's;http://server.iad.liveperson.net/hc/88956865/;https://server.iad.liveperson.net/hc/88956865/;g'
That is, using ; instead of / as the delimiter, so you don't have to escape the / every time using \.
Had to run the command logged in as root because sed -i creates temporary files in /tmp and needed write access.
Thanks:Used jim's syntax with the semicolons which worked fine. ooga, I did not have to escape the literal periods.
I want to edit a file from the command line, because opening it in vim or other editors takes forever (a large file). I want to add a string ('chr') to the beginning of every line that is not commented out with a #. The command I am using is this:
cat '/home/me/37.vcf' | sed s/^/chr/>'sp.vcf'
But it adds a chr to the beginning of EVERY line and a > to the END of every line. I don't want either of those things to occur.
Can anyone offer any suggestions to improve my results?
To apply the substitution to only the lines that don't start with a #:
sed '/^[^#]/s/^/chr/' file > output
Note: the command cat is for concatenating files, it is useless here.
You can syntax error in your sed command. Use this syntactically correct sed command:
sed -E 's/^([^#]|$)/chr/' /home/me/37.vcf > sp.vcf
OR on Linux:
sed -r 's/^([^#]|$)/chr/' /home/me/37.vcf > sp.vcf
This might work for you (GNU sed):
sed '/^\s*#/!s/^/chr/' file > new_file