I have following sed command working in linux box. but the same is not working in solaris box. Please correct me what the problem is ?
a=`sed ':a;N;$!ba;s/\n/ /g' file.txt`
Throws,
-bash-3.00$ a=`sed ':a;N;$!ba;s/\n/ /g' file.txt`
Label too long: :a;N;$!ba;s/\n/ /g
With sed on solaris you'll have to break it up like this:
sed -e ':a' -e 'N;$!ba' -e 's/\n/ /g' file.txt
According to man page:
b label Branch to the : command bearing the label.
If label is empty, branch to the end of
the script. Labels are recognized unique
up to eight characters.
Since they recognize up to eight characters, if your label is shorter than you'll need to split your sed in multiple expressions.
In the original sed, I think a label had to be on a line by itself. From my very ancient sed & awk Nutshell book, it states:
A lable is any sequence of up to seven character. A label is put on a line by itself and beginning with a colon:
:label
That means, you need to separate it from the rest of the script with multiple -e arguments or see if you have nawk or gawk on your Solaris box. Alternatively, since it appears you just want to replace all newlines with spaces, there are better tools for the job, such as tr, the translate utility, which should at least be as ubiquitous as sed:
a=`tr '\n' ' ' <file.txt`
Related
Specify the command that removes multiple spaces from a text file, leaving a single space in their place. Extra requirements : Original file to be modified.
Managed to pull out those 3 commands:
awk '{$2=$2};1' filename.txt
tr -s '[:space:]' < filename.txt > filename.new && mv filename.new filename.txt
sed -i 's/\s\+/ /g' filename.txt
Not sure if using a 'temporary file' is the best way to do the trick. Is there any more efficient way to do the problem ? Doesn't matter if it is tr / sed / awk or anything else, you can post all of them.
Example input:
I'm just giving spaces
Output :
I'm just giving spaces
Edit: Still looking for more answers
I'd use ed over the non-standard sed -i (And non-portable RE in your example) if you want to alter the original file:
printf "%s\n" '1,$s/[[:space:]]\{2,\}/ /g' w | ed -s filename.txt
or with perl:
perl -pi -e 's/\s{2,}/ /g' filename.txt
The {2,} regular expression construct (\{2,\} for POSIX Basic Regular Expressions like sed and ed use) matches 2 or more of the previous token.
Both of these match any whitespace characters, not just space, because that's how your examples work. If the goal is to only compress multiple spaces, not spaces + tabs, switch out the [[:space:]] and \s for just a single space.
(Anything that modifies a file "in place", be it ed, sed -i, perl -i, or a regular editor, has a good chance that it's going to be using a temporary file under the hood, by the way. They just handle it for you so you don't have to do it manually like with your tr example.)
I am trying to replace string into number from the file
So, I have variable &FLOW which need to change to 001, ex :
cat file.txt
This is file ' PISP &FLOW'
PISD.DATA.CYCLE&FLOW..ONE
desired output
This is file ' PISP 001'
PISD.DATA.CYCLE001.ONE
I tried below commands in a script :
for item in file.txt
do
sed 's/\&FLOW/\./001/g' $item
sed 's/\&FLOW/001/g' $item
done
It is giving error. The second sed command is working, but I need to run first the beginning sed command otherwise after running first the second sed command, it would ignore the beginning sed command.
Any help would be appreciated!
Use a single sed command and use -i to actually modify the file contents and you need to pass file.txt as the input for the sed command:
sed -i 's/&FLOW\.\{0,1\}/001/g' file.txt
See the online demo. If you are using it in Mac OS, you need sed -i '' 's/&FLOW\.\{0,1\}/001/g' file.txt. Also see sed edit file in place.
Pattern details
It is a POSIX BRE compliant pattern matching
&FLOW - a literal &FLOW substring
\.\{0,1\} - 0 or 1 occurrence of a . char.
try this:
for item in file.txt
do
sed 's/\&FLOW\./001/g' $item
sed 's/\&FLOW/001/g' $item
done
You had a redundant / in after FLOW
This might also work:
sed -i 's/\&FLOW[\.]?/001/g' file.txt
I need to edit a text file by adding a blank line above every line starting with a period.
Before
Corn
.Apple
Words.
.Orange
Bean
After
Corn
.Apple
Words.
.Orange
Bean
Here is what I have so far.
This adds a spaces after every period. There are more in the actual file.
cat File.txt | sed -r 's/([.]+)/\n\1/g'
This displays the lines that start with a period
while read -r line; do
if [[ "$line" == "."* ]]; then
echo "$line"
fi
done < File.txt
How do I merge them together?
This produces the output that you want:
$ sed 's/^[.]/\n./' file
Corn
.Apple
Words.
.Orange
Bean
If you want to change the file in-place, use sed's -i option:
sed -i 's/^[.]/\n./' file
For Mac OSX or other BSD system, use:
sed -i '' 's/^[.]/\n./' file
We use ^ which matches only at the beginning of a line. Since we are matching a period at the beginning of the line, it is not necessary to capture a group with parentheses: we know the match is a period. All that we need to do is add a newline before that period.
with sed
sed 's/^\./\n\./'
with awk
awk '/^\./{print ""} 1'
or
awk 'sub(/^\./,"\n.") 1'
Using RegExp it could be:
cat File.txt | sed -r 's/^(\..+)/\n\1/g'
I think an awk script is going to work best.
/^\./ {print "";}
{print $0;}
Put that into a file, in this case, called "awkfile" and run it like `awk -f awkfile File.txt'
$ sed -e '/^\./i\\' pru.txt
Corn
.Apple
Words.
.Orange
Bean
this command line instructs sed(1) to search for lines beginning with a dot, and then insert a blank line before it. Look in sed(1) manual page for how to use the insert, replace and append commands.
Say for instance I'm searching a line that is like this:
Color asdf
and I use grep to find that line, like grep asdf file.txt
How would I then display Color? Learning linux is hard.
With the command line tool sed you can replace stings by using regular expressions:
echo "Color asdf" | sed 's/\([^ ]*\).*/\1/'
This part: \([^ ]*\).* is a regular expresion. The first part of the regex: [^ ]*, matches any character except a space as many times as possible and what's between the \( and \) is being captured in the variable \1. Then you also match the remaining part of the string with .* and replace all of that with only the first word which was captured by \([^ ]*\) by using \1 in the replace part of the sed command.
Here some more info about sed:
http://linux.about.com/od/commands/a/Example-Uses-Of-Sed-Cmdsedxa.htm
You could use sed:
sed -n 's/[[:space:]][[:space:]]*asdf$//p' file.txt
Details:
The -n option tells sed not to print the pattern space automatically. Basically, it doesn't output anything unless you tell it to.
The s command of sed replaces text. Here, if a line ends with asdf, preceded by at least one whitespace character, we replace all of that with nothing and then print the line (notice the p flag at the end of the s command). The printing is only done if something was actually replaced. More information about the s command can be found e. g. in the GNU sed manual.
Edit for clarity: When using single quotes, parameter expansion does not work and thus, variables won't be replaced. To use variables, use double quotes:
search=asdf
sed -n "s/[[:space:]][[:space:]]*${search}\$//p" file.txt
If you'd really like to use grep here, you could pipe the output from grep into cut:
grep -h asdf *.txt | cut -s -d -f 1
Note that there have to be two spaces after the -d option to cut - the first tells cut to use a blank as the field delimiter (I'm assuming your fields are blank-delimited rather than tab-delimited), while the second separates the -d option from the following option (-f).
But, yeah, sed or awk are probably your friends here... :-)
you can color pattern in the line using grep
grep --colour -o 'asdf' file.txt
edit: the -o option will print only the patterns
I am trying to delete empty lines using sed:
sed '/^$/d'
but I have no luck with it.
For example, I have these lines:
xxxxxx
yyyyyy
zzzzzz
and I want it to be like:
xxxxxx
yyyyyy
zzzzzz
What should be the code for this?
You may have spaces or tabs in your "empty" line. Use POSIX classes with sed to remove all lines containing only whitespace:
sed '/^[[:space:]]*$/d'
A shorter version that uses ERE, for example with gnu sed:
sed -r '/^\s*$/d'
(Note that sed does NOT support PCRE.)
I am missing the awk solution:
awk 'NF' file
Which would return:
xxxxxx
yyyyyy
zzzzzz
How does this work? Since NF stands for "number of fields", those lines being empty have 0 fields, so that awk evaluates 0 to False and no line is printed; however, if there is at least one field, the evaluation is True and makes awk perform its default action: print the current line.
sed
'/^[[:space:]]*$/d'
'/^\s*$/d'
'/^$/d'
-n '/^\s*$/!p'
grep
.
-v '^$'
-v '^\s*$'
-v '^[[:space:]]*$'
awk
/./
'NF'
'length'
'/^[ \t]*$/ {next;} {print}'
'!/^[ \t]*$/'
sed '/^$/d' should be fine, are you expecting to modify the file in place? If so you should use the -i flag.
Maybe those lines are not empty, so if that's the case, look at this question Remove empty lines from txtfiles, remove spaces from start and end of line I believe that's what you're trying to achieve.
I believe this is the easiest and fastest one:
cat file.txt | grep .
If you need to ignore all white-space lines as well then try this:
cat file.txt | grep '\S'
Example:
s="\
\
a\
b\
\
Below is TAB:\
\
Below is space:\
\
c\
\
"; echo "$s" | grep . | wc -l; echo "$s" | grep '\S' | wc -l
outputs
7
5
Another option without sed, awk, perl, etc
strings $file > $output
strings - print the strings of printable characters in files.
With help from the accepted answer here and the accepted answer above, I have used:
$ sed 's/^ *//; s/ *$//; /^$/d; /^\s*$/d' file.txt > output.txt
`s/^ *//` => left trim
`s/ *$//` => right trim
`/^$/d` => remove empty line
`/^\s*$/d` => delete lines which may contain white space
This covers all the bases and works perfectly for my needs. Kudos to the original posters #Kent and #kev
The command you are trying is correct, just use -E flag with it.
sed -E '/^$/d'
-E flag makes sed catch extended regular expressions. More info here
You can say:
sed -n '/ / p' filename #there is a space between '//'
You are most likely seeing the unexpected behavior because your text file was created on Windows, so the end of line sequence is \r\n. You can use dos2unix to convert it to a UNIX style text file before running sed or use
sed -r "/^\r?$/d"
to remove blank lines whether or not the carriage return is there.
This works in awk as well.
awk '!/^$/' file
xxxxxx
yyyyyy
zzzzzz
You can do something like that using "grep", too:
egrep -v "^$" file.txt
My bash-specific answer is to recommend using perl substitution operator with the global pattern g flag for this, as follows:
$ perl -pe s'/^\n|^[\ ]*\n//g' $file
xxxxxx
yyyyyy
zzzzzz
This answer illustrates accounting for whether or not the empty lines have spaces in them ([\ ]*), as well as using | to separate multiple search terms/fields. Tested on macOS High Sierra and CentOS 6/7.
FYI, the OP's original code sed '/^$/d' $file works just fine in bash Terminal on macOS High Sierra and CentOS 6/7 Linux at a high-performance supercomputing cluster.
If you want to use modern Rust tools, you can consider:
ripgrep:
cat datafile | rg '.' line with spaces is considered non empty
cat datafile | rg '\S' line with spaces is considered empty
rg '\S' datafile line with spaces is considered empty (-N can be added to remove line numbers for on screen display)
sd
cat datafile | sd '^\n' '' line with spaces is considered non empty
cat datafile | sd '^\s*\n' '' line with spaces is considered empty
sd '^\s*\n' '' datafile inplace edit
Using vim editor to remove empty lines
:%s/^$\n//g
For me with FreeBSD 10.1 with sed worked only this solution:
sed -e '/^[ ]*$/d' "testfile"
inside [] there are space and tab symbols.
test file contains:
fffffff next 1 tabline ffffffffffff
ffffffff next 1 Space line ffffffffffff
ffffffff empty 1 lines ffffffffffff
============ EOF =============
NF is the command of awk you can use to delete empty lines in a file
awk NF filename
and by using sed
sed -r "/^\r?$/d"