I'm using a shell & TCL script to login to a switch and get the output of certain commands and in some places I can see ^D coming up. I tried to use the dos2unix utility but still it didn't go away.
Eth1/37 NOM: xcvrAbsen routed auto auto --
^DEth1/38 NOM: xcvrAbsen routed auto auto --
Eth1/39 NOM: xcvrAbsen routed auto auto --
Eth101/1/45 eth 1000 NOM:NO_PATCHING CABLE
^DEth101/1/46 eth 1000 NOM:NO_PATCHING CABLE
Eth101/1/47 eth 1000 NOM:NO_PATCHING CABLE
How can this be eliminated, are there any standard tools like dos2unix which can get rid of such data?
What I'm trying to do is to compare two files which are from the same switch and the same command and the same output, but due to these ^D, Vimdiff shows it as different lines.
How to get this eliminated?
Command I'm using is something like this:
$cdir/ciscocmd -Y -u $operator -p $password -s $password -t $switch -r rfc_sa_commands | sed 's/^^D//' > $switch.$NOW
dos2unix removes carriage returns, no other control characters.
The tool to remove all occurrences of an arbitrary character is called tr.
tr -d '\004' <inputfile >outputfile
This assumes you have literal ctrl-D characters, not sequences of caret ^ and D. The tr utility cannot remove a specific sequence; it just processes individual characters. To remove a sequence, you'd need
sed 's/\^D//g' inputfile >outputfile
where the backslash is required because the caret alone has a special meaning in regular expressions (it matches beginning of line). Doubling it does not escape it; ^^ probably still just matches beginning of line, though it's not really well-defined, and could introduce apparently random behavior.
Even if the special character is visible as '^D', it may be NOT catchable like this.
Interesting readings, are:
https://en.wikipedia.org/wiki/ASCII#Character_groups
https://en.wikipedia.org/wiki/End-of-Transmission_character
I think a way to do it would be:
<your command>|sed -e 's/\x04//g'
Does it solve your issue?
Related
want to search and replace in vim, the /find finds the pattern but :s%//g will not?
have a script that monitors software raid (if interested check it out https://dwaves.org/2019/09/06/linux-server-monitor-software-raid-mail-notification-on-failure/)
echo "=== smart status of all drives ==="| tee -a /scripts/monitor/raid_status_mail.log
# want to search and replace the /path/to/file.sh with $LOGFILE
# searching for the pattern works like charm
/\/scripts\/monitor\/raid_status_mail.log
# but replacing it won't
:s%/\/scripts\/monitor\/raid_status_mail\.log/\$LOGFILE/g
# what does one do wrong?
should replace /scripts/monitor/raid_status_mail.log with $LOGFILE
The substitution operation needs to be prefixed with %s and not the other way around as s%. So doing
%s/\/scripts\/monitor\/raid_status_mail\.log/\$LOGFILE/g
should work as expected. Or just the Vim's equivalent ex in command line mode as
printf '%s\n' "%s/\/scripts\/monitor\/raid_status_mail\.log/\$LOGFILE/g" w q | ex -s file
You inverted the beginning s%. Use %s instead.
Also, you use / as separation for the different fields, it works but makes the command less readable. You can replace the separation character by anything else. You could use : for example:
%s:/scripts/monitor/raid_status_mail.log:$LOGFILE:g
One last tip: install vim-over
This will highlight your searches in live while replacing something in vim.
I'm currently working on a Unix box and came across this post which I found helpful, in order to learn about cat command in Unix. At the bottom of the page found this line saying: -A = Equivalent to -vET
As I'm new into Unix, I'm unaware of what does this mean actually? For example lets say I've created a file called new using cat and then apply this command to the file:
cat -A new, I tried this command but an error message comes up saying it's and illegal option.
To cut short, wanted to know what does cat -A really mean and how does it effect when I apply it to a file. Any help would be appreciated.
It means show ALL.
Basically its a combination of -vET
E : It will display '$' at the end of every line.
T : It will display tab character as ^I
v : It will use ^ and M-notation
^ and M-notation:
(Display control characters except for LFD(LineFeed or NewLine) and TAB using '^' notation and precede characters that have the high bit set with
'M-') M- notation is a way to display high-bit characters as low bit ones by preceding them with M-
You should read about little-endian and big-endian if you like to know more about M notation.
For example:
!http://i.imgur.com/0DGET5k.png?1
Check your manual page as below and it will list all options avaialable with your command and check is there -A present, if it is not present it is an illegal option.
man cat
It displays non-printing characters
In Mac OS you need to use -e flag and
-e Display non-printing characters (see the -v option), and display a dollar sign (`$') at the end of each line.
So, im making a small script to do an entire task for me. The task is to get the output of the dmidecode -Fn into a text file and then take a part of the dmidecode output, in my example, the Address (0xE0000) as the file name of the txt.
My script goes as follows and does work, i have tested it. The only little issue that i have, is that the file name of the txt appears as "? 0xE0000.txt"
My question is, why am i getting a question mark followed by a space in the name?
#!/bin/bash
directory=$(pwd)
name=$(dmidecode|grep -i Address|sed 's/Address://')
inxi -Fn > $directory/"$name".txt
The quotes in the "$name".txt is to avoid an "ambiguous redirect" error i got when running the script.
Update #Just Somebody
root#server:/home/user/Desktop# dmidecode | sed -n 's/Address://p'
0xE0000
root#server:/home/user/Desktop#
Solution
The use of |sed -n 's/^.*Address:.*0x/0x/p' got rid of the "? " in 0xE0000.txt
A big thanks to everyone!
You've got a nonprinting char in there. Try:
dmidecode |grep -i Address|sed 's/Address://'| od -c
to see exactly what you're getting.
UPDATE: comments indicate there's a tab char in there that needs to be cleaned out.
UPDATE 2: the leading tab is before the word Address. Try:
name=$(dmidecode |grep -i Address|sed 's/^.*Address:.*0x/0x/')
or as #just_somebody points out:
name=$(dmidecode|sed -n 's/^.*Address:.*0x/0x/p')
UPDATE 3
This changes the substitution regex to replace
^ (start of line) followed by .* (any characters (including tab!)) followed by Address: followed by .* (any characters (including space!)) followed by 0x (which are always at the beginning of the address since it's in hex)
with
0x (because you want that as part of the result)
If you want to learn more, read about sed regular expressions and substitutions.
I am trying to understand how
sed 's/\^\[/\o33/g;s/\[1G\[/\[27G\[/' /var/log/boot
worked and what the pieces mean. The man page I read just confused me more and I tried the info sai Id but had no idea how to work it! I'm pretty new to Linux. Debian is my first distro but seemed like a rather logical place to start as it is a root of many others and has been around a while so probably is doing stuff well and fairly standardized. I am running Wheezy 64 bit as fyi if needed.
The sed command is a stream editor, reading its file (or STDIN) for input, applying commands to the input, and presenting the results (if any) to the output (STDOUT).
The general syntax for sed is
sed [OPTIONS] COMMAND FILE
In the shell command you gave:
sed 's/\^\[/\o33/g;s/\[1G\[/\[27G\[/' /var/log/boot
the sed command is s/\^\[/\o33/g;s/\[1G\[/\[27G\[/' and /var/log/boot is the file.
The given sed command is actually two separate commands:
s/\^\[/\o33/g
s/\[1G\[/\[27G\[/
The intent of #1, the s (substitute) command, is to replace all occurrences of '^[' with an octal value of 033 (the ESC character). However, there is a mistake in this sed command. The proper bash syntax for an escaped octal code is \nnn, so the proper way for this sed command to have been written is:
s/\^\[/\033/g
Notice the trailing g after the replacement string? It means to perform a global replacement; without it, only the first occurrence would be changed.
The purpose of #2 is to replace all occurrences of the string \[1G\[ with \[27G\[. However, this command also has a mistake: a trailing g is needed to cause a global replacement. So, this second command needs to be written like this:
s/\[1G\[/\[27G\[/g
Finally, putting all this together, the two sed commands are applied across the contents of the /var/log/boot file, where the output has had all occurrences of ^[ converted into \033, and the strings \[1G\[ have been converted to \[27G\[.
I want to add hugepages to grub.conf file through bash script. I can use this command
sed -i '0,/quiet\+/s//\0\ transparent_hugepage=never hugepages=23/' /boot/grub/grub.conf
However, this will add to the boot option 0 everytime. I want to check what is the default boot option (default=n) and add it to nth boot option.
Eg: grub.conf
default=1
title CentOS (3.12.8)
...
kernel ........ quiet
title CentOS (2.6.3)
....
kernel ........ quiet
Here I would want to add huge pages for CentOS (2.6.3) after quiet since its the default but
sed -i '0,/quiet\+/s//\0\ transparent_hugepage=never hugepages=23/' /boot/grub/grub.conf
will add it to CentOS (3.12.8)
Any suggestions?
Thank you
I suspect this could be possible with just one sed command if you like really arcane programming challenges. Otherwise you could do it a bit like the following, using two sed commands:
INPUT=/boot/grub/grub.conf
POSITION=`sed -n -e '/^default=\(.*\)/\1/p' "$INPUT"`
POSITION=$(( $POSITION + 1)) # Because 'default' is 0-based
REPLACE="transparent_hugepage=never hugepages=23"
sed \
'/^kernel /{x;s/$/Z/;/^Z\{'$POSITION'\}$/{x;s/ quiet/ quiet '$REPLACE'/p;d};x}'\
-i "$INPUT"
Explanation
The first sed command extracts the position of the default, this is zero based so we need to add 1.
The second sed command is rather more complicated:
For each time it finds a line starting with "kernel ", appends a 'Z' character to the hold space - i.e. {x;s/$/Z/;
Thus the number of Z characters in the hold space is used to detect when the entry that needs to be edited is reached - i.e. ^/Z\{'$POSITION'\}$/ - if $POSITION is 2, this becomes ^/Z\{2\}$/ (Note the substitution of $POSITION is performed by the shell)
When this is detected, switch back, perform the actual regex replace you need to do, then delete the hold space - i.e. {x;s/ quiet/ quiet «extra text» /p;d}
The above works because the sed command s always operates in pattern space
Lastly, switch back to the pattern space and keep going - i.e. ;x}
Take care with spaces and quotes, etc.