Replace a string from a set of string in a file - SED - linux

I am having a trouble replacing string coming from a set of string from a File
So here's how I would like my script to run
I have these files:
macadd.file
00:90:8F:56:63:88
00:90:8F:56:63:77
00:90:8F:56:63:6B
00:90:8F:56:63:86
00:90:8F:56:63:87
00:90:8F:56:64:E5
test.file
host Accountant { hardware ethernet 00:90:8F:56:63:88; fixed-address 192.168.10.29; }
host Accountant { hardware ethernet 00:90:8F:56:63:88; fixed-address 192.168.10.29; }
host Accountant { hardware ethernet 00:90:8F:56:63:88; fixed-address 192.168.10.29; }
host Accountant { hardware ethernet 00:90:8F:56:63:88; fixed-address 192.168.10.29; }
host Accountant { hardware ethernet 00:90:8F:56:63:88; fixed-address 192.168.10.29; }
host Accountant { hardware ethernet 00:90:8F:56:63:88; fixed-address 192.168.10.29; }
So I've tried replacing it using
for i in $(cat macadd); do sed -i "s/00:90:8F:56:63:88/$i/g" test;done
However, it only changes it from the first string in macadd.file
You can see that 00:90:8F:56:63:88 is all the entry there. What I would like to happen is like this, the result:
host Accountant { hardware ethernet 00:90:8F:56:63:88; fixed-address 192.168.10.29; }
host Accountant { hardware ethernet 00:90:8F:56:63:77; fixed-address 192.168.10.29; }
host Accountant { hardware ethernet 00:90:8F:56:63:6B; fixed-address 192.168.10.29; }
host Accountant { hardware ethernet 00:90:8F:56:63:86; fixed-address 192.168.10.29; }
host Accountant { hardware ethernet 00:90:8F:56:63:87; fixed-address 192.168.10.29; }
host Accountant { hardware ethernet 00:90:8F:56:64:E5; fixed-address 192.168.10.29; }

Actually for the example you present in the question something without the substitution would be more simple and clear:
<macadd.file xargs -I{} printf "host Accountant { hardware ethernet %s; fixed-address 192.168.10.29; }\n" {}
That is for every line in the input file call printf to substitute only the MAC address part. More trouble comes when you have more MAC addresses and more resulting IP addresses.

The first pass across causes sed to replace all the hits with the first value. If that's not what you want, use a different sed script. Perhaps like this:
sed 's!.*!s/00:90:8F:56:63:88/&/;n;!' macadd.file
This produces on standard output a new sed script, which you can pass to (... pause ...) sed.
sed 's!.*!s/00:90:8F:56:63:88/&/;n;' macadd.file |
sed -f - -i test.file
This is a slightly demanding pattern the first time you see it (and sed -f - does not work on all platforms) but once you get the idea, having a script generate a script is a very powerful and versatile tool for your toolbox.
... On MacOS, you can't use sed -f - and also -i requires an argument. If you have Bash (or some other shell which supports process substitution), here's a workaround:
sed -f <(sed 's!.*!s/00:90:8F:56:63:88/&/;n;!' macadd.file) -i '' test.file

Straightforwardly with AWK:
awk 'NR==FNR{ a[NR]=$1; next }
a[FNR]{ sub("00:90:8F:56:63:88", a[FNR]) }1' macadd tesfile > tmp_f && mv tmp_f testfile
NR - the total number of input records read so far
FNR - the number of records that have been read so far from the current input file
The final testfile contents:
host Accountant { hardware ethernet 00:90:8F:56:63:88; fixed-address 192.168.10.29; }
host Accountant { hardware ethernet 00:90:8F:56:63:77; fixed-address 192.168.10.29; }
host Accountant { hardware ethernet 00:90:8F:56:63:6B; fixed-address 192.168.10.29; }
host Accountant { hardware ethernet 00:90:8F:56:63:86; fixed-address 192.168.10.29; }
host Accountant { hardware ethernet 00:90:8F:56:63:87; fixed-address 192.168.10.29; }
host Accountant { hardware ethernet 00:90:8F:56:64:E5; fixed-address 192.168.10.29; }

If MAC adresses are not always the same in test.file
You can try :
sed -E '
s/((([[:xdigit:]]){2}:){5}[[:xdigit:]]{2})(.*)((([[:xdigit:]]){2}:){5}[[:xdigit:]]{2})(.*)/\4\1\8/
# ((([[:xdigit:]]){2}:){5}[[:xdigit:]]{2}) catch the MAC addresse
s/^\t//
# remove tab which come from paste
/^((([[:xdigit:]]){2}:){5}[[:xdigit:]]{2})/d
# remove lines if maccad.file have more lines than test.file
'<<< $(paste macadd.file test.file)>test.file

Related

Why U disk couldnot be addressed when unplug a usb speaker from the usb 3.0 port?

My board has a usb3.0 port and a usb2.0 port. The bug occurred in the usb3.0 port. The Linux kernel version is 3.10.65.
The script that makes the bug:
plug a USB speaker in the usb3.0 port.
play audio in the USB speaker continuously.
unplug the USB speaker, Not stop audio play when unplug.
plug a usb3.0 disk in the same that usb3.0 port.
then the USB host controller couldn't address the U Disk.
System report the following log:
#
#
#
# ./test.sh
[ 68.036091] usb 3-1: USB disconnect, device number 2
[ 68.041659] cannot submit urb (err = -19)
[ 78.860028] xhci-hcd xhci-hcd.0.auto: Timeout while waiting for address device command
[ 84.070030] xhci-hcd xhci-hcd.0.auto: Timeout while waiting for address device command
[ 84.280026] usb 4-1: device not accepting address 2, error -62
[ 89.460027] xhci-hcd xhci-hcd.0.auto: Timeout while waiting for address device command
[ 94.670025] xhci-hcd xhci-hcd.0.auto: Timeout while waiting for address device command
[ 94.880020] usb 4-1: device not accepting address 3, error -62
[ 100.060036] xhci-hcd xhci-hcd.0.auto: Timeout while waiting for address device command
[ 105.270044] xhci-hcd xhci-hcd.0.auto: Timeout while waiting for address device command
[ 105.480024] usb 4-1: device not accepting address 4, error -62
[ 110.660024] xhci-hcd xhci-hcd.0.auto: Timeout while waiting for address device command
[ 115.870026] xhci-hcd xhci-hcd.0.auto: Timeout while waiting for address device command
[ 116.080020] usb 4-1: device not accepting address 5, error -62
[ 116.086478] hub 4-0:1.0: unable to enumerate USB device on port 1
After a long time debug, I find if I add the debug info , the bug never happen. But I don't know why.
void usb_set_device_state(struct usb_device *udev,
enum usb_device_state new_state)
{
unsigned long flags;
int wakeup = -1;
spin_lock_irqsave(&device_state_lock, flags);
if (udev->state == USB_STATE_NOTATTACHED)
; /* do nothing */
else if (new_state != USB_STATE_NOTATTACHED) {
/* root hub wakeup capabilities are managed out-of-band
* and may involve silicon errata ... ignore them here.
*/
if (udev->parent) {
if (udev->state == USB_STATE_SUSPENDED
|| new_state == USB_STATE_SUSPENDED)
; /* No change to wakeup settings */
else if (new_state == USB_STATE_CONFIGURED)
wakeup = (udev->quirks &
USB_QUIRK_IGNORE_REMOTE_WAKEUP) ? 0 :
udev->actconfig->desc.bmAttributes &
USB_CONFIG_ATT_WAKEUP;
else
wakeup = 0;
}
if (udev->state == USB_STATE_SUSPENDED &&
new_state != USB_STATE_SUSPENDED)
udev->active_duration -= jiffies;
else if (new_state == USB_STATE_SUSPENDED &&
udev->state != USB_STATE_SUSPENDED)
udev->active_duration += jiffies;
udev->state = new_state;
} else
recursively_mark_NOTATTACHED(udev);
if (new_state == USB_STATE_NOTATTACHED) {
printk("[%s %d] .\n", __func__, __LINE__);
printk("[%s %d] .\n", __func__, __LINE__);
printk("[%s %d] .\n", __func__, __LINE__);
}
spin_unlock_irqrestore(&device_state_lock, flags);
if (wakeup >= 0)
device_set_wakeup_capable(&udev->dev, wakeup);
}
enter code here

How to insert lines after second pattern with linux sed command

I want to insert this block:
host client3 {
hardware ethernet c0:03:03:bc:30:fa;
}
after this block:
subnet 11.10.0.0 netmask 255.255.255.0 {
range 11.10.1.2 11.10.1.254;
group {
filename "10M-5M-OKS2016NOV.cm";
The line: filename "10M-5M-OKS2016NOV.cm";
apears multiple times in the file. But only once inside subnet 11.10.0.0 netmask 255.255.255.0 {
Until now I can print the subnet block until the "filename":
sed -n -e :a -e '/subnet 11\.10\.0\.0 netmask 255\.255\.255\.0/,/}/{/filename "10M-5M-OKS2016NOV\.cm";/!{$!{N;ba};};p;}' dhcpd.conf
but when I try:
sed -n -e :a -e '/subnet 11\.10\.0\.0 netmask 255\.255\.255\.0/,/}/{/filename "10M-5M-OKS2016NOV\.cm";/!{$!{N;ba};};a\ \thost client3 {\n\thardware ethernet c0:03:03:bc:30:fa;\n\t}\n;}' dhcpd.conf
I get:
sed: -e expression #1, char 0: unmatched `{'
subnet 10.10.0.0 netmask 255.255.255.0 {
range 10.10.0.2 10.10.0.254;
group {
filename "10M-5M-OKS2016NOV.cm";
host client1 {
hardware ethernet a0:b4:3d:bc:df:fa;
}
host client2 {
hardware ethernet 90:6e:bb:ba:cd:d4;
}
}
}
subnet 11.10.0.0 netmask 255.255.255.0 {
range 11.10.1.2 11.10.1.254;
group {
filename "10M-5M-OKS2016NOV.cm";
host client1 {
hardware ethernet c0:14:e3:bc:df:fa;
}
host client2 {
hardware ethernet 90:6e:fb:ba:3d:04;
}
}
}
subnet 12.10.0.0 netmask 255.255.255.0 {
range 12.10.2.2 12.10.2.254;
group {
filename "10M-5M-OKS2016NOV.cm";
host client1 {
hardware ethernet c0:a4:3d:bc:df:fa;
}
host client2 {
hardware ethernet 90:6e:bb:ca:3d:04;
}
}
}
Please try something like:
#!/bin/bash
# define newline and tab characters for replacement
NL=$'\n'
NL="\\$NL"
TAB=$'\t'
TAB="\\$TAB"
sed '
:l
N
$!b l
# first of all slurp all lines in the pattern space
# and perform the replacement over the lines
s/subnet 11\.10\.0\.0 netmask 255\.255\.255\.0[^}]*filename "10M-5M-OKS2016NOV\.cm";/&'"$NL$TAB"'host client3 {'"$NL$TAB$TAB"'hardware ethernet c0:03:03:bc:30:fa;'"$NL$TAB"'}/g
' dhcpd.conf
It yields the following output by using the posted lines as dhcpd.conf,
subnet 10.10.0.0 netmask 255.255.255.0 {
range 10.10.0.2 10.10.0.254;
group {
filename "10M-5M-OKS2016NOV.cm";
host client1 {
hardware ethernet a0:b4:3d:bc:df:fa;
}
host client2 {
hardware ethernet 90:6e:bb:ba:cd:d4;
}
}
}
subnet 11.10.0.0 netmask 255.255.255.0 {
range 11.10.1.2 11.10.1.254;
group {
filename "10M-5M-OKS2016NOV.cm";
host client3 {
hardware ethernet c0:03:03:bc:30:fa;
}
host client1 {
hardware ethernet c0:14:e3:bc:df:fa;
}
host client2 {
hardware ethernet 90:6e:fb:ba:3d:04;
}
}
}
subnet 12.10.0.0 netmask 255.255.255.0 {
range 12.10.2.2 12.10.2.254;
group {
filename "10M-5M-OKS2016NOV.cm";
host client1 {
hardware ethernet c0:a4:3d:bc:df:fa;
}
host client2 {
hardware ethernet 90:6e:bb:ca:3d:04;
}
}
}
It initially slurps all the lines at first to process multi lines efficiently.
It assumes the right curly brace } does not appear in the search target block
to achieve the shortest match in regex.
Hope this helps.
This might work for you (GNU sed):
sed '/subnet 11\.10\.0\.0 netmask 255\.255\.255\.0/{:a;n;/filename "10M-5M-OKS2016NOV\.cm";/!ba;p;s/\S.*/host client3 {/p;s// hardware ethernet c0:03:03:bc:30:fa;/p;s//}/}' file
This finds the first line containing subnet 11.10.0.0 netmask 255.255.255.0 and then reads on further until the line containing filename "10M-5M-OKS2016NOV.cm";. After printing that line it uses the line as a template to format the required detail.
Another solution, using a preformed insertion file:
cat <<\! | sed '/subnet 11\.10\.0\.0 netmask 255\.255\.255\.0/!b;:a;n;/filename "10M-5M-OKS2016NOV\.cm";/!ba;r /dev/stdin' file
host client3 {
hardware ethernet c0:03:03:bc:30:fa;
}
!
sed is great as a stream editor, that means to process multiple times the same actions. Here you just want to insert once a bloc of text. That would be much simpler (more readable and maintenable) with ed:
ed dhcpd.conf <<EOF
/subnet 11.10.0.0/
/filename/
a
host client3 {
hardware ethernet c0:03:03:bc:30:fa;
}
.
w
q
EOF
Beware: ed is a file editor. That means that the dhcpd.conf file will be changed by the above script. Make sure to have a backup if things goes wrong...

Why is the probe function in my kernel module not being called?

While following, among others, this tutorial (1) and reading certain chapters in the Linux Device Drivers book, I cannot get the pr_debug() statements in the probe function to show any output in dmesg.
Here's my code:
#include <linux/module.h> /*included for all kernel modules*/
#include <linux/kernel.h> /*included for KERN_DEBUG*/
#include <linux/init.h> /*included for __init and __exit macros*/
#include <linux/usb.h>
#include <linux/usb/input.h>
#include <linux/hid.h>
#define VENDOR_ID 0x0930
#define DEVICE_ID 0x6545
MODULE_LICENSE("GPL");
MODULE_AUTHOR("dev");
MODULE_DESCRIPTION("eusb driver");
static struct usb_device_id eusb_table[] = {
{ USB_DEVICE(VENDOR_ID, DEVICE_ID) },
{ } /* Terminating entry */
};
MODULE_DEVICE_TABLE (usb, eusb_table);
static int eusb_probe(struct usb_interface *interface, const struct usb_device_id *id)
{
pr_debug("USB probe function called\n");
return 0;
}
static void eusb_disconnect(struct usb_interface *interface)
{
pr_debug("USB disconnect function called\n");
}
static struct usb_driver eusb_driver = {
//.owner = THIS_MODULE,
.name = "eusb",
.probe = eusb_probe,
.disconnect = eusb_disconnect,
.id_table = eusb_table
};
static int __init eusb_init(void)
{
int result = 0;
pr_debug("Hello world!\n");
result = usb_register(&eusb_driver);
if(result){
pr_debug("error %d while registering usb\n", result);}
else{pr_debug("no error while registering usb\n");}
return 0;
}
static void __exit eusb_exit(void)
{
usb_deregister(&eusb_driver);
pr_debug("Exiting module\n");
}
module_init(eusb_init);
module_exit(eusb_exit);
and Makefile:
obj-m := eusb.o
CFLAGS_eusb.o := -DDEBUG
KDIR := /lib/modules/$(shell uname -r)/build
PWD := $(shell pwd)
default:
$(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules
Make finishes without errors, after insmod I can see the module listed in lsmod and the pr_debug() in the init and exit functions show output in dmesg.
When inserting the device however the probe function seems to not get called (or the pr_debug() statements do not show any output in dmesg).
Dmesg output:
[ 7777.521236] Hello world!
[ 7777.521264] usbcore: registered new interface driver eusb
[ 7777.521266] no error while registering usb
[ 7780.597087] usb 1-6: USB disconnect, device number 9
[ 7797.686970] usb 1-6: new high-speed USB device number 10 using xhci_hcd
[ 7797.857324] usb 1-6: New USB device found, idVendor=0930, idProduct=6545
[ 7797.857328] usb 1-6: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 7797.857330] usb 1-6: Product: DataTraveler 2.0
[ 7797.857331] usb 1-6: Manufacturer: Kingston
[ 7797.857333] usb 1-6: SerialNumber: 08606E6D407FED10571E5067
[ 7797.858787] usb-storage 1-6:1.0: USB Mass Storage device detected
[ 7797.858902] scsi host11: usb-storage 1-6:1.0
[ 7798.931417] scsi 11:0:0:0: Direct-Access Kingston DataTraveler 2.0 PMAP PQ: 0 ANSI: 4
[ 7798.931824] sd 11:0:0:0: Attached scsi generic sg3 type 0
[ 7800.184749] sd 11:0:0:0: [sdc] 60964864 512-byte logical blocks: (31.2 GB/29.0 GiB)
[ 7800.186338] sd 11:0:0:0: [sdc] Write Protect is off
[ 7800.186343] sd 11:0:0:0: [sdc] Mode Sense: 23 00 00 00
[ 7800.187948] sd 11:0:0:0: [sdc] No Caching mode page found
[ 7800.187952] sd 11:0:0:0: [sdc] Assuming drive cache: write through
[ 7800.220477] sdc: sdc1 sdc2 sdc3
[ 7800.225068] sd 11:0:0:0: [sdc] Attached SCSI removable disk
[ 7802.798403] ISO 9660 Extensions: Microsoft Joliet Level 3
[ 7802.799507] ISO 9660 Extensions: RRIP_1991A
I have tried with another device, tried with printk() instead of pr_debug(). I found several questions on SO with the same problem, but my code is as far as I can tell almost/completely the same as the code in the answers.
I have also tried with:
USB_INTERFACE_INFO(
USB_INTERFACE_CLASS_HID,
USB_INTERFACE_SUBCLASS_BOOT,
USB_INTERFACE_PROTOCOL_KEYBOARD)
instead of USB_DEVICE() (the other device was a keyboard).
I noticed that some answers talk about platform_driver instead of usb_driver, but I think this is not relevant for me, as it is never mentioned in the tutorials.
Where am I going wrong?
Is your purpose just educational? Otherwise the driver is already there and it's called usb-storage. Seems even that driver enumerates your device first. Try to disable usb-storage and load your driver again. Or better use the virtual machine (KVM / Qemu) to try your drivers.
As you can see your Hello world! was printed which implies the module was loaded successfully. Often all the modules claiming to handle the device via usb_device_id* table are loaded. See 9.3.2.3. Module Loading.
Now, the probe function is called only for the driver the device was bound to. In the case where there is no priority set and two drivers are capable of binding, any of them might get the first opportunity.
I have written a blog post which might clear things further (if not, do comment here).
Further, you might want to go through these two relevant mailing list threads
https://lists.kernelnewbies.org/pipermail/kernelnewbies/2011-October/003639.html
https://marc.info/?l=linux-usb&m=162123892329464&w=2

Linux kernel cannot receive multicast

I built a Linux kernel with CONFIG_IP_MULTICAST=y,however no UDP multicast package received in this kernel while UDP unicast works well.
ethtool -S eth0 | grep multicast
txmulticastframes_g: 0
txmulticastframes_gb: 0
rxmulticastframes_g: 0
Any hints how can I solve this problem?
Thx. Forrest G
=================================================================================
Additional:
tcpdump can get the packet
root#JHI # ./tcpdump port 3702
device eth0 entered promiscuous mode
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
-7:-48:-19.4151 IP 192.168.42.212.3702 > 239.255.255.250.3702: UDP, length 787
-7:-48:-19.4661 IP 192.168.42.212.3702 > 239.255.255.250.3702: UDP, length 803
^C
2 packets captured
2 padevice eth0 left promiscuous mode
ckets received by filter
0 packets dropped by kernel
I write this WS-devicediscovery function with gSOAP. It works on X86 machine. When running on ARM device, it can send out igmp packet but not receive anything.
void wsdd()
{
struct soap *soap_udp;
struct ip_mreq mreq;
soap_udp=soap_new();
soap_init1(soap_udp, SOAP_IO_UDP|SOAP_IO_FLUSH);
if (!soap_valid_socket(soap_bind(soap_udp, NULL, 3702, 100)))
{
soap_print_fault(soap_udp, stderr);
}
mreq.imr_multiaddr.s_addr = inet_addr("239.255.255.250");
mreq.imr_interface.s_addr = htonl(INADDR_ANY);
if(setsockopt(soap_ud->master,IPPROTO_IP,IP_ADD_MEMBERSHIP,&mreq,sizeof(mreq))==-1) {
perror("membership error\n");
}
int loop = 1;
int sock_opt = 1;
if ((setsockopt(soap_udp->master, SOL_SOCKET, SO_REUSEADDR, (void *) &sock_opt,
sizeof (sock_opt))) == -1) {
printf("setsockopt\n");
}
if ((setsockopt(soap_udp->master, IPPROTO_IP, IP_MULTICAST_LOOP,
&loop, sizeof (loop))) == -1) {
printf("setsockopt\n");
}
while(1){
soap_accept(soap_udp);
soap_serve(soap_udp);
soap_end(soap_udp);
}
}
Then I have tried these things but still not work
route add -net 224.0.0.0 netmask 240.0.0.0 eth0
echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter
Problem solved by changing..
ifconfig eth0 promisc
Can anyone explain the principle?

systemtap tcp connections script error

i am trying to run this script , and it shows no tcp connections any ideas? OS: centos 6.3
#! /usr/bin/env stap
probe begin {
printf("%6s %16s %6s %6s %16s\n",
"UID", "CMD", "PID", "PORT", "IP_SOURCE")
}
probe kernel.function("tcp_accept").return?,
kernel.function("inet_csk_accept").return? {
sock = $return
if (sock != 0)
printf("%6d %16s %6d %6d %16s\n", uid(), execname(), pid(),
inet_get_local_port(sock), inet_get_ip_source(sock))
}
OUTPUT:
[root#server src]# ./tcp_con.stp
Missing separate debuginfos, use: debuginfo-install kernel-2.6.32-279.1.1.el6.centos.plus.x86_64
UID CMD PID PORT IP_SOURCE
Follow systemtap's advice and install kernel-debuginfo.
The question marks after both kernel.function() probes let stap quietly drop both of those.

Resources