I have large data file with some data as :
01 01 00 2c 00 82 03 00 02 00 00 00 07 08 07 08
07 08 07 08 07 08 07 08 07 08 07 08 07 08 07 08
07 08 07 08 07 08 07 08 07 08 07 08 07 08 07 08
07 08 07 08 07 08 07 08 07 08 07 08 07 08 07 08
0f 08 08 08 0a 08 07 08 0f 08 08 08 08 08 08 08
08 08 08 08 08 07 08 07 0a 07 07 07 0f 07 08 07
08 07 08 07 08 07 08 07 08 07 08 07 0a 07 07 07
..
.....
I would like to delete every first n characters from every 6th row
I have found a command :
sed 's/^.\{,n\}//' file
But this command deletes first n chars from each row, which I do not want to happen.
Could someone suggest the right command?
GNU sed allows you to use address in form of
first~step
(...)matches every stepth line starting with line first(...)
therefore 1~6 does pertain to 1st, 7th, 13th and so on lines, 2~6 pertains to 2nd, 8th, 14th and so on lines, let file.txt content be
01 01 00 2c 00 82 03 00 02 00 00 00 07 08 07 08
07 08 07 08 07 08 07 08 07 08 07 08 07 08 07 08
07 08 07 08 07 08 07 08 07 08 07 08 07 08 07 08
07 08 07 08 07 08 07 08 07 08 07 08 07 08 07 08
0f 08 08 08 0a 08 07 08 0f 08 08 08 08 08 08 08
08 08 08 08 08 07 08 07 0a 07 07 07 0f 07 08 07
08 07 08 07 08 07 08 07 08 07 08 07 0a 07 07 07
..
.....
and n be equal to 5 then, you might do
sed '1~6 s/^.....//' file.txt
which gives output
00 2c 00 82 03 00 02 00 00 00 07 08 07 08
07 08 07 08 07 08 07 08 07 08 07 08 07 08 07 08
07 08 07 08 07 08 07 08 07 08 07 08 07 08 07 08
07 08 07 08 07 08 07 08 07 08 07 08 07 08 07 08
0f 08 08 08 0a 08 07 08 0f 08 08 08 08 08 08 08
08 08 08 08 08 07 08 07 0a 07 07 07 0f 07 08 07
08 07 08 07 08 07 08 07 08 07 0a 07 07 07
..
.....
(tested GNU sed 4.7)
awk -v n=17 '(NR%6)==1 { print substr($0, n+1); next } 1' file
The condition uses modulo arithmetic on the line number NR to select every sixth line, starting from the first. The final 1 causes the other lines to be printed normally.
You haven't revealed the value of n so I guessed.
gawk/nawk solution. mawks would need a slightly diff approach
for N in $(jot - 2 28 7); do
jot -b '0123456789012345678901234567890123456789' 8 |
gawk -v FS='^.{'"$N"'}' 'BEGIN { _^=OFS=_ } NR%6!=_ || NF=NF' | gcat -n
echo "${N}"
done
1 23456789012345678901234567890123456789
2 0123456789012345678901234567890123456789
3 0123456789012345678901234567890123456789
4 0123456789012345678901234567890123456789
5 0123456789012345678901234567890123456789
6 0123456789012345678901234567890123456789
7 23456789012345678901234567890123456789
8 0123456789012345678901234567890123456789
2
1 9012345678901234567890123456789
2 0123456789012345678901234567890123456789
3 0123456789012345678901234567890123456789
4 0123456789012345678901234567890123456789
5 0123456789012345678901234567890123456789
6 0123456789012345678901234567890123456789
7 9012345678901234567890123456789
8 0123456789012345678901234567890123456789
9
1 678901234567890123456789
2 0123456789012345678901234567890123456789
3 0123456789012345678901234567890123456789
4 0123456789012345678901234567890123456789
5 0123456789012345678901234567890123456789
6 0123456789012345678901234567890123456789
7 678901234567890123456789
8 0123456789012345678901234567890123456789
16
1 34567890123456789
2 0123456789012345678901234567890123456789
3 0123456789012345678901234567890123456789
4 0123456789012345678901234567890123456789
5 0123456789012345678901234567890123456789
6 0123456789012345678901234567890123456789
7 34567890123456789
8 0123456789012345678901234567890123456789
23
Related
fmt.Println(hex.Dump([]byte{1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0}))
It shows:
00000000 01 02 03 04 05 06 07 08 09 00 01 02 03 04 05 06 |................|
00000010 07 08 09 00 |....|
How to add an indent before the hex addresses to all lines? like
00000000 01 02 03 04 05 06 07 08 09 00 01 02 03 04 05 06 |................|
00000010 07 08 09 00 |....|
Split the lines by \n then put an indent before each line will do, but is there any built-in tool does this? I tried hex.Dumper and tabwriter.NewWriter, but they just treat the indent as normal data and printed along with the hex dump.
hex.Dump() returns a formatted string. If you want it to be formatted like indenting every line, replace all newline characters with a newline + indentation (e.g. tab). Plus prepend the indentation to the start of it so the first line will also be indented (which is not preceeded by a newline char).
For replacing, use strings.ReplaceAll().
See this example:
s := hex.Dump([]byte{1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0})
fmt.Println(s)
s = "\t" + strings.ReplaceAll(s, "\n", "\n\t")
fmt.Println(s)
Output (try it on the Go Playground):
00000000 01 02 03 04 05 06 07 08 09 00 01 02 03 04 05 06 |................|
00000010 07 08 09 00 01 02 03 04 05 06 07 08 09 00 01 02 |................|
00000020 03 04 05 06 07 08 09 00 |........|
00000000 01 02 03 04 05 06 07 08 09 00 01 02 03 04 05 06 |................|
00000010 07 08 09 00 01 02 03 04 05 06 07 08 09 00 01 02 |................|
00000020 03 04 05 06 07 08 09 00 |........|
lakhal#lakhal-ThinkPad-T430s:~$ hexdump -C /dev/bus/usb/003/014
00000000 12 01 00 02 e0 00 00 40 e8 04 63 68 ff ff 02 03 |.......#..ch....|
00000010 04 01 09 02 4b 00 02 01 00 c0 30 08 0b 00 02 e0 |....K.....0.....|
00000020 01 03 07 09 04 00 00 01 e0 01 03 05 05 24 00 10 |.............$..|
00000030 01 05 24 01 00 01 04 24 02 00 05 24 06 00 01 07 |..$....$...$....|
00000040 05 83 03 08 00 09 09 04 01 00 02 0a 00 00 06 07 |................|
00000050 05 81 02 00 02 00 07 05 02 02 00 02 00 |.............|
0000005d
The file allows you to read certain USB descriptors from the device. The USB device descriptor, which is defined by the USB specification, is located at the beginning of the file.
For more details, see the file usb/core/devio.c in the source code of your kernel. In particular, look at the function usbdev_read, which implements the kernel side of the read system call for these special files. I cannot find official documentation at the moment.
I believe some parts of the file are "holes", meaning that you won't be able to read any data from them, and you might get meaningless data depending on whether the program you uses initializes the buffers it uses for reading from the file.
for loop need to print numbers (not in a string format) like
05 06 07 08 09 10 11
For example:
print(format(5, '02d'))
Output:
'05'
I need to generate numbers (not string) in for loop
for i in range(int(format(5, '02d')),10):
print(i)
The following code is printing 5 6 7 8 9 instead 05 06 07 08 09
You should try something like below:
for i in range(5,10):
print(format(i, '02d'))
or
for i in range(5,10):
print('%02d' % i)
Output:
05
06
07
08
09
I am currently attempting to configure snmptrapd to do trap forwarding.With my current configuration, snmptrapd is receiving and logging
incoming traps, but it it does not appear to be forwarding them.
The contents of my snmptrapd.conf file are as follows:
disableAuthorization yes
traphandle default /usr/sbin/snmptthandler
authCommunity log public
forward default 127.0.0.1:1062
When I run snmptrapd as follows:
snmptrapd -f -Le -d
I can see traps being received as follow result,
Received 67 byte packet from UDP: [191.50.2.21]49801->[191.50.3.75]:162
0000: 30 41 02 01 00 04 08 54 4E 50 49 47 50 29 2A A4 0A.....USPIGP)*.
0016: 32 06 08 2B 06 01 06 03 01 01 05 40 04 C0 A8 06 2..+.......#....
0032: 0C 02 01 04 02 01 00 43 04 3D F3 FC 72 30 14 30 .......C.=..r0.0
0048: 12 06 0A 2B 06 01 04 01 09 02 01 05 00 40 04 B9 ...+.........#..
0064: 5E 6F 01
When I send a test trap from localhost,
Sending 96 bytes to UDP: [127.0.0.1]:1062->[0.0.0.0]:0
0000: 30 5E 02 01 00 04 06 70 75 62 6C 69 63 A4 51 06 0^.....public.Q.
0016: 09 2B 06 01 06 03 01 01 05 05 40 04 7F 00 00 01 .+........#.....
0032: 02 01 06 02 01 11 43 04 02 12 FA 93 30 32 30 30 ......C.....0200
0048: 06 09 2B 06 01 06 03 01 01 05 05 04 23 4A 75 73 ..+.........#Jus
0064: 74 20 61 20 74 65 73 74 2E 2E 2E 62 6C 61 62 6C t a test...blabl
0080: 61 62 6C 61 62 6C 2E 2E 2E 2E 2E 2E 2E 2E 2E 2E ablabl..........
I would appreciate any assistance with further debugging and
ultimately addressing this issue.
Thank you.
Andrew
If you want to forward the trap you need to enable the net type:
authCommunity log,net public
i have a file with .raw extension (raw image format), but i cannot open it. i tried to analyze it with the commands file filename.raw, exiv2 print, exiftool, ufraw, dcraw. since it appears to be a binary file, i tried to extract information using strings filename.raw, hexdump and xxd. it is surprising that the file does not appear to have any metadata. i suspect the file extension to be wrong, and the file to be manipulated.
how else can i determine the type? how can i extract any information?
thank you very much for any suggestions.
EDIT: a sample file can be downloaded
here.
Hmm... well,
//Judging from the number of bytes in the file:
// 1447600
//This number is exactly.
// 3 x 499 x 967
//Which is very odd, but it means if this is an image, then it must be 3 bytes per pixel and either 499 x 967 or 967 x 499
EDIT: resolution is wrong. Probably bits per channel as well, this is what I get for using bc in the middle of the night instead of WA.
Actual factors are 2^4*5^2*7*11*47, which means it cannot be 3 Bytes per pixel. There might be some stream compression going on? But probably not, the number is too nice.
Looking at some of the data, most of it is quite noisy, which fits with a camera with a noisy digital sensor:
00000000 0e 08 06 02 04 06 00 02 00 04 01 04 0e 03 05 01 |................|
00000010 03 01 06 04 00 03 01 05 01 00 02 00 06 02 04 01 |................|
00000020 00 07 0c 0d 05 01 0a 0d 03 08 11 08 0c 03 06 06 |................|
00000030 08 02 07 04 01 01 07 02 05 06 10 04 07 01 04 02 |................|
00000040 02 01 0c 09 07 02 05 01 0d 04 00 05 0e 01 00 07 |................|
00000050 00 02 04 0a 11 04 07 04 01 00 0d 07 0f 06 03 02 |................|
00000060 06 04 0a 0b 03 08 02 0c 00 08 00 04 06 09 03 03 |................|
00000070 04 00 0a 04 01 06 00 00 02 08 0a 07 04 09 01 0b |................|
00000080 01 0c 0d 04 03 03 05 00 02 11 09 00 07 08 08 05 |................|
00000090 04 07 02 0c 00 13 01 06 07 02 08 0a 07 05 02 01 |................|
This area, at the beginning of the file, is likely to be close to all black.
So, of the common 3 bytes per pixel formats, you have BGR24, RGB24 or YUV24.
Because it's simpler to deal with RGB or BGR formats, let's whip up some short code to dump this to a png, using libpng.
First try:
#include <png.h>
#include <exception>
#include <memory>
int main() {
FILE * fpout;
FILE * fpin;
png_structp png_ptr = nullptr;
png_infop info_ptr = nullptr;
const unsigned width = 499;
const unsigned height = 967;
const unsigned channels = 3;
fpin = fopen("sample.raw", "rb");
if (!fpin)
throw std::exception();
std::unique_ptr<char[]> data(new char[width*height*channels]);
fread(data.get(), width*height*channels, 1, fpin);
fclose(fpin);
fpout = fopen("output.png", "wb");
if (!fpout)
throw std::exception();
png_ptr = png_create_write_struct(PNG_LIBPNG_VER_STRING, nullptr, nullptr, nullptr);
if (!png_ptr) {
fclose (fpout);
throw std::exception();
}
info_ptr = png_create_info_struct(png_ptr);
if (!info_ptr) {
png_destroy_write_struct(&png_ptr, nullptr);
fclose (fpout);
throw std::exception();
}
if (setjmp(png_jmpbuf (png_ptr))) {
png_destroy_write_struct(&png_ptr, &info_ptr);
fclose (fpout);
throw std::exception();
}
png_set_IHDR(png_ptr,
info_ptr,
width,
height,
8,
PNG_COLOR_TYPE_RGB,
PNG_INTERLACE_NONE,
PNG_COMPRESSION_TYPE_DEFAULT,
PNG_FILTER_TYPE_DEFAULT);
/* Initialize rows of PNG. */
std::unique_ptr<png_bytep[]> row_ptrs(new png_bytep[height]);
size_t stride = width * 8 * channels / 8;
for (size_t i = 0; i < height; ++i) {
size_t q = i * stride;
row_ptrs[i] = (png_bytep)data.get() + q;
}
/* Write the image data to "fp". */
png_init_io(png_ptr, fpout);
png_set_rows(png_ptr, info_ptr, row_ptrs.get());
png_write_png(png_ptr, info_ptr, PNG_TRANSFORM_IDENTITY, nullptr);
png_destroy_write_struct(&png_ptr, &info_ptr);
fclose (fpout);
}
Gives us:
http://i.imgur.com/Iwi3M.jpg
This is heartening, because it seems that we merely have the resolution wrong. So examining the image, it looks like the bar repeats itself every 467 pixels or there abouts. However, it does not look like there is anything interesting in the data. So it's likely not simple RBG or BGR or YUV encoding, otherwise, it would look nicer.
Raw Therapee doesn't recognize it. So, I'm asking you now. What camera did you use to make this?