editing hex / binary files in limited linux - linux

I have a very limited linux w/ few basic linux commands.
I need to replace a few chars in a hex / binary file:
INPUT:
# hexdump -C block.bin
00000000 11 11 50 04 42 00 00 00 58 00 00 00 3c 0e e2 d4 |..P.B...X...<...|
00000010 50 0b 00 00 00 80 00 00 00 00 00 00 00 00 d0 d7 |P...............|
00000020 1f 09 00 00 00 00 02 00 00 00 00 04 ff ff ff ff |................|
00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000040 28 31 98 5b d3 0e 05 00 00 00 00 00 00 00 00 00 |(1.[............|
00000050 00 00 00 00 00 00 00 00 64 00 00 00 00 00 10 00 |........d.......|
00000060 00 ff ff ff ff 00 00 00 00 03 01 0d 03 01 0d 01 |................|
00000070 00 00 00 00 0c 00 01 02 00 00 00 00 00 ff ff ff |................|
00000080 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
*
00000180 00 00 20 00 00 00 ff ff ff ff 01 00 00 0c 00 00 |.. .............|
00000190 04 00 00 00 02 00 00 04 00 00 00 00 00 00 ff ff |................|
000001a0 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
*
00000200 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
I need to change the following:
00000060 00 ff ff ff ff 00 00 00 00 03 01 0d 03 01 0d 01 |................|
00000070 00 00 00 00 0c 00 01 02 00 00 00 00 00 ff ff ff |................|
to (and rewrite back to the same file):
00000060 00 ff ff ff ff 00 00 00 00 03 01 0d 03 01 0d 02 |................|
00000070 01 00 00 00 0c 00 01 02 00 00 00 00 00 ff ff ff |................|
Like I've said, I have a handful of commands: hexdump, od, vi, nano, awk, sed, python.
Looking through the internet, many solutions require 3rd party installs or the use of 'xxd'. Both I cannot use.
Any suggestions?
Thanks!

I don't think trying this with sed or awk is a sane idea, so we're stuck with python, which is certainly powerful enough for this task. I'm thinking along these lines:
#!/usr/bin/python
# open file in binary mode for reading and writing
f = open("block.bin", "r+b")
# seek to position and read two bytes
f.seek(0x6f)
data = f.read(2)
# seek to position again
f.seek(0x6f)
# and write the transformed characters back
for d in data:
f.write(chr(ord(d) + 1))
f.close()

Related

Get VLAN tag or untag via SNMP on Procurve Switch

How can I get via SNMP if the VLAN where a port belongs is tagged or untagged on a Procurve switch ?
I've found with this OID : .1.3.6.1.2.1.17.7.1.4.3.1.4 that it returns a series of hex code that should tell me which port belongs to a vlan (this is an extract for vlan 1 and 100):
snmpwalk -v2c -c public 192.168.0.1 .1.3.6.1.2.1.17.7.1.4.3.1.4
SNMPv2-SMI::mib-2.17.7.1.4.3.1.4.1 = Hex-STRING: 00 00 00 00 00 00 FF FF FF FF FF FF FF FF FF 00
00 03 FF FF FF FF FF FF FF FF FF FF FF FF FF FF
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
FF FF FF FF FF FF FF FF 80 00 00 00
SNMPv2-SMI::mib-2.17.7.1.4.3.1.4.100 = Hex-STRING: FF FF FF FF E0 00 00 00 00 00 00 00 00 00 00 00
03 F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00
My questions are:
How can I get the ifindex from this HEX code?
How can I understand it the port is tagged or untagged?
Thanks.
1.3.6.1.2.1.17.7.1.4.3.1.4 is dot1qVlanStaticUntaggedPorts, part of dot1qVlanStaticTable, defined in Q-BRIDGE-MIB. If a port is listed by this object, it is untagged. If a port is listed in dot1qVlanStaticEgressPorts, also part of dot1qVlanStaticTable, but not listed in dot1qVlanStaticUntaggedPorts, then it is tagged.
Both dot1qVlanStaticEgressPorts and dot1qVlanStaticUntaggedPorts are of type PortList, also defined in Q-BRIDGE-MIB. Each bit of each octet corresponds to one ifIndex, with the most significant bit of the first octet being ifIndex=1, the next bit being ifIndex=2, and so on. If the bit is 1, it is a member; if it is 0 then it is not.
Strange that the agent would be returning such long values (= many, many, many ports), though.

How to decode the section table in an ELF?

I'm analyzing this tiny ELF file:
00000000 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 |.ELF............|
00000010 02 00 3e 00 01 00 00 00 78 00 40 00 00 00 00 00 |..>.....x.#.....|
00000020 40 00 00 00 00 00 00 00 98 00 00 00 00 00 00 00 |#...............|
00000030 00 00 00 00 40 00 38 00 01 00 40 00 03 00 02 00 |....#.8...#.....|
00000040 01 00 00 00 05 00 00 00 00 00 00 00 00 00 00 00 |................|
00000050 00 00 40 00 00 00 00 00 00 00 40 00 00 00 00 00 |..#.......#.....|
00000060 7e 00 00 00 00 00 00 00 7e 00 00 00 00 00 00 00 |~.......~.......|
00000070 00 00 20 00 00 00 00 00 31 c0 ff c0 cd 80 00 2e |.. .....1.......|
00000080 73 68 73 74 72 74 61 62 00 2e 74 65 78 74 00 00 |shstrtab..text..|
00000090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
000000d0 00 00 00 00 00 00 00 00 0b 00 00 00 01 00 00 00 |................|
000000e0 06 00 00 00 00 00 00 00 78 00 40 00 00 00 00 00 |........x.#.....|
000000f0 78 00 00 00 00 00 00 00 06 00 00 00 00 00 00 00 |x...............|
00000100 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 |................|
00000110 00 00 00 00 00 00 00 00 01 00 00 00 03 00 00 00 |................|
00000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000130 7e 00 00 00 00 00 00 00 11 00 00 00 00 00 00 00 |~...............|
00000140 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 |................|
00000150 00 00 00 00 00 00 00 00 |........|
00000158
I found documentation on the ELF header and the program header and decoded both of those, but I'm having problems decoding what's after this (starting with 31 c0 ff c0 cd 80 00 2e). Judging by the "shstrtab" text, I am looking at the section table, but what does 31 c0 ff c0 cd 80 00 2e mean? Where is this part documented?
OK, judging by the information in the first 16 bytes of the header:
00000000 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 |.ELF............|
E L F | | '--- Pudding :) ---'
| '--- Little-endian (ELFDATA2LSB)
'------ 64-bit (ELFCLASS64)
we're dealing with a 64-bit ELF with little-endian encoding of multi-byte numbers. So the ELF header is the first 4 rows in the hex editor. We're interested in these fields in the last two rows of it:
Prog Hdr Tab offset Sect Hdr Tab offset
.----------^----------. .----------^----------.
00000020 40 00 00 00 00 00 00 00 98 00 00 00 00 00 00 00 |#...............|
00000030 00 00 00 00 40 00 38 00 01 00 40 00 03 00 02 00 |....#.8...#.....|
'-.-' '-.-' '-.-' '-.-' '-.-'
PHT entry size ---' | | | '-- Sect names in #2
PHT num entries ----------' | '-- SHT num entries
'-------- SHT entry size
So we know that the Program Headers Table starts at offset 0x40 in the file (right after this header) and contains 1 entry of size 0x38 (56 bytes). So it ends at offset 0x40 + 1*0x38 = 0x78 (this is the first byte after this table, and this is also where your "mysterious data" begins, so keep this in mind).
The Section Headers Table starts at offset 0x98 in the file and contains 3 entries of size 0x40 (64 bytes), that is, each entry in SHT takes 4 consecutive rows in a hex editor, and the entire table is 3*4 = 12 such rows, so the offset 0x158 is the first byte after this table. But this is just the end of the file, so there's nothing more after the SHT.
The SHT entry at index 2 (the third=last one) should be a string table that contains the names for the sections.
So let's look at those sections now, shall we?
Section #2
Let's start with section #2, since it is supposed to contain the string table with the names for all the sections, so it will be very useful in further analysis. Here's its header (the last one in the table):
Name index Type=SHT_STRTAB (bingo!)
Flags .----^----. .----^----.
00000118 .----------^----------. 01 00 00 00 03 00 00 00 |........|
00000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000130 7e 00 00 00 00 00 00 00 11 00 00 00 00 00 00 00 |~...............|
'----------.----------' '----------.----------'
Starting offset Size
00000140 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 |................|
00000150 00 00 00 00 00 00 00 00 |........|
00000158
So this is indeed a string table (0x03 = SHT_STRTAB). It starts from offset 0x7E in the file and takes 0x11 (17) consecutive bytes. The first byte after the string table is therefore 0x8F. This byte is not a part of any section (garbage).
The string table
So let's see what's in the section containing the string table, so that we could name our sections:
0000007E 00 2e |..|
00000080 73 68 73 74 72 74 61 62 00 2e 74 65 78 74 00 |shstrtab..text.|
0000008F
Here's the string table, with addresses relative to its beginning:
+0 +1 +2 +3 +4 +5 +6 +7 +8 +9 +A +B +C +D +E +F
00: 00 2E 73 68 73 74 72 74 61 62 00 2e 74 65 78 74
10: 00
or the same in ASCII, with the NULL characters marked as ∎:
+0 +1 +2 +3 +4 +5 +6 +7 +8 +9 +A +B +C +D +E +F
00: ∎ . s h s t r t a b ∎ . t e x t
10: ∎
So we have just 3 full string in it, with the following relative offsets:
00: "" (Just the empty string)
01: ".shstrtab" (Name for this section)
0B: ".text" (Name for the section that contains the executable code)
(Keep in mind, though, that sections can also address substrings inside those strings, if they share the common ending.)
We can now verify that this section (#2) is indeed named .shstrtab: its name index was 0x01 after all, wasn't it? ;)
Section #1
Now let's take apart section #1's header:
Name index Type=SHT_PROGBITS
Flags .----^----. .----^----.
000000d8 .----------^----------. 0b 00 00 00 01 00 00 00 |........|
000000e0 06 00 00 00 00 00 00 00 78 00 40 00 00 00 00 00 |........x.#.....|
000000f0 78 00 00 00 00 00 00 00 06 00 00 00 00 00 00 00 |x...............|
'----------.----------' '----------.----------'
Starting offset Size
00000100 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 |................|
00000110 00 00 00 00 00 00 00 00 |........|
00000118
So this section is named .text (note the name index 0x0B) and it is of type SHT_PROGBITS, so it contains some program-defined data; the executable code in this case. It starts from the offset 0x78 in the file and takes the next 6 bytes, so the first byte after this section is at offset 0x7E (where the string table begins). Here's its contents:
00000070 31 c0 ff c0 cd 80 |1.....|
0000007E
But wait! Remember where your "mysterious data" starts? Yes! It's the 0x78 offset! :) So this "mysterious data" is actually your executable payload :) After decoding it as Intel x86-64 opcodes we get this tiny little program:
31 C0 xor %eax,%eax ; Clear the EAX register to 0 (the short way).
FF C0 inc %eax ; Increase the EAX, so now it contains 1.
CD 80 int $0x80 ; Interrupt 0x80 is the system call on Linux.
which is basically equivalent to calling exit(0) in C ;) because the syscall interrupt expects the operation number in EAX, which in this case is sys_exit (operation number 1).
So yeah, mystery solved :) But let's continue anyway, to learn something more, and this way we'll find out where this piece of code will be loaded in memory.
Section #0
And finally section #0. It has some part missing, but I assume it was all 0s, since the first section is always a NULL section after all. Here's its (butchered) header:
00000098 00 00 00 00 00 00 00 00 | ........|
*
000000d0 00 00 00 00 00 00 00 00
But it's of no use to us. Nothing interesting here.
Program Headers Table
The last thing what's left to decode is the Program Headers Table, which – according to the information from the ELF header – starts from the offset 0x40 and takes 56 bytes, the first byte after it being at offset 0x78. Here's the dump:
Type=PHT_EXEC Flags=RX Starting offset in file
.----^----. .----^----. .----------^----------.
00000040 01 00 00 00 05 00 00 00 00 00 00 00 00 00 00 00 |................|
00000050 00 00 40 00 00 00 00 00 00 00 40 00 00 00 00 00 |..#.......#.....|
'----------.----------' '----------.----------'
Virtual address Physical address
Size in file Size in memory
.----------^----------. .----------^----------.
00000060 7e 00 00 00 00 00 00 00 7e 00 00 00 00 00 00 00 |~.......~.......|
00000070 00 00 20 00 00 00 00 00
00000078 '----------.----------'
Alignment
So it says that we load the first 126 (0x7E) bytes of the file into a memory segment of the same size, and the memory segment is supposed to start from the virtual address 0x400000. Our code starts from the offset 0x78 in the file and the first byte after it has the offset 0x7E, so it basically loads the entire beginning of the file, with the ELF header and the program header table into memory, as well as our executable payload at the end of it, and stops loading afterwards, ignoring the rest of the file.
So if the beginning of the file is loaded at address 0x400000, and our program starts 120 (0x78) bytes from its beginning, it will be located at the address 0x400078 in memory :>
Now let's see what entry point is specified in the ELF header for our program:
Executable x86-64 Version=1 Program's entry point
.-^-. .-^-. .----^----. .----------^----------.
00000010 02 00 3e 00 01 00 00 00 78 00 40 00 00 00 00 00 |..>.....x.#.....|
Bingo! :> It's 0x400078, so it points at the start of our little piece of code in the memory image.
And that's all, folks! ;)

Do modern operating systems ignore LBA/CHS information in the MBR boot sector?

I've wiped all the CHS/LBA information and bootcode on a spare flash drive leaving only the bare sector count information:
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000100 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000130 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000150 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000160 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000180 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000190 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 80 ff |................|
000001c0 ff ff 0c ff ff ff 00 08 00 00 00 f8 d4 01 00 00 |................|
000001d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000001f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 aa |..............U.|
Leaving only this:
80 ff ff ff 0c ff ff ff 00 08 00 00 00 f8 d4 01
The first flag saying it is bootable, the 4th that it is FAT32, and the last 2, 4 byte values the sector start and count in Big-endian. The FF's are CHS/LBA info I set all to FF
And, the disk still works fine. Partition shows up, I can mount it, read from it, write etc.
I'm on Linux. How does Windows handle the LBA/CHS info? Is CHS/LBA irrelevant in this day and age?

JPEG EOF vs EOI marker

The JPEG file signature states that the JPEG EOI (End of Image) marker is given by the final FF D9. However I very occasionally come across JPEGs that also have a small amount of data following this EOI (in this example 02 24 ...):
FF D8 FF E1
...
...
FF D9
02 24 02 24 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 48 00 00 00 00 00 00 00 00 00 00 00 00 27 10 00 00 00 00 00 00 00 00 27 10 00 00 00 00 27 10 00 00 27 10 00 00 00 00 00 00 00 00 27 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 41 58 53 21 00 1E 5D 32 00 00 00 00
Does the JPEG standard allow for data following the EOI marker or is this additional binary data corruption?
If this is not corruption what information is encoded in these final bytes?
The JPEG standard only defines a JPEG data stream. That is what goes between the SOI and EOI markets. Anything outside that is outside JPEG.
JPEG does not define a file format so technically there is nothing wrong with having extra data at the begin or the end. However, most decoders will puke if you have extra data before the SOI marker.

hexadecimal string to wireshark pcap

I have a hex string which I want to convert into a Wireshark pcap. Wireshark accepts a hex dump in the following format:
0000 00 00 00 00 00 aa 00 00 00 00 00 01 88 47 00 3e
0010 80 0a 00 00 d1 0a 10 00 89 02 20 01 05 46 00 00
0020 00 01 00 02 04 03 6d 64 31 02 03 6d 61 57 00 00
0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0060 00 00 00 00 00
I have the following stream:
0000000000AA0000000000018847003E800A0000D10A100089022001054600000001000204036D643102036D615700000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
I have been experimenting with od -Ax -tc1 -v command in linux but can not seem to get the correct output. Does anyone know how this can be done?
First put the hex stream into a text file "a.txt",
0000000000AA0000000000018847003E800A0000D10A100089022001054600000001000204036D643102036D615700000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
then convert the text file into a binary file:
xxd -r -p a.txt a.bin
using od and text2pcap to convert the binary file to a pcap file.
od -Ax -tx1 -v a.bin | text2pcap - a.pcap
You can use the following script. This is just a workaround. od and hexdump will interpret 0 as the character '0', and use the byte 48, so you may not get proper output. The script reads two chars for 16 times, and offset is incremented accordingly. Pretty simple to understand I guess:)
#!/bin/bash
off=0
while [ 1 ]
do
printf "%04x " $off
for ((i=0;i<16;i++))
do
read -n 2 a
[ $? -ne 0 ] && echo && exit
echo -n "$a "
done
echo
off=`expr $off + 16`
done <test
where test is the file that holds the stream. I got the following output for your input.
0000 00 00 00 00 00 AA 00 00 00 00 00 01 88 47 00 3E
0010 80 0A 00 00 D1 0A 10 00 89 02 20 01 05 46 00 00
0020 00 01 00 02 04 03 6D 64 31 02 03 6D 61 57 00 00
0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0060 00 00 00 00 00

Resources