My PSSH-Box for PlayReady is below in base64 and hex format.
"playready": {
"pssh-base64": "AAAD4nBzc2gAAAAAmgTweZhAQoarkuZb4IhflQAAA8LCAwAAAQABALgDPABXAFIATQBIAEUAQQBEAEUAUgAgAHgAbQBsAG4AcwA9ACIAaAB0AHQAcAA6AC8ALwBzAGMAaABlAG0AYQBzAC4AbQBpAGMAcgBvAHMAbwBmAHQALgBjAG8AbQAvAEQAUgBNAC8AMgAwADAANwAvADAAMwAvAFAAbABhAHkAUgBlAGEAZAB5AEgAZQBhAGQAZQByACIAIAB2AGUAcgBzAGkAbwBuAD0AIgA0AC4AMAAuADAALgAwACIAPgA8AEQAQQBUAEEAPgA8AFAAUgBPAFQARQBDAFQASQBOAEYATwA+ADwASwBFAFkATABFAE4APgAxADYAPAAvAEsARQBZAEwARQBOAD4APABBAEwARwBJAEQAPgBBAEUAUwBDAFQAUgA8AC8AQQBMAEcASQBEAD4APAAvAFAAUgBPAFQARQBDAFQASQBOAEYATwA+ADwASwBJAEQAPgB2ADgAMgByAHMARwBsADUATgAwAE8AdQBLADgAcgBPAEgAQQBLAHgAcABRAD0APQA8AC8ASwBJAEQAPgA8AEwAQQBfAFUAUgBMAD4AaAB0AHQAcAA6AC8ALwBwAHIALQBrAGUAeQBvAHMALgBsAGkAYwBlAG4AcwBlAGsAZQB5AHMAZQByAHYAZQByAC4AYwBvAG0ALwBjAG8AcgBlAC8AcgBpAGcAaAB0AHMAbQBhAG4AYQBnAGUAcgAuAGEAcwBtAHgAPAAvAEwAQQBfAFUAUgBMAD4APABEAFMAXwBJAEQAPgBWAGwAUgA3AEkAZABzAEkASgBFAHUAUgBkADAANgBMAGEAcQBzADIAagB3AD0APQA8AC8ARABTAF8ASQBEAD4APABDAFUAUwBUAE8ATQBBAFQAVABSAEkAQgBVAFQARQBTACAAeABtAGwAbgBzAD0AIgAiAD4APABDAEkARAA+AHYAOAAyAHIAcwBHAGwANQBOADAATwB1AEsAOAByAE8ASABBAEsAeABwAFEAPQA9ADwALwBDAEkARAA+ADwARABSAE0AVABZAFAARQA+AHMAbQBvAG8AdABoADwALwBEAFIATQBUAFkAUABFAD4APAAvAEMAVQBTAFQATwBNAEEAVABUAFIASQBCAFUAVABFAFMAPgA8AEMASABFAEMASwBTAFUATQA+ADgARQBLAEQAcQBJAHgAUwAwAFEAWQA9ADwALwBDAEgARQBDAEsAUwBVAE0APgA8AC8ARABBAFQAQQA+ADwALwBXAFIATQBIAEUAQQBEAEUAUgA+AA==",
"pssh-hex": "000003e270737368000000009a04f07998404286ab92e65be0885f95000003c2c203000001000100b8033c00570052004d00480045004100440045005200200078006d006c006e0073003d00220068007400740070003a002f002f0073006300680065006d00610073002e006d006900630072006f0073006f00660074002e0063006f006d002f00440052004d002f0032003000300037002f00300033002f0050006c00610079005200650061006400790048006500610064006500720022002000760065007200730069006f006e003d00220034002e0030002e0030002e00300022003e003c0044004100540041003e003c00500052004f00540045004300540049004e0046004f003e003c004b00450059004c0045004e003e00310036003c002f004b00450059004c0045004e003e003c0041004c004700490044003e004100450053004300540052003c002f0041004c004700490044003e003c002f00500052004f00540045004300540049004e0046004f003e003c004b00490044003e007600380032007200730047006c0035004e0030004f0075004b00380072004f00480041004b007800700051003d003d003c002f004b00490044003e003c004c0041005f00550052004c003e0068007400740070003a002f002f00700072002d006b00650079006f0073002e006c006900630065006e00730065006b00650079007300650072007600650072002e0063006f006d002f0063006f00720065002f007200690067006800740073006d0061006e0061006700650072002e00610073006d0078003c002f004c0041005f00550052004c003e003c00440053005f00490044003e0056006c005200370049006400730049004a004500750052006400300036004c0061007100730032006a0077003d003d003c002f00440053005f00490044003e003c0043005500530054004f004d004100540054005200490042005500540045005300200078006d006c006e0073003d00220022003e003c004300490044003e007600380032007200730047006c0035004e0030004f0075004b00380072004f00480041004b007800700051003d003d003c002f004300490044003e003c00440052004d0054005900500045003e0073006d006f006f00740068003c002f00440052004d0054005900500045003e003c002f0043005500530054004f004d0041005400540052004900420055005400450053003e003c0043004800450043004b00530055004d003e00380045004b00440071004900780053003000510059003d003c002f0043004800450043004b00530055004d003e003c002f0044004100540041003e003c002f00570052004d004800450041004400450052003e00"
}
I use mspr:pro tag in the manifest.
As far as I understand, when using mspr:pro , you enter PlayReady Object not PSSH-Box ( for PSSH-BOX cenc:ppsh tag using in manifest ).
How can I build Playready Object from PSSH ?
Just strip the PSSH box header.
PSSH box syntax:
aligned(8) class ProtectionSystemSpecificHeaderBox extends FullBox(‘pssh’, version, flags=0)
{
unsigned int(8)[16] SystemID;
if (version > 0)
{
unsigned int(32) KID_count;
{
unsigned int(8)[16] KID;
} [KID_count]
}
unsigned int(32) DataSize; unsigned
int(8)[DataSize] Data;
}
PSSH hex from your example:
// PSSH, big endian
00 00 03 E2 70 73 73 68 // box header, 994 bytes "pssh"
00 00 00 00 // full box header, version 0, flags 0
9A 04 F0 79 98 40 42 86 AB 92 E6 5B E0 88 5F 95 // PlayReady system ID
00 00 03 C2 // data size, 962 bytes
// PlayReady Object, little-endian
C2 03 00 00 // object length, 962 bytes
01 00 // record count, 1
01 00 // record type, 1 - PRH
B8 03 // record length, 952 bytes
3C 00 57 00 52 [...] // record value, "<WRMHEADER xmlns=..."
You can also decode this online using: https://tools.axinom.com/decoders/PsshBox
Reference:
ISO/IEC 23001-7
PlayReady Header Specification
So I've got really big .bin files with bytes written in them. They have 96-bit numbers written in them as two's complement numbers (still no ASCII, only bytes). Now I have to write an assembly program to sort the numbers in this files and save it to another file (don't ask why, assembly class). I've done it for a file with 32k numbers, like this:
./main < inSort32Kx96b.bin > XD.bin
diff outSort32Kx96b.bin XD.bin
The file outSort32Kx96b.bin is given to me by my teacher. So now diff doesn't output anything, they are identical (I can check that with hexdump or mcview). But I got another file inSort1Kx96b.bin. And I also sort it. But then diff says:
Binary files outSort1Kx96b.bin and XD.bin differ
#Edit:
cmp gave the number of byte where those 2 files differ. Now I can see the difference:
So the difference appears from 0x000017C0. Then I don't know now. If they are written as two's complement numbers in little endian order, then which one is bigger, f.e
00 00 00 00 00 00 00 81 00 00 00 00
or
00 00 00 00 00 00 00 7F 00 00 00 00
?
A hexdump shows you the single bytes in big endian order. If a bunch of bytes have to be interpreted as a number, only the byte order is reversed.
little endian big endian (C notation)
AB CD EF = 0xEFCDAB
01 02 03 04 05 06 07 08 09 10 11 12 = 0x121110090807060504030201
Let's translate your examples to big endian order:
0x000000008100000000000000
0x000000007F00000000000000
You can see that the first number is bigger.
"Two's complement number" is not a very clear expression. Better is "signed integer". The sign which shows whether the number is positive or negativ, is the very first bit of the number. This bit can be found at the begginning of a big endian number and the end of a little endian number.
Positive:
00 00 00 00 00 00 00 81 00 00 00 00 = 0x000000008100000000000000
00 00 00 00 00 00 00 81 00 00 00 10 = 0x100000008100000000000000
00 00 00 00 00 00 00 81 00 00 00 7F = 0x7F0000008100000000000000
Negativ:
00 00 00 00 00 00 00 81 00 00 00 80 = 0x800000008100000000000000
00 00 00 00 00 00 00 81 00 00 00 CD = 0xCD0000008100000000000000
00 00 00 00 00 00 00 81 00 00 00 F0 = 0xF00000008100000000000000
Just curious. This obviously isn't a very good solution for actual programming, but say I wanted to make an executable in Bless (a hex editor).
My architecture is x86. What's a very simple program I can make? A hello world? An infinite loop? Similar to this question, but in Linux.
Decompile a NASM hello world and understand every byte in it
Version of this answer with a nice TOC and more content: http://www.cirosantilli.com/elf-hello-world (hitting the 30k char limit here)
Standards
ELF is specified by the LSB:
core generic: http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/elf-generic.html
core AMD64: http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-AMD64/LSB-Core-AMD64/book1.html
The LSB basically links to other standards with minor extensions, in particular:
generic (both by SCO):
System V ABI 4.1 (1997) http://www.sco.com/developers/devspecs/gabi41.pdf, no 64 bit, although a magic number is reserved for it. Same for core files.
System V ABI Update DRAFT 17 (2003) http://www.sco.com/developers/gabi/2003-12-17/contents.html, adds 64 bit. Only updates chapters 4 and 5 of the previous document: the others remain valid and are still referenced.
architecture specific:
IA-32: http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-IA32/LSB-Core-IA32/elf-ia32.html, points mostly to http://www.sco.com/developers/devspecs/abi386-4.pdf
AMD64: http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-AMD64/LSB-Core-AMD64/elf-amd64.html, points mostly to http://www.x86-64.org/documentation/abi.pdf
A handy summary can be found at:
man elf
Its structure can be examined in a human readable way via utilities like readelf and objdump.
Generate the example
Let's break down a minimal runnable Linux x86-64 example:
section .data
hello_world db "Hello world!", 10
hello_world_len equ $ - hello_world
section .text
global _start
_start:
mov rax, 1
mov rdi, 1
mov rsi, hello_world
mov rdx, hello_world_len
syscall
mov rax, 60
mov rdi, 0
syscall
Compiled with:
nasm -w+all -f elf64 -o 'hello_world.o' 'hello_world.asm'
ld -o 'hello_world.out' 'hello_world.o'
Versions:
NASM 2.10.09
Binutils version 2.24 (contains ld)
Ubuntu 14.04
We don't use a C program as that would complicate the analysis, that will be level 2 :-)
Hexdumps
hd hello_world.o
hd hello_world.out
Output at: https://gist.github.com/cirosantilli/7b03f6df2d404c0862c6
Global file structure
An ELF file contains the following parts:
ELF header. Points to the position of the section header table and the program header table.
Section header table (optional on executable). Each has e_shnum section headers, each pointing to the position of a section.
N sections, with N <= e_shnum (optional on executable)
Program header table (only on executable). Each has e_phnum program headers, each pointing to the position of a segment.
N segments, with N <= e_phnum (optional on executable)
The order of those parts is not fixed: the only fixed thing is the ELF header that must be the first thing on the file: Generic docs say:
ELF header
The easiest way to observe the header is:
readelf -h hello_world.o
readelf -h hello_world.out
Output at: https://gist.github.com/cirosantilli/7b03f6df2d404c0862c6
Bytes in the object file:
00000000 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 |.ELF............|
00000010 01 00 3e 00 01 00 00 00 00 00 00 00 00 00 00 00 |..>.............|
00000020 00 00 00 00 00 00 00 00 40 00 00 00 00 00 00 00 |........#.......|
00000030 00 00 00 00 40 00 00 00 00 00 40 00 07 00 03 00 |....#.....#.....|
Executable:
00000000 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 |.ELF............|
00000010 02 00 3e 00 01 00 00 00 b0 00 40 00 00 00 00 00 |..>.......#.....|
00000020 40 00 00 00 00 00 00 00 10 01 00 00 00 00 00 00 |#...............|
00000030 00 00 00 00 40 00 38 00 02 00 40 00 06 00 03 00 |....#.8...#.....|
Structure represented:
typedef struct {
unsigned char e_ident[EI_NIDENT];
Elf64_Half e_type;
Elf64_Half e_machine;
Elf64_Word e_version;
Elf64_Addr e_entry;
Elf64_Off e_phoff;
Elf64_Off e_shoff;
Elf64_Word e_flags;
Elf64_Half e_ehsize;
Elf64_Half e_phentsize;
Elf64_Half e_phnum;
Elf64_Half e_shentsize;
Elf64_Half e_shnum;
Elf64_Half e_shstrndx;
} Elf64_Ehdr;
Manual breakdown:
0 0: EI_MAG = 7f 45 4c 46 = 0x7f 'E', 'L', 'F': ELF magic number
0 4: EI_CLASS = 02 = ELFCLASS64: 64 bit elf
0 5: EI_DATA = 01 = ELFDATA2LSB: big endian data
0 6: EI_VERSION = 01: format version
0 7: EI_OSABI (only in 2003 Update) = 00 = ELFOSABI_NONE: no extensions.
0 8: EI_PAD = 8x 00: reserved bytes. Must be set to 0.
1 0: e_type = 01 00 = 1 (big endian) = ET_REl: relocatable format
On the executable it is 02 00 for ET_EXEC.
1 2: e_machine = 3e 00 = 62 = EM_X86_64: AMD64 architecture
1 4: e_version = 01 00 00 00: must be 1
1 8: e_entry = 8x 00: execution address entry point, or 0 if not applicable like for the object file since there is no entry point.
On the executable, it is b0 00 40 00 00 00 00 00. TODO: what else can we set this to? The kernel seems to put the IP directly on that value, it is not hardcoded.
2 0: e_phoff = 8x 00: program header table offset, 0 if not present.
40 00 00 00 on the executable, i.e. it starts immediately after the ELF header.
2 8: e_shoff = 40 7x 00 = 0x40: section header table file offset, 0 if not present.
3 0: e_flags = 00 00 00 00 TODO. Arch specific.
3 4: e_ehsize = 40 00: size of this elf header. TODO why this field? How can it vary?
3 6: e_phentsize = 00 00: size of each program header, 0 if not present.
38 00 on executable: it is 56 bytes long
3 8: e_phnum = 00 00: number of program header entries, 0 if not present.
02 00 on executable: there are 2 entries.
3 A: e_shentsize and e_shnum = 40 00 07 00: section header size and number of entries
3 E: e_shstrndx (Section Header STRing iNDeX) = 03 00: index of the .shstrtab section.
Section header table
Array of Elf64_Shdr structs.
Each entry contains metadata about a given section.
e_shoff of the ELF header gives the starting position, 0x40 here.
e_shentsize and e_shnum from the ELF header say that we have 7 entries, each 0x40 bytes long.
So the table takes bytes from 0x40 to 0x40 + 7 + 0x40 - 1 = 0x1FF.
Some section names are reserved for certain section types: http://www.sco.com/developers/gabi/2003-12-17/ch4.sheader.html#special_sections e.g. .text requires a SHT_PROGBITS type and SHF_ALLOC + SHF_EXECINSTR
readelf -S hello_world.o:
There are 7 section headers, starting at offset 0x40:
Section Headers:
[Nr] Name Type Address Offset
Size EntSize Flags Link Info Align
[ 0] NULL 0000000000000000 00000000
0000000000000000 0000000000000000 0 0 0
[ 1] .data PROGBITS 0000000000000000 00000200
000000000000000d 0000000000000000 WA 0 0 4
[ 2] .text PROGBITS 0000000000000000 00000210
0000000000000027 0000000000000000 AX 0 0 16
[ 3] .shstrtab STRTAB 0000000000000000 00000240
0000000000000032 0000000000000000 0 0 1
[ 4] .symtab SYMTAB 0000000000000000 00000280
00000000000000a8 0000000000000018 5 6 4
[ 5] .strtab STRTAB 0000000000000000 00000330
0000000000000034 0000000000000000 0 0 1
[ 6] .rela.text RELA 0000000000000000 00000370
0000000000000018 0000000000000018 4 2 4
Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), l (large)
I (info), L (link order), G (group), T (TLS), E (exclude), x (unknown)
O (extra OS processing required) o (OS specific), p (processor specific)
struct represented by each entry:
typedef struct {
Elf64_Word sh_name;
Elf64_Word sh_type;
Elf64_Xword sh_flags;
Elf64_Addr sh_addr;
Elf64_Off sh_offset;
Elf64_Xword sh_size;
Elf64_Word sh_link;
Elf64_Word sh_info;
Elf64_Xword sh_addralign;
Elf64_Xword sh_entsize;
} Elf64_Shdr;
Sections
Index 0 section
Contained in bytes 0x40 to 0x7F.
The first section is always magic: http://www.sco.com/developers/gabi/2003-12-17/ch4.sheader.html says:
If the number of sections is greater than or equal to SHN_LORESERVE (0xff00), e_shnum has the value SHN_UNDEF (0) and the actual number of section header table entries is contained in the sh_size field of the section header at index 0 (otherwise, the sh_size member of the initial entry contains 0).
There are also other magic sections detailed in Figure 4-7: Special Section Indexes.
SHT_NULL
In index 0, SHT_NULL is mandatory. Are there any other uses for it: What is the use of the SHT_NULL section in ELF? ?
.data section
.data is section 1:
00000080 01 00 00 00 01 00 00 00 03 00 00 00 00 00 00 00 |................|
00000090 00 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00 |................|
000000a0 0d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
000000b0 04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
80 0: sh_name = 01 00 00 00: index 1 in the .shstrtab string table
Here, 1 says the name of this section starts at the first character of that section, and ends at the first NUL character, making up the string .data.
.data is one of the section names which has a predefined meaning http://www.sco.com/developers/gabi/2003-12-17/ch4.strtab.html
These sections hold initialized data that contribute to the program's memory image.
80 4: sh_type = 01 00 00 00: SHT_PROGBITS: the section content is not specified by ELF, only by how the program interprets it. Normal since a .data section.
80 8: sh_flags = 03 7x 00: SHF_ALLOC and SHF_EXECINSTR: http://www.sco.com/developers/gabi/2003-12-17/ch4.sheader.html#sh_flags, as required from a .data section
90 0: sh_addr = 8x 00: in what virtual address the section will be placed during execution, 0 if not placed
90 8: sh_offset = 00 02 00 00 00 00 00 00 = 0x200: number of bytes from the start of the program to the first byte in this section
a0 0: sh_size = 0d 00 00 00 00 00 00 00
If we take 0xD bytes starting at sh_offset 200, we see:
00000200 48 65 6c 6c 6f 20 77 6f 72 6c 64 21 0a 00 |Hello world!.. |
AHA! So our "Hello world!" string is in the data section like we told it to be on the NASM.
Once we graduate from hd, we will look this up like:
readelf -x .data hello_world.o
which outputs:
Hex dump of section '.data':
0x00000000 48656c6c 6f20776f 726c6421 0a Hello world!.
NASM sets decent properties for that section because it treats .data magically: http://www.nasm.us/doc/nasmdoc7.html#section-7.9.2
Also note that this was a bad section choice: a good C compiler would put the string in .rodata instead, because it is read-only and it would allow for further OS optimizations.
a0 8: sh_link and sh_info = 8x 0: do not apply to this section type. http://www.sco.com/developers/gabi/2003-12-17/ch4.sheader.html#special_sections
b0 0: sh_addralign = 04 = TODO: why is this alignment necessary? Is it only for sh_addr, or also for symbols inside sh_addr?
b0 8: sh_entsize = 00 = the section does not contain a table. If != 0, it means that the section contains a table of fixed size entries. In this file, we see from the readelf output that this is the case for the .symtab and .rela.text sections.
.text section
Now that we've done one section manually, let's graduate and use the readelf -S of the other sections.
[Nr] Name Type Address Offset
Size EntSize Flags Link Info Align
[ 2] .text PROGBITS 0000000000000000 00000210
0000000000000027 0000000000000000 AX 0 0 16
.text is executable but not writable: if we try to write to it Linux segfaults. Let's see if we really have some code there:
objdump -d hello_world.o
gives:
hello_world.o: file format elf64-x86-64
Disassembly of section .text:
0000000000000000 <_start>:
0: b8 01 00 00 00 mov $0x1,%eax
5: bf 01 00 00 00 mov $0x1,%edi
a: 48 be 00 00 00 00 00 movabs $0x0,%rsi
11: 00 00 00
14: ba 0d 00 00 00 mov $0xd,%edx
19: 0f 05 syscall
1b: b8 3c 00 00 00 mov $0x3c,%eax
20: bf 00 00 00 00 mov $0x0,%edi
25: 0f 05 syscall
If we grep b8 01 00 00 on the hd, we see that this only occurs at 00000210, which is what the section says. And the Size is 27, which matches as well. So we must be talking about the right section.
This looks like the right code: a write followed by an exit.
The most interesting part is line a which does:
movabs $0x0,%rsi
to pass the address of the string to the system call. Currently, the 0x0 is just a placeholder. After linking happens, it will be modified to contain:
4000ba: 48 be d8 00 60 00 00 movabs $0x6000d8,%rsi
This modification is possible because of the data of the .rela.text section.
SHT_STRTAB
Sections with sh_type == SHT_STRTAB are called string tables.
They hold a null separated array of strings.
Such sections are used by other sections when string names are to be used. The using section says:
which string table they are using
what is the index on the target string table where the string starts
So for example, we could have a string table containing: TODO: does it have to start with \0?
Data: \0 a b c \0 d e f \0
Index: 0 1 2 3 4 5 6 7 8
And if another section wants to use the string d e f, they have to point to index 5 of this section (letter d).
Notable string table sections:
.shstrtab
.strtab
.shstrtab
Section type: sh_type == SHT_STRTAB.
Common name: section header string table.
The section name .shstrtab is reserved. The standard says:
This section holds section names.
This section gets pointed to by the e_shstrnd field of the ELF header itself.
String indexes of this section are are pointed to by the sh_name field of section headers, which denote strings.
This section does not have SHF_ALLOC marked, so it will not appear on the executing program.
readelf -x .shstrtab hello_world.o
Gives:
Hex dump of section '.shstrtab':
0x00000000 002e6461 7461002e 74657874 002e7368 ..data..text..sh
0x00000010 73747274 6162002e 73796d74 6162002e strtab..symtab..
0x00000020 73747274 6162002e 72656c61 2e746578 strtab..rela.tex
0x00000030 7400 t.
The data in this section has a fixed format: http://www.sco.com/developers/gabi/2003-12-17/ch4.strtab.html
If we look at the names of other sections, we see that they all contain numbers, e.g. the .text section is number 7.
Then each string ends when the first NUL character is found, e.g. character 12 is \0 just after .text\0.
.symtab
Section type: sh_type == SHT_SYMTAB.
Common name: symbol table.
First the we note that:
sh_link = 5
sh_info = 6
For SHT_SYMTAB sections, those numbers mean that:
strings that give symbol names are in section 5, .strtab
the relocation data is in section 6, .rela.text
A good high level tool to disassemble that section is:
nm hello_world.o
which gives:
0000000000000000 T _start
0000000000000000 d hello_world
000000000000000d a hello_world_len
This is however a high level view that omits some types of symbols and in which the symbol types . A more detailed disassembly can be obtained with:
readelf -s hello_world.o
which gives:
Symbol table '.symtab' contains 7 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
1: 0000000000000000 0 FILE LOCAL DEFAULT ABS hello_world.asm
2: 0000000000000000 0 SECTION LOCAL DEFAULT 1
3: 0000000000000000 0 SECTION LOCAL DEFAULT 2
4: 0000000000000000 0 NOTYPE LOCAL DEFAULT 1 hello_world
5: 000000000000000d 0 NOTYPE LOCAL DEFAULT ABS hello_world_len
6: 0000000000000000 0 NOTYPE GLOBAL DEFAULT 2 _start
The binary format of the table is documented at http://www.sco.com/developers/gabi/2003-12-17/ch4.symtab.html
The data is:
readelf -x .symtab hello_world.o
Which gives:
Hex dump of section '.symtab':
0x00000000 00000000 00000000 00000000 00000000 ................
0x00000010 00000000 00000000 01000000 0400f1ff ................
0x00000020 00000000 00000000 00000000 00000000 ................
0x00000030 00000000 03000100 00000000 00000000 ................
0x00000040 00000000 00000000 00000000 03000200 ................
0x00000050 00000000 00000000 00000000 00000000 ................
0x00000060 11000000 00000100 00000000 00000000 ................
0x00000070 00000000 00000000 1d000000 0000f1ff ................
0x00000080 0d000000 00000000 00000000 00000000 ................
0x00000090 2d000000 10000200 00000000 00000000 -...............
0x000000a0 00000000 00000000 ........
The entries are of type:
typedef struct {
Elf64_Word st_name;
unsigned char st_info;
unsigned char st_other;
Elf64_Half st_shndx;
Elf64_Addr st_value;
Elf64_Xword st_size;
} Elf64_Sym;
Like in the section table, the first entry is magical and set to a fixed meaningless values.
STT_FILE
Entry 1 has ELF64_R_TYPE == STT_FILE. ELF64_R_TYPE is continued inside of st_info.
Byte analysis:
10 8: st_name = 01000000 = character 1 in the .strtab, which until the following \0 makes hello_world.asm
This piece of information file may be used by the linker to decide on which segment sections go.
10 12: st_info = 04
Bits 0-3 = ELF64_R_TYPE = Type = 4 = STT_FILE: the main purpose of this entry is to use st_name to indicate the name of the file which generated this object file.
Bits 4-7 = ELF64_ST_BIND = Binding = 0 = STB_LOCAL. Required value for STT_FILE.
10 13: st_shndx = Symbol Table Section header Index = f1ff = SHN_ABS. Required for STT_FILE.
20 0: st_value = 8x 00: required for value for STT_FILE
20 8: st_size = 8x 00: no allocated size
Now from the readelf, we interpret the others quickly.
STT_SECTION
There are two such entries, one pointing to .data and the other to .text (section indexes 1 and 2).
Num: Value Size Type Bind Vis Ndx Name
2: 0000000000000000 0 SECTION LOCAL DEFAULT 1
3: 0000000000000000 0 SECTION LOCAL DEFAULT 2
TODO what is their purpose?
STT_NOTYPE
Then come the most important symbols:
Num: Value Size Type Bind Vis Ndx Name
4: 0000000000000000 0 NOTYPE LOCAL DEFAULT 1 hello_world
5: 000000000000000d 0 NOTYPE LOCAL DEFAULT ABS hello_world_len
6: 0000000000000000 0 NOTYPE GLOBAL DEFAULT 2 _start
hello_world string is in the .data section (index 1). It's value is 0: it points to the first byte of that section.
_start is marked with GLOBAL visibility since we wrote:
global _start
in NASM. This is necessary since it must be seen as the entry point. Unlike in C, by default NASM labels are local.
SHN_ABS
hello_world_len points to the special st_shndx == SHN_ABS == 0xF1FF.
0xF1FF is chosen so as to not conflict with other sections.
st_value == 0xD == 13 which is the value we have stored there on the assembly: the length of the string Hello World!.
This means that relocation will not affect this value: it is a constant.
This is small optimization that our assembler does for us and which has ELF support.
If we had used the address of hello_world_len anywhere, the assembler would not have been able to mark it as SHN_ABS, and the linker would have extra relocation work on it later.
SHT_SYMTAB on the executable
By default, NASM places a .symtab on the executable as well.
This is only used for debugging. Without the symbols, we are completely blind, and must reverse engineer everything.
You can strip it with objcopy, and the executable will still run. Such executables are called stripped executables.
.strtab
Holds strings for the symbol table.
This section has sh_type == SHT_STRTAB.
It is pointed to by sh_link == 5 of the .symtab section.
readelf -x .strtab hello_world.o
Gives:
Hex dump of section '.strtab':
0x00000000 0068656c 6c6f5f77 6f726c64 2e61736d .hello_world.asm
0x00000010 0068656c 6c6f5f77 6f726c64 0068656c .hello_world.hel
0x00000020 6c6f5f77 6f726c64 5f6c656e 005f7374 lo_world_len._st
0x00000030 61727400 art.
This implies that it is an ELF level limitation that global variables cannot contain NUL characters.
.rela.text
Section type: sh_type == SHT_RELA.
Common name: relocation section.
.rela.text holds relocation data which says how the address should be modified when the final executable is linked. This points to bytes of the text area that must be modified when linking happens to point to the correct memory locations.
Basically, it translates the object text containing the placeholder 0x0 address:
a: 48 be 00 00 00 00 00 movabs $0x0,%rsi
11: 00 00 00
to the actual executable code containing the final 0x6000d8:
4000ba: 48 be d8 00 60 00 00 movabs $0x6000d8,%rsi
4000c1: 00 00 00
It was pointed to by sh_info = 6 of the .symtab section.
readelf -r hello_world.o gives:
Relocation section '.rela.text' at offset 0x3b0 contains 1 entries:
Offset Info Type Sym. Value Sym. Name + Addend
00000000000c 000200000001 R_X86_64_64 0000000000000000 .data + 0
The section does not exist in the executable.
The actual bytes are:
00000370 0c 00 00 00 00 00 00 00 01 00 00 00 02 00 00 00 |................|
00000380 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
The struct represented is:
typedef struct {
Elf64_Addr r_offset;
Elf64_Xword r_info;
Elf64_Sxword r_addend;
} Elf64_Rela;
So:
370 0: r_offset = 0xC: address into the .text whose address this relocation will modify
370 8: r_info = 0x200000001. Contains 2 fields:
ELF64_R_TYPE = 0x1: meaning depends on the exact architecture.
ELF64_R_SYM = 0x2: index of the section to which the address points, so .data which is at index 2.
The AMD64 ABI says that type 1 is called R_X86_64_64 and that it represents the operation S + A where:
S: the value of the symbol on the object file, here 0 because we point to the 00 00 00 00 00 00 00 00 of movabs $0x0,%rsi
A: the addend, present in field r_added
This address is added to the section on which the relocation operates.
This relocation operation acts on a total 8 bytes.
380 0: r_addend = 0
So in our example we conclude that the new address will be: S + A = .data + 0, and thus the first thing in the data section.
Program header table
Only appears in the executable.
Contains information of how the executable should be put into the process virtual memory.
The executable is generated from object files by the linker. The main jobs that the linker does are:
determine which sections of the object files will go into which segments of the executable.
In Binutils, this comes down to parsing a linker script, and dealing with a bunch of defaults.
You can get the linker script used with ld --verbose, and set a custom one with ld -T.
do relocation on text sections. This depends on how the multiple sections are put into memory.
readelf -l hello_world.out gives:
Elf file type is EXEC (Executable file)
Entry point 0x4000b0
There are 2 program headers, starting at offset 64
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
LOAD 0x0000000000000000 0x0000000000400000 0x0000000000400000
0x00000000000000d7 0x00000000000000d7 R E 200000
LOAD 0x00000000000000d8 0x00000000006000d8 0x00000000006000d8
0x000000000000000d 0x000000000000000d RW 200000
Section to Segment mapping:
Segment Sections...
00 .text
01 .data
On the ELF header, e_phoff, e_phnum and e_phentsize told us that there are 2 program headers, which start at 0x40 and are 0x38 bytes long each, so they are:
00000040 01 00 00 00 05 00 00 00 00 00 00 00 00 00 00 00 |................|
00000050 00 00 40 00 00 00 00 00 00 00 40 00 00 00 00 00 |..#.......#.....|
00000060 d7 00 00 00 00 00 00 00 d7 00 00 00 00 00 00 00 |................|
00000070 00 00 20 00 00 00 00 00 |.. ..... |
and:
00000070 01 00 00 00 06 00 00 00 | ........|
00000080 d8 00 00 00 00 00 00 00 d8 00 60 00 00 00 00 00 |..........`.....|
00000090 d8 00 60 00 00 00 00 00 0d 00 00 00 00 00 00 00 |..`.............|
000000a0 0d 00 00 00 00 00 00 00 00 00 20 00 00 00 00 00 |.......... .....|
Structure represented http://www.sco.com/developers/gabi/2003-12-17/ch5.pheader.html:
typedef struct {
Elf64_Word p_type;
Elf64_Word p_flags;
Elf64_Off p_offset;
Elf64_Addr p_vaddr;
Elf64_Addr p_paddr;
Elf64_Xword p_filesz;
Elf64_Xword p_memsz;
Elf64_Xword p_align;
} Elf64_Phdr;
Breakdown of the first one:
40 0: p_type = 01 00 00 00 = PT_LOAD: TODO. I think it means it will be actually loaded into memory. Other types may not necessarily be.
40 4: p_flags = 05 00 00 00 = execute and read permissions, no write TODO
40 8: p_offset = 8x 00 TODO: what is this? Looks like offsets from the beginning of segments. But this would mean that some segments are intertwined? It is possible to play with it a bit with: gcc -Wl,-Ttext-segment=0x400030 hello_world.c
50 0: p_vaddr = 00 00 40 00 00 00 00 00: initial virtual memory address to load this segment to
50 8: p_paddr = 00 00 40 00 00 00 00 00: initial physical address to load in memory. Only matters for systems in which the program can set it's physical address. Otherwise, as in System V like systems, can be anything. NASM seems to just copy p_vaddrr
60 0: p_filesz = d7 00 00 00 00 00 00 00: TODO vs p_memsz
60 8: p_memsz = d7 00 00 00 00 00 00 00: TODO
70 0: p_align = 00 00 20 00 00 00 00 00: 0 or 1 mean no alignment required TODO what does that mean? otherwise redundant with other fields
The second is analogous.
Then the:
Section to Segment mapping:
section of the readelf tells us that:
0 is the .text segment. Aha, so this is why it is executable, and not writable
1 is the .data segment.
As mentioned in my comment, you will essentially be writing your own elf-header for the executable eliminating the unneeded sections. There are still several required sections. The documentation at Muppetlabs-TinyPrograms does a fair job explaining this process. For fun, here are a couple of examples:
The equivalent of /bin/true (45 bytes):
00000000 7F 45 4C 46 01 00 00 00 00 00 00 00 00 00 49 25 |.ELF..........I%|
00000010 02 00 03 00 1A 00 49 25 1A 00 49 25 04 00 00 00 |......I%..I%....|
00000020 5B 5F F2 AE 40 22 5F FB CD 80 20 00 01 |[_..#"_... ..|
0000002d
Your classic 'Hello World!' (160 bytes):
00000000 7f 45 4c 46 01 01 01 03 00 00 00 00 00 00 00 00 |.ELF............|
00000010 02 00 03 00 01 00 00 00 74 80 04 08 34 00 00 00 |........t...4...|
00000020 00 00 00 00 00 00 00 00 34 00 20 00 02 00 28 00 |........4. ...(.|
00000030 00 00 00 00 01 00 00 00 74 00 00 00 74 80 04 08 |........t...t...|
00000040 74 80 04 08 1f 00 00 00 1f 00 00 00 05 00 00 00 |t...............|
00000050 00 10 00 00 01 00 00 00 93 00 00 00 93 90 04 08 |................|
00000060 93 90 04 08 0d 00 00 00 0d 00 00 00 06 00 00 00 |................|
00000070 00 10 00 00 b8 04 00 00 00 bb 01 00 00 00 b9 93 |................|
00000080 90 04 08 ba 0d 00 00 00 cd 80 b8 01 00 00 00 31 |...............1|
00000090 db cd 80 48 65 6c 6c 6f 20 77 6f 72 6c 64 21 0a |...Hello world!.|
000000a0
Don't forget to make them executable...
I made TDataset descendant asynchronous, instead of sleep or ProcessMessages in main thread it works by events from network thread. So when Recordset is ready it calls
procedure TMySqlQuery.OrdinalOnDataReady(Sender: TObject);
begin
AddToLog('OrdinalOnDataReady');
FDataAvailable := true; // used in IsCursorOpen
inherited Open;
if Assigned(FParentOnDataReady) then
FParentOnDataReady(self);
end;
It works, but sometime I have issues with GetRecord by Open called from this thread and DBGrid's GetFieldData called from main thread by DBGrid's DrawCells from Form's ProcessMessages. By logging both functions I see
[17:10:39] RecordToBuffer row 0
[17:10:39] len = 17 buf :
00 E0 10 C3 00 0A 00 00 00 F0 10 C3 00 0B 00 00 | .ïœ.ï“.....ÿ€.ï“....
00 | .
[17:10:40] RecordToBuffer row 1
[17:10:40] len = 17 buf :
00 00 FF C3 00 25 00 00 00 10 11 C3 00 0B 00 00 | ..ÿï“.%.....ï“....
00 | .
[17:10:40] ActiveBuffer
[17:10:40] len = 17 buf :
00 E0 10 C3 00 0A 00 00 00 F0 10 C3 00 0B 00 00 | .ïœ.ï“.....ÿ€.ï“....
00 | .
...
more ActiveBuffer
...
[17:10:40] ActiveBuffer
[17:10:40] len = 17 buf :
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
00 | .
[17:10:40] len = 8 buf :
00 00 00 00 00 00 00 00 | ........
and on break by assert when ActiveBuffer column data is nil I can see that DBGrid attempts to read row higher than GetRecord readed into own internal FBuffers. For example if assertion fired at GetFieldData row 3 - FBuffers filled up to row 2 from total 36 rows available in Recordset. When I debugging step by step GetRecord with F8 there is no errors utnil I press F9 and get assertion at another record.
I'm not quite understand how exactly DBGrid works with TDataset (even stack trace is huge), but can this thread races be solved?
Solution was quite simple: since data in TDataset's FBuffers (from Data.DB) filled with 0 if not initialized it is possible to find filled ActiveBuffer by GetRecord or not by adding yet another marker byte into record and assigning not 0 in GetRecord.
So if DBGrid attempts to read uninitialized data I checking marker in GetFieldData, if 0 result false and exit. Since DBGrid extracting data to same cells more than once I still have it filled with proper data.
It is workaround but it works.
I have a node application which runs on a raspberry pi that keeps track of a bunch of UPnP-players (Sonos), which I would like to be able to control through a physical remote. I have a couple of airmouses, which has small keyboards as well as volume buttons that I would like to use.
I have tried to get a grip on how to read out physical key strokes on a linux machine, and come to the conclusion that I need to read events from the input device, which in my case would be:
/dev/input/by-id/usb-Dell_Dell_QuietKey_Keyboard-event-kbd
How to find the device and stuff like that is not a problem, the real issue is how to interpret the data that you read from it.
I know that you would receive a C struct, like this:
struct input_event {
struct timeval time;
unsigned short type;
unsigned short code;
unsigned int value;
};
But I'm not sure how I would go about reading this from node. If I could run an external app that would be triggered from pre-defined keystrokes, and then invoke an HTTP-request against my node, that would be my second option, a python script or some native daemon. I have however looked at some hotkey-daemons, but none of them worked.
If would of course be nice if I could contain it within node somehow.
EDIT: So I did some testing, and made a simple snippet:
var fs = require('fs');
var buffer = new Buffer(16);
fs.open('/dev/input/by-id/usb-HJT_Air_Mouse-event-kbd', 'r', function (err, fd) {
while (true) {
fs.readSync(fd, buffer, 0, 16, null);
console.log(buffer)
}
});
This outputs something like this (for space):
<Buffer a4 3e 5b 51 ab cf 03 00 04 00 04 00 2c 00 07 00>
<Buffer a4 3e 5b 51 c3 cf 03 00 01 00 39 00 01 00 00 00>
<Buffer a4 3e 5b 51 cb cf 03 00 00 00 00 00 00 00 00 00>
<Buffer a4 3e 5b 51 ba 40 06 00 04 00 04 00 2c 00 07 00>
<Buffer a4 3e 5b 51 cd 40 06 00 01 00 39 00 00 00 00 00>
<Buffer a4 3e 5b 51 d2 40 06 00 00 00 00 00 00 00 00 00>
I realize that the first four bytes are some sort of timestamp, and the following 3 bytes could be something like a micro/millisecond thing.
Another odd thing is that not all keypresses produces output, but a subsequent press might sent twice as much data, and most of the time it starts blasting out data which would stop after subsequent keypresses (or after about 20 seconds or so). I'm not really sure how to interpret that. I have tried to read the source for this daemon https://github.com/baskerville/shkd/blob/master but C is not my strongest language and I can't identify how he handles it (or if it should even be handled). And that daemon didn't even work for me (compiled it on a raspberry pi).
Well, let's have a look at that struct.
struct input_event {
struct timeval time;
unsigned short type;
unsigned short code;
unsigned int value;
};
A struct timeval has this structure:
struct timeval
{
__time_t tv_sec; /* Seconds. */
__suseconds_t tv_usec; /* Microseconds. */
};
The definition of those time types are
typedef signed long time_t;
typedef signed long suseconds_t;
A signed long is 4 bytes (well, not if you just follow the spec, but in practice, it is), so the first 8 bytes are a typestamp. Next, you have a type and a code. Both are short, so in practice, they're 2 bytes each. Now there's just the value left, and that's an int again, which will be four bytes. Also, a compiler could theoretically add padding between the fields here, but I'm pretty sure he won't.
So, first chop the bytes you've read into chunks of 4+4+2+2+4=16 bytes. Each of those chunks is an event. This fits your sample data. Next, extract the values from the buffer (as little endian values because you're on an ARM system – on a normal PC, you'd need big endian) and interpret the values. For instructions on how to do that, read http://www.mjmwired.net/kernel/Documentation/input/event-codes.txt. The values of the constants aren't written down there, but you can usually find those using grep -R NAME_OF_CONSTANT /usr/include.
Let's chop up
<Buffer a4 3e 5b 51 ab cf 03 00 04 00 04 00 2c 00 07 00>
as an example.
<Buffer a4 3e 5b 51 ab cf 03 00 04 00 04 00 2c 00 07 00>
| tv_sec | tv_usec |type |code | value |
tv_sec in hex is 0x515b3ea4 (reversed order because it's little endian), which is 1364934308 in decimal. A simple unix time converter reports that this means 02.04.2013 - 22:25:08. Looks good!
tv_usec is 0x0003cfab=249771, so actually, the event happened 249771 microseconds after that time.
Type is 0x0004=4. /usr/include/linux/input.h tells us that this is a EV_MSC.
Given the type, we can also see the the code, 0x0004=4, means MSC_SCAN.
The value is 0x0007002c. Turns up nowhere in input.h. Hmm.
I think what you're looking for is fs.createReadStream, so you can install some event handlers.
You can parse input events into structs by using the Buffer.readX routines:
var i = 0;
while((buf.length-i) >= 16) {
var event = {
tssec: buf.readUInt32LE(i+0),
tsusec: buf.readUInt32LE(i+4),
type: buf.readUInt16LE(i+8),
code: buf.readUInt16LE(i+10),
value: buf.readUInt32LE(i+12)
};
i += 16;
}