How to specify 3GB using bit shift in python? - python-3.x

I am allocating memory in the Jetson TX2. It has 8GB of RAM.
I need to specify the maximun GPU memory size available for TensorRT.
max_workspace_size_bytes = (has to be an integer)
I have seen some examples using these "values":
1<<20 = 1048576 (decimal)
= 0001 0000 0000 0000 0000
1<<30 = 1073741824
= 0001 0000 0000 0000 0000 0000 0000
But if I have 8GB of RAM, how can "1048576" or "1073741824" represent a part of RAM?
I have used this to allocate 3GB:
3*(10**9)
But I would like to understand the other way of representing a number.

Perhaps you're having a "Gb vs Gib" problem. Usually, 3 Gigas of RAM refers to 3,221,225,472 bytes instead of 3,000,000,000.
The first value is 3 * (2^10)*(2^10)*(2^10), a nice 3 (11) followed by 30 zeros in binary representation, while the second is 3 * (10^3)*(10^3)*(10^3), which is a mess in binary.
This convention of using powers of 2 instead of powers of 10 is the reason why you'll see people writing a 3Gb as 3 << 30:
3 << 30 == 3 * (1 << 10) * (1 << 10) * (1 << 10)
== 3 * (2**10 * 2**10 * 2**10)
== 3 * (2**30)
There's a related question and a good Wikipedia article about this issue if you want to learn more.

You can sum them up.
((1<<30)+(1<<31))
Or bitwise OR them.
((1<<30) | (1<<31))
Or shift a larger value than 1, e.g. 3.
(3<<30)

3GB = 3,221,225,472
1100 0000 0000 0000 0000 0000 0000 0000
3<<30 = 3GB

Related

elf aarch64 golfed with sys_write

To better understand the ELF format and the ARM aarch64, I'm trying to create my elf binary without compilers, just echoing bytes with bash.
Will can see my effort here: http://www.github.com/glaudiston/elf
I have succeeded in achieving a fully working elf with sys_write and sys_exit syscalls for x64.
But for aarch64, it's not working as I expect it to:
# cat make-elf.sh
#!/bin/bash
#
# depends on:
# - elf_fn.sh (github.com/glaudiston/elf)
# - base64 (gnu-coreutils)
#
. elf_fn.sh
instructions="";
instructions="${instructions}\nwrite $(echo -en "hello world\n" | base64 -w0)";
instructions="${instructions}\nexit 3";
write_elf elf "${instructions}";
It generates:
$ xxd elf
00000000: 7f45 4c46 0201 0100 0000 0000 0000 0000 .ELF............
00000010: 0200 b700 0100 0000 7800 0100 0000 0000 ........x.......
00000020: 4000 0000 0000 0000 0000 0000 0000 0000 #...............
00000030: 0000 0000 4000 3800 0100 0000 0000 0000 ....#.8.........
00000040: 0100 0000 0500 0000 0000 0000 0000 0000 ................
00000050: 0000 0100 0000 0000 0000 0000 0000 0000 ................
00000060: 7800 0000 0000 0000 7800 0000 0000 0000 x.......x.......
00000070: 0000 0000 0000 0000 2000 80d2 010c 0058 ........ ......X
00000080: 8201 80d2 0808 80d2 0100 00d4 6000 80d2 ............`...
00000090: a80b 80d2 0100 00d4 6865 6c6c 6f20 776f ........hello wo
000000a0: 726c 640a
$ ./make-elf.sh 0 && ./elf; echo $?
3
$ cat elf | base64 -w0; echo
f0VMRgIBAQAAAAAAAAAAAAIAtwABAAAAeAABAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAEAAOAABAAAAAAAAAAEAAAAFAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAeAAAAAAAAAB4AAAAAAAAAAAAAAAAAAAAIACA0gEMAFiCAYDSCAiA0gEAANRgAIDSqAuA0gEAANRoZWxsbyB3b3JsZAo=
It returns the expected exit code, with no illegal exceptions, but the sys_write call is not printing anything.
Hiding all ELF overhead, we have this:
00000078: 2000 80d2 010c 0058 ......X
00000080: 8201 80d2 0808 80d2 0100 00d4 6000 80d2 ............`...
00000090: a80b 80d2 0100 00d4 6865 6c6c 6f20 776f ........hello wo
000000a0: 726c 640a rld.
The exit call is working as expected, so I can hide it too:
00000078: 2000 80d2 010c 0058 ......X
00000080: 8201 80d2 0808 80d2 0100 00d4 ............
00000090: 6865 6c6c 6f20 776f hello wo
000000a0: 726c 640a rld.
So we have the data hello world.\n starting at position 98. I am very confused about how to do the sys_write call here. In x64 I can set to the next data address that in this case should be 65688(composed of PH_VADDR_V(65536) + ELF_HEADER_SIZE(64) + ELF_BODY_SIZE(32) (without DATA_SECTION)")
To the output fd I do set in r0 the value 1 with 2000 80d2
To the data address I am using 010c that is little endian representation of 0c01 this bits: 00001100000 00001 The last 5 bits are the r1 register, used to data address.
Given I only have 11 bits
Here I've used the LDR (0058) but I've tried MOV (here 80d2) too. With no success
I've tried any value from 0 to 2048 where it starts to reports Illegal instruction and exit code 132.
I think maybe aarch64 does not allow the same trick I've used in x64 to print data without a labeled data section. I'll work on creating it, but this is just a guess and I really want to understand why this is not printing nothing.
So, your string is at absolute address 0x10098 and you need to get this address into the x1 register.
First of all, LDR is not what you want. It is, as the name suggests, a load (read) from memory. You don't want your instruction to access memory at all, it just wants to put the value 0x10098 into the register.
MOV is closer, which writes an immediate value into the register, but the problem is that the immediate is limited to 16 bits, and you need 17. Because instructions are 32 bits, there are only so many bits available for an immediate. My guess is that you overflowed this and ended up changing opcode bits instead, so you encoded a totally different instruction. (Don't guess at encodings! Look them up. This would have shown you the 16-bit limit.)
For getting arbitrary immediate values into a register, the intended approach is a sequence of MOV/MOVK instructions to write 16 bits at a time. Here you would just need two of them:
0: d2801301 mov x1, #0x98 // #152
4: f2a00021 movk x1, #0x1, lsl #16
Though since we are using a extra word, the address of the string will also shift, so you'd have to adjust accordingly.
However, for addresses in particular, AArch64 provides pc-relative address generation instructions, ADR/ADRP. These let you add an immediate value to the current value of the program counter (i.e. the address of the currently executing instruction) and write the result to a register. As a bonus, they allot more bits for the immediate (though you will no longer need them).
Here we can use ADR. Its opcode is 0 at bit 31, and 10000 at bits 24-28. The destination register is bits 0-4, we want 00001. The immediate gets its low two bits at bits 29-30, and the higher bits at 5-23. The ADR instruction will be at absolute address 0x1007c and we want 0x10098, so the displacement is 0x1c = 0b11100. Thus the encoding we want is
0 00 10000 0000000000000000111 00001 = 0x100000e1
Some general tips:
Try writing code with an assembler first, so that you can learn the instruction set and be able to focus on experimenting with what the instructions do, instead of also getting bogged down in how they are encoded. If you want to come back and do the encoding by hand later, fine, but with an assembler you'll also have a way to check your work.
Use a debugger to single-step your program. That would have showed you that your LDR was giving you a totally bogus value and might have been a hint that it didn't do what you think it did.
Use strace to see what system calls your program makes. That would show you (I think, I didn't test) that write does get invoked but with the wrong address.

jumping indices of values in buffer

I am currently testing some things with an accelerometer and its iio buffer and there is something that confuses me.
The sensor does have four different scan elements: x, y, z and a timestamp.
The indices of those values are:
x = 0, y = 1, z = 2 and time = 3. So far so good.
If I enable all available scan elements the order of the entries is set according to the description.
everything enabled:
0000010 f758 011c 3f64 c0b0 be90 0bfe 499f 0004
0000020 f724 0134 3f58 c0b0 3f2f 10ab 499f 0004
But once I have gaps, for example if I disable the scan element for y, the z value jumps onto index 1 and my buffer looks like this:
x, z and time:
0000010 f720 3f70 0000 0000 722a 5c13 4946 0004
0000020 f728 3f74 0000 0000 0958 60c0 4946 0004
z and time:
0000010 3f6c 0000 0000 0000 ca0b 6ef1 48be 0004
0000020 3f44 0000 0000 0000 edf7 739e 48be 0004
only x and z:
0000010 f720 3f48 f748 3f54 f744 3f5c f75c 3f68
0000020 f750 3f78 f738 3f80 f718 3f64 f700 3f50
I could not find further information on this but I am a bit confused and surprised that the scan elements do not respect their given index once the timestamp is activated and there is an index gap. Is this the normal behavior or is this some stuff that the current sensor driver mixes up?

bit representation in python

Hi I have a question about the bit representation in python
when I use bit operation 1<<31, then we can see the bits are
1000 0000 0000 0000 0000 0000 0000 0000
python will print this value as 2147483648
but when I give a variable value like a = -2**31
the bits are also
1000 0000 0000 0000 0000 0000 0000 0000
but python will print -2147483648
so if the bits are the same , how python decide to use 2147483648 or -2147483648 ?
In python integers do not have a limited precision. Which means among other things, that the numbers are not stored in twos compliment binary. The sign is NOT stored in the bit representation of the number.
So all of -2**31, 2**31 and 1<<31 will have the same bit representation for the number. The sign part of the -2**31 is not part of the bitwise representation of the number. The sign is separate.
You can see this if you try this:
>>> bin(5)
'0b101'
>>> bin(-5)
'-0b101'
The representation isn't really the same. You can use int.to_bytes to check it:
(1 << 31).to_bytes(32, 'big', signed=True)
b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x80\x00\x00\x00'
(-2 ** 31).to_bytes(32, 'big', signed=True)
b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x80\x00\x00\x00'
Also, be careful about the - operator, which have the lower priority here:
-2 ** 31 == -(2 ** 31)

n-th bit of the binary representation

I have a problem that asks
"find the K-th bit of the binary representation of an integer N"
where 0 <= K <= 31.
The answer states that when N=1 K=0 , the k-th bit is 1
and also
when N=2 K=1 , the k-th bit is 1 as well. How is this so?
The little endian binary representation of 1 is 0000 0000 0000 0001 and the little endian binary representation of 2 is 0000 0000 0000 0010.

what left and right shift operator and how they works

//((Hello everybody!
i am C# beginner can any one tell me the function of left and right shift operator and their working way w.r.t the following program. I read it somewhere but confuse.
thanks ))
using System;
class clc
{
public static void Main() // the Main method
{
int x = 7, y = 2, z, r;
z = x << y ; //left shift operator
r = x >> y; // right shift operator
Console.WriteLine("\n z={3}\tr={4} ",z,r);
}
}
To understand the shift operations you must understand binary numbers.
Let's take your example for left shift:
z = 7 << 2;
32 bit integer 7 is 0000 0000 0000 0000 0000 0000 0000 0111 in binary. You must move the bits to the left beginning from the right. The bits that are shifted out of either end are discarded.
Shifting it by 1 will result 0000 0000 0000 0000 0000 0000 0000 1110
Shifting it by 1 one more time will result 0000 0000 0000 0000 0000 0000 0001 1100 which is 28 in integer representation.
Read this good wikipedia article Binary number

Resources