Serial communication : Send a list from Python3 to Arduino - python-3.x

I don't manage to deal with my problem even if I have read a lot about it on internet these last few days.
I try to communicate a variable length list from my Python3 program to my Arduino Leonardo.
Actually the length of these lists is variable but there are only three possible length :
first possibility : [0, 0, 1, 176, 1, 0, 0]
second possibility : [0, 1, 11, 255]
third possibility : [0, 2, 0]
(most of the values inside these lists are variables)
My Python3 code :
with Serial(port = port, baudrate=9600, timeout=1, writeTimeout=1) as port_serie :
if port_serie.isOpen() :
for value in Datas_To_Send : #Envoi des données
s = struct.pack('!{0}B'.format(len(Datas_To_Send)), *Datas_To_Send)
port_serie.write(s)
This code sends binary values like this one :
b'\x00\x00\x01\xb0\x01\x00\x00'
(the original list to send was : [0, 0, 1, 176, 1, 0, 0])
The problem is that I absolutely don't know how to find back my original list of values with my Arduino code...
My Arduino code (quite basic) :
void changeSettings() {
if ( Serial.available() > 0 ) {
int byte_read = Serial.read() ;
Serial.println(byte_read, DEC) ;
The output of this code is a pure conversion of each character from the ASCII to the Decimal...
Output (for the binary value I gave as example) :
98
39
92
120
48
48
92
120
48
48
92
120
48
49
92
120
98
48
92
120
48
49
92
120
48
48
92
120
48
48
39
10
Do you have any idea to find the first list back ?
Thank you

It seems you need to transmit either 7, 4 or 3 values, correct?
Are all the values under 256?
So, i would send 1 byte that is either 7, 4 or 3 followed by either 7, 4 or 3 bytes of the list's elements. If any item in your list is greater than 255 and less than 65,536 you'll need to send 2 bytes per element.

Well, I found the solution thanks to #MarkSetchel's and Olmagzar's (Discord user) tips.
Python3 code :
if Datas_To_Send :
long = len(Datas_To_Send)
Datas_To_Send.insert(0, long)
with Serial(port = '/dev/cu.usbmodem14101', baudrate=9600, timeout=1, writeTimeout=1) as port_serie :
if port_serie.isOpen() :
s = struct.pack('!{0}B'.format(len(Datas_To_Send)), *Datas_To_Send)
port_serie.write(s)
port_serie.close()
So I add list length directly at the first position of my list "Datas_To_Send".
Like that I just have to read it first on Arduino's side to know how many items I have to read.
Arduino code :
void changeSettings() {
if (Serial.available() > 0)
{
unsigned char len = Serial.read();
unsigned char Datas[len] ;
for (int i = 0; i < len - 1; i++)
{
unsigned char byte_read = Serial.read();
Datas[i] = byte_read ;
}
}
}

Related

Python 3.8: Bytes to a list of hex

I'm currently banging by head bloody over this problem I have:
I have a method where I receive a byte input of a defined length of 8:
def ConvertToHexAndSend(serialnumber):
if len(serialnumber) != 8:
throw a good exception...
Some good converter code here..
The method shall then take the serialnumber and split it into a list with the size 4.
Example:
serialnumber = b'12345678'
Expected output:
[0x12, 0x34, 0x56, 0x78]
What I have tried so far:
new_list = list(serialnumber) # Gives a list of [49, 50, 51, 52, 53, 54, 55, 56] and then:
first byte = new_list[0] << 8 + new_list[1] # gives some riddish value
and
hex(serialnumber[0]) # Gives '0x32' string
and
first_byte = serialnumber[0] + serialnumber[1] # Gives int 99
But no success so far.. any idea?
You want something like:
result = list(bytes.fromhex(serialnumber.decode('ascii')))
Your bytes are represent a hex-string. So use the bytes.fromhex to read that (converting the bytes to str first), and you can use list to get a list of int objects from the new resulting bytes object

Why is Node.js complaining ERR_OUT_OF_RANGE if the offset is within the file's range?

I'm trying to overwrite some of the contents of a pre-generated 1GB file via a 4 byte buffer at an iterating offset.
Best I can tell, I'm using the correct flag:
const fd = fs.openSync(dataPath, "r+") // also tried "a+"
And the file size looks within range:
let stats = fs.statSync(dataPath)
let fileSizeInBytes = stats["size"]
let fileSizeInMegabytes = fileSizeInBytes / 1000000
console.log("fileSizeInMegabytes", fileSizeInMegabytes) // => fileSizeInMegabytes 1000
But when I try to write the updates:
const bufferSize = 74
let pointer = (timestampSet.size * 4) + 4
for (let j = 0; j < timestampSet.size; j++) {
pointer += mapIterator.next().value * bufferSize
const pointerBuffer = Buffer.alloc(4)
pointerBuffer.writeUInt32BE(pointer, 0) // <Buffer 00 2e 87 e4>
console.log("writing", pointerBuffer, "to file", dataPath, "at offset", j * 4)
// writing <Buffer 00 2e 87 e4> to file E://data.odat at offset 4
fs.writeSync(fd, pointerBuffer, j * 4, 4)
}
fs.close(fd).then(() => {
console.log("write stream closed")
})
iterateProcess()
I get the error:
RangeError [ERR_OUT_OF_RANGE]: The value of "length" is out of range. It must be <= 0. Received 4
Why is this error happening if the file has the correct size and the correct flag is used?
You seem to misinterpret the writeSync arguments. offset refers to the position in the buffer, not in the file. For the address in the file, use position.
The error message comes from the fact that the system cannot find 4 bytes in the buffer starting from the location in the buffer you specified.
Your code should read:
fs.writeSync(fd, pointerBuffer, 0, 4, j*4)
From the docs:
offset determines the part of the buffer to be written, and length is an integer specifying the number of bytes to write.
position refers to the offset from the beginning of the file where this data should be written. If typeof position !== 'number', the data will be written at the current position. [..]

How to convert to ascii using node.js

I am trying to convert length message to ascii.
My length message is like
var ll = "0170";
In node js , is there some kind of function which converts into ascii?
Please help?
Here's a simple function(ES6) which converts a string into ASCII characters using charCodeAt()
const toAscii = (string) => string.split('').map(char=>char.charCodeAt(0)).join(" ")
console.log(toAscii("Hello, World"))
Output:
-> 72 101 108 108 111 44 32 87 111 114 108 100
You could create a prototype function aswell. There are many solutions :)
you can't have an ascii code for a whole string.
An ascii code is an integer value for a character, not a string. Then for your string "0170" you will get 4 ascii codes
you can display these ascii codes like this
var str = "0170";
for (var i = 0, len = str.length; i < len; i++) {
console.log(str[i].charCodeAt());
}
Ouput : 48 49 55 48
use charCodeAt() function to covert asscii format.
var ll = "0170";
function ascii (a) { return a.charCodeAt(); }
console.log(ascii(ll[0]),ascii(ll[1]), ascii(ll[2]), ascii(ll[3]) )
result:
48 49 55 48

C++ converting hex number to human readable string representation

I have a char[32] that consist of ASCII characters, some of which may be non printable characters, however, they are all in valid range of 0 to 255.
Let's say, it contains these numbers:
{ 9A 2E 5C 66 26 3D 5A 88 76 26 F1 09 B5 32 DE 10 91 3E 68 2F EA 7B C9 52 66 23 9D 77 16 BB C9 42 }
I'd like to print out or store in std::string as "9A2E5C66263D5A887626F109B532DE10913E682FEA7BC95266239D7716BBC942", however, if I print it out using printf or sprintf, it will yield the ASCII representative of each number, and instead it will print out "ö.\f&=Zàv&Ò µ2fië>h/Í{…Rf#ùwª…B", which is the correct ASCII character representation, since: ö = 9a, . = 2e, ...
How do I reliably get the string representation of the hex numbers? ie: I'd expect a char[64] which contains "9A2E5C66263D5A887626F109B532DE10913E682FEA7BC95266239D7716BBC942" instead.
Thanks!
void bytesToHex(char* dest, char* src, int size) {
for (unsigned int i = 0; i < size; i++) {
sprintf(&dest[i * 2], "%02x", src[i]);
}
}
You'd have to allocate your own memory here.
It would be used like this:
char myBuffer[32]
char result[65];
bytesToHex(result, myBuffer, 32);
result[64] = 0;
// print it
printf(result);
// or store it in an std::string
std::string str = string(result);
I tried:
char szStringRep[64];
for ( int i = 0; i < 32; i++ ) {
sprintf( &szStringRep[i*2], "%x", szHash[i] );
}
it works as intended, but if it encountered a < 0x10 number, it will null terminated as it is printing 0
You can encode your string to another system like Base64, which is widely used when using encryption algorithms.

How does ASN.1 encode an object identifier?

I am having trouble understanding the basic concepts of ASN.1.
If a type is an OID, does the corresponding number get actually encoded in the binary data?
For instance in this definition:
id-ad-ocsp OBJECT IDENTIFIER ::= { id-ad 1 }
Does the corresponding 1.3.6.1.5.5.7.48.1 get encoded in the binary exactly like this?
I am asking this because I am trying to understand a specific value I see in a DER file (a certificate), which is 04020500, and I am not sure how to interpret it.
Yes, the OID is encoded in the binary data. The OID 1.3.6.1.5.5.7.48.1 you mention becomes 2b 06 01 05 05 07 30 01 (the first two numbers are encoded in a single byte, all remaining numbers are encoded in a single bytes as well because they're all smaller than 128).
A nice description of OID encoding is found here.
But the best way to analyze your ASN.1 data is to paste in into an online decoder, e.g. http://lapo.it/asn1js/.
If all your digits are less than or equal to 127 then you are very lucky because they can be represented with a single octet each. The tricky part is when you have larger numbers which are common, such as 1.2.840.113549.1.1.5 (sha1WithRsaEncryption). These examples focus on decoding, but encoding is just the opposite.
1. First two 'digits' are represented with a single byte
You can decode by reading the first byte into an integer
var firstByteNumber = 42;
var firstDigit = firstByteNumber / 40;
var secondDigit = firstByteNumber % 40;
Produces the values
1.2
2. Subsequent bytes are represented using Variable Length Quantity, also called base 128.
VLQ has two forms,
Short Form - If the octet starts with 0, then it is simply represented using the remaining 7 bits.
Long Form - If the octet starts with a 1 (most significant bit), combine the next 7 bits of that octet plus the 7 bits of each subsequent octet until you come across an octet with a 0 as the most significant bit (this marks the last octet).
The value 840 would be represented with the following two bytes,
10000110
01001000
Combine to 00001101001000 and read as int.
Great resource for BER encoding, http://luca.ntop.org/Teaching/Appunti/asn1.html
The first octet has value 40 * value1 + value2. (This is unambiguous,
since value1 is limited to values 0, 1, and 2; value2 is limited to
the range 0 to 39 when value1 is 0 or 1; and, according to X.208, n is
always at least 2.)
The following octets, if any, encode value3, ...,
valuen. Each value is encoded base 128, most significant digit first,
with as few digits as possible, and the most significant bit of each
octet except the last in the value's encoding set to "1." Example: The
first octet of the BER encoding of RSA Data Security, Inc.'s object
identifier is 40 * 1 + 2 = 42 = 2a16. The encoding of 840 = 6 * 128 +
4816 is 86 48 and the encoding of 113549 = 6 * 1282 + 7716 * 128 + d16
is 86 f7 0d. This leads to the following BER encoding:
06 06 2a 86 48 86 f7 0d
Finally, here is a OID decoder I just wrote in Perl.
sub getOid {
my $bytes = shift;
#first 2 nodes are 'special';
use integer;
my $firstByte = shift #$bytes;
my $number = unpack "C", $firstByte;
my $nodeFirst = $number / 40;
my $nodeSecond = $number % 40;
my #oidDigits = ($nodeFirst, $nodeSecond);
while (#$bytes) {
my $num = convertFromVLQ($bytes);
push #oidDigits, $num;
}
return join '.', #oidDigits;
}
sub convertFromVLQ {
my $bytes = shift;
my $firstByte = shift #$bytes;
my $bitString = unpack "B*", $firstByte;
my $firstBit = substr $bitString, 0, 1;
my $remainingBits = substr $bitString, 1, 7;
my $remainingByte = pack "B*", '0' . $remainingBits;
my $remainingInt = unpack "C", $remainingByte;
if ($firstBit eq '0') {
return $remainingInt;
}
else {
my $bitBuilder = $remainingBits;
my $nextFirstBit = "1";
while ($nextFirstBit eq "1") {
my $nextByte = shift #$bytes;
my $nextBits = unpack "B*", $nextByte;
$nextFirstBit = substr $nextBits, 0, 1;
my $nextSevenBits = substr $nextBits, 1, 7;
$bitBuilder .= $nextSevenBits;
}
my $MAX_BITS = 32;
my $missingBits = $MAX_BITS - (length $bitBuilder);
my $padding = 0 x $missingBits;
$bitBuilder = $padding . $bitBuilder;
my $finalByte = pack "B*", $bitBuilder;
my $finalNumber = unpack "N", $finalByte;
return $finalNumber;
}
}
OID encoding for dummies :) :
each OID component is encoded to one or more bytes (octets)
OID encoding is just a concatenation of these OID component encodings
first two components are encoded in a special way (see below)
if OID component binary value has less than 7 bits, the encoding is just a single octet, holding the component value (note, most significant bit, leftmost, will always be 0)
otherwise, if it has 8 and more bits, the value is "spread" into multiple octets - split the binary representation into 7 bit chunks (from right), left-pad the first one with zeroes if needed, and form octets from these septets by adding most significant (left) bit 1, except from the last chunk, which will have bit 0 there.
first two components (X.Y) are encoded like it is a single component with a value 40*X + Y
This is a rewording of ITU-T recommendation X.690, chapter 8.19
This is a simplistic Python 3 implementation of the of above, resp. a string form of an object identifier into ASN.1 DER or BER form.
def encode_variable_length_quantity(v:int) -> list:
# Break it up in groups of 7 bits starting from the lowest significant bit
# For all the other groups of 7 bits than lowest one, set the MSB to 1
m = 0x00
output = []
while v >= 0x80:
output.insert(0, (v & 0x7f) | m)
v = v >> 7
m = 0x80
output.insert(0, v | m)
return output
def encode_oid_string(oid_str:str) -> tuple:
a = [int(x) for x in oid_str.split('.')]
oid = [a[0]*40 + a[1]] # First two items are coded by a1*40+a2
# A rest is Variable-length_quantity
for n in a[2:]:
oid.extend(encode_variable_length_quantity(n))
oid.insert(0, len(oid)) # Add a Length
oid.insert(0, 0x06) # Add a Type (0x06 for Object Identifier)
return tuple(oid)
if __name__ == '__main__':
oid = encode_oid_string("1.2.840.10045.3.1.7")
print(oid)

Resources