Trouble displaying signed unsigned bytes with python - python-3.x

I have a weird problem! I made a client / server Python code with Bluetooth in series, to send and receive byte frames (for example: [0x73, 0x87, 0x02 ....] )
Everything works, the send reception works very well !
The problem is the display of my frames, I noticed that the bytes from 0 to 127 are displayed, but from 128, it displays the byte but it adds a C2 (194) behind, for example: [0x73, 0x7F, 0x87, 0x02, 0x80 ....] == [115, 127, 135, 2, 128 ....] in hex display I would have 73 7F C2 87 2 C2 80 .. , we will notice that he adds a byte C2 from nowhere!
I think that since it is from 128! that it is due to a problem of signed (-128 to 127) / unsigned (0 to 255).
Anyone have any indication of this problem?
Thank you

0xc2 and 0xc3 are byte values that appear when encoding character values between U+0080 and U+00FF as UTF-8. Something on the transmission side is trying to send text instead of bytes, and something in the middle is (properly) converting the text to UTF-8 bytes before sending. The fix is to send bytes instead of text in the first place.

Related

Convert binary <something> of hex bytes to list of decimal values

I have the following binary (something):
test = b'40000000111E0C09'
Every two digits is a hexadecimal number I want out, so the following is clearer than the above:
test = b'40 00 00 00 11 1E 0C 09'
0x40 = 64 in decimal
0x00 = 0 in decimal
0x11 = 17 in decimal
0x1E = 30 in decimal
You get the idea.
How can I use struct.unpack(fmt, binary) to get the values out? I ask about struct.unpack() because it gets more complicated... I have a little-endian 4-byte integer in there... The last four bytes were:
b'11 1E 0C 09'
What is the above in decimal, assuming it's little-endian?
Thanks a lot! This is actually from a CAN bus, which I'm accessing as a serial port (frustrating stuff..)
Assuming you have string b'40000000111E0C09', you can use codecs.decode() with hex parameter to decode it to bytes form:
import struct
from codecs import decode
test = b'40000000111E0C09'
test_decoded = decode(test, 'hex') # from hex string to bytes
for i in test_decoded:
print('{:#04x} {}'.format(i, i))
Prints:
0x40 64
0x00 0
0x00 0
0x00 0
0x11 17
0x1e 30
0x0c 12
0x09 9
To get last four bytes as UINT32 (little-endian), you can do then (struct docs)
print( struct.unpack('<I', test_decoded[-4:]) )
Prints:
(151789073,)

How to convert binary to byte for iso8583?

I am trying to understand iso8583 and how to send it to our payment provider. I see that the first piece of the message structure is the Message Type Identifier. I used 0100 for authorization, which seems to be correct. However it also seems that I am not sending it in the right format. I simply sent the binary representation, meaning I sent and then the rest of the message. Do I need to convert 0100 into bytes before sending it? I am using node.js. Any help is appreciated, as this is a very complex topic.
You have to use ascii format. In your case you have to send authorization using 4 bytes 0100 ascii the representation in hex is 0x30 0x31 0x30 0x30
What Panagiotis was saying is the MTI (and the rest of the message) need to be convert from ASCII to Hexadecimal. Something like this would do it:
let message = "0100..."
let msgArr = message.split( "" )
let convertedMessage = ""
for ( let index in msgArr )
{
convertedMessage += String.fromCharCode( msgArr[ index ]).toString( 16 )
}
// use convertedMessage to send your request

What is meaning of the response status word 0x61xx from a smart card?

I wrote a Java Card applet that saves some data into the APDU buffer at offset ISO7816.OFFSET_CDATA and sends those bytes as a response.
Util.arrayCopy(Input_Data, (short)0, buffer, (short) ISO7816.OFFSET_CDATA, (short)Datalength);
apdu.setOutgoing();
apdu.setOutgoingLength((short)(DataLength) );
apdu.sendBytesLong(buffer, ISO7816.OFFSET_CDATA, (short)(DataLength));
I tested this in a simulator without any problem. But when I test this on a real smart card (Java Card v2.2.1 manufactured by Gemalto), I get the status word 0x6180 as response.
My command APDU is 00 40 00 00 80 Data, where data has a length of 128 bytes, so I have 4+128 bytes in the buffer and (260-(4+128)) byte is null.
Your simulator probably uses T=1 transport protocol, but your real card does not. It uses T=0 protocol, which means it can either receive data, or send data in a single APDU.
Status word 0x6180 indicates there are 0x80 bytes to receive from the card. Generally, 61XX means XX bytes to receive.
How to receive them? Well, there is a special APDU command called GET RESPONSE. You should call it each time you get 61XX status word. Use XX as the Le byte of your GET RESPONSE APDU
APDU -> 61 XX
00 C0 00 00 XX -> your data 90 00
A few other notes on your code:
Datalength vs DataLength?
Copy your output data to 0 instead of ISO7816.OFFSET_CDATA
Why do you cast DataLength to short each time? Is it short? Do not cast then. Is it byte? You cast it in a wrong way then, because unsigned byte value > 0x80 will be cast to a negative short. The correct cast from an unsigned byte to a short is (short) (DataLength & 0xFF)
Use setOutgoingAndSend whenever you can. It is much simpler.
Use arrayCopyNonAtomic instead of arrayCopy whenever you are not copying to a persistent array. Performance of arrayCopyNonAtomic is much better.

Converting hex values in buffer to integer

Background: I'm using node.js to get the volume setting from a device via serial connection. I need to obtain this data as an integer value.
I have the data in a buffer ('buf'), and am using readInt16BE() to convert to an int, as follows:
console.log( buf )
console.log( buf.readInt16BE(0) )
Which gives me the following output as I adjust the external device:
<Buffer 00 7e>
126
<Buffer 00 7f>
127
<Buffer 01 00>
256
<Buffer 01 01>
257
<Buffer 01 02>
258
Problem: All looks well until we reach 127, then we take a jump to 256. Maybe it's something to do with signed and unsigned integers - I don't know!
Unfortunately I have very limited documentation about the external device, I'm having to reverse engineer it! Is it possible it only sends a 7-bit value? Hopefully there is a way around this?
Regarding a solution - I must also be able to convert back from int to this format!
Question: How can I create a sequential range of integers when 7F seems to be the largest value my device sends, which causes a big jump in my integer scale?
Thanks :)
127 is the maximum value of a signed 8-bit integer. If the integer is overflowing into the next byte at 128 it would be safe to assume you are not being sent a 16 bit value, but rather 2 signed 8-bit values, and reading the value as a 16-bit integer would be incorrect.
I would start by using the first byte as a multiplier of 128 and add the second byte, this will give the series you are seeking.
buf = Buffer([0,127]) //<Buffer 00 7f>
buf.readInt8(0) * 128 + buf.readInt8(1)
>127
buf = Buffer([1,0]) //<Buffer 01 00>
buf.readInt8(0) * 128 + buf.readInt8(1)
>128
buf = Buffer([1,1]) //<Buffer 01 01>
buf.readInt8(0) * 128 + buf.readInt8(1)
>129
The way to get back is to divide by 128, round it down to the nearest integer for the first byte, and the second byte contains the remainder.
i = 129
buf = Buffer([Math.floor(i / 128), i % 128])
<Buffer 01 01>
Needed to treat the data as two signed 8-bit values. As per #forrestj the solution is to do:
valueInt = buf.readInt8(0) * 128 + buf.readInt8(1)
We can also convert the int value into the original format by doing the following:
byte1 = Math.floor(valueInt / 128)
byte2 = valueInt % 128

Confused about "three successive writes: bytes 10, bytes 32, bytes 54"?

I am learning SMSC smc91cx driver code, and I learned how to program test code for smc91c111 nic by the instructions of Application Note 9-6. I cannot understand the following instructions under "Transmitting A Packet":
Write the destination address (three successive writes: bytes 10, bytes 32, bytes 54)
Write 0xFFFF, 0xFFFF, 0xFFFF
Write the source address (three successive writes: bytes 10, bytes32, bytes 54)
Write 0x0000, 0x0000, 0x0000
I cannot make sense of these instructions. Should I write 10 bytes size of 0xFF plus 32 bytes size plus 54 bytes size to the buffer, or just write 0xFF in 10th byte postion, 32th, 54th byte postion?
But if so, why would you write 0x0000 to the same position?
Rather than allocating several different registers to write to, that chip has you write to the same DATA register serially until you set all the info. The DATA register is 2 bytes wide, but a MAC address is 6 bytes, numbered 0-5. So you have to write it 2 bytes at a time: bytes number 1 and 0 first, followed by bytes number 3 and 2, then bytes number 5 and 4. Then write 0xFFFF 3 times to the DATA register, then repeat for the source address and the 0x0000s.

Resources