I have the following binary (something):
test = b'40000000111E0C09'
Every two digits is a hexadecimal number I want out, so the following is clearer than the above:
test = b'40 00 00 00 11 1E 0C 09'
0x40 = 64 in decimal
0x00 = 0 in decimal
0x11 = 17 in decimal
0x1E = 30 in decimal
You get the idea.
How can I use struct.unpack(fmt, binary) to get the values out? I ask about struct.unpack() because it gets more complicated... I have a little-endian 4-byte integer in there... The last four bytes were:
b'11 1E 0C 09'
What is the above in decimal, assuming it's little-endian?
Thanks a lot! This is actually from a CAN bus, which I'm accessing as a serial port (frustrating stuff..)
Assuming you have string b'40000000111E0C09', you can use codecs.decode() with hex parameter to decode it to bytes form:
import struct
from codecs import decode
test = b'40000000111E0C09'
test_decoded = decode(test, 'hex') # from hex string to bytes
for i in test_decoded:
print('{:#04x} {}'.format(i, i))
Prints:
0x40 64
0x00 0
0x00 0
0x00 0
0x11 17
0x1e 30
0x0c 12
0x09 9
To get last four bytes as UINT32 (little-endian), you can do then (struct docs)
print( struct.unpack('<I', test_decoded[-4:]) )
Prints:
(151789073,)
Related
[SOLVED] Could someone help on this? I use minimalmodbus on python 3.11.1 to query a USB<->RS485 transceiver (see picture below) that is connected to a PZEM-016 (for measuring AC power consumption: voltage, current, power, energy, frequency...).
When used with the Simply Modbus Master 8.1.2, I can read the data without any problem.
With Python and minimalmodbus, I can only get up to 7 registers and of course the data is wrong (I can't pass the 7th register).
Here is the code
import minimalmodbus
#######################################
# CONSTANTS declaration
#######################################
# Communication constants
PORT_NAME ='COM3'
BAUD_RATE = 9600
BYTE_SIZE = 8
STOP_BITS = 1
TIMEOUT_VAL = 0.7
SLAVE_ADDR = 1
READ_MODE_NB = 4 # minimalmodus default to 3 PZEM requires 4 or it issues an error
WRITE_FCT_NB = 16 #HEX: 0x10
DEBUG_VAL = True
ECHO_VAL = False # Changed back from True to False after solution found
# Register constants
BASE_REG = 0
VOLT_REG = 0 #0x0000
CURR_LOW_REG = 1 #0x0001
CURR_HIGH_REG = 2 #0x0002
POWER_LOW_REG = 3 #0x0003
POWER_HIGH_REG = 4 #0x0004
ENGY_LOW_REG = 5 #0x0005
ENGY_HIGH_REG = 6 #0x0006
FREQCY_REG = 7 #0x0007
POWER_FACTOR_REG = 8 #0x0008
ALARM_REG = 9 #0x0009
#######################################
# Variables initialization
#######################################
instrument = ''
response = ''
voltage = ''
current_high = ''
current_low = ''
power_high = ''
power_low = ''
energy_high = ''
energy_low = ''
frequency = ''
power_factor = ''
alarm_status = ''
#######################################
# Set up the instrument
#######################################
instrument = minimalmodbus.Instrument(PORT_NAME, SLAVE_ADDR)
#######################################
# Explicit instrument settings
#######################################
instrument.serial.baudrate = BAUD_RATE # baud rate
instrument.serial.bytesize = BYTE_SIZE # data bits
instrument.serial.parity = minimalmodbus.serial.PARITY_NONE # parity
instrument.serial.stopbits = STOP_BITS # stop bit
instrument.serial.timeout = TIMEOUT_VAL # seconds
instrument.mode = minimalmodbus.MODE_RTU # communication protocol
# instrument.address = SLAVE_ADDR # slave address when only one slave in the bus
instrument.debug = DEBUG_VAL
instrument.handle_local_echo = ECHO_VAL
instrument.close_port_after_each_call = True
instrument.clear_buffers_before_each_transaction = True
#######################################
# Typical request format of the master
# Slave address: x001
# Command type: 0x04
# First register address high = left bytes: 0x00
# First register address low = right bytes: 0x00 (=> 0x0000 = 0)
# Number of registers high = left bytes: 0x00
# Number of registers low = right bytes: 0x0A (=> 0x000A = 10)
# CRC check high = left bytes: 0x70
# CRC check low = right bytes: 0x0D (=> 0x700D)
# 01 04 00 00 00 0A 70 0D
# Typical reply format of the slave
# Slave address: 0x01
# Command type: 0x04
# Number of bytes: 0x14 (= 20 bytes = 2 bytes per requested register)
# Register 1: Voltage 8 bits
# Register 1 data high byte = left byte: 0x09
# Register 1 data low byte = right byte: 0x2E (=> 0x092E = 2350 = 235.0V)
# Register 2 + 3: Current 16 bits
# Register 2 data low byte = right bits: 0x01
# Register 2 data high byte = left bits: 0x78
# Register 3 data low byte = right bits: 0x00
# Register 3 data high byte = left bits: 0x00 (=> 0x00000178 = 376 = 0.376A)
# Register 4 + 5: Power 16 bits
# Register 4 data low byte = right bits: 0x01
# Register 4 data high byte = left bits: 0xE7
# Register 5 data low byte = right bits: 0x00
# Register 5 data high byte = left bits: 0x00 (=> 0x000001E7 = 487 = 48.7W)
# Register 6 + 7: Energy 16 bits
# Register 6 data low byte = right bits: 0x02
# Register 6 data high byte = left bits: 0xEA
# Register 7 data low byte = right bits: 0x00
# Register 7 data high byte = left bits: 0x00 (=> 0x000002EA = 746 = 746Wh)
# Register 8: Frequency 8 bits
# Register 8 data high byte = left byte: 0x01
# Register 8 data low byte = right byte: 0xF4 (=> 0x01F4 = 500 = 50.0Hz)
# Register 9: Power factor 8 bits
# Register 9 data high byte = left byte: 0x00
# Register 9 data low byte = right byte: 0x37 (=> 0x0037 = 55 = 0.55)
# Register 10: Alarm status 8 bits
# Register 10 data high byte = left byte: 0x00
# Register 10 data low byte = right byte: 0x00 (=> 0x0000 = 0 = No alarm)
# CRC check high = left bytes: 0xE5 (= 229)
# CRC check low = right bytes: 0xFD (= 253) (=> 0xE5FD)
# LED lights turned on
# 01 04 14 09 2E 01 78 00 00 01 E7 00 00 02 EA 00 00 01 F4 00 37 00 00 39 51
# LED lights turned off
# 01 04 14 09 34 00 00 00 00 00 05 00 00 02 E4 00 00 01 F3 00 64 00 00 E5 FD
#######################################
print('instrument: ', instrument)
#######################################
# Read and print registers' data
#######################################
response = instrument.read_registers(VOLT_REG, 10, READ_MODE_NB)
print('Response raw: ', response)
With the wrong read mode value, I get an error message with the read_registers() command
>>> ReadSerial.py
< MinimalModbus debug mode. Will write to instrument (expecting 25 bytes back): 01 03 00 00 00 0A C5 CD (8 bytes)
MinimalModbus debug mode. Clearing serial buffers for port COM3
MinimalModbus debug mode. No sleep required before write. Time since previous read: 167521078.00 ms, minimum silent period: 4.01 ms.
MinimalModbus debug mode. Closing port COM3
MinimalModbus debug mode. Response from instrument: 01 83 02 C0 F1 (5 bytes), roundtrip time: 0.8 ms. Timeout for reading: 700.0 ms.
< Traceback (most recent call last):
File "...\ReadSerial.py", line 127, in <module>
response = instrument.read_registers(VOLT_REG, 10, READ_MODE_NB)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "...\minimalmodbus.py", line 904, in read_registers
returnvalue = self._generic_command(
^^^^^^^^^^^^^^^^^^^^^^
File "...\minimalmodbus.py", line 1245, in _generic_command
payload_from_slave = self._perform_command(functioncode, payload_to_slave)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "...\minimalmodbus.py", line 1329, in _perform_command
payload_from_slave = _extract_payload(
^^^^^^^^^^^^^^^^^
File "...\minimalmodbus.py", line 1880, in _extract_payload
_check_response_slaveerrorcode(response)
File "...\minimalmodbus.py", line 3538, in _check_response_slaveerrorcode
raise error
minimalmodbus.IllegalRequestError: Slave reported illegal data address
I matched everything I could with the Simply Modbus Master example, except for the query mode, which value should be 4 (minimalmodbus default read mode is 3).
Update from above
I finally found the error in my code, which was very stupid, but errors are often stupid like a missing ";".
I still have to figure out why the read_register() command doesn't work (without the "s" for reading one register at a time). I can't pass the 7th register without errors and the returned data is wrong.
I leave the question and the answer for the community, in case someone uses the same PZEM-014 -> PZEM0-17 and looks for a python solution to retrieve the data
I'm experiencing some weird behavior in NodeJS v14.16.1 on Windows 10.
When I append 4 bytes to a file, it's only appending 2 bytes:
const fs = require('fs');
let infoFile = fs.openSync('field.info', 'a');
fs.writeSync(infoFile, Buffer.from([0x06, 0x00, 0x04, 0x00]))
fs.closeSync(infoFile);
Output for field.info:
06 04
Expected output:
06 00 04 00
Buffer.from([0x06, 0x00, 0x04, 0x00, 0x00]) produces the expected output:
06 00 04 00 00
The .toString() on the Buffer's produce strings of the correct length, so I can only imagine it's an issue in fs.writeSync. I can't find anything in the documentation that would indicate the issue...
Edit
fs.writeSync reports 4 bytes written but only 2 exist in the file.
This is most certainly due to the editor you opening the file with, interpreting it as UTF-16. Thus, you see only two characters, because each character is representing two bytes. Showing the file properties in windows explorer clearly shows the correct file size.
I started digging into Java NIO API and as a first try I wanted to read a JPEG file magic number.
Here's the code
import java.nio.ByteBuffer;
import java.nio.CharBuffer;
import java.nio.channels.FileChannel;
import java.nio.charset.Charset;
import java.io.FileInputStream;
public class JpegMagicNumber {
public static void main(String[] args) throws Exception {
FileChannel file = new FileInputStream(args[0]).getChannel();
ByteBuffer buffer = ByteBuffer.allocate(6);
file.read(buffer);
buffer.flip();
System.out.println(Charset.defaultCharset().decode(buffer).toString());
file.close();
buffer.clear();
}
}
I expect to get the magic number chars back but all I get is garbage data into the terminal.
Am I doing something wrong ?
Short answer: There is nothing particularly defective with the code. JPEG just has 'garbage' up front.
Long answer: JPEG internally is made up of segments, one after the other. These segments start with a 0xFF byte, followed by an identifier byte, and then an optional payload/content.
Example start:
FF D8 FF E0 00 10 4A 46 49 46 00 01 01 00 00 01 00 01 00 00 FF E1
The image starts with the Start Of Image (SOI) segment, 0xFF 0xD8, which has no payload.
The next segment is 'application specific', 0xFF 0xE0. Two bytes follow with the length of the payload (these two bytes included!).
0x4A 0x46 0x49 0x46 : JFIF ← perhaps what you were looking for?
JPEG doesn't have a magic number in the sense you were perhaps looking for, like 'PK' for zip or '‰PNG' for PNG. (The closest thing is 0xFF 0xD8 0xFF for the SOI and the first byte of the next segment.)
So your code does correctly read the first six bytes of the file, decodes them into characters per your native platform, and prints them out, but a JPEG header just looks that way.
I have a weird problem! I made a client / server Python code with Bluetooth in series, to send and receive byte frames (for example: [0x73, 0x87, 0x02 ....] )
Everything works, the send reception works very well !
The problem is the display of my frames, I noticed that the bytes from 0 to 127 are displayed, but from 128, it displays the byte but it adds a C2 (194) behind, for example: [0x73, 0x7F, 0x87, 0x02, 0x80 ....] == [115, 127, 135, 2, 128 ....] in hex display I would have 73 7F C2 87 2 C2 80 .. , we will notice that he adds a byte C2 from nowhere!
I think that since it is from 128! that it is due to a problem of signed (-128 to 127) / unsigned (0 to 255).
Anyone have any indication of this problem?
Thank you
0xc2 and 0xc3 are byte values that appear when encoding character values between U+0080 and U+00FF as UTF-8. Something on the transmission side is trying to send text instead of bytes, and something in the middle is (properly) converting the text to UTF-8 bytes before sending. The fix is to send bytes instead of text in the first place.
Background: I'm using node.js to get the volume setting from a device via serial connection. I need to obtain this data as an integer value.
I have the data in a buffer ('buf'), and am using readInt16BE() to convert to an int, as follows:
console.log( buf )
console.log( buf.readInt16BE(0) )
Which gives me the following output as I adjust the external device:
<Buffer 00 7e>
126
<Buffer 00 7f>
127
<Buffer 01 00>
256
<Buffer 01 01>
257
<Buffer 01 02>
258
Problem: All looks well until we reach 127, then we take a jump to 256. Maybe it's something to do with signed and unsigned integers - I don't know!
Unfortunately I have very limited documentation about the external device, I'm having to reverse engineer it! Is it possible it only sends a 7-bit value? Hopefully there is a way around this?
Regarding a solution - I must also be able to convert back from int to this format!
Question: How can I create a sequential range of integers when 7F seems to be the largest value my device sends, which causes a big jump in my integer scale?
Thanks :)
127 is the maximum value of a signed 8-bit integer. If the integer is overflowing into the next byte at 128 it would be safe to assume you are not being sent a 16 bit value, but rather 2 signed 8-bit values, and reading the value as a 16-bit integer would be incorrect.
I would start by using the first byte as a multiplier of 128 and add the second byte, this will give the series you are seeking.
buf = Buffer([0,127]) //<Buffer 00 7f>
buf.readInt8(0) * 128 + buf.readInt8(1)
>127
buf = Buffer([1,0]) //<Buffer 01 00>
buf.readInt8(0) * 128 + buf.readInt8(1)
>128
buf = Buffer([1,1]) //<Buffer 01 01>
buf.readInt8(0) * 128 + buf.readInt8(1)
>129
The way to get back is to divide by 128, round it down to the nearest integer for the first byte, and the second byte contains the remainder.
i = 129
buf = Buffer([Math.floor(i / 128), i % 128])
<Buffer 01 01>
Needed to treat the data as two signed 8-bit values. As per #forrestj the solution is to do:
valueInt = buf.readInt8(0) * 128 + buf.readInt8(1)
We can also convert the int value into the original format by doing the following:
byte1 = Math.floor(valueInt / 128)
byte2 = valueInt % 128