I have posted several question regarding a similar problem, here and here but after thinking and troubleshooting over the weekend, I'd like to explore more possible cause, but lack the knowledge.
How should I check ion my current code is writing anything to the sensor I have connected via USB-RS232. Here is my code.
port = serial.Serial("/dev/ttyUSB0", baudrate=9600, timeout=20, bytesize=8, rtscts =1,dsrdtr=1)
f_w = open('/home/ryan/python_serial_output.txt','r+')
f_o = open('/home/ryan/python_serial_parse.txt','w')
port.send_break()
sys_reply = port.read(100000)
sys_reply_str = sys_reply.decode('cp437')
print(sys_reply_str)
sys_reply_str_haha = sys_reply_str.replace("\r","")
sys_reply_str_haha = sys_reply_str_haha.replace("\n","")
i = list(sys_reply_str_haha)
if str(i[-1]) == '>':
port.reset_input_buffer()
print("ip_b_reset")
port.reset_output_buffer()
print("op_b_reset")
print(port.writable())
ip = 'CR1'
ip_en = ip.encode('cp437')
port.write(ip_en)
read_syscheck = port.read(1000)
read_syscheck_str = read_syscheck.decode('cp437')
print(read_syscheck_str)
With the same decoding format I tried writing to the port and should receive back a reply
>CR1
*[Parameters set to FACTORY defaults]*
Instead, I got
Sensor 2009
All rights reserved.
Firmware Version: 34.11
>
ip_b_reset
op_b_reset
True
CR1
7F7FD30000000030C6A87F9978B9302....
and that is why I conclude I might not be writing to the sensor.
PS: The data stream 7F7FD30000000030C6A87F9978B9302.... is always present and would appear when timed out. Hence I am unsure if this is the start of an actual reading? Or just a random readout.
Related
Background: I want to use ble for server/client software. But in this case, the server should be the peripheral with multiple connections. For that i start multiple services, up to 10. I think that's the limit for peripheral connections on my chip. The different services are the same in functionality but should connect to only one client. I see no other possibility to differentiate the clients in the notification function.
Problem: For that i restart the advertising after each connection. In the advertisement, I set the uuid information field “ServiceUUIDs” for the next free service to connect. This is working after restarting my pc. In my flutter app, I’m able to read this field perfectly in the advertisment. But when I restart the python script, the uuid is no longer advertised and the field “ServiceUUIDs” is empty although {'Type': 'peripheral', 'ServiceUUIDs': dbus.Array(['0000ac01-0000-1000-8000-00805f9b34fb'], signature=dbus.Signature('s')), 'LocalName': dbus.String('Hello'), 'Discoverable': dbus.Boolean(True)} is set in the bluetooth_constants.ADVERTISING_MANAGER_INTERFACE. Also restarting the bluetooth module is not working. Therefore I must restart my pc everytime I want to restart my server... After connecting to the server, the client can scan all services and does not know, which one of the 10 services is free to connect.
My python code is the same as in the dbus tutorial:
class Advertisement(dbus.service.Object):
PATH_BASE = '/org/bluez/ldsg/advertisement'
def __init__(self, bus, index, advertising_type):
self.path = self.PATH_BASE + str(index)
self.bus = bus
self.ad_type = advertising_type
self.service_uuids = ['0000ac01-0000-1000-8000-00805f9b34fb']
self.manufacturer_data = None
self.solicit_uuids = None
self.service_data = None
self.local_name = 'Hello'
self.include_tx_power = False
self.data = None #{0x26: dbus.Array([0x01, 0x01, 0x01], signature='y')}
self.discoverable = True
dbus.service.Object.__init__(self, bus, self.path)
def get_properties(self):
properties = dict()
properties['Type'] = self.ad_type
if self.service_uuids is not None:
properties['ServiceUUIDs'] = dbus.Array(self.service_uuids,
signature='s')
if self.solicit_uuids is not None:
properties['SolicitUUIDs'] = dbus.Array(self.solicit_uuids,
signature='s')
if self.manufacturer_data is not None:
properties['ManufacturerData'] = dbus.Dictionary(
self.manufacturer_data, signature='qv')
if self.service_data is not None:
properties['ServiceData'] = dbus.Dictionary(self.service_data,
signature='sv')
if self.local_name is not None:
properties['LocalName'] = dbus.String(self.local_name)
if self.discoverable is not None and self.discoverable == True:
properties['Discoverable'] = dbus.Boolean(self.discoverable)
if self.include_tx_power:
properties['Includes'] = dbus.Array(["tx-power"], signature='s')
if self.data is not None:
properties['Data'] = dbus.Dictionary(
self.data, signature='yv')
print(properties)
return {bluetooth_constants.ADVERTISING_MANAGER_INTERFACE: properties}
def start_advertising():
global adv
global adv_mgr_interface
# we're only registering one advertisement object so index (arg2) is hard coded as 0
print("Registering advertisement",adv.get_path())
adv_mgr_interface.RegisterAdvertisement(adv.get_path(), {},
reply_handler=register_ad_cb,
error_handler=register_ad_error_cb)
bus = dbus.SystemBus()
adv_mgr_interface = dbus.Interface(bus.get_object(bluetooth_constants.BLUEZ_SERVICE_NAME,adapter_path), bluetooth_constants.ADVERTISING_MANAGER_INTERFACE)
adv = Advertisement(bus, 0, 'peripheral')
start_advertising()
Maybe there is an other way to send additional information in the advertisment? I also tried "Data" and "ServiceData" but then the advertisement throws an error. I haven't found a good tutorial for this. I'm only looking for a stable way to give the additional information for the next free service in the advertisment.
Thank You!
I have light detector sensors connected to a data acquisition box and it is connected to my laptop via RS232 usb cable. I have stabilized serial communication to that port in python. But when I try to read the data it just keeps on running and it doesn't display any value. I have tried this same think in MATALB and it works properly, so I know that port and sensors are working fine. I am just not able to read the data in python. I have three ways(shown below in python code) but nothing works. Please help me.
Here is my python code:
import serial
from serial import Serial
s = serial.Serial('COM3') # open serial port
print(ser.name)
# Set the serial port to desired COM
s = serial.Serial(port = 'COM3', bytesize = serial.EIGHTBITS, parity = serial.PARITY_NONE, baudrate = 9600, stopbits = serial.STOPBITS_ONE)
# Read the data(method 1)
while 1:
while (s.inWaiting() > 0):
try:
inc = s.readline().strip()
print(inc)
# Read the data(method 2)
data = str(s.read(size = 1))
print(data)
# Read all the data(method 3)
while i <10:
b = s.readlines(100) # reading all the lines
in matlab it gave values with a space but in same line
That indicates there are no newline characters sent by the device.
I've never used your device or the Python serial module. But from the docs, this is what I would do:
from time import sleep
while s.is_open():
n = s.in_waiting()
if n == 0:
print "no data"
sleep(1)
continue
try:
inc = s.read(n)
print(inc)
catch serial.SerialException as oops:
print(oops)
catch serial.SerialTimeoutException as oops:
print(oops)
print("serial port closed")
This gives you feedback for each condition: port open/closed, data ready/not, and the data, if any. From there you can figure out what to do.
The documentation says inWaiting is deprecated, so I used in_waiting.
I wouldn't be surprised if the problem lies in configuring the serial port in the first place. Serial ports are tricky like that.
I need to find out for a script I'm writing who is the true owner of a file in an smb share (mounted using mount -t cifs of course on my server and using net use through windows machines).
Turns out it is a real challenge finding this information out using python on a linux server.
I tried using many many smb libraries (such as smbprotocol, smbclient and others), nothing worked.
I find few solutions for windows, they all use pywin32 or another windows specific package.
And I also managed to do it from bash using smbcalcs but couldn't do it cleanly but using subprocess.popen('smbcacls')..
Any idea on how to solve it?
This was unbelievably not a trivial task, and unfortunately the answer isn't simple as I hoped it would be..
I'm posting this answer if someone will be stuck with this same problem in the future, but hope maybe someone would post a better solution earlier
In order to find the owner I used this library with its examples:
from smb.SMBConnection import SMBConnection
conn = SMBConnection(username='<username>', password='<password>', domain=<domain>', my_name='<some pc name>', remote_name='<server name>')
conn.connect('<server name>')
sec_att = conn.getSecurity('<share name>', r'\some\file\path')
owner_sid = sec_att.owner
The problem is that pysmb package will only give you the owner's SID and not his name.
In order to get his name you need to make an ldap query like in this answer (reposting the code):
from ldap3 import Server, Connection, ALL
from ldap3.utils.conv import escape_bytes
s = Server('my_server', get_info=ALL)
c = Connection(s, 'my_user', 'my_password')
c.bind()
binary_sid = b'....' # your sid must be in binary format
c.search('my_base', '(objectsid=' + escape_bytes(binary_sid) + ')', attributes=['objectsid', 'samaccountname'])
print(c.entries)
But of course nothing will be easy, it took me hours to find a way to convert a string SID to binary SID in python, and in the end this solved it:
# posting the needed functions and omitting the class part
def byte(strsid):
'''
Convert a SID into bytes
strdsid - SID to convert into bytes
'''
sid = str.split(strsid, '-')
ret = bytearray()
sid.remove('S')
for i in range(len(sid)):
sid[i] = int(sid[i])
sid.insert(1, len(sid)-2)
ret += longToByte(sid[0], size=1)
ret += longToByte(sid[1], size=1)
ret += longToByte(sid[2], False, 6)
for i in range(3, len(sid)):
ret += cls.longToByte(sid[i])
return ret
def byteToLong(byte, little_endian=True):
'''
Convert bytes into a Python integer
byte - bytes to convert
little_endian - True (default) or False for little or big endian
'''
if len(byte) > 8:
raise Exception('Bytes too long. Needs to be <= 8 or 64bit')
else:
if little_endian:
a = byte.ljust(8, b'\x00')
return struct.unpack('<q', a)[0]
else:
a = byte.rjust(8, b'\x00')
return struct.unpack('>q', a)[0]
... AND finally you have the full solution! enjoy :(
I'm adding this answer to let you know of the option of using smbprotocol; as well as expand in case of misunderstood terminology.
SMBProtocol Owner Info
It is possible to get the SID using the smbprotocol library as well (just like with the pysmb library).
This was brought up in the github issues section of the smbprotocol repo, along with an example of how to do it. The example provided is fantastic and works perfectly. An extremely stripped down version
However, this also just retrieves a SID and will need a secondary library to perform a lookup.
Here's a function to get the owner SID (just wrapped what's in the gist in a function. Including here in case the gist is deleted or lost for any reason).
import smbclient
from ldap3 import Server, Connection, ALL,NTLM,SUBTREE
def getFileOwner(smb: smbclient, conn: Connection, filePath: str):
from smbprotocol.file_info import InfoType
from smbprotocol.open import FilePipePrinterAccessMask,SMB2QueryInfoRequest, SMB2QueryInfoResponse
from smbprotocol.security_descriptor import SMB2CreateSDBuffer
class SecurityInfo:
# 100% just pulled from gist example
Owner = 0x00000001
Group = 0x00000002
Dacl = 0x00000004
Sacl = 0x00000008
Label = 0x00000010
Attribute = 0x00000020
Scope = 0x00000040
Backup = 0x00010000
def guid2hex(text_sid):
"""convert the text string SID to a hex encoded string"""
s = ['\\{:02X}'.format(ord(x)) for x in text_sid]
return ''.join(s)
def get_sd(fd, info):
""" Get the Security Descriptor for the opened file. """
query_req = SMB2QueryInfoRequest()
query_req['info_type'] = InfoType.SMB2_0_INFO_SECURITY
query_req['output_buffer_length'] = 65535
query_req['additional_information'] = info
query_req['file_id'] = fd.file_id
req = fd.connection.send(query_req, sid=fd.tree_connect.session.session_id, tid=fd.tree_connect.tree_connect_id)
resp = fd.connection.receive(req)
query_resp = SMB2QueryInfoResponse()
query_resp.unpack(resp['data'].get_value())
security_descriptor = SMB2CreateSDBuffer()
security_descriptor.unpack(query_resp['buffer'].get_value())
return security_descriptor
with smbclient.open_file(filePath, mode='rb', buffering=0,
desired_access=FilePipePrinterAccessMask.READ_CONTROL) as fd:
sd = get_sd(fd.fd, SecurityInfo.Owner | SecurityInfo.Dacl)
# returns SID
_sid = sd.get_owner()
try:
# Don't forget to convert the SID string-like object to a string
# or you get an error related to "0" not existing
sid = guid2hex(str(_sid))
except:
print(f"Failed to convert SID {_sid} to HEX")
raise
conn.search('DC=dell,DC=com',f"(&(objectSid={sid}))",SUBTREE)
# Will return an empty array if no results are found
return [res['dn'].split(",")[0].replace("CN=","") for res in conn.response if 'dn' in res]
to use:
# Client config is required if on linux, not if running on windows
smbclient.ClientConfig(username=username, password=password)
# Setup LDAP session
server = Server('mydomain.com',get_info=ALL,use_ssl = True)
# you can turn off raise_exceptions, or leave it out of the ldap connection
# but I prefer to know when there are issues vs. silently failing
conn = Connection(server, user="domain\username", password=password, raise_exceptions=True,authentication=NTLM)
conn.start_tls()
conn.open()
conn.bind()
# Run the check
fileCheck = r"\\shareserver.server.com\someNetworkShare\someFile.txt"
owner = getFileOwner(smbclient, conn, fileCheck)
# Unbind ldap session
# I'm not clear if this is 100% required, I don't THINK so
# but better safe than sorry
conn.unbind()
# Print results
print(owner)
Now, this isn't super efficient. It takes 6 seconds for me to run this one a SINGLE file. So if you wanted to run some kind of ownership scan, then you probably want to just write the program in C++ or some other low-level language instead of trying to use python. But for something quick and dirty this does work. You could also setup a threading pool and run batches. The piece that takes longest is connecting to the file itself, not running the ldap query, so if you can find a more efficient way to do that you'll be golden.
Terminology Warning, Owner != Creator/Author
Last note on this. Owner != File Author. Many domain environments, and in particular SMB shares, automatically alter ownership from the creator to a group. In my case the results of the above is:
What I was actually looking for was the creator of the file. File creator and modifier aren't attributes which windows keeps track of by default. An administrator can enable policies to audit file changes in a share, or auditing can be enabled on a file-by-file basis using the Security->Advanced->Auditing functionality for an individual file (which does nothing to help you determine the creator).
That being said, some applications store that information for themselves. For example, if you're looking for Excel this answer provides a method for which to get the creator of any xls or xlsx files (doesn't work for xlsb due to the binary nature of the files). Unfortunately few files store this kind of information. In my case I was hoping to get that info for tblu, pbix, and other reporting type files. However, they don't contain this information (which is good from a privacy perspective).
So in case anyone finds this answer trying to solve the same kind of thing I did - Your best bet (to get actual authorship information) is to work with your domain IT administrators to get auditing setup.
I'm trying to write a small code which displays data received from a game (F1 2019) over UDP.
The F1 2019 game send out data via the UDP. I have been able to receive the packets and have separated the header and data and now unpacked the data according to the structure in which the data is sent using the rawutil module.
The struct in which the packets are sent can be found here:
https://forums.codemasters.com/topic/38920-f1-2019-udp-specification/
I'm only interested in the telemetry packet.
import socket
import cdp
import struct
import array
import rawutil
from pprint import pprint
# settings
ip = '0.0.0.0'
port = 20777
# listen for packets
listen_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
listen_socket.bind((ip, port))
while True:
# Receiving data
data, address = listen_socket.recvfrom(65536)
header = data[:20]
telemetry = data[20:]
# decode the header
packetFormat, = rawutil.unpack('<H', header[:2])
gameMajorVersion, = rawutil.unpack('<B', header[2:3])
gameMinorVersion, = rawutil.unpack('<B', header[3:4])
packetVersion, = rawutil.unpack('<B', header[4:5])
packetId, = rawutil.unpack('<B', header[5:6])
sessionUID, = rawutil.unpack('<Q', header[6:14])
sessionTime, = rawutil.unpack('<f', header[14:18])
frameIdentifier, = rawutil.unpack('<B', header[18:19])
playerCarIndex, = rawutil.unpack('<B', header[19:20])
# print all info (just for now)
## print('Packet Format : ',packetFormat)
## print('Game Major Version : ',gameMajorVersion)
## print('Game Minor Version : ',gameMinorVersion)
## print('Packet Version : ',packetVersion)
## print('Packet ID : ', packetId)
## print('Unique Session ID : ',sessionUID)
## print('Session Time : ',sessionTime)
## print('Frame Number : ',frameIdentifier)
## print('Player Car Index : ',playerCarIndex)
## print('\n\n')
#start getting the packet data for each packet starting with telemetry data
if (packetId == 6):
speed, = rawutil.unpack('<H' , telemetry[2:4])
throttle, = rawutil.unpack('<f' , telemetry[4:8])
steer, = rawutil.unpack('<f' , telemetry[8:12])
brake, = rawutil.unpack('<f' , telemetry[12:16])
gear, = rawutil.unpack('<b' , telemetry[17:18])
rpm, = rawutil.unpack('<H' , telemetry[18:20])
print (speed)
The UDP specification states that the speed of the car is sent in km/h. However when I unpack the packet, the speed is a multiple of 256, so 10 km/h is 2560 for example.
I want to know if I'm unpacking the data in the wrong way? or is it something else that is causing this.
The problem is also with the steering for example. the spec says it should be between -1.0 and 1.0 but the actual values are either very large or very small.
screengrab here: https://imgur.com/a/PHgdNrx
Appreciate any help with this.
Thanks.
I recommend you don't use the unpack method, as with big structures (e.g. MotionPacket has 1343 bytes) your code will immediately get very messy.
However, if you desperately want to use it, call unpack only once, such as:
fmt = "<HBBBBQfBB"
size = struct.calcsize(fmt)
arr = struct.unpack("<HBBBBQfBB", header[:size])
Alternatively, have a look at ctypes library, especially ctypes.LittleEndianStructure where you can set the _fields_ attribute to a sequence of ctypes (such as uint8 etc, without having to translate them to relevant symbols as with unpack).
https://docs.python.org/3.8/library/ctypes.html#ctypes.LittleEndianStructure
Alternatively alternatively, have a look at namedtuples.
Alternatively alternatively alternatively, there's a bunch of python binary IO libs, such as binio where you can declare a structure of ctypes, as this is a thin wrapper anyway.
To fully answer your question, the structure seems to be:
struct PacketHeader
{
uint16 m_packetFormat; // 2019
uint8 m_gameMajorVersion; // Game major version - "X.00"
uint8 m_gameMinorVersion; // Game minor version - "1.XX"
uint8 m_packetVersion; // Version of this packet type, all start from 1
uint8 m_packetId; // Identifier for the packet type, see below
uint64 m_sessionUID; // Unique identifier for the session
float m_sessionTime; // Session timestamp
uint m_frameIdentifier; // Identifier for the frame the data was retrieved on
uint8 m_playerCarIndex; // Index of player's car in the array
};
Meaning that the sequence of symbols for unpack should be: <H4BQfLB, because uint in ctypes is actually uint32, where you had uint8.
I also replaced BBBB with 4B. Hope this helps.
Haider I wanted to read car speed from Formula 1 2019 too. I found your question, from your question I had some tips and solved my issue. And now i think i must pay back. The reason you get speed multiplied with 256 is you start from wrong byte and this data is formatted little endian. The code you shared starts from 22nd byte to read speed, if you start it from 23rd byte you will get correct speed data.
I have been trying to send a value from a Python program via serial to an Arduino, but I have been unable to get the Arduino to store and echo back the value to Python. My code seems to match that I've found in examples online, but for whatever reason, it's not working.
I am using Python 3.5 on Windows 10 with an Arduino Uno. Any help would be appreciated.
Arduino code:
void readFromPython() {
if (Serial.available() > 0) {
incomingIntegrationTime = Serial.parseInt();
// Collect the incoming integer from PySerial
integration_time = incomingIntegrationTime;
Serial.print("X");
Serial.print("The integration time is now ");
// read the incoming integer from Python:
// Set the integration time to what you just collected IF it is not a zero
Serial.println(integration_time);
Serial.print("\n");
integration_time=min(incomingIntegrationTime,2147483648);
// Ensure the integration time isn't outside the range of integers
integration_time=max(incomingIntegrationTime, 1);
// Ensure the integration time isn't outside the range of integers
}
}
void loop() {
readFromPython();
// Check for incoming data from PySerial
delay(1);
// Pause the program for 1 millisecond
}
Python code:
(Note this is used with a PyQt button, but any value could be typed in instead of self.IntegrationTimeInputTextbox.text() and the value is still not receieved and echoed back by Arduino).
def SetIntegrationTime(self):
def main():
# global startMarker, endMarker
#This sets the com port in PySerial to the port with the Genuino as the variable arduino_ports
arduino_ports = [
p.device
for p in serial.tools.list_ports.comports()
if 'Genuino' in p.description
]
#Set the proper baud rate for your spectrometer
baud = 115200
#This prints out the port that was found with the Genuino on it
ports = list(serial.tools.list_ports.comports())
for p in ports:
print ('Device is connected to: ', p)
# --------------------------- Error Handling ---------------------------
#Tell the user if no Genuino was found
if not arduino_ports:
raise IOError("No Arduino found")
#Tell the user if multiple Genuinos were found
if len(arduino_ports) > 1:
warnings.warn('Multiple Arduinos found - using the first')
# ---------------------------- Error Handling ---------------------------
#=====================================
spectrometer = serial.Serial(arduino_ports[0], baud)
integrationTimeSend = self.IntegrationTimeInputTextbox.text()
print("test value is", integrationTimeSend.encode())
spectrometer.write(integrationTimeSend.encode())
for i in range(10): #Repeat the following 10 times
startMarker = "X"
xDecoded = "qq"
xEncoded = "qq"
while xDecoded != startMarker: #Wait for the start marker (X))
xEncoded = spectrometer.read() #Read the spectrometer until 'startMarker' is found so the right amound of data is read every time
xDecoded = xEncoded.decode("UTF-8");
print(xDecoded);
line = spectrometer.readline()
lineDecoded = line.decode("UTF-8")
print(lineDecoded)
#=====================================
spectrometer.close()
#===========================================
#WaitForArduinoData()
main()
First, this is a problem:
incomingValue = Serial.read();
Because read() returns the first byte of incoming serial data reference. On the Arduino the int is a signed 16-bit integer, so reading only one byte of it with a Serial.read() is going to give you unintended results.
Also, don't put writes in between checking if data is available and actual reading:
if (Serial.available() > 0) {
Serial.print("X"); // Print your startmarker
Serial.print("The value is now ");
incomingValue = Serial.read(); // Collect the incoming value from
That is bad. Instead do your read immediately as this example shows:
if (Serial.available() > 0) {
// read the incoming byte:
incomingByte = Serial.read();
That's two big issues there. Take care of those and let's take a look at it after those fundamental issues are corrected.
PART 2
Once those are corrected, the next thing to do is determine which side of the serial communication is faulty. Generally what I like to do is determine one side is sending properly by having its output show up in a terminal emulator. I like TeraTerm for this.
Set your python code to send only and see if your sent values show up properly in a terminal emulator. Once that is working and you have confidence in it, you can attend to the Arduino side.