I have data acquisition devices I would like to pull information from. I've started a small project in Python 3.5 using pyserial to communicate to a device. I can send commands and receive data.
import serial
ser = serial.Serial()
ser.port = 'COM1'
ser.baudrate = 9600
ser.parity = PARITY_NONE
ser.timeout=.5
ser.open()
ser.write(b'#02\r')
print(ser.readline())
ser.close()
This sends a command to retrieve data in the buffer, and when I use the readline command, I pull in data.
b'>-999999-999999-999999-999999 -999999\r'
I've created an excel sheet to host data tables and test criteria which I am judging performance of our machines on. This was initially for manual user input, but I decided I'd try and see if I can automate this directly in excel. I've poured over several webpages, found several companies that would ask for payment for code- etc. I've finally settled on work done by The Scarms which uses the WIN32API to deal with serial I/O vs. the original mscomm32.ocx driver.
I've been able to bring his files into my project, and used the sample code to start. I can send a message, and visually verify it from the device I'm communicating through, but I don't get any reply from my end data acquisition device.
strData = "#02\r"
lngSize = Len(strData)
lngStatus = CommWrite(intPortID, strData)
The variable strData is a string. When sending a message using pyserial, it's prefaced by "b" which (to my knowledge) signals it to be encoded to bits before sent through the serial port.
I've been trying to look through the modCOMM code that gets added to VBA from the code provided by the link above, but I can't seem to get an input at all. Am I sending the information incorrectly using the WIN32API?
How do I send this command over the bus properly in order to get a response from the end device?
The end device in question is an Advantech ADAM 4017+.
Related
I am using a barcode scanner as part of a project, everything works correctly until I exit the program then I can't communicate anymore with the barcode scanner. This holds true for whatever program I'm running, be it one of my own or just using screen to monitor the transmissions. As soon as I exit, the only way to make the scanner work again is to unplug and replug.
The scanner (this one) is always mounted correctly (usually at /dev/ttyACM0) and communicates by SSI over USB CDC.
I’ve tried monitoring with pyserial’s miniterm and with screen /dev/ttyACM0 9600 but the same problem arises (f.e. screen just says [screen is terminating])
Mind you, everything works well on another computer so I believe it might be an issue with the Jetson rather than the scanner.
In the project I’m trying to run, I use pyserial to interact with the device. Here is an extract of the code to give you an idea of how I use it:
import serial
serial_port = "/dev/ttyACM0"
baud_rate = 9600
with serial.Serial(serial_port, baud_rate, timeout=0.1) as device_serial:
device_serial.flush()
while True:
try:
# read a line from the serial port
barcode_byte_string = device_serial.readline()
if len(barcode_byte_string) > 0:
# convert the byte string to a string and strip the newline character
barcode = barcode_byte_string.decode("utf-8").rstrip()
# publish the barcode to the topic
self.publish_barcode(barcode, serial_port)
except serial.SerialException as e:
# exit with error code 1. This will cause the application to be restarted.
sys.exit(1)
except Exception as e:
break
I have a WebSocket created using Python WebSocket.
I have a JS client.
The client uses .getUserMedia to capture only audio stream. The client uses MediaRecorder(stream), and .ondataavailable to send the WebSocket a message as a blob.
WebSocket receives the message successfully using message = websocket.recv(), and I am able to write the received audio bytes to a wav file by extending a list.
audio_bytes.extend(message)
if len(audio_bytes) > 20000:
by = io.BytesIO(bytes(audio_bytes))
sound = AudioSegment.from_file(by).export('new_sample.wav', format='wav')
Works fine.
However, I don't want to keep all the bytes in a list for the duration of running this script.
When I clear the list and extend the bytes as before, not picking up the stream from the start causes an issue, or if I slice the original list and try to write to a WAV file using the same method as before.
audio_bytes.extend(message)
if len(audio_bytes) > 20000:
audio_bytes2 = audio_bytes[3000:12000]
by = io.BytesIO(bytes(audio_bytes2))
sound = AudioSegment.from_file(by).export('new_sample2.wav', format='wav')
I want to be able to basically clear the list after the length reaches > 20000, and use y, sr = librosa.load('new_sample.wav'), do further analysis and repeat.
I'm not very experienced in working with audio. I know the headers take up a certain amount of bytes at the start (around 54?).
I stored the first 1000 elements of the list in a header variable header = audio_bytes[:1000] from the first message received in the WebSocket, and extended this header to the new list I created when the length of message received reaches > 20000.
I tested this, and although the first 1000 bytes stayed the same, I was still receiving from FFmpeg:
Invalid data found when processing input
I have also written the bytes directly to a .txt file and loaded the .txt file directly with librosa, again, successful when its the full stream but if I chunk the stream and try to load a chunk at the end, librosa gives the error
Format not recognised
First, thank for fixing my post. I'm still not sure how to include a sketch. I've been reading posts here for many months, but never posted one before.
My headless RasPi is running two sketches of mine, one reads data from a pm2.5 sensor (PMS7003) and the other is the program listed above that sends information to another Pi, the client, that turns on a pm2.5 capable air filter. (I live in California) The program that reads the PMS7003 sorts the data, called max_index, into one of six categories, 0 thru 5 and saves the current category to a text file. I'm using the 'w' mode during the write operation, so there is only one character in the text file at any time. The server program listed above reads the text file and sends it to a client that turns on the air filter for categories above 2. The client sends the word "done" back to the server to end the transaction.
Until you mentioned it, I didn't realize my mistake, clientsocket.recv(2). I'll fix that and try again.
So, the listener socket should go outside the while loop, leaving the send and receive inside???
Troubleshooting: I start the two programs using nice nohup python3 xxx.py & nice nohup python3 yyy.py. The program that reads the PMS7003 continues running and updating the text file with current category, but the server program falls out of existence after a few days. top -c -u pi reveals only the PMS7003 program running, while the server program is missing. Also, there's nothing in nohup.out or in socketexceptions.txt and I tried looking through system logs in /var/log but was overwhelmed by information and found nothing that made any sense to me.
Since writing to the socketexceptions.txt file is not in a try/except block, the crash might be happening there.
import socket
import time
index = " "
clientsocket = ""
def getmaxindex():
try:
with open('/home/pi/pm25/fan.txt','r')as f:
stat = f.read() #gets max_index from pm25b.py
return(stat)
except:
with open("/home/pi/pm25/socketexceptions.txt",'a')as f:
f.write("Failed to read max index")
def setup(index):
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR,1)
s.bind(("192.168.1.70", 5050))
except:
with open("/home/pi/pm25/socketexceptions.txt",'a')as f:
f.write("Failed to bind")
try:
s.listen(1)
clientsocket, address = s.accept()
clientsocket.send(index)
rx = clientsocket.recv(2)
if rx == "done":
clientsocket.close()
except:
with open("/home/pi/pm25/socketexceptions.txt",'a')as f:
f.write("Failed to communicate with flient")
while True:
index = getmaxindex().encode('utf-8')
setup(index)
time.sleep(5)
enter code here
It is unknown what program is supposed to do and where exactly you run into problems, since there is only a code dump and no useful error description (what does "stop" mean - hang or exit, where exactly does it stop). But the following condition can never be met:
rx = clientsocket.recv(2)
if rx == "done":
The will receive at most 2 bytes (recv(2)) which is definitely not enough to store the value "done".
Apart from that it makes not real sense to recreate the same listener socket again and again, just to accept a single client and exchange some data. Instead the listener should only be created once and multiple accept should be called on the same listener socket, where each will result in a new client connection.
I'm looking at trying to read pcap files from various CTF events.
Ideally, I would like something that can do the breakdown of information such as wireshark, but just being able to read the timestamp and return the packet as a bytestring of some kind would be welcome.
The problem is that there is little or no python 3 support with all the commonly cited libraries: dpkt, pylibpcap, pcapy, etc.
Does anyone know of a pcap library that works with python 3?
to my knowledge, there are at least 2 packages that seems to work with Python 3: pure-pcapfile and dpkt:
pure-pcapfile is easy to install in python 3 using pip. It's very easy to use but still limited to decoding Ethernet and IP data. The rest is left to you. But it works right out of the box.
dpkt doesn't work right out of the box and needs some manipulation before. They are porting it to Python 3 and plan to have a Python 2 and 3 compatible version for version 2.0. Unfortunately, it's not there yet. However, it is way more complete than pure-pcapfile and can decode many protocols. If your packet embeds several layers of protocols, it will decode them automatically for you. The only problem is that you need to make a few corrections here and there to make it work (as the time of writing this comment).
pure-pcapfile
the only one that I found working for Python 3 so far is pcapfile. You can find it at https://pypi.python.org/pypi/pypcapfile/ or install it by doing pip3 install pypcapfile.
There are just basic functionalities but it works very well for me and has been updated quite recently (at the time of writing this message):
from pcapfile import savefile
file = open('mypcapfile.pcp' , 'rb')
pcapfile = savefile.load_savefile(file,verbose=True)
If everything goes well, you should see something like this:
[+] attempting to load mypcapfile.pcap
[+] found valid header
[+] loaded 1234 packets
[+] finished loading savefile.
A few remarks now. I'm using Python 3.4.3. And doing import pcapfile will not import anything from it (I'm still a beginner with Python) but the only basic information and functions from the package. Next, you have to explicitly open your file in read binary mode by passing 'rb' as the mode in the open() function. In the documentation they don't say it explicitly.
The rest is like in the documentation:
packet = pcapfile.packets[12]
to access the packet number 12 (the 13th packet then, the first one being at 0). And you have basic functionalities like
packet.timestamp
to get a timestamp or
packet.raw()
to get raw data.
The documentation mentions functions to do packet decoding of some standard formats like Ethernet and IP.
dpkt
dpkt is not available for Python 3 so you need to do the following, assuming you have access to a command line. The code is available on https://github.com/kbandla/dpkt.git and you must download it before:
git clone https://github.com/kbandla/dpkt.git
cd dpkt
git checkout --track origin/migrate_py3
git pull
This 4 commands do the following:
clone (download) the code from its git repository on github
go into the newly created directory named dpkt
switch to the branch name migrate_py3 which contains the Python 3 code. As you can see from the name of this branch, it's still experimental. So far it works for me.
(just in case) download again the code
then copy the directory named dpkt in your project or wherever Python 3 can find it.
Later on, in Python 3 here is what you have to do to get started:
import dpkt
file = open('mypcapfile.pcap','rb')
will open your file. Don't forget the 'rb' binary mode in Python 3 (same thing as in pure-pcapfile).
pcap = dpkt.pcap.Reader(file)
will read and decode your file
for ts, buf in pcap:
eth = dpkt.ethernet.Ethernet(buf)
print(eth)
will, for example, decode Ethernet packet and print them. Then read the documentation on how to use dpkt. If your packets contain IP or TCP layer, then dpkt.ethernet.Ethernet(buf) will decode them as well. Also note that in the for loop, we have access to the timestamps in ts.
You may want to iterate it in a less constrained form and doing as follows will help:
(ts,buf) = next(pcap)
eth = dpkt.ethernet.Ethernet(buf)
where the first line get the next tuple from the pcap file. If pcap is False then you read everything.
How can I manage to periodically read the output of a script while it is running?
In the case of youtube-dl, it sends download information (progress/speed/eta) about the video being downloaded to the terminal.
With the following code I am able to capture the total result of the scripts output (on linux) to a temporary file:
tmpFile = io.open("/tmp/My_Temp.tmp", "w+")
f = io.popen("youtube-dl http://www.youtube.com/watch?v=UIqwUx_0gJI", 'r')
tmpFile:write(f:read("*all"))
Instead of waiting for the script to complete and writing all the data at the end, I would like able to capture "snapshots" of the latest information that youtube-dl has sent to the terminal.
My overall goal is to capture the download information in order to design a progress bar using Iup.
If there are more intelligent ways of capturing download information I will be happy to take advice as well.
Regardless, if it is possible to use io.popen(), os.execute(), or other tools in such a way I would still like to know how to capture the real time console output.
This works fine both on Windows and Linux. Lines are displayed in real-time.
local pipe = io.popen'ping google.com'
for line in pipe:lines() do
print(line)
end
pipe:close()
UPD :
If previous code didn't work try the following (as dualed suggested):
local pipe = io.popen'youtube-dl with parameters'
repeat
local c = pipe:read(1)
if c then
-- Do something with the char received
io.write(c) io.flush()
end
until not c
pipe:close()