1

I am trying to establish a reliable serial connection between embedded device and Python. Which works fine without termination, if Python code is started at the right moment while the serial port is in idle between messages. But it is not practical especially at high data rates. So I decided to do it right and added a carriage return 'x0D' at the end of the source messages to solve the timing problem. But I'm still not sure how to implement it properly in Python.

Note: Size of each serial message is 17 bytes = 4 sensor values payload and CR.

enter image description here

The required data rate is 2.3kHz, but it starts to become unreliable already at 200Hz:

import serial
import struct

serialPort = 'COM3'
serialBaud = 230400
rawData = bytearray(16)
numSensors = 4
dataNumBytes = 4
data = [0 for x in range(numSensors)]

ser = serial.Serial(serialPort, serialBaud, timeout=1)
ser.reset_input_buffer()

while(True):
    rawData = bytearray(ser.read(17).split(b'\x0D')[0])
    for i in range(numSensors):
        bytedata = rawData[(i * dataNumBytes):(dataNumBytes + i * dataNumBytes)]
        data[i], = struct.unpack('f', bytedata)
    print(data)

ser.close()

E.g. at a data rate of 200Hz it works after a few attempts, but then throws an error:

\Python\temp> python .\print_data.py
Traceback (most recent call last):
  File ".\print_data.py", line 22, in <module>
    data[i], = struct.unpack('f', bytedata)
struct.error: unpack requires a buffer of 4 bytes

\Python\temp> python .\print_data.py
Traceback (most recent call last):
  File ".\print_data.py", line 22, in <module>
    data[i], = struct.unpack('f', bytedata)
struct.error: unpack requires a buffer of 4 bytes

\Python\temp> python .\print_data.py
[-487.5, 0.963982343673706, -0.021746518090367317, -0.26979273557662964]
[0.3199999928474426, 0.963982343673706, -0.02106693759560585, -0.26979273557662964]
[0.30000001192092896, 0.9643221497535706, -0.020047569647431374, -0.2680937945842743]
[0.2800000011920929, 0.9622833728790283, -0.018688412383198738, -0.269113153219223]
[0.0, 0.9626231789588928, -0.01800883375108242, -0.27047231793403625]
Traceback (most recent call last):
  File ".\print_data.py", line 22, in <module>
    data[i], = struct.unpack('f', bytedata)
struct.error: unpack requires a buffer of 4 bytes

Additionally I have tried with readline without success.

I would like to know what I am doing wrong.

martineau
  • 119,623
  • 25
  • 170
  • 301
stardust
  • 343
  • 3
  • 17
  • 1
    You might be able to make the code fault-tolerant by putting the call to `struct.unpack()` in a `try`/`except struct.error` — or alternatively by checking the length of `rawData ` which might not be `17` if a timeout occurred. It's a little strange that it looks like you're not getting enough data — are you sure the device can generate data as fast as you're trying to read it? – martineau Mar 11 '20 at 20:26
  • 3
    Binary messages should not be using a line-termination character nor be read as lines. The binary data can easily trigger false end-of-line conditions. For *"reliable serial connection"* the message needs to include check bytes to validate the data integrity; CRC16 should be good enough. See https://stackoverflow.com/questions/16177947/identification-of-packets-in-a-byte-stream/16180135#16180135 You might want to add a start-of message byte to facilitate the lexical scan, but it's not required. – sawdust Mar 12 '20 at 00:40
  • Good to know. I have little to no knowledge of check sum. But I would like to try to implement it. If you know a good example please share. – stardust Mar 12 '20 at 09:55
  • Now I can generate CRC16 with both Arduino and Python. But not sure how to implement frame synchronization. – stardust Mar 14 '20 at 18:35
  • I think it would be enough to detect idle between frames, as it takes couple of time longer that message itself. But start/stop bits is definitely the right way. – stardust Mar 14 '20 at 21:27

0 Answers0