Skip to Content

Wiki source code of Telemetry

Last modified by Even Kruse Hvidsten on 2026/06/12 12:29
Show last authors
1 {{warning}}
2 work needing to be still done is referenced in the respective section; once a task has been dealt with delete it; once all task have been done delete this warning
3 {{/warning}}
4
5 (% class="wikigeneratedid" id="H" %)
6 (((
7 {{warning}}
8 RTFM, if you change anything UPDATE THE FUCKING MANUAL
9 {{/warning}}
10 )))
11
12 = Table of contents =
13
14 {{toc depth="3" reference="Software.Low-Voltage.Telemetry.WebHome"/}}
15
16 = Description =
17
18 (% class="box warningmessage" %)
19 (((
20 Specify STM32 board used
21 )))
22
23 The telemetry software section of the project comprised such tasks
24
25 * creating software for relaying CANBUS messages through MQTT via 4G using a Raspberry pi
26 * creating software for reading CANBUS messages from main CANBUS loop using STM32 chip and sending via serial to raspberry
27 * lowering boot time of Raspberry pi
28 * writing script to be launched at boot to connect to the internet via modem
29 * making the SD used by the pi read-only; this is needed as micro-sd cards are failure prone, especially on Rpi hardware
30
31 === Remaining work ===
32
33 here is listed what remains to be done by order of importance
34
35 1. stm32 board code for canbus reading
36 1. 4g modem script
37 1. try lowering boot time by profiling
38 1. make cmake script to compile everything at once, until now ran gcc command with manual linking
39 11. if changes to the location of the json config file are made act accordingly and refer to the section about message relay via rpi
40 11. update section "how to compile" if changed
41 1. maybe make build system to create directly the whole image with the necessary software; this way is storage device fails it is easier to rebuild the whole system
42
43 = Architecture description =
44
45 == Canbus message reader STM32 ==
46
47 Most of it is still **not** implemented, but shouldn't take long to implement.
48
49 A simple loop reading the canbus port and relaying it to the serial interface will be more than enough; depending on board used package filtering could be done either on the STM board or raspberry pi
50
51 == Message Relay via raspberry pi ==
52
53 This is the meaty boy of the project; the file to refer to is mqtt_usb_sender.c.
54
55 Most of the C code created is commented using doxygen, so
56
57 * if modification are made, comment using the same system (i'm seeing you not writing comments 🙂)
58 * for specific function description look at the code; there will be a comment explaining it
59
60 Typical flow of the app
61
62 1. it loads the config found at {{code language="none"}}MQTTUsbSenderConfig.json{{/code}}; this json file will have to be in the same folder level as the main C compiled object.
63 1. will create the connect with the MQTT broker
64 1. will either enter normal or test operating mode
65 11. normal mode will run until killed
66 11. test mode will force join all active thread and print the number of messages each one of them sent
67 1. will launch a thread per serial connection
68 1. such thread will
69 11. setup the serial port (which in linux is simply a file descriptor) by what specified in {{code language="none"}}MQTTUsbSenderConfig.json{{/code}}
70 11. open a reading loop in which will read up to 256 chars, format it into an MQTT message and send it asynchronouslyù
71
72 === Libraries used (and why) ===
73
74 * PahoMQTT
75 ** simple, has C and python implementations
76 ** well documented
77 * CJson 
78 ** more complete libraries exist, but here we need to read just a small config file
79 ** can be linked very simply as it is a single file
80
81 === JSON config file format ===
82
83 {{warning}}
84 remember that if you need to change the config file format you also need to change the parser function {{code language="none"}}loadInitialConfig{{/code}} and the wiki description
85 {{/warning}}
86
87 this is the typical format for the description of serial ports and the broker used by the application
88
89 {{code language="JSON"}}
90 {
91 "ADDRESS": "tcp://192.168.204.107:1883",
92 "CLIENTID": "racingpi",
93 "TESTTIMEOUT": 20,
94 "INPUT": [
95 {
96 "PORT": "/dev/ttyUSB0",
97 "TOPIC": "testTopic",
98 "IN_SPEED": 115200,
99 "OUT_SPEED": 115200,
100 "comment": "hello there,general kenobi"
101 }
102 ]
103 }
104
105
106 {{/code}}
107
108 each single serial port is an element of the list {{code language="none"}}INPUT{{/code}} of this format, comments can be omitted
109
110 {{code language="JSON"}}
111 {
112 "PORT": "/dev/ttyUSB0",
113 "TOPIC": "programmingSocks",
114 "IN_SPEED": 115200,
115 "OUT_SPEED": 115200,
116 "comment": "eat past go fasta"
117 }
118 {{/code}}
119
120 === Test mode ===
121
122 Mostly used for performance evaluation over a short test; the duration of such test is specified in {{code language="none"}}MQTTUsbSenderConfig.json{{/code}}.
123
124 If modifications are needed remember to take into consideration {{code language="none"}}pthread_setcanceltype{{/code}} as this will make sure it will wait for a system call to join the thread when a cancellation was asked; this way you should avoid most errors/lost messages.
125
126 == Connecting the pi to the internet ==
127
128 Still **not** implemented; just a bash script to be added to the startup script referenced in the next section.
129
130 reference material can be found here:
131
132 * reference here [[https:~~/~~/www.jeffgeerling.com/blog/2022/using-4g-lte-wireless-modems-on-raspberry-pi/>>https://www.jeffgeerling.com/blog/2022/using-4g-lte-wireless-modems-on-raspberry-pi/]]
133 * for more info about modem etc refer to [[Electrical>>Electrical.WebHome]]
134
135 == Boot script ==
136
137 While looking around the github page you might find a .sh script called mqtt_boot.sh; this bash script is run by systemD at boot.
138
139 It can be modified to include also the commands to connect via 4g in the future.
140
141 If the underling OS is wiped/changed drastically remember to re-setup such script to launched at boot; asking any LLM how to do it should be enough.
142
143 == Boot time optimization ==
144
145 For now it is simply an headless (IE no desktop interface software like x11, gnome, Etc..) installation of Raspberry OS; this setup starts in around 15-20s.
146
147 Further optimization can be done by profiling the boot to strip not needed functionality; reference to how here
148
149 * [[https:~~/~~/forums.raspberrypi.com/viewtopic.php?t=277010>>https://forums.raspberrypi.com/viewtopic.php?t=277010]]
150 * [[https:~~/~~/forums.raspberrypi.com/viewtopic.php?t=373140>>https://forums.raspberrypi.com/viewtopic.php?t=373140]]
151 * Another option could be using an nvme/usb3 storage sistem
152
153 = How to compile the project =
154
155 for now gcc is used to compile with manual linking
156
157 First follow the installation guides for pahoMQTT anc CJson libraries.
158
159 Then here the typical command i used until now
160
161 {{code language="none"}}
162 gcc -o mqtt_usb_sender mqtt_usb_sender.c -lpaho-mqtt3c -lcjson
163 {{/code}}
164
165 = FAQ =
166
167 ==== **the setup is the one of the original creator, how do i access the pi?** ====
168
169 until now work was done via ssh (credential are user: racingpi , password: 1234) using my personal wi.fi hotspot; the simpler way is to use the physical Lan port of the py, then if needed change the wifi config to the new user.
170
171 ==== **I see some errors in the log from the a serial interface, what is wrong?** ====
172
173 It is mostly a matter of flags used when reading the serial port, refer to the documentation of the attached serial device.
174
175 Also remember that most Linux distros could have some serial issues because of braille devices driver (IE BRLTTY); disabling them could help.
176
177 ==== **I would like to test the software but i don't have a serial device with me** ====
178
179 you're in luck; you can use {{code language="none"}}serialSim.py{{/code}} to simulate a serial device; you just need to open on Linux two fake serial port, put one of them in the sim and the other in the config, then launch the python script
180
181 ==== **I don't have linux** ====
182
183 It's the year of desktop linux; jokes aside, you can compile on the pi, use wsl or just adapt when needed with some wizardry.
184
185
186 = Local testing =
187
188 == Setup ==
189
190 === Connecting to the RPi ===
191
192 Until the PCIe to USB adapter for the modem arrives, use one of the Align computers with LAN port and set the ethernet port from client to a router
193
194 {{code language="none"}}
195 nmcli con mod "[name_of_connection]" ipv4.method shared
196 {{/code}}
197
198 To find the name of the ethernet connection, run
199
200 {{code language="none"}}
201 nmcli con show
202 {{/code}}
203
204 Check that there is a connection between the computer and RPi over ethernet
205
206 {{code language="none"}}
207 nmcli dev status
208 {{/code}}
209
210 Connect to the RPi over ssh, run, and use password 1234
211
212 {{code language="none"}}
213 ssh racingpi@[ip_address]
214 {{/code}}
215
216 To find the IP of the RPi, check the ARP cache with
217
218 {{code language="none"}}
219 ip neigh show
220 {{/code}}
221
222 If its not cached, run
223
224 {{code language="none"}}
225 cd /var/lib/NetworkManager
226 cat dnsmasq-[dev_name].leases // dev_name = DEVICE (nmcli con show)
227 {{/code}}
228
229 === Broker ===
230
231 Currently testing with a local broker running on Align PC (AR-D02), to use the one running on Align server, update the address in the config file on the RPi. To start, stop or check the status of the broker on AR-D02, run
232
233 {{code language="none"}}
234 docker start emqx
235 docker stop emqx
236 docker ps
237 {{/code}}
238
239
240 === Simulating serial-device ===
241
242 To test without telemetry PCB, simulating the serial-device connection, use the networking tool socat to establish a virtual connection through a pseudo-terminal. Run this on the computer sending the fake data. If using the {{code language="none"}}serialSim.py{{/code}} script, the link name can be found in the file, look for VN200_PORT.
243
244 {{code language="none"}}
245 socat pty,raw,echo=0,link=[some_name] TCP-LISTEN:7000,reuseaddr,fork
246 {{/code}}
247
248 Connect the RPi to the other end with (run this on the RPi)
249
250 {{code language="none"}}
251 socat pty,raw,echo=0,link=[some_name] TCP:[some_ip]:7000
252 {{/code}}
253
254 Set the link name to the port name in the config file, and the ip is the default gateway address. The default gateway address is the address assigned to the computer's ethernet interface when using the port as a router. Run {{code language="none"}}ip addr show{{/code}} and look for the ipv4 of the ethernet port device name.
255
256
257 == Running ==
258
259 After the connection is established, run the program on RPi
260
261 {{code language="none"}}
262 ./mqtt_usb_sender
263 {{/code}}
264
265 Then run the python script to send fake data. To see the messages the broker is distributing, subscribe to the same topic (see the config file for name), run
266
267 {{code language="none"}}
268 mosquitto_sub -h localhost -p 1883 -t "[name]"
269 {{/code}}
270
271
272
273