Tag Archives: Raspberry Pi5

Moving Vision Processing to My 4-Wheel Robot, Part 2

Leave a reply

Posted 03 August 2026,

Part 1 in this series laid out the strategy for adding vision processing capabilities to my 4-wheel robot. This post continues that effort.

The first step was to replace the Teensy 3.6 main controller with a Teensy 4.1. As part of this process, I also replaced the glued-on ‘Gnd’ and ‘+5V’ bussbars with a bit more finished product, as you can see in the following photos:

The next step is moving the old 4-wheel firmware to the ‘new-improved’ 4-wheel hardware. I started this by porting “C:\Users\Frank\Documents\Arduino\WallE3_Git\WallE3_Git\WallE3_Git.ino” to “C:\Users\Frank\Documents\Robot_Projects\VisionEnhaned4WheelRobot_V1\VisionEnhaned4WheelRobot_V1.ino”. This took a while, and I’m not sure I wouldn’t have been better off just cloning the entire WallE3_Git.ino file, but I “got it done’.

05 August 2026 Update:

It took a while, but I now have a working 4-wheel drive robot program running again on the 4-wheel chassis with a Teensy 4.1 main controller The only small ‘gotcha’ I encountered during the Teensy 3.5 -> Teensy 4.1 controller switch was the control lines for the right motor driver. It turns out the ‘speed’ (PWM) pin used on the Teensy 3.5 isn’t PWM-capable on the Teensy 4.1, so this required some minor pin swapping, as shown below:

Using the manual control features built accessible via the ‘CheckforUserInput() function, I was able to perform left/right 10 degree turns and speed up/slow down, all from my keyboard. I don’t have the second deck connected up yet, but that is next on the agenda.

Upgraded 4-wheel robot demonstrating basic motor

06 August 2026 Update:

4-wheel robot with Raspberry Pi5 and OAKD-Lite camera mounted on 2nd deck

At this point I have a complete 4-wheel drive robot assembled with a Raspberry Pi5 and OAKD-Lite camera, and I can send it basic (left/right/faster/slower/forward/reverse) movement commands via a serial port. In previous work on the 2-wheel robot I demonstrated the ability to send vision-processing-derived movement commands to the main processor from the Pi5 via a serial port.

It occurs to me that with the above capability, The robot doesn’t really need anything else for navigation and obstacle avoidance. The OAKD-Lite can provide visual depth information to the Pi5, and the Pi5 can send basic movement commands to the robot. In addition, it seems to me that I might be able to eliminate the entire ‘homing-to-charger’ software and hardware blocks, as the camera should be able to tell the Pi5 where the charger dock is located in the field of view, and the Pi5 should be able to tell the robot which way to turn and by how much, and when to speed up and slow down. This could make the robot a LOT simpler (as Elon Musk said about Tesla FSD “Once you have vision solved, you no longer need anything else’).

08 August 2026 Update:

The robot is alive! With Grok’s help I now have a python script running on the Pi5 that can direct the robot to turn left or right depending on image processing by the OAKD-Lite camera.

First, the Python shell script ‘restart_camera.sh’ is executed to get the OAKD-Lite camera ready:

Once the camera is ready, then the demo program ‘clearest_direction_node.py’ is executed on the Pi5 (with ‘VisionEnhanced4WheelRobot.ino already running on the Teensy 4.1)

Based on a rudimentary analysis of the depth (distance) distribution in the camera field of view, the ‘clearest_direction_node.py’ Python script sends left/right/stop commands to the Teensy 4.1 from the Pi5’s serial port to the Teensy 4.1’s ‘Serial1’ port. The commands use the already-existing robot manual motion control command structure shown below:

Here’s a short video showing the action

At this point in the project, we have demonstrated that visual ‘depth’ information can be used to guide the robot, and we have separately demonstrated that the Teensy firmware can be updated ‘Over The Air’ (OTA) via the wifi connection to the Pi5. I believe the next step should be to add wifi OTA to the ‘clearest_direction_node.py’ script in order to remove the requirement for physically attaching a USB cable to the Teensy 4.1 to update the firmware.

Stay tuned!

Frank

2 Wheel Robot with Vision Processing, Part III

Leave a reply

Posted 28 June 2026

When I work on significant projects like the vision-enhanced robot project, I generally hold two different but related mental maps for the overall project.

One map describes the physical and/or logical entities that are needed for the overall project to succeed, such as wheel odometers, battery packs, power regulation/distribution, the OAKD-Lite camera, the pi5, wheel motors and drivers, etc.

The second mental map describes the software (here the term ‘software’ includes both the pi5 software and related teensy firmware) pieces needed to give the robot the ability to do what we want, primarily the enhanced navigation possible with vision processing via the OAKD-Lite camera and associated software. This mental map (at least for me) seems to be project oriented, where each project addresses different, mostly independent software capability implementations (Wifi_OTA, with its precursor projects ‘SerialPassthroughDemo’, Wifi_OTA_Demo, etc).

I tend to work on a big project like this from the top (system-view) and bottom (small sub-projects that will be later integrated into the overall project) at the same time. I have learned over the years that creating (and later modifying as needed) a clear top-down system architecture is absolutely crucial to improving the chances of getting someplace that looks like where you wanted to go. This systems architecture is the ‘aspirational view’ (to borrow a modern Elon-ism) of the project’s long-term goal. In the case of the vision-enhanced robot (VER), the goal is similar to the one for my 4-wheel robot, i.e. “Autonomous navigation around our home”. However, instead of ‘wall-following’ as the 4-wheel robot did, this project will utilize vision processing at the primary navigation technology.

Over the last few weeks, Grok Code and I have been working at the bottom of the systems architecture with things like getting OTA updates for Teensy firmware working, both via the PC -> Bluetooth -> HC-05 -> Teensy Serial channel and most recently, via the PC -> Wifi -> Pi5 -> Teensy Serial channel. After (mostly) getting the Wifi_OTA capability going, I decided it was time to stop and make sure we had a Git Repo structure consistent with the top-down view. After the requisite amount of fumbling around, we (me and Grok Code) came up with the following structure.

my_vision_robot/
├── .gitignore
├── README.md
├── docs/ ← Overall project documentation
│
├── hardware/ ← Schematics, BOMs, mechanical (future)
│
├── shared/ ← Code used by multiple projects
│ ├── firmware/ ← Shared Teensy code/libraries
│ └── software/ ← Shared Pi5 Python modules
│
├── software/
│ ├── SerialPassthroughDemo/
│ │ ├── README.md
│ │ ├── pi5/
│ │ ├── teensy/
│ │ └── docs/
│ │
│ ├── Wifi_OTA/
│ │ ├── Wifi_OTA.py
│ │ ├── README.md
│ │ ├── pi5/ (if needed later)
│ │ ├── teensy/
│ │ └── docs/
│ │
│ └── Vision_Navigation/ ← Future main vision project
│ ├── pi5/
│ ├── teensy/
│ └── docs/
│
├── tests/ ← System/integration tests
└── tools/ ← General one-off utilities

Now the challenge is to move all my sub-project files from our current quite-messy repo structure into this one. Standby!

07 July 2026 Update: Wrapping up the Wifi_OTA Demo project

This little project had more than its share of bumps and bruises, but I think Grok Code and I have finally got it done. The project has four major parts; three on my PC and one on the pi5.

  • The Wifi_OTA Visual Studio/Visual Micro project on my PC
  • A ‘board.txt’ file in the same folder as the Wifi_OTA project. This file tells VS what to do after the build step
  • A small Python script that copies the .HEX output from the compile to the ‘latest.hex’ file on the pi5
  • A Python program on the pi5 that watches for updates to ‘latest.hex’ and when one is detected, passes that along the the teensy on its Serial1 port

Here is the Wifi_OTA.ino file: All it does is blink the built-in LED a couple of times and then waits for a ‘U’ character to start the flash update process.

Here is the ‘board.txt’ file that calls the PostBuild.py program when an F5 debug compile is completed:

And here is the Python script that copies the .HEX output from the compiler over to ‘latest.hex’ on the pi5:

Here’s a short video showing the Wifi_OTA update process. The video starts just after I pressed F5 to start the debug compile on my PC. After about two seconds, the built-in LED on the Teensy 4.1 (lower-left foreground) goes OFF when the HEX file transfer starts. The file transfer takes about 30 sec, and then a few seconds after the file transfer finishes, the built-in LED on the Teensy 4.1 blinks twice and then stays ON, confirming that the update was successful.

A significant part of this little sub-project was getting the Git repository set up and running, on my PC, on the pi5, and on GitHub. The same folder structure is used in all three locations, but the pi5 side only updates entries in the ‘pi5’ subfolders, and the PC side only updates the Teensy firmware. When I do an update either on the PC or the pi5, I follow the same steps each time:

  • git pull origin main <<— synchs the local repo with the master on Github
  • git add -A <<– adds anything new to the local repo
  • git commit -m “put my update description here”
  • git push origin main <<– synchs the Github repo with the updated local repo.

    At this point I believe I have the basic infrastructure in place to proceed with the real project of adding vision-processing-based navigation capabilities to the robot
  • The OAKD-Lite camera is installed and confirmed working
  • The pi5 is installed and connected to the OAKD-Lite camera and via Serial1 to the Teensy4.1.
  • Teensy firmware can be updated via the new Wifi_OTA update channel
  • The Hall-effect wheel encoders are installed and confirmed working.
  • The next big step is to integrate the ROS (Robot Operating System) with the OAKD-Lite camera and the Teensy.

Stay Tuned!

Teensy Firmware OTA Update Via SSH to Pi5

Leave a reply

Posted 21 June 2026

After getting OTA via Bluetooth to the onboard HC-05 going for the 2-wheel robot, I had an epiphany; For vision processing the 2-wheel robot uses an on-board Raspberry Pi5 with a Wi-Fi connection to my local network (and thence to my PC), so why use the HC-05 link at all? The Wi-Fi connection is much more robust than the BT/HC-05 link and is available throughout the house. The pi5 has a serial port, so in theory I could simply write a small python script to pipe characters back and forth between its Wi-Fi port and its serial port, just as the HC-05 does between its BT port and its serial port. How hard could it be?

Grok Code and I have been working on this issue for a while now and have gotten to the point where we can transfer the .HEX file from the pi5 to the Teensy once, but not multiple times. This indicates that the firmware update did not happen correctly. Also, we haven’t yet figured out how to automatically transfer the HEX file resulting from a Visual Studio compile to the pi5 so that it can be passed to the Teensy via the pi5’s serial port so we are bypassing this step by using SCP (or a copy/paste using VS Code) to create a duplicate of the HEX file on the pi5; then all the pi5 script has to do is pass lines from the local HEX file to the Teensy via serial.

Grok Code and I have been trying to troubleshoot this problem, and we don’t seem to be getting anywhere. Grok does not really know how to troubleshoot in an organized manner – it is more of a ‘random walk’ process. This post is intended to document my own troubleshooting efforts.

First, what is the basic problem? The basic problem is that multiple transfers of a HEX firmware file to the Teensy using the established BT/HC-05/Serial2 succeed, but the same process using the Wi-Fi/pi5/Serial1 link appears to succeed the first time but fails on the second attempt. Since the firmware HEX files in the two cases are identical, the problem must be somewhere in the pi5 script, either in the way lines are read from the local HEX file or in the way lines are transferred to the Teensy.

A basic assumption in the above is that the HEX file transferred to the Teensy via BT/HC-05 and the HEX file transferred to the Teensy via Wi-Fi/Pi5 are identical, so I decided to start there. Are they really identical?

  • Compiled firmware on VS, copy/pasted (using VS Code) from “C:\Users\Frank\Documents\Arduino\Wifi_OTA_Demo\obj\x64\Debug\Wifi_OTA_Demo.hex” to “/home/pi/my_vision_robot/tests/Wifi_OTA_Demo/Wifi_OTA_Demo.hex”. Then I copy/pasted from the pi5 file to notepad++ and compared with the original – they matched perfectly.
  • I modified FxUtil.cpp’s update_firmware() to add the line “out->println(line);” then updated Teensy firmware using USB connector to establish ‘known-good’ baseline. Then used pi5 script to transfer its local copy of the firmware to the Teensy, logging the transfer via VS serial monitor. The file as logged going into the Teensy and the source file on the pi5 also match perfectly. This pretty much eliminates a corrupted file transfer as the source of the problem.
  • Then I performed the same procedure except using the BT/HC-05 channel instead of the Wifi/Pi5 channel.

Here’s the Wifi_OTA_Demo.ino file used to run the above tests:

25 June 2026 Update:

Grok Code and finally managed to get the Teensy OTA update via Pi5/Serial1 working. Here is the final Python script on the Pi5:

And here is the Teensy sketch used for the test:

To verify that the OTA update code performed properly, I ran the update three times in a row. The first time replaces whatever sketch was on the Teensy with my test sketch (that also includes the code that supports further updates), and the second and third updates confirm that the first (and the second) updates contained the crucial update support code.

At this point the ‘Wifi_OTA_Demo’ project is pretty much finished. The only step remaining is to fully automate the process.

27 June 2026 Update:

Now I have the update process updated to the point where I can press ‘F5’ in Visual Studio in an OTA-equipped Teensy project and have the resulting .hex file automatically copied over to the pi5. Here are the relevant files:

board.txt:

Updated 06/27/26 Post-Build: Copy .hex to Pi5

PostBuild_OTA.py:

The next step is to put these two steps (copy the .hex file to the pi5, and then launch the update code that xfers the .hex file to the Teensy) together into one seamless automated process.

08 August 2026 Update:

I recently got my 4-wheel robot going again and moved the Raspberry Pi5/OAKD-Lite over to it from the 2-wheel robot, and ported my WallE3_Git.ino program to it as well. Due to some changes made to file locations on both my PC and on the Pi5, some changes to the OTA-related files were necessary as well. Here are the updated files:

Board.txt (lives in the same folder as the project & .ino file):

C:\Users\Frank\Documents\Robot_Projects\my_vision_robot\software\Wifi_OTA\teensy\Wifi_OTA\PostBuild_OTA.py:

C:\Users\Frank\Documents\Robot_Projects\my_vision_robot\software\Wifi_OTA\pi5\Wifi_OTA.py:

These files are also saved on my Github site in the my_vision_robot repo.

These updates went very smoothly, all things considered. The only minor hitch in the get-along was getting the new .hex file copied over from my PC to the Pi5 via SCP without having to enter the Pi5 password. Here’s the relevant material from Grok:

The password prompt

That’s expected if SSH key authentication isn’t set up (or the key isn’t loaded). scp is falling back to password auth, which blocks a fully automatic F5 experience.

Fix (one-time setup on the PC):

  1. Make sure you have an SSH key pair:

Bash

# in PowerShell or Git Bash

ls ~/.ssh/id_rsa.pub

# or

ls ~/.ssh/id_ed25519.pub

  1. Copy the public key to the Pi (if not already done):

Bash

ssh-copy-id pi@RobotPi5

(or manually append the .pub contents to ~/.ssh/authorized_keys on the Pi)

  1. Test that it works without a password:

Bash

scp some_small_file pi@RobotPi5:~/my_vision_robot/firmware/test.txt

After that, the post-build scp should run silently. (this still failed):

No SSH key exists yet. Generate one (ed25519 is preferred):

In PowerShell:

PowerShell

ssh-keygen -t ed25519 -C “frank@vision-robot”

  • Press Enter to accept the default file location (C:\Users\Frank\.ssh\id_ed25519)
  • You can set a passphrase or leave it empty (empty = fully automatic, no prompts)

Then copy the public key to the Pi:

PowerShell

type $env:USERPROFILE\.ssh\id_ed25519.pub | ssh pi@RobotPi5 “mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys”

(It will ask for the Pi password one last time.)

After that, test:

PowerShell

scp $env:USERPROFILE\.ssh\id_ed25519.pub pi@RobotPi5:~/test_key.txt

If that works without asking for a password, F5 will also be password-free.