Author Archives: paynterf

Moving Vision Processing to My 4-Wheel Robot, Part 2

Leave a reply

Posted 03 August 2026,

Part 1 in this series laid out the strategy for adding vision processing capabilities to my 4-wheel robot. This post continues that effort.

The first step was to replace the Teensy 3.6 main controller with a Teensy 4.1. As part of this process, I also replaced the glued-on ‘Gnd’ and ‘+5V’ bussbars with a bit more finished product, as you can see in the following photos:

The next step is moving the old 4-wheel firmware to the ‘new-improved’ 4-wheel hardware. I started this by porting “C:\Users\Frank\Documents\Arduino\WallE3_Git\WallE3_Git\WallE3_Git.ino” to “C:\Users\Frank\Documents\Robot_Projects\VisionEnhaned4WheelRobot_V1\VisionEnhaned4WheelRobot_V1.ino”. This took a while, and I’m not sure I wouldn’t have been better off just cloning the entire WallE3_Git.ino file, but I “got it done’.

05 August 2026 Update:

It took a while, but I now have a working 4-wheel drive robot program running again on the 4-wheel chassis with a Teensy 4.1 main controller The only small ‘gotcha’ I encountered during the Teensy 3.5 -> Teensy 4.1 controller switch was the control lines for the right motor driver. It turns out the ‘speed’ (PWM) pin used on the Teensy 3.5 isn’t PWM-capable on the Teensy 4.1, so this required some minor pin swapping, as shown below:

Using the manual control features built accessible via the ‘CheckforUserInput() function, I was able to perform left/right 10 degree turns and speed up/slow down, all from my keyboard. I don’t have the second deck connected up yet, but that is next on the agenda.

Upgraded 4-wheel robot demonstrating basic motor

06 August 2026 Update:

4-wheel robot with Raspberry Pi5 and OAKD-Lite camera mounted on 2nd deck

At this point I have a complete 4-wheel drive robot assembled with a Raspberry Pi5 and OAKD-Lite camera, and I can send it basic (left/right/faster/slower/forward/reverse) movement commands via a serial port. In previous work on the 2-wheel robot I demonstrated the ability to send vision-processing-derived movement commands to the main processor from the Pi5 via a serial port.

It occurs to me that with the above capability, The robot doesn’t really need anything else for navigation and obstacle avoidance. The OAKD-Lite can provide visual depth information to the Pi5, and the Pi5 can send basic movement commands to the robot. In addition, it seems to me that I might be able to eliminate the entire ‘homing-to-charger’ software and hardware blocks, as the camera should be able to tell the Pi5 where the charger dock is located in the field of view, and the Pi5 should be able to tell the robot which way to turn and by how much, and when to speed up and slow down. This could make the robot a LOT simpler (as Elon Musk said about Tesla FSD “Once you have vision solved, you no longer need anything else’).

08 August 2026 Update:

The robot is alive! With Grok’s help I now have a python script running on the Pi5 that can direct the robot to turn left or right depending on image processing by the OAKD-Lite camera.

First, the Python shell script ‘restart_camera.sh’ is executed to get the OAKD-Lite camera ready:

Once the camera is ready, then the demo program ‘clearest_direction_node.py’ is executed on the Pi5 (with ‘VisionEnhanced4WheelRobot.ino already running on the Teensy 4.1)

Based on a rudimentary analysis of the depth (distance) distribution in the camera field of view, the ‘clearest_direction_node.py’ Python script sends left/right/stop commands to the Teensy 4.1 from the Pi5’s serial port to the Teensy 4.1’s ‘Serial1’ port. The commands use the already-existing robot manual motion control command structure shown below:

Here’s a short video showing the action

Here’s another short video showing the robot actually making 10-deg turns left & right in response to a simulated obstacle in the field of view:

And here is some of the telemetry output from the above run:

At this point in the project, we have demonstrated that visual ‘depth’ information can be used to guide the robot, and we have separately demonstrated that the Teensy firmware can be updated ‘Over The Air’ (OTA) via the wifi connection to the Pi5. I believe the next step should be to add wifi OTA to the ‘clearest_direction_node.py’ script in order to remove the requirement for physically attaching a USB cable to the Teensy 4.1 to update the firmware.

08 August 2026 Update:

Adding Wifi OTA to the current ‘clearest_direction_node.py’ configuration turned out to be relatively easy, as this ability had already been developed as a stand-alone demo back in June 2026. All we had to do was update the ‘board.txt’ file to point to the new location of ‘PostBuild_OTA.py’ on my PC, and make sure ‘Wifi_OTA.py’ was running on the Pi5 so it would look for changes in “/home/pi/my_vision_robot/firmware/latest.hex” and trigger the update.

09 August 2026 Update:

Thinking while drifting off to sleep last night, I realized that the addition of the Pi5 to the robot means that all the navigation and obstacle avoidance tasks (including possibly navigating to and connecting with the charger module) will be handled by the Pi5, which significantly simplifies the Teensy 4.1 code. All the Teensy code has to do now is respond to motion commands from the Pi5, monitor the various hardware sensors and update the rear-facing LED array.

Teensy Responsibilities:

  • Run the motors in response to motion commands from Pi5 on Serial1
  • Update the ‘state-of-charge’ (and left/right steering) LEDs
  • Monitor the output from three different 1NA169 current sensors (I_chg, I_tot, I_upper-deck)
  • Monitor the output from two different Vbatt voltage sensors (V_batt, V_upper-deck)
  • Monitor the ‘charger connected’ output from the TPS5100 battery pack
  • Monitor the rear-facing VL53L1X LIDAR for rear collision avoidance
  • Run the speaker
  • Home in on and connect to the charger module (may be done by the Pi5 instead)

Pi5 Responsibilities:

  • Manage all navigation and obstacle avoidance tasks
  • Possibly manage charger module homing (April tag?)
  • Create map (SLAM)?
  • Respond to rear-facing obstacle alerts from Teensy
  • Pass robot telemetry data on Serial1 to PC
  • Pass Pi5-software-related telemetry data to VS Code terminal

10 August 2026 Update:

Good progress on the vision-enhanced robot project today. Yesterday we installed two additional cases in the switch block that accepts manual control commands for the robot, to allow the user to command left/right turns of an arbitrary number of degrees at an arbitrary turn rate. The command format consists of the letter ‘L’ (left/CCW) or ‘R’ (Right/CW) followed by the number of degrees to turn (and optionally a specified turn rate in deg/sec), but the commands failed. Today (while waiting in the cellphone lot for my wife) I figured out what was wrong with the teensy command parsing code. Now I can type “L30.5” at a VS Code prompt that gets transmitted via SSH over Wifi to the Pi5, then via the dedicated serial to the Teensy. Then the Teensy calls ‘SpinTurn(30.5, rate = 30) to actually effect the turn. Here’s a short video showing a left (CCW) turn of approximately 90 deg at a 90 deg/sec rate, followed by another left turn of 90 deg at the default 30 deg/sec rate, followed by a right turn of 180 deg at the default 30 deg/sec rate:

We also implemented a replacement for my old telemetry display using the Visual Studio serial port monitor. In theory I could maintain that feature as I still have the HC-05 Bluetooth serial module installed on the robot, but I would rather put everything over wifi for better range & reliability. What we did instead was to have WallE_5.py log everything that comes to it on its serial port (Teeny ‘Serial1’) to a ‘telemetry.log’ file, which allows me to start up a new Pi5 terminal session in its own window and run

to display whatever telemetry information the teensy is sending to ‘Serial1’. Here’s a short sampling of the telemetry data setup:

15 August 2026 Update:

I discovered that the Teensy 4.1 was unresponsive – yuk! After some research and troubleshooting, I realized that I had fallen into a classsic Teensy 3.5->4.1 transition trap; Teensy 3.5 is 5V tolerant on (almost) all pins, while the 4.5 is not. I didn’t really look hard at the circuit before making the 3.5->4.1 change, but almost all the peripheral hardware produces 5V signals on the MCU pins – oops!

So, I have spent the last few days going backwards – from Teensy 4.1 to Teensy 3.5. I took the opportunity to improve wire/pin labelling while doing the reverse transition. Here is the updated schematic:

And here are some photos showing the wire labelling:

I created a small test program, originally to just test the rear-facing VL53L1X LIDAR, but later expanded to test all sensors. Here’s the code:

And here’s the output showing all three current sensors, the battery voltage, and the rear-facing VL53L1X LIDAR output (variations in the rear distanced are due to my waving my hand in front of the sensor).

I wasn’t quite sure what to make of the zero readings for Bot_I, but then I realized that when the USB cable is plugged into the Teensy 3.5, it provides all the current needed for the first deck peripherals, meaning the current through the Bot_I INA169 sensor is essentially zero.

Next I ported the MPU-6050 support code from the VisionEnhanced4WheelRobot project into my little demo program and put some code in ‘loop()’ to add the IMU heading output to the sensor line. This didn’t work at first, but with Grok’s help I traced the problem to a disconnected power/ground lead from the main power distribution bus to the MPU6050. Once this was connected, the demo program started working right away:

I also installed and tested a front-facing red laser diode and exercised the on-board speaker. At this point I believe I have exercised/tested all the sensors, the laser diode and the speaker. I haven’t run the motors with this program as I believe that will happen when I get back to the ‘mainline’ VisionEnhanced4WheelRobot.ino program.

16 August 2026 Update:

I back-ported the code from my VL53L1X_Demo program back to my VisionEnhanced4WheelRobot program and after the normal number of mistakes, got it going. Here’s a small bit of output from loop(), with me rotating the robot back and forth to show variation in the ‘Hdg’ and ‘RearCm’ values:

Then I fired up the VS Code connection to the pi5, started WallE_5.py supervisor program and verified that I could send motor commands in the WallE_5.py window and have the robot respond appropriately – yay. I also verified that ‘tail -f ./logs/telemetry.log’ in a separate VS Code terminal window produced the same telemetry output as above.

17 August 2026 Update:

I finally have the ‘VisionEnhanced4WheelRobot’ program running again with a Teensy 3.5. The last little bit was undoing the change in motor control pin assignments for the right-hand motor driver. Going from Teenys 3.5 to 4.1 I had to change the pin assignments to place the ‘speed’ pin on a PWM-able output pin, and going back I didn’t initially undo this pin assignment – all fixed now.

At this point the robot firmware is pretty much complete – it correctly monitors all three current sensors, the battery voltage, the current heading, and the rear distance reading. I think the next step is to work on the python side to add more navigation and obstacle-avoidance ‘smarts’.

Stay tuned!

Frank

Generating Pull Request for Vin’s MissChanger Klipper-Toolchanger Repo

Leave a reply

Posted 29 July 2026

In a prior post, Grok help me figure out why I was having so much trouble trying to change my Voron 2.4 printer from ‘dock’ to ‘no_dock’ configuration. Solving this problem exposed a pretty significant flaw in the way that config_switch.py managed the process of changing the printer.cfg file to reflect the new configuration. After getting everything to work, I suggested we generate a pull request to Vin’s Klipper-toolchanger repo to make the config swap process a bit more dependable. This post is intended to document this effort.

We started out by making sure we had all the preparatory work done.

  • I copied yet another set of config files to my PC
  • 1. Update my clone of Vin-y’s klipper-toolchanger repo on my printer, so all I had to do was update it with: cd ~/klipper-toolchanger, git pull. Now my local repo is the same as Vin-y’s main branch

    cd ~/klipper-toolchanger git status # just to see where you are git pull origin main
  • Created a new branch so we can make our changes without touching the main branch.Just type this one command and show me the result:bash

    git checkout -b fix-config-switch-in-place
  • In VS Code on my PC, I navigated to /home/pi/klipper-toolchanger/klipper/extras/config_switch.py and replaced its contents with the file generated by Grok.

Then we restarted the printer. Klipper came up cleanly, so that’s a good sign!

After testing the new script with some round-trip ‘TOGGLE_CONFIG_MODE‘ commands, the new script looks like it is performing correctly.

Grok then guided me through the process of updating the local repo:

Create a new branch to make the changes without changing the ‘main’ branch:

Then in VS Code we overwrote the existing klipper-toolchanger / klipper / extras / config_switch.py with the new version (see above), saved the file and then restarted Klipper with “sudo systemctl restart klipper”

Testing:

At the start of the test program, ‘printer.cfg’ contains the ‘no_dock’ configuration. Then I ran SAVE_CONFIG_MODE fro the mainsail console, and got the following output:

Most notably, now the ‘config_no_dock.cfg’ file has both the ‘#;<‘ and ‘#;>’ markers – yay!

To go the other way, we edited the ‘config_wt_dock.cfg’ file to add the closing marker ‘#;>’ to the end of the file, and then ran TOGGLE_CONFIG_MODE again to swap out the ‘no_dock’ “Session Variables” with teh ‘with_dock’ ones and then checked to make sure the Voron would reboot successfully (it did – yay!) and the “session variables” and ‘SAVE_CONFIG’ sections were correct.

Pull Request:

First we committed all the changes (principally the revised ‘config_switch.py’ script) to the local repo:

Then before we do anything else, we need to fork Vin-y’s klipper-toolchanger repo to my Github account with

When the push is performed, Github creates a link that can be used for the pull request:

Here’s what the pull request looks like on Github, just before clicking the ‘Create Pull Request button:

And here’s what it looks like after creating the pull request:

It feels good to be able to contribute something of worth to the MISSChanger community, as I am certain that others have run into similar problems with TOGGLE_CONFIG_MODE.

Stay Tuned,

Frank

New Current Sense & Step-Down Regulator Board

Leave a reply

In my old 4-wheel robot setup, I had a 5V regulator built around an LF50CV low-voltage dropout 3-pin regulator. Unfortunately, with the added current drain of the Pi5 and OAKD-Lite Camera, the LF50CV started dropping the output voltage to the point where the Pi5 started to get flaky. In discussing this with Grok, he suggested The Pololu D24V50F5 step-down regulator shown below:

I did some initial testing on the module:

  • At 9.1V at the ‘Switched Battery input, the Vbatt monitor voltage was 3.3V – the upper limit for inputs to a Teensy 4.1. Since 9.1 is well above even the high charging limit (~8.4V), this seems pretty safe.
  • At a measured Irun = 0.2A, Irun_out measured 90mV
  • At a measured Irun = 1.5A, Irun_out measured 0.4V
  • at a measured run = 1.14A, Irun_out measured 0.255V

Then attached a USB Type C male pigtail to the regulator output and then plugged it directly into the Raspberry Pi5. The Pi5 booted right up and I was able to access it from my PC using VS Code. Regulator output voltage with Pi5 connected was 4.99V at a measured input current of about 0.55A.

Here’s a photo showing the current sense/regulator module already hot-glued to its 3D-printed mounting plate.

30 July 2026 Update:

Yesterday the plan was for the existing LF50CV-based LDO 5V voltage regulator to be replaced with the above D24V50F5 DC-DC step-down regulator. However, as I was waiting for sleep last night, I was thinking about the increased 5V current level required by the Raspberry Pi5 and the OAKD-Lite camera and the fact that in the current (no pun intended) design, that 5V power comes from a module on the lower deck. So even with the higher current capacity (nominally 5A max) provided by the D24V50F5 regulator, there is still going to be a significant voltage drop in the approximately 2 meters of 24-guage stranded hookup wire and the TE Connectivity ‘MATE-N-LOK inter-deck connector. As Grok reminded me, there is not only the 1m line TO the Pi5/Camera, there is also the ground wire coming BACK for a total of approx 2m and two times through the connector. Grok noted that the total round-trip IR drop at 3A would be about 0.5V, which would place the input voltage at the Pi5 at about 4.5V, way too low for reliable operation.

That made me realize that even if I replaced the LF50CV-based LDO 5V voltage regulator with the D24V50F5 5A module, it *still* wouldn’t provide a clean 5V to the Pi5. It was at that point that the way to skin this cat was to leave the LF50CV-based LDO 5V voltage regulator in place on the bottom deck and move the D24V50F5 5A module to the top deck where the wire length to the Pi5 would be more like 0.1m instead of 2m – problem solved!

Well, not entirely, as I still have to get +Vbatt (nominally 8V) up to (and back from) the second deck, with the same 2m + 2x through the connector as before. But now that same 0.5V IR drop is applied to Vbatt instead of +5V, and that makes the problem go away entirely – yay!

Looking at a recent system schematic:

I see that +5V from the LF50CV goes through the interdeck connector on pin 1 (red), and the return path is on pin 2 (black). I believe all I would have to do is move the red wire from the output to the input of the LF50CV module. No, not that simple – the LF50CV output goes directly to the Teensy main controller 5V buss, and a separate wire goes from that 5V buss up through the interdeck connector to a terminal strip. That wire will have to be disconnected from the Teensy 5V buss and re-connected to Vbatt at the input to the LF50CV module. At the other end, the wire will have to be disconnected from the terminal strip and routed instead to the ‘Batt’ terminal of the D24V50F0 module. The output (both +5 and GND) from the D24V50F0 module must be routed back to the terminal strip. At this point, the two 5V regulator modules will be tied to the same (GND) reference, but supply entirely different circuits. Here’s a photo of the D24V50F0 module installed on the upper deck.

With this setup I was able to run the Pi5 for several hours before the +5V supply dropped to low to continue.

I need at least four additional lines from the Teensy 4.1 to the 2nd deck. Two lines for Serial1-Pi5 comms and two for Vbatt/Irun sensors on the 2nd deck 5V regulator board. The existing Brn/Wht wire pair has already been designated on the schematic for Serial1-Pi5 comms, and the existing but unassigned Blu/Gry pair can be used for the sensor lines. Note that the robot now sports *three* 1NA169 current sensor modules (the third one is in the base with the battery pack), and two of those also provide (Vbatt – 6V) voltage outputs. The pre-existing gray wire on the second deck was used to reset the Teensy 3.5 used to control the VL531LX LIDAR sensors but since the Teensy 3.5 (and all but one of the VL53L1X sensors) was removed, the Gry wire is no longer used. Here’s the updated schematic:

Building an Improved Under-monitor Accessories Tray

Leave a reply

Posted 25 July 2026

For quite a while I have used the accessories tray shown below, but I have become increasingly unhappy with the quality with it; in particular the small tray sections aren’t quite big enough to handle all my USB sticks, micro-SD cards and holders, spring-clips, etc. So, after trying in vain to find a better model on-line, I decided I would design and 3D print one for myself, using my 300×300 Voron 3D printer in the ‘no_dock’ configuration.

Here’s where I am at the moment:

preliminary design for under-monitor accessory tray. Note that this is only 1/2 of the finished product

I started out with the idea of using removable partitions to fix the problem of having too-small or too-large bins for whatever I wanted to store. The available free distance under the riser is about 270mm so I made the width (the ‘Y’ axis above) slightly wider than original at 260mm. The above model is about 237mm long (the ‘X’ direction above), but is only 1/2 length, as the full length won’t fit on the Voron’s 300x300mm print bed, so I’ll have to figure out some way of connecting the two halves.

26 July 2026 Update:

The idea I came up with to connect the two halves of the accessory tray was to place holes in the ‘front’ half of the tray, and matching posts in the ‘back’ half of the tray, as shown in the screenshot below:

Now that I have my ‘no_dock’ printer configuration working properly, I printed up the small section of the back half of the tray, along with just a small section of the mating end of the front half of the tray, just to make sure the mating idea would work. It seemed to work pretty well, as shown in the following photos:

Mating parts shown; posts on the back half, holes on the front
Parts mated together. The gap at the top is due to the front half piece lifting off the heated bed

The posts on the back half came out a little scruffy due to small diameter and their length, but still worked well after a bit of cleanup. In the final version I think I will increase the diameter a bit and reduce the length. The gap shown in the mated photo above I believe was due to the way I printed the part, and shouldn’t be a factor in the finished product – we’ll see!

The next step was to create the rear section sketch. First I tried do this by simply copy/pasting the entire ‘2-bin_Tray’ section, but the result was ‘too many’ broken references, as shown below:

‘Too many’ broken references

But after some consultation with Grok about using the ‘use’ feature, I simply created a new sketch on the same plane as the original ‘2-bin Tray’ sketch, then activated the ‘Use’ feature and dragged a rectangle to select the entire original sketch. This produced a new, unlinked sketch which I moved to a non-interfering location with the transform tool, as shown below:

A better ‘copy/paste’ using the ‘Use’ tool, selecting everything, and then using the ‘Transform’ tool to move it

The rear section of the tray will only be 120mm long instead of 240, as the total length of the tray is 360mm. Here are some photos of the constituent parts:

Assembled tray, waiting on ‘PET-G Gloop’
Pegs in the smaller part fit into holes in the bigger one
Removable partitions in three lengths

09 August 2026 Update:

After the ‘Gloop’ arrived I was able to glue the front & back sections together with a liberal application of ‘gloop’ to both sides of the junction and to the top and bottom of the tray once the sections were joined. Then transferred all my junk from my old tray to the new one and installed it under my monitor riser, as shown in the following photos:

Moving Vision Processing to My 4-Wheel Robot, Part 1

Leave a reply

Posted 08 July 2026

In this previous post, I describes my work with Grok Code to successfully demonstrate the use of vision processing with a Luxonis OAKD-Lite camera and a Raspberry Pi5 to generate ‘best clear direction’ advisories for robot motion control.

After achieving the above milestone, I started thinking about the next steps. Clearly I want to move on to achieve actual robot navigation utilizing the above ‘best direction’ information, but the current 2-wheel robot battery pack doesn’t really provide enough power for the Pi5, the OAKD-Lite camera, and the wheel motors. Consequently, I started thinking about moving the vision processing project from the 2-wheel robot onto my existing 4-wheel platform to take advantage of the latter’s larger battery pack. Looking at the pinouts for the Teensy 3.6 and Teensy 4.1 along with the 2-wheel and 4-wheel schematics, I developed the following pin assignment spreadsheet:

Pin assignments for the 2-wheel to 4-wheel transfer

In the above (hopefully self-explanatory) spreadsheet there was only one significant conflict. On the current 2-wheel robot pins 29-32 are occupied by the four Hall-effect wheel encoders, and on the current 4-wheel robot these pins are used by four of the six battery charge indicator LED’s, and that function carries over into the new vision-enhanced robot. Easy enough resolution, just move the wheel encoder pins to 7-10 (actually even easier, I don’t need to move the encoders over until I actually have software/firmware that needs them, and that might be a while).

I started out thinking I would 3D print another ‘2nd deck’ plate so I wouldn’t have to disassemble the original one. Then I came to my senses and realized that I had originally constructed the 2nd deck expressly so it *could* be disassembled – duh! Here’s the mostly disassembled 2nd deck plate:

Top. Terminal strip and rear LIDAR are reused
Bottom: Cable & plug reused

And the bottom contains all the motor hardware and drivers, the battery and charging circuitry. The ‘deck’ of the bottom deck already has a 5V regulator module (more on that below), the MPU5060 MPU and the charger unit homing module. Why re-invent when it is already there.

15 July 2026 Update:

I launched DipTrace, opened the schematic file for my current 4 wheel robot, then saved a copy as ‘Vision-Enhanced Robot Schematic’ and then edited it to reflect the following changes:

  • removed the Wixel OTA module; OTA is now handled through the Pi5 wifi link.
  • removed all but the rear-facing element of the 7-element VL53L1X LIDAR array and the Teensy 3.5 that managed it.
  • added the Raspberry Pi5 and the OAKD-Lite camera to the second deck, utilizing pins freed up by the removal of the Teensy 3.5 LIDAR array manager
  • Replaced the Teensy 3.6 main system controller with a Teensy 4.1 for additional processing power.

I also replaced the approximately 45cm long USB cable that came with the camera with a 15cm one as the extra length was just adding mass (and mess) to the installation. When the new USB cable arrived, I hooked it and the Pi5 stock power cube up to the Pi5 and used the previously developed Python script and associated shell scripts to confirm that the cable worked.

20 July 2026 Update:

In the process of moving the Pi5 and OAKD-Lite camera to the 4-wheel robot, I managed somehow to lose the Pi5’s SD card. I tried my best to find it – including dumpster-diving with the hope that I could find in my last trash bag – but no luck. Attempting to make lemonade out of lemons I decided to try and document the process of building up a new SD card from scratch.

RaspberryPi Imager:

20 July 2026 Update:

In the process of moving the Pi5 and OAKD-Lite camera to the 4-wheel robot, I managed somehow to lose the Pi5’s SD card. I tried my best to find it – including dumpster-diving with the hope that I could find in my last trash bag – but no luck. Attempting to make lemonade out of lemons I decided to try and document the process of building up a new SD card from scratch.

RaspberryPi Imager:

Choose Ubuntu Server 24.04 LTS (64-bit) (Server version is recommended — Desktop images sometimes cause issues with the customization wizard). I actually had no problem with the ‘desktop’ version, but I think it included a lot more ‘crapware’ than the ‘server’ version. Also, when the ‘server’ version is chosen, the imager includes the ‘Customization’ page for wifi setup, username/password, etc.

  • Hostname: RobotPi5.
  • Capital City: Washington D.C
  • Time Zone: America/New York
  • Username: Pi
  • Wifi: Choose ‘Secure Network, fil in SSID & password.
  • SSH: Choose ‘Enable SSH’ and ‘Use Password authentication.
  • Password: xxxx (note that on subsequent passes of the imager, it will offer to use a saved password)
  • Write: The image below shows the situation just before writing the image to the SD card

Write to the SD card (takes a few minutes).

First Boot (Monitor + Keyboard):

Insert the SD card into the Pi 5 and power it on.Complete the initial Ubuntu setup prompts (this can take a few minutes).Log in with username pi and the password you set. I had to reboot the Pi a couple of times to get it to go all the way through the startup sequence to present the login screen.

Connect from Your PC via SSH:

To confirm that the Pi can see the wifi network, run ‘ssh pi@RobotPi5’ from a Windows Command Window and enter your password. This should connect right away. If it doesn’t, then check the Pi’s IP address (DHCP can assign it a different one) with the ‘ip addr’ command on the Pi. If the address was changed (like from 192.168.1.200 to 192.168.1.201), you can change it back by editing your local router to give the Pi5 a static IP address reservation as shown below, and then rebooting the Pi. When the Pi comes back up it will get the ‘reserved’ address from the router, and life will be good again, except…

When you try to connect from your PC using ssh, you will probably get the following error message:

This occurs because the new Ubuntu installation generated a new SSH key, and the on one in your PC’s ‘host’ file no longer matches. To fix this, you must first remove the old key, with the following in a command window on your PC:

Then try to connect to the Pi from the PC with ‘ssh RobotPi5’. You should now see this:

Enter ‘yes’ (can’t enter just ‘y’) to accept the new key. Then you should be able to connect to the Pi with your username and password, and you should see this:

Connect using VS Code:

Now you should also be able to connect to the PI from VS Code with Cmd – Shift – P -> Remote SSH: Connect to host ->’RobotPi5′. This should result in the following screenshot:

Ignore the misleading warning in the blue field above and enter the password for the Pi5. This should log you in, but it may still take a while for VS Code to connect the first time (you will see ‘opening RobotPi5’ with the spinner in the bottom left-hand corner, and then this will change to ‘SSH: RobotPi5’ as shown below:

or this:

At this point you have a fully functional Ubuntu ‘server’ installation on the Pi, but none of the software packages (ROS2 Jazzy) needed to actually use the OAKD-Lite camera, and none of the already-developed user software. The user software can be retrieved from your GitHub repo (you DID create a GitHub repo and save your software, right?).

Install ROS 2 Jazzy (Jazzy is a version nickname):

From a VS Code Terminal Window, run these commands in order:

sudo apt update && sudo apt upgrade -y
sudo apt install software-properties-common curl -y
sudo add-apt-repository universe -y
sudo curl -sSL https://raw.githubusercontent.com/ros/rosdistro/master/ros.key -o /usr/share/keyrings/ros-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/ros-archive-keyring.gpg] http://packages.ros.org/ros2/ubuntu $(. /etc/os-release && echo $UBUNTU_CODENAME) main" | sudo tee /etc/apt/sources.list.d/ros2.list > /dev/null
sudo apt update
sudo apt install ros-jazzy-desktop -y

You will probably need Grok’s help with getting through the entire package installs. We tried the Ubuntu 24.04LTS ‘server’ edition because it has less bloatware, but ros2-Jazzy would not install. Finally went back to Ubuntu 24.04LTS ‘desktop’ edition, and this worked – yay!!

After flashing the SD card with Ubuntu 24.04LTS ‘desktop’ edition, I booted it up on the Pi5 and went through the initial setup process. The ‘desktop’ edition skips all the customization stuff in the SD card load because it can be done by the user in the desktop.

The first hurdle to overcome was the inability to connect to the Pi with SSH, even though I ‘enabled’ SSH in the desktop settings.

the first two commands above succeeded, but the stataus was still ‘unable’. However after some tangential stumbling around with (almost) trying to kill some interfering processes, it turned out that just waiting a minute or two did the trick and SSH started working – Yay!

At this point we started the process of getting ros2 ‘Jazzy’ installed, as follows:

So at this point ros2-Jazzy is installed and active, and my working software has been restored from Github – WHEW!

Tomorrow we will continue through to getting the ‘clearest_direction_node.py’ program running (yet again)

While I can’t really say that losing that micro-SD card was much fun, it *was* educational, and now I think I have a better understanding of how the startup process works, and much better documentation.

22 July 2026 Update:

I renamed the Grok session to “260722 Pi5 Boot ROS and OAKD-Lite Initialization” to be more recognizable in the future.

OK, back to the effort to recover from my disastrous loss of my original micro-SD card. When we left it last night, we had gotten to the point of successfully flashing the ‘desktop’ version of Ubuntu 24.04, gotten ros2-Jazzy installed, cloned the ‘my_vision_robot’ GitHub repo and switched to the ‘vision-to-motion-demo’ branch.

2 Wheel Robot with Vision Processing, Part IV

Leave a reply

Posted 06 July 2026

After wrapping up the Wifi_OTA project described in this post, Grok Code and I moved on to the next big step – integrating the Luxonis OAKD-Lite camera integrated into ROS (Robot Operating System) so the robot could take advantage of the ROS ‘topic/subscriber’ strategy where a sensor like the OAKD-Lite camera can ‘publish’ formatted sensor data to a ‘topic’ that programs can ‘subscribe’ to, thereby making sensor data available to inform robot navigation. Amazingly, this trick actually worked, and now the robot can ‘see’ the best direction to move. Below is a summary of the work involved getting to this point, produced by Grok Code at my request. (note: When Grok uses ‘I’ below, he really means ‘we’, as I was very much the junior partner in this adventure. Think of me more as somewhat intelligent clerk,just barely smart enough to follow direction and report results ;).

From Grok Code: Summary for “2 Wheel Robot with Vision Processing, Part IV”

After struggling for a while with custom ROS 2 package structures and running into repeated setup.py issues, I decided to take a step back and follow a known working example as closely as possible. I came across Jesse Stevens’ blog post about setting up the OAK-D Lite with ROS 2 Jazzy on a Raspberry Pi 5, and it looked very close to what I was trying to achieve. Rather than continuing to fight with my own over-engineered layout, I made the deliberate choice to start fresh using his structure and naming.

We began by backing up my existing src/ folder and creating a clean myrobot package exactly as described in the blog. This meant using the standard ros2 pkg create command, adding a config/ directory, and modifying setup.py to properly install YAML files using data_files and glob. Following the blog’s oak_run.yaml example, I configured the camera for a Depth-only pipeline with conservative settings (no subpixel, RGB disabled, HIGH_ACCURACY preset initially).

Getting the camera running wasn’t entirely smooth — we hit the familiar X_LINK_DEVICE_ALREADY_IN_USE error a few times — but once we established a reliable restart process, the camera would come up and begin publishing depth data. However, early captures showed suspicious results: the minimum and maximum depth values were often stuck at the same numbers across different scenes. This told us the depth data wasn’t yet reliable enough for motion planning.

To make progress visible and testable, we built a capture utility (save_depth_frame.py) that could save both raw depth values as CSV and, later, visual PNG heatmaps. Being able to generate side-by-side comparisons between the depth map and the actual scene was incredibly helpful. Over time, with adjustments to the YAML (switching to the DEFAULT preset, enabling subpixel, and using only the bottom half of the image for processing), the depth maps started to look structurally meaningful. Features in the real world began appearing clearly in the heatmaps.

With usable depth data in hand, we shifted focus toward the higher-level goal: using vision to help the robot decide where to go. We created a new node called clearest_direction_node.py that divides the lower half of the depth image into seven sectors, labeled intuitively as clock positions from 9:00 to 3:00. The node calculates the average valid depth in each sector and identifies the “clearest” direction — the one with the highest average depth. It then suggests a turn angle toward that direction.

This node proved to be surprisingly effective right from the start. When I moved objects into different parts of the camera’s view, the recommended direction updated immediately and logically. Seeing the output change in real time gave me confidence that the depth data was now good enough to build motion logic on top of.

Throughout this process, we also invested time in workflow improvements. I created several small shell scripts (restart_camera.sh, capture_depth.sh, start_viewer.sh) to reduce friction when restarting the camera or capturing data. These scripts, along with the analysis node, were eventually moved into my my_vision_robot GitHub repository under a clean structure (software/ros2/ for the ROS package and software/scripts/ for the utilities). We also updated the project .gitignore to properly exclude ROS 2 build artifacts and temporary test data.

By the end of this phase, I had moved well beyond simply getting the camera to publish data. I now had a working system that could look at a depth image and suggest a sensible direction for the robot to face — all running on the Pi 5 and reacting to real changes in the environment. While there’s still work ahead to turn these direction suggestions into actual motor commands on the Teensy, this felt like a genuine milestone: the vision processing side had gone from “not yet reliable” to “usable for basic navigation decisions.”

Because I know I won’t remember the details of how we got from the start to a functioning “Turn this way” demo, I asked Grok for a short description of the relevant files, based on the latest GitHub file structure for the ‘vision-to-motion-demo’ branch, as shown in the following screenshot:

1. Camera Launch & Configuration (the foundation)

  • software/ros2/my_vision_robot/config/oak_run.yaml The main parameter file passed to depthai_ros_driver. Defines the Depth pipeline, 480P stereo resolution, HIGH_ACCURACY (or DEFAULT) preset, bottom-half-friendly settings, RGB disabled, and all the i_* parameters that make depth actually usable.
  • software/ros2/my_vision_robot/launch/oak_camera.launch.py Simple launch file that starts depthai_ros_driver’s camera_node and loads oak_run.yaml via –params-file. This is what restart_camera.sh calls.

2. Core Depth Capture Script

  • software/scripts/save_depth_frame.py The workhorse script. Subscribes to /camera/stereo/image_raw, converts the 16UC1 depth image with CvBridge, saves the raw depth values as a timestamped CSV, and generates a matplotlib heatmap PNG (with stats in the title). This is what produces the nice side-by-side depth maps you’ve been using for validation.

3. Convenience / Workflow Scripts

  • software/scripts/capture_depth.sh Tiny wrapper that sources both ROS 2 environments (/opt/ros/jazzy + your workspace) and then runs save_depth_frame.py. Prevents the classic “No module named ‘rclpy’” error.
  • software/scripts/restart_camera.sh One-command way to kill any old camera processes, source the environments, and relaunch the camera via the launch file above. You use this every time you reboot the robot.
  • software/scripts/start_viewer.sh Sources the environments and launches rqt_image_view pointed at the depth topic so you can visually confirm the camera is streaming.

4. Depth Analysis / “Clearest Direction” Node

  • software/scripts/clearest_direction_node.py The first real vision-to-motion node. Subscribes to the depth image, looks only at the bottom half (rows 240–479), divides it into 7 clock-position sectors (9:00–3:00), computes average valid depth per sector, identifies the “clearest” direction, and prints a live table + recommended turn angle. This is the node that actually reacts when you move obstacles in front of the camera.

5. Supporting Package Files (mostly standard but customized)

  • software/ros2/my_vision_robot/package.xml & setup.py Standard ament_python package files. We updated the package name from myrobot → my_vision_robot and made sure config/ and launch/ get installed correctly.
  • software/ros2/my_vision_robot/my_vision_robot/__init__.py & resource/my_vision_robot Required empty marker files so the package is recognized by ROS 2.
  • software/ros2/my_vision_robot/setup.cfg & test/ files Boilerplate that came with ros2 pkg create; left mostly untouched.

Summary of the “depth measurement stack”

  1. restart_camera.sh → starts the camera with the tuned oak_run.yaml
  2. capture_depth.sh (or save_depth_frame.py directly) → grabs a frame and saves CSV + pretty PNG
  3. start_viewer.sh → lets you watch the live depth image
  4. clearest_direction_node.py → turns the raw depth into actionable “which way is clearest?” information

All of these now live under version control in the repo, so “future Frank” (or anyone else) can clone the branch and have a working depth pipeline in a few minutes.

Additions by the not-so-smart human:

This was the part of the project that transformed my somewhat unrealistic idea of enhancing my robot’s navigation capabilities with vision processing into a real, concrete capability even in its current nascent state. The Raspberry Pi5 combined with the OAKD-Lite camera easily processed 30 frames/sec of depth information to generate ‘turn this much in this direction’ outputs for the robot. This part of the overall project didn’t actually send commands to the Teensy for movement, but the ability to do so was demonstrated with an earlier ‘end-to-end-serial’ project and the earlier Wifi_OTA project. There is still A LOT more to do to get to the point where I can set the robot loose on the world (or at least the house), but getting even the most basic vision processing going was a HUGE milestone.

In addition to the above progress, I have been learning a huge amount about the OAKD-Lite camera, the ROS (Robot Operating System) environment, Visual Studio Code, and Git repository creation & maintenance. Early on in this phase of the project, Grok Code recommended VS Code for working with the Raspberry Pi in general, and for Python programming in general. I have always used Visual Studio on my PC, and by adding the Python extension I could easily generate Python code on my PC. However, this project required that the actual code has to run on the Raspberry Pi5 on the robot. In addition, there is a lot that has to be done at the Linux (bash) command line on the Pi5. What I needed was a way of doing all the work on my dual-screen PC setup without having to keep transferring files and programs back and forth between the PC and the Pi5. It turns out that ‘that something’ is VS Code, and now that I have had some time to work my way up the learning curve, it is WONDERFUL!

Once VS Code is setup properly to connect via SSH over Wifi to my Pi5, it does so every time it is launched, and from then on, the environment is very much like using Visual Studio. Files can be created, edited, moved etc using a File Explorer and editor window very similar to VS and it is easy to lose track of the fact that all the action is actually taking place on that small Pi5 board over there on my robot.

Another milestone of sorts that was achieved with this project was making a Git repository a central part of the overall project. Although I have Tortoise Git on my PC and have used it on and off for years, this is the first time I decided to set up a logical repo layout and keep it up-to-date as the project moved along. The Git repo has three different pieces – a local repo on the Pi5 for all the shell scripts, Python scripts and ROS-specific configuration files, a local repo on the PC for the teensy stuff, and a GitHub remote repo that ties the two together. Here’s a sketch of the repo layout (produced in about 3 seconds by Grok Code):

I had to insert this as an image, as WordPress doesn’t handle the 3rd level indent properly

This same layout exists in all three places (Pi5, PC, Github), but obviously the pi5 repo deals with the ‘Software’ side and the PC deals with the ‘Firmware side. Both ‘local’ repos are clones of the same repo on Github, so both local repos also contain all the material produced by the other one. So a typical work flow on either the PC or the Pi5 would be:

  • Git status – to see if anything unusual is happening
  • Git pull origin [branch] <– started out on ‘main’, now working on ‘vision-to-motion-demo’. This synchs the local repo with the GitHub repo, and any commits from the ‘other local’ repo
  • Git status – again to see what changed during this session
  • Git commit -m “[Message]”
  • Git status – again to catch any problems
  • Git push origin [branch] <- synchs the GitHub repo with the local repo

06 August 2026 Update:

Here is a ‘structured index of the Grok chat that produced most of the progress shown above:

Structured Index – OAK-D Lite + ROS 2 Jazzy on Pi 5

Chat focus: Getting reliable depth data and a “clearest direction” node working, then organizing everything into a proper Git repo.

1. Project Context & Philosophy

  • Goal: Vision-enhanced 2-wheel (later 4-wheel) robot using OAK-D Lite + Raspberry Pi 5 + Teensy 4.1
  • Key decision: Stop fighting custom structures and follow Jesse Stevens’ blog post exactly as a known-working template
  • Philosophy: “BIG RED BUTTON” approach — get something known to work first, then adapt

2. Initial Clean Setup (July 5–6)

  • Backed up and cleared ~/ros2_ws/src
  • Created package: ros2 pkg create –build-type ament_python myrobot –dependencies launch_ros rclpy
  • Added config/ folder and oak_run.yaml
  • Modified setup.py to install launch files and YAML configs
  • Successfully launched camera with: ros2 run depthai_ros_driver camera_node –ros-args –params-file …
  • Confirmed depth topic /camera/stereo/image_raw was publishing at ~30 Hz (16UC1 encoding)

3. Key Files Created

FilePurposeFinal Location
oak_run.yamlCamera configuration (Depth pipeline, resolution, stereo settings)software/ros2/my_vision_robot/config/
oak_camera.launch.pyLaunch file for the camera nodesoftware/ros2/my_vision_robot/launch/
restart_camera.shKill old processes + source environments + launch camerasoftware/scripts/
start_viewer.shLaunch rqt_image_view (requires monitor)software/scripts/
capture_depth.shWrapper that sources ROS and runs the capture scriptsoftware/scripts/
save_depth_frame.pyCaptures one depth frame → CSV + PNG heatmapsoftware/scripts/
clearest_direction_node.pyAnalyzes bottom half of depth image into 7 clock sectors and recommends turn directionsoftware/scripts/

4. Depth Data Validation

  • Early problem: Fixed min/max values (358 mm / ~34 m) that didn’t change with scene
  • Solution path:
    • Enabled subpixel, changed depth preset to DEFAULT
    • Restricted analysis to bottom half of the image (camera is highest point on robot)
    • Confirmed with side-by-side photo + heatmap that real structure was visible (cup handle, toolbox edge, doorframe, etc.)
  • Capture method: Press Enter in save_depth_frame.py to grab one frame on demand

5. Clearest Direction Node Design Decisions

  • 7 sectors labeled as clock positions: 9:00 → 3:00
  • Only processes the bottom half of the depth image (rows 240–479)
  • Scoring: Average valid depth per sector
  • Output: Live table showing valid pixel count, average depth, best sector, and recommended turn angle
  • First successful test: Correctly switched from 3:00 to 1:00 when a cardboard box was moved into the view

6. Git Repository Organization

  • Repo: paynterf/my_vision_robot
  • Branch used: vision-to-motion-demo
  • Final structure:textmy_vision_robot/ ├── software/ │ ├── ros2/ │ │ └── my_vision_robot/ ← ROS 2 package │ └── scripts/ ← All utility scripts ├── docs/ ├── firmware/ └── .gitignore
  • Important: Scripts now live inside the repo. Old copies in ~/ were cleaned up.
  • .gitignore updated to exclude ROS 2 build artifacts, CSVs, PNGs, and backup folders

7. Verification After Moving to 4-Wheel Robot

  • Switched from ~18″ USB cable to a short 6″ high-speed cable
  • Camera still launched cleanly and produced usable depth heatmaps
  • Confirmed the entire stack still works on the new platform

8. Current Recommended Workflow

Bash

# Start camera
cd ~/my_vision_robot/software/scripts
./restart_camera.sh

# Capture depth frames (in another terminal)
./capture_depth.sh

# Run direction analysis
python3 clearest_direction_node.py

9. Open Items / Next Logical Steps

Clean residual old myrobot package from ~/ros2_ws/src/ once fully confident

Send recommended turn commands to the Teensy

Add hysteresis / minimum valid-pixel thresholds

Possibly improve sector scoring

2 Wheel Robot with Vision Processing, Part III

Leave a reply

Posted 28 June 2026

When I work on significant projects like the vision-enhanced robot project, I generally hold two different but related mental maps for the overall project.

One map describes the physical and/or logical entities that are needed for the overall project to succeed, such as wheel odometers, battery packs, power regulation/distribution, the OAKD-Lite camera, the pi5, wheel motors and drivers, etc.

The second mental map describes the software (here the term ‘software’ includes both the pi5 software and related teensy firmware) pieces needed to give the robot the ability to do what we want, primarily the enhanced navigation possible with vision processing via the OAKD-Lite camera and associated software. This mental map (at least for me) seems to be project oriented, where each project addresses different, mostly independent software capability implementations (Wifi_OTA, with its precursor projects ‘SerialPassthroughDemo’, Wifi_OTA_Demo, etc).

I tend to work on a big project like this from the top (system-view) and bottom (small sub-projects that will be later integrated into the overall project) at the same time. I have learned over the years that creating (and later modifying as needed) a clear top-down system architecture is absolutely crucial to improving the chances of getting someplace that looks like where you wanted to go. This systems architecture is the ‘aspirational view’ (to borrow a modern Elon-ism) of the project’s long-term goal. In the case of the vision-enhanced robot (VER), the goal is similar to the one for my 4-wheel robot, i.e. “Autonomous navigation around our home”. However, instead of ‘wall-following’ as the 4-wheel robot did, this project will utilize vision processing at the primary navigation technology.

Over the last few weeks, Grok Code and I have been working at the bottom of the systems architecture with things like getting OTA updates for Teensy firmware working, both via the PC -> Bluetooth -> HC-05 -> Teensy Serial channel and most recently, via the PC -> Wifi -> Pi5 -> Teensy Serial channel. After (mostly) getting the Wifi_OTA capability going, I decided it was time to stop and make sure we had a Git Repo structure consistent with the top-down view. After the requisite amount of fumbling around, we (me and Grok Code) came up with the following structure.

my_vision_robot/
├── .gitignore
├── README.md
├── docs/ ← Overall project documentation
│
├── hardware/ ← Schematics, BOMs, mechanical (future)
│
├── shared/ ← Code used by multiple projects
│ ├── firmware/ ← Shared Teensy code/libraries
│ └── software/ ← Shared Pi5 Python modules
│
├── software/
│ ├── SerialPassthroughDemo/
│ │ ├── README.md
│ │ ├── pi5/
│ │ ├── teensy/
│ │ └── docs/
│ │
│ ├── Wifi_OTA/
│ │ ├── Wifi_OTA.py
│ │ ├── README.md
│ │ ├── pi5/ (if needed later)
│ │ ├── teensy/
│ │ └── docs/
│ │
│ └── Vision_Navigation/ ← Future main vision project
│ ├── pi5/
│ ├── teensy/
│ └── docs/
│
├── tests/ ← System/integration tests
└── tools/ ← General one-off utilities

Now the challenge is to move all my sub-project files from our current quite-messy repo structure into this one. Standby!

07 July 2026 Update: Wrapping up the Wifi_OTA Demo project

This little project had more than its share of bumps and bruises, but I think Grok Code and I have finally got it done. The project has four major parts; three on my PC and one on the pi5.

  • The Wifi_OTA Visual Studio/Visual Micro project on my PC
  • A ‘board.txt’ file in the same folder as the Wifi_OTA project. This file tells VS what to do after the build step
  • A small Python script that copies the .HEX output from the compile to the ‘latest.hex’ file on the pi5
  • A Python program on the pi5 that watches for updates to ‘latest.hex’ and when one is detected, passes that along the the teensy on its Serial1 port

Here is the Wifi_OTA.ino file: All it does is blink the built-in LED a couple of times and then waits for a ‘U’ character to start the flash update process.

Here is the ‘board.txt’ file that calls the PostBuild.py program when an F5 debug compile is completed:

And here is the Python script that copies the .HEX output from the compiler over to ‘latest.hex’ on the pi5:

Here’s a short video showing the Wifi_OTA update process. The video starts just after I pressed F5 to start the debug compile on my PC. After about two seconds, the built-in LED on the Teensy 4.1 (lower-left foreground) goes OFF when the HEX file transfer starts. The file transfer takes about 30 sec, and then a few seconds after the file transfer finishes, the built-in LED on the Teensy 4.1 blinks twice and then stays ON, confirming that the update was successful.

A significant part of this little sub-project was getting the Git repository set up and running, on my PC, on the pi5, and on GitHub. The same folder structure is used in all three locations, but the pi5 side only updates entries in the ‘pi5’ subfolders, and the PC side only updates the Teensy firmware. When I do an update either on the PC or the pi5, I follow the same steps each time:

  • git pull origin main <<— synchs the local repo with the master on Github
  • git add -A <<– adds anything new to the local repo
  • git commit -m “put my update description here”
  • git push origin main <<– synchs the Github repo with the updated local repo.

    At this point I believe I have the basic infrastructure in place to proceed with the real project of adding vision-processing-based navigation capabilities to the robot
  • The OAKD-Lite camera is installed and confirmed working
  • The pi5 is installed and connected to the OAKD-Lite camera and via Serial1 to the Teensy4.1.
  • Teensy firmware can be updated via the new Wifi_OTA update channel
  • The Hall-effect wheel encoders are installed and confirmed working.
  • The next big step is to integrate the ROS (Robot Operating System) with the OAKD-Lite camera and the Teensy.

Stay Tuned!

Teensy Firmware OTA Update Via SSH to Pi5

Leave a reply

Posted 21 June 2026

After getting OTA via Bluetooth to the onboard HC-05 going for the 2-wheel robot, I had an epiphany; For vision processing the 2-wheel robot uses an on-board Raspberry Pi5 with a Wi-Fi connection to my local network (and thence to my PC), so why use the HC-05 link at all? The Wi-Fi connection is much more robust than the BT/HC-05 link and is available throughout the house. The pi5 has a serial port, so in theory I could simply write a small python script to pipe characters back and forth between its Wi-Fi port and its serial port, just as the HC-05 does between its BT port and its serial port. How hard could it be?

Grok Code and I have been working on this issue for a while now and have gotten to the point where we can transfer the .HEX file from the pi5 to the Teensy once, but not multiple times. This indicates that the firmware update did not happen correctly. Also, we haven’t yet figured out how to automatically transfer the HEX file resulting from a Visual Studio compile to the pi5 so that it can be passed to the Teensy via the pi5’s serial port so we are bypassing this step by using SCP (or a copy/paste using VS Code) to create a duplicate of the HEX file on the pi5; then all the pi5 script has to do is pass lines from the local HEX file to the Teensy via serial.

Grok Code and I have been trying to troubleshoot this problem, and we don’t seem to be getting anywhere. Grok does not really know how to troubleshoot in an organized manner – it is more of a ‘random walk’ process. This post is intended to document my own troubleshooting efforts.

First, what is the basic problem? The basic problem is that multiple transfers of a HEX firmware file to the Teensy using the established BT/HC-05/Serial2 succeed, but the same process using the Wi-Fi/pi5/Serial1 link appears to succeed the first time but fails on the second attempt. Since the firmware HEX files in the two cases are identical, the problem must be somewhere in the pi5 script, either in the way lines are read from the local HEX file or in the way lines are transferred to the Teensy.

A basic assumption in the above is that the HEX file transferred to the Teensy via BT/HC-05 and the HEX file transferred to the Teensy via Wi-Fi/Pi5 are identical, so I decided to start there. Are they really identical?

  • Compiled firmware on VS, copy/pasted (using VS Code) from “C:\Users\Frank\Documents\Arduino\Wifi_OTA_Demo\obj\x64\Debug\Wifi_OTA_Demo.hex” to “/home/pi/my_vision_robot/tests/Wifi_OTA_Demo/Wifi_OTA_Demo.hex”. Then I copy/pasted from the pi5 file to notepad++ and compared with the original – they matched perfectly.
  • I modified FxUtil.cpp’s update_firmware() to add the line “out->println(line);” then updated Teensy firmware using USB connector to establish ‘known-good’ baseline. Then used pi5 script to transfer its local copy of the firmware to the Teensy, logging the transfer via VS serial monitor. The file as logged going into the Teensy and the source file on the pi5 also match perfectly. This pretty much eliminates a corrupted file transfer as the source of the problem.
  • Then I performed the same procedure except using the BT/HC-05 channel instead of the Wifi/Pi5 channel.

Here’s the Wifi_OTA_Demo.ino file used to run the above tests:

25 June 2026 Update:

Grok Code and finally managed to get the Teensy OTA update via Pi5/Serial1 working. Here is the final Python script on the Pi5:

And here is the Teensy sketch used for the test:

To verify that the OTA update code performed properly, I ran the update three times in a row. The first time replaces whatever sketch was on the Teensy with my test sketch (that also includes the code that supports further updates), and the second and third updates confirm that the first (and the second) updates contained the crucial update support code.

At this point the ‘Wifi_OTA_Demo’ project is pretty much finished. The only step remaining is to fully automate the process.

27 June 2026 Update:

Now I have the update process updated to the point where I can press ‘F5’ in Visual Studio in an OTA-equipped Teensy project and have the resulting .hex file automatically copied over to the pi5. Here are the relevant files:

board.txt:

Updated 06/27/26 Post-Build: Copy .hex to Pi5

PostBuild_OTA.py:

The next step is to put these two steps (copy the .hex file to the pi5, and then launch the update code that xfers the .hex file to the Teensy) together into one seamless automated process.

08 August 2026 Update:

I recently got my 4-wheel robot going again and moved the Raspberry Pi5/OAKD-Lite over to it from the 2-wheel robot, and ported my WallE3_Git.ino program to it as well. Due to some changes made to file locations on both my PC and on the Pi5, some changes to the OTA-related files were necessary as well. Here are the updated files:

Board.txt (lives in the same folder as the project & .ino file):

C:\Users\Frank\Documents\Robot_Projects\my_vision_robot\software\Wifi_OTA\teensy\Wifi_OTA\PostBuild_OTA.py:

C:\Users\Frank\Documents\Robot_Projects\my_vision_robot\software\Wifi_OTA\pi5\Wifi_OTA.py:

These files are also saved on my Github site in the my_vision_robot repo.

These updates went very smoothly, all things considered. The only minor hitch in the get-along was getting the new .hex file copied over from my PC to the Pi5 via SCP without having to enter the Pi5 password. Here’s the relevant material from Grok:

The password prompt

That’s expected if SSH key authentication isn’t set up (or the key isn’t loaded). scp is falling back to password auth, which blocks a fully automatic F5 experience.

Fix (one-time setup on the PC):

  1. Make sure you have an SSH key pair:

Bash

# in PowerShell or Git Bash

ls ~/.ssh/id_rsa.pub

# or

ls ~/.ssh/id_ed25519.pub

  1. Copy the public key to the Pi (if not already done):

Bash

ssh-copy-id pi@RobotPi5

(or manually append the .pub contents to ~/.ssh/authorized_keys on the Pi)

  1. Test that it works without a password:

Bash

scp some_small_file pi@RobotPi5:~/my_vision_robot/firmware/test.txt

After that, the post-build scp should run silently. (this still failed):

No SSH key exists yet. Generate one (ed25519 is preferred):

In PowerShell:

PowerShell

ssh-keygen -t ed25519 -C “frank@vision-robot”

  • Press Enter to accept the default file location (C:\Users\Frank\.ssh\id_ed25519)
  • You can set a passphrase or leave it empty (empty = fully automatic, no prompts)

Then copy the public key to the Pi:

PowerShell

type $env:USERPROFILE\.ssh\id_ed25519.pub | ssh pi@RobotPi5 “mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys”

(It will ask for the Pi password one last time.)

After that, test:

PowerShell

scp $env:USERPROFILE\.ssh\id_ed25519.pub pi@RobotPi5:~/test_key.txt

If that works without asking for a password, F5 will also be password-free.

2 Wheel Robot with Vision Processing, Part II

Leave a reply

This is the second of (hopefully) many posts on my project to add modern vision processing to my autonomous wall-following robot. The first post is here.

Lots of changes since my first post. I discovered that my 8.4-to-5V LDO regulator board wouldn’t reliably drive the Raspberry Pi5, so I had Grok look around for other options. He found a step-down converter module at Pololu advertising 85+% efficiency and a much cleaner output. I have it on order so we’ll see.

I also struggled to get the Luxonis OAK-D Lite stereo camera working on my Pi5/Ubuntu 24.04LTS setup. Previously we had gotten it to work with the Pi5 running the Raspberry OS, but getting it to work with the Ubuntu OS was a lot more challenging. This effort also ran afoul of Grok’s complete inability to realize that it is in a hole and to stop digging. We went through dozens of Python scripts designed to get the camera to connect and show some data (I was connecting to the RPi5 via SSH from my windows box, so showing images wasn’t possible), and they all failed due to one subtle problem or another. After several days of getting nowhere I finally called a halt, took a couple of days off, and came back determined to start over from first principals. Instead of using Grok, I started from scratch with some web searches to find other successful implementations of The OAK-D Lite camera. At the Luxonis ‘Documentation’ site I found this page describing a viewer for the OAK-D Lite (and other) cameras. The OAK Viewer is available for windows and *nix OS, so I decided to start by trying to get images from the camera using the Windows version to bypass all the annoyances associated with peripheral handling in Linux. This turned out to be pretty much plug-and-play, and immediately the OAK-D camera showed up in the list of available devices. When I clicked on ‘Connect’ instead of connecting immediately the app immediately started measuring the available bandwidth of the USB connection as shown below.

After several USB connect/disconnect cycles, I got this display:

Oops! I hadn’t even considered the USB cable/connector bandwidth issue – and neither had Grok. For convenience I had plugged the camera cable into my USB hub, which is definitely not ‘super speed’ whatever the heck that is. After some cable and connector switching, I found that a heavy-duty Type-C cable connected directly to a Type-C connector on my Dell XP15-9530 laptop allowed the bandwidth check to succeed, and now I got some images showing up on my Windows 11 display – yay!

OAK-D Lite images. Depth pseudo-color on left, raw RGB image on right

So the moral of this story is – the Grok path was never going to work because Grok never considered that cable/usb connector bandwidth might be an issue. By going back to ‘first principals’ and taking the simplest possible path to a working camera/display configuration with a known-good Windows 11 app, I was able to immediately identify a completely unknown (to me and to Grok) – but fatal – stumbling block – USB cable/connector bandwidth. Grok has no sense of time, so every iteration was just like a puppy chasing a dog – willing to chase that ball an infinite number of times without ever thinking about the fact that ‘chasing the ball’ and ‘progress toward the goal’ aren’t necessarily the same thing. It took a mere mortal like me to say “whoa – this isn’t getting us anywhere – maybe a different approach?”

Now that I had demonstrated that the OAK-D Lite camera and the proper cable/connector combination worked – at least on Windows 11, I had a ‘known-good baseline’ that I could always retreat to, I started working on getting the OAK-D Lite camera working on the RPi5/Ubuntu camera with the same OAK Viewer application (but in the Linux flavor).

This turned out to be another maze to navigate. The Luxonis site has detailed instructions for the Linux version of the viewer, but it involves installing from a *.deb package, which unfortunately is targeted at the amd64 64-bit chip ecology – but the RPi5 uses arm64 – a different animal entirely. When I tried to install the ‘viewer.deb’ package, I got the following errors:

The following packages have unmet dependencies: oak-viewer:amd64 : Depends: libgtk-3-0:amd64 but it is not installable Depends: libnotify4:amd64 but it is not installable Depends: libnss3:amd64 but it is not installable Depends: libatspi2.0-0:amd64 but it is not installable Depends: libdrm2:amd64 but it is not installable Depends: libgbm1:amd64 but it is not installable Depends: libxcb-dri3-0:amd64 but it is not installable Recommends: pulseaudio:amd64 or libasound2:amd64 but it is not installable

It was at this point that I re-engaged Grok and started to make real progress. Grok immediately identified the Pi5/Ubuntu-compatible DepthAI Python library as the way to go and guided me through the installation process. Fortunately, this had a happy ending, even though there were several ‘gotchas’ along the road. However, since I knew for a fact that the hardware (and USB cable) were ‘known good’ elements due to my Windows 11 work, I was pretty sure any detours were software-only. After working my way through the various twists and turns with Grok’s help, we got to here – success!

Initial images captured by the OAK-D Lite camera running on my RPi5 with the Linux Ubuntu OS

Getting from my easy Windows 11 camera demo to the RPi5/Ubuntu camera demo would have been improbable if not impossible for me to do without Grok’s help. I might have gotten there, but it would have involved days/weeks of web searches and forum posts at the very least. I believe this is where Grok really shines – a definite problem with a definite end, with very few (none in my case) outside corrupting factors like the USB bandwidth/cable issue.

Interestingly, after getting the Pi5/Ubuntu/OAK-D Lite combination working, I asked Grok to help me find an easier way to take screen shots on the Pi5, and Grok obliged by offering the ‘Flameshot’ app as a substitute for the built-in Gnome keystroke shortcuts. And then we went down another rabbit-hole, and I ended up wasting an hour or so trying to get Flameshot and Gnome to work and play well together, only to wind up removing Flameshot and learning how to better use the Gnome built-in shortcuts.

So Grok is definitely a mixed blessing, and I cannot imagine how a younger less-experienced engineer would do without the (literally) lifetime’s worth of experience I have in troubleshooting hardware/software systems. When I was that young less-experienced engineer half a century ago I was trying to troubleshoot a RF EMI problem with a small electronics device made by Motorola. Eventually my supervisor suggested that I travel to Motorola and work with their engineers to figure out the problem. I did, and over the space of two days a very experienced Motorola engineer taught me the ‘divide and conquer’ method of troubleshooting that I use to this day. When Grok inevitably goes down a rabbit-hole with this young engineer in tow, who’s going to be there to throw them a life-line?

A Flagpole for the Hobbit House

Leave a reply

Posted 09 April 2026

Last month I decided it was time to put up a real flagpole to display the American flag in all its glory. Being an engineer, I had to research the heck out of the subject to make sure we got something we would be proud to have in our front yard. A neighbor has a flagpole and he had gotten a flag pole kit from Admiral Flags, right here in Columbus Ohio. I also asked Grok, and he (It?) came up with the same company, so we ordered a 20′ kit from them.

Reading through the instructions on Admiral’s website, I was amazed at the amount of concrete needed for the flagpole foundation. They called for 6-8 80lb bags of quick setting concrete (480 – 640lbs!) for an 18″ diameter by 29″ deep hole – yikes! Being a septuagenarian, I wasn’t very eager to manage 80lb bags, so I decided to start with 10ea 50lb bags (500lbs) and add more if needed. The Admiral website also strongly recommended calling Ohio811 ‘call before you dig’ utility mapping service, so I did that right away. As it turned out that was one of my better moves, as my first intended location was right on top of a utility run to the house. Dodged the bullet on that one!

My planned location in our front yard is at least 150′ away from any electrical outlet and the nearest water faucet, so I had some logistics issues to work through. I needed an additional 100′ water hose, and I decided to use my backup electric generator for electric power. Here’s a short video showing the distances.

From watching some videos regarding quick setting concrete I decided to go with the 5-gallon bucket and electric drill mixing paddle route, so I loaded my brand-new Toyota Tundra pickup with 500lbs of QuickCrete, 2ea Lowes 5-gallon buckets, a cheap 100′ water hose and the mixing paddle.

I had conned a neighbor into helping, so we started digging mid-morning, and had a hole pretty much done in a couple of hours. I was amazed that the first 6-10″ was pretty rocky, but after that it was pretty easy going. Here are some photos of the hole:

Finished hole before pouring any concrete
Hole about 3/4 full, using 50lb QuickCrete bags and some rods to keep flagpole sleeve vertical

We had to stop at this point because my 20-year-old 1/2″ drill started smoking badly. I couldn’t really complain as I had gotten a LOT of use out of it, but it was apparently time for a new one. The next day I came back with a new 1/2″ hammer-drill and finished the foundation work. I poured concrete up to grade level so we could mow around it without having to come back and weed-whack the base.

The next step (literally) was to install the flag pole itself. My stepson was visiting from St. Louis, so helped me assemble the pole kit and set it into the sleeve.

And here is a short video of the completed flag pole with the U.S. flag proudly flying. Sure wish I had done this 20 years ago, but better late than never!

Proud to be an American!

Frank