Tag Archives: WallE3

The Robot Rises Again – Adding Vision Processing

Leave a reply

Posted 04 April 2026

Well today I exceeded my boredom threshold, so something had to give. I worked out on my basketball court for an hour, flew a glider race in the Condor3 Soaring Flight Simulator, played an online bridge tournament on BBO, and I still hadn’t used up half the day. My workbench is empty, my two 3D printers are silent, and I can’t think of a thing to do. However, while I paced around with my hands in my pockets feeling sorry for myself, I caught sight of Wall-E3, my autonomous wall-following robot gathering dust in its charging cradle in the corner. That robot has gone through a number of iterations, starting with this post in early 2015 and ending with this post in late 2024. The goal of this almost decade-long project was to create a robot that could autonomously navigate around our house, scaring cats and humans alike and recharging itself whenever it got hungry. It did a really good job at the scaring part but unfortunately, I never got it to the point where it wouldn’t get stuck in one of our junk-filled guest bedrooms; close, but no cigar 😒.

Looking at my robot, I realized that in the intervening decade, a lot had changed in the indoor navigation realm, in particular the ability to exploit video camera images for navigation in reasonably constrained environments. So, I decided to revisit my robot project to see if I could utilize the new technology to get closer to my goal of true autonomous navigation.

Part of my interest in starting a new run at the household navigation project has been my positive experience over the last year with Grok, X’s AI. Grok, despite some real issues that need to be kept in mind, is a GREAT research assistant. In the past, I might spend most of a week working my way through internet links and going down various rabbit holes trying to solve a subtle software or hardware problem but Grok can do the same thing in a few seconds. At this point I knew next to nothing about vision-based navigation systems, but I knew for sure that Grok would be able to distill the netiverse down to reasonably digestible chunks pretty quickly.

I started with the following prompt for Grok:

In less than thirty seconds, Grok produced the following:

Absolutely—switching to vision processing is a smart move for handling those tricky “special cases” where pure distance sensors (like your LIDAR + gyro combo) fall short. Cameras give you way more information: color, texture, edges, and even semantic understanding (e.g., “that’s a chair leg” vs. just “something 30cm away”). This helps with low obstacles, reflective surfaces, dynamic items, or cluttered home environments that confuse LIDAR.

Since you already have a solid 6DOF gyro (IMU), you can fuse it with vision for excellent Visual-Inertial Odometry (VIO) or Visual SLAM (vSLAM). This is basically what modern robot vacuums do (vSLAM), but fully customizable for your wheeled robot.

Quick Reality Check Before Diving InVision isn’t perfect either:

  • Sensitive to lighting changes (solution: good indoor lights or IR-capable cameras).
  • Needs decent compute (Raspberry Pi 5 or equivalent works; older Pis may struggle with heavy SLAM).
  • Depth from vision is usually “good enough” indoors but not as precise as LIDAR in perfect conditions.

Hybrid tip: You can keep your existing LIDAR/gyro as a backup while testing vision.Recommended Hardware to Get Started (2026-Friendly Options)You’ll replace/add a camera that provides both color and depth (RGB-D) or stereo for 3D perception. Top practical picks for small wheeled robots:

  1. Best overall: Luxonis OAK-D series (OAK-D Lite or Pro)
  2. Budget depth option: Arducam T2 RGBD ToF Camera (~$100)
  3. Classic: Intel RealSense D435/D455 (if you can find one) or similar RGB-D.

Mount the camera forward-facing (maybe add a second for 360° if budget allows). Keep your gyro—it will make odometry rock-solid when fused.What’s your current brain? (Raspberry Pi 4/5, Jetson, Arduino + companion computer, etc.?) Budget for the camera? That’ll let me refine this exactly.Software Stack – The Modern Way (ROS2 Recommended). The gold standard for this is ROS 2 (Robot Operating System) + Nav2 navigation stack. It handles mapping, localization, path planning, and obstacle avoidance out of the box.

Core vision libraries:

  • OpenCV → Basic start (edge detection, simple avoidance).
  • RTAB-Map or ORB-SLAM3 → Full visual SLAM (builds a 3D map while localizing).
  • VINS-Fusion or OpenVINS → Perfect VIO that fuses your existing 6DOF gyro with camera data (highly accurate odometry).

All run great on Pi 5 with Ubuntu 24.04 + ROS 2 Jazzy (or newer).

This was a lot like drinking from a firehose, but within an hour or so I had a Raspberry Pi 5 kit and two different cameras (a cheap one arriving tomorrow, and a not-so-cheap one arriving in another week or so). In the meantime I asked Grok to hunt up some tutorials and examples so I could start getting up to speed on all things ROS 2 Jazzy Robot Operating System) and SLAM (Simultaneous Localization and Mapping).

Hopefully in the next week or so I will have some hardware & software to play with, and maybe make a step or two in the direction of upgrading WallE3 to WallE4.

08 April 2026 Update:

Worked with Grok to start adding vision capability to my autonomous 4-wheel wall-tracking robot. The plan is to use a Luxonis OAK-D-Lite Fixed-Focus camera and a Raspberry Pi5 running Ubuntu 24.04 and ROS (Robot Operating System) to add vision processing to the robot. Because there was a long lead time for the Luxonis camera, we started with a cheaper (and faster delivery) alternative, the Arducam ToF Camera. Grok insisted the Arducam would work as a near-term solution until the Luxonis camera arrived, but it didn’t work out quite that way. The Arducam worked OK with the default Pi5 OS without ROS, but when we tried to add ROS and the drivers for the Arducam, we ran into a dead end. Then Grok was *sure* that we could simply replace the Pi5 OS with Ubuntu 24.04 LTS and everything would work – – – NOT! So now I’m waiting for the Luxonis so we can move forward again.

When I braced Grok about the readily apparent difference between what Grok thought was going to work fine and the actual result, I asked Grok to search for ANY successful implementation using Pi5, ROS and Arducam, and Grok could not come up with a single confirmed successful implementation. This is a real problem with Grok, one that the devs should be concerned about, but nobody seems to care.

Grok now uses several ‘agents’ to independently research user questions, and the eventual answer is a consensus (I think) of their findings. I have suggested that at least one of the agents should be dedicated to attempting to falsify the results of the others, but I have seen nothing to indicate this has actually gotten through to the devs. In this case, all the ‘red team’ agent would have to do is perform the above search for successful implementations and come up with ZERO, thereby making it obvious that starting with the Arducam instead of waiting for the Luxonis was probably not going to end well. On a positive note, I asked Grok to do the same search with the Luxonis instead of the Arducam, and it came back with several confirmed successful implementations.

Until Grok implements the ‘red team’ facility, users need to be aware that Grok will quite happily drive a user off a cliff, insisting all the way down that “this next fix will surely solve the problem”. Has anyone else had his happen?

16 April 2026 Update:

A lot has happened since my last post on this subject. Working with Grok, we finally got the OAK-D camera working on the Pi5, with real-time imagery displayed on a monitor connected to the Pi5. Of course, this is a LOOOONG way from actually implementing visual navigation on a robot, but ya gotta start somewhere!

When considering how to move the Pi5 and Oak-D camera setup onto the robot, I decided to refresh my old 2-wheel (plus castering wheel) robot rather than trying to shoehorn the Pi5 and camera setup onto my 4-wheel robot. The 2-wheel robot already as a set of nice wheel motors, a pair of DRV8871 motor drivers and a battery/charger setup, and it isn’t being used for anything else at the moment. So, I stripped off the Uno Mega 2560 controller, installed the Pi5 and the camera at one end, and a Teensy 4.1 MCU on the other. I also installed an 8V-to-5V LDO voltage regulator that I had hanging around from a previous project. Here are some photos

Then I started working on the idea of installing magnets on the wheels and Hall-effect sensors on the chassis to produce wheel rotation rate information to integrate with vision processing. Based on a conversation with Grok, I ordered some A3144 Hall-effect sensors from Amazon. When they came in, I implemented a small plugboard circuit to test the sensor, as shown in the following short video:

After I got this working, I created a printed magnet carrier for the wheels, and a small perfboard circuit mount for the sensor. I tested the setup by running the small motor test program Grok wrote for the Teensy 4.1, and the results are shown in the following video:

LED illuminates when the sensor is triggered by a wheel magnet. Scope in the background shows the detection waveform

Here’s a scope trace grab of a typical half-speed wheel motor sequence:

From the above trace, the measured frequency is 4.823Hz. There are four magnets, which implies that the wheel is turning at about 1.2RPS, or about 72RPM. The wheel diameter is about 67mm, which gives a wheel circumference of about 21cm. At the 1.2 RPS rate, this works out to about 0.24m/s.

Of course I’m not anywhere near done yet, as I need two sensors per wheel to be able to determine rotation direction so three more sensor circuits, and a second wheel magnet carrier and magnets for the other wheel. Still though, it’s a good start!

22 April 2026 Update:

I finally got all four A3144 Hall-effect sensor modules and the second wheel magnet carrier mounted and tested. As shown in the following photos, The Hall-effect sensor modules are mounted approximately 90 degrees from each other to provide effective direction sensing.

23 April 2026 Update:

After finishing the Hall-effect sensor project, I worked on implementing a wireless serial connection to the robot for remote status monitoring and for remote ‘Over The Air’ (OTA) programming. I have had this capability for years on the 4-wheel robot and it really paid off, so I didn’t want to get too far down the road with the 2-wheel version without having it. As Grok pointed out, I could use the wireless ethernet capability of the Raspberry Pi5 for this; wire a serial port on the Pi to a hardware serial port on the Teensy 4.1 and send updated .HEX files to the Teensy that way. However, I decided not to do this for three reasons; first and foremost, I disliked the idea of tying Teensy 4.1 program updates to a working Pi5, as that added another huge set of variables; secondly, the Pi5 draws a LOT of power and so I don’t want to have it running until absolutely required, and thirdly I completely solved the Teensy 3x/4x OTA problem some five years back (see this post, this post and this post) and can readily port the solution to the current configuration.

I mounted one of my spare HC-05 Bluetooth Serial modules to the back of my new camera support structure with hot glue and connected it to Serial2 (pins 7 & 8) on the Teensy 4.1. After the usual cussing and screwups getting the Tx & Rx lines connected properly, I was able to demonstrate simple pass-through serial-to-serial capability, so then I moved on to getting OTA working. First I created an Arduino project called ‘2WheelRobot_HC05_OTA1’. Then I cloned Joe Pasquariello’s FlasherX repository to my PC and copied the latest versions of FlashTxx.h/.cpp, FxUtil.h/.cpp, and FlasherTxx.ino from my local repo to the local folder of that project. Next I ported the ‘CheckForUserInput()’ function from one of my 4Wheel robot projects into the OTA1 project, removing everything but the ‘U’ (Update Firmware) case. After the usual number of mistakes and cussing, I got this working to the point where I could manually send a new .HEX file to the Teensy 4.1 using the HC-05 Bluetooth serial link and TerraTerm, as shown in the following screengrab:

The next step is to automate this procedure as I did back in 2021. I should be able to use the same ‘board.txt’ and ‘TeensyOTA1.ttl’ files from back then. At this point I decided to try my ‘symlink’ trick instead of copying the files directly. Here is the relevant text from ‘Symbolic Link HOWTO.txt’:

//04/19/24 Arduino projects now use an extra folder layer, so instead of

mklink C:\Users\Frank\Documents\Arduino\230918_WallE3_MoveTo_Test_V1\TeensyOTA1.ttl “C:\Users\Frank\Documents\Arduino\Robot Common Files\TeensyOTA1.ttl”
mklink C:\Users\Frank\Documents\Arduino\230918_WallE3_MoveTo_Test_V1\board.txt “C:\Users\Frank\Documents\Arduino\Robot Common Files\board.txt”

The commands need to be:

mklink C:\Users\Frank\Documents\Arduino\230918_WallE3_MoveTo_Test_V1\230918_WallE3_MoveTo_Test_V1\TeensyOTA1.ttl “C:\Users\Frank\Documents\Arduino\Robot Common Files\TeensyOTA1.ttl”
mklink C:\Users\Frank\Documents\Arduino\230918_WallE3_MoveTo_Test_V1\230918_WallE3_MoveTo_Test_V1\board.txt “C:\Users\Frank\Documents\Arduino\Robot Common Files\board.txt”

I opened a command prompt and pasted the first mklink command in, and then edited it to point to my ‘2WheelRobot_HC05_OTA1’ folder. When I was done, I had this command line:

After executing these commands, I confirmed that the ‘board.txt’ and ‘TeensyOTA1.ttl’ files now appear in the ‘C:\Users\Frank\Documents\Arduino\2WheelRobot_HC05_OTA1\2WheelRobot_HC05_OTA1’ folder – yay!

To confirm proper operation, I changed the ‘LED_ONOFF_MSEC’ parameter from 250 to 1000 in the .ino file, changed the COM port to COM12 and hit F5.

Rats! Close – but no Cigar! It started off OK and got to the point where TeraTerm should send the HEX file, and then I got this error message:

Hmm, turns out it actually worked – now the LED is blinking at 1000mSec ON, 1000mSec OFF, instead of 250 ON/OFF. So I tried again and this time I got “unable to connect to Teensy on COM12

25 April 2026 Update:

With a lot of help from Grok, I finally got to the point where I can now do ‘over-the-air’ (OTA) program updates to my 2-wheel robot via the onboard HC-05 BT serial port connected to the Teensy 4.1 Serial2 port (pins 7 & 8). For all the details, see the ’25 April 2026 Update’ section in this post.

At this point in the project, I have a 2 wheel robot with the following characteristics:

  • Teensy 4.1 processor and HC-05 BT Serial (Serial2) allowing OTA program updates
  • Dual DRV8871 motor controllers
  • 7.4V (dual Lithium-Oxide) battery pack with 5V LDO voltage regulator
  • Raspberry Pi5 micro-computer with ROS and other relevant software installed
  • Luxonis OAK-D Lite camera connected to the Pi5 and functionally tested
  • 4ea A3144 Hall-effect sensors mounted in physically quadrature pairs on each wheel, opposite a 4-magnet disc for speed & direction sensing.

At the moment the robot has no distance sensing capability, although the plan is to mount a pair of VL53L1X ToF sensors for side-distance monitoring.

the Pi5 and camera are not yet connected, as their current draw is significant and may require an upgraded battery and regulator for practical use. Here are some photos of the current robot configuration:

Here is the current system schematic. Note that theRaspberry Pi5, Luxonis OAK-D camera and MPU6050 6DoF gyro have not yet been incorporated into the system.

And here is the complete Teensy 4.1 sketch that was used to verify OTA programming capabilities.

27 April 2026 Update:

I ran into a problem with the way the ‘quadrature’ Hall-effect sensor module was mounted on the wheel motors. I had simply hot-glued them to the motor casing, but after a day or so the module fell off, so I needed something a little more permanent.

I designed a printed piece to snap-fit onto the motor body, with a slot in the top for the Hall-effect sensor perfboard, as shown below:

Here is a photo showing the Hall-effect sensor modules mounted on the underside of the robot.

02 May 2026 Update:

As it turns out, the above ‘physical quadrature’ arrangement was optimally bad, because it does not provide the 00 01 10 11 activation sequence required for accurate speed and direction sensing. After figuring this out, I wound up manually adjusting the position of one of the sensor modules to get the proper sequence, and then hot gluing the module down at that point. The following photo shows the somewhat messy current arrangement.

My plan is to design another version of the motor mount for a more permanent installation.

I also spent some time neatening up the topside of the robot and updating the schematic, as shown in the following photos.

04 May 2026 Update:

Grok and I worked together to remove a lot of unneeded PID code and add forward & reverse tick-count based motion control. Now the robot will move forward or backward a specified distance with a command such as “D, 0.3” or “D, -0.5” (meters). It will also stop in response to an ‘S’ command. Here’s the full code:

I also had Grok generate a Python script to automate the task of creating symlinks to my custom ‘board.txt’ and ‘TeensyOTA1.ttl’ files necessary for OTA updates. Now I can double-click a desktop shortcut, point to a folder, and have the above two files appear magically as symlinks to the files in my new “C:\Users\Frank\Documents\Arduino\Robot Common Files 2026” folder. This is wonderful time-saver and I only wish I had done this earlier. Here’s the code:

05 May 2026 Update:

I added a tick-count reset ‘R/r’ operation to the available commands in the ‘CheckForUserInput() function. At this point the robot will go forwards and backwards by a set distance (tick count), will stop on command, and the tick count can be reset to zero at any time. Here is the ‘final’ code:

The next big step in the Teensy code will be to integrate the MPU6050 6DoF gyro into the code so that the robot can make accurate PID-controlled specified-angle turns

Stay tuned,

Frank

Return of the Robot

Leave a reply

Posted 14 October 2024

It’s been almost four months since I did anything with Wall-E3, my autonomous wall-following robot. I’ve been busy with building a new 3D printer (Prusa Mk4) quarrelling with (and losing to) my other 3D printer (Flashforge Creator Pro 2), and some other stuff, but I’m now between projects so I want to spend some more time with Wall-E3. I ran a couple of field tests on my normal home track, and saw that the robot is still having trouble with managing transitions from one wall to another, especially at the end of the kitchen counter (the ‘B’ position shown in the diagram in this post).

Looking through the telemetry log, I was struck by the fact that it appears that the HandleAnomalousConditions() function always called with ANOMALY_CODE == ANOMALY_NONE. Looking through the code, I’m not quite sure where this happens. There is code in UpdateAllEnvironmentParameters(), but it shouldn’t execute unless all the other ‘elseif’s fail. Looking further, I found this code in the HandleAnomalousConditions() function:

However, it doesn’t appear that HandleExcessSteervalCase() is being called at all – hmm.

20 October 2024 Update:

So I think I figured some of this out. Here’s the relevant code in loop():

So the program simply loops through UpdateAllEnvironmentParameters() and HandleAnomalousConditions. The Update function retrieves all sensor information and uses it to update the global variable gl_LastAnomalyCode. HandleAnomalousConditions then uses the value of gl_LastAnomalyCode in a CASE statement to figure out what to do. Here’s the relevant code from HandleAnomalousConditions

The problem I was seeing was, HandleAnomalousConditions() was being called each time, but the ANOMALY_EXCESS_STEER_VAL case was never executed. Somehow, the anomaly code was being changed to ANOMALY_NONE before the call to HandleAnomalousConditions().

I think I finally figured out what was happening. The very first time through loop(), UpdateAllEnvironmentParameters() is called and (normally) sets gl_LastAnomalyCode to ANOMALY_NONE. Then HandleAnomalousConditions() is called and the ANOMALY_NONE case executes. This launches either the TrackRight or TrackLeft functions, which have their own internal loop that doesn’t exit until a non-NONE anomaly is detected. When this happens, the tracking loop exits, but because it started in the ANOMALY_NONE case section of HandleAnomalousConditions(), the next instruction to be executed is the one immediately after HandleAnomalousConditions() exits! The next relevant instruction is the call to UpdateAllEnvironmentParameters() at the top of loop(), which can (and apparently does) modify gl_LastAnomalyCode from ANOMALY_EXCESS_STEER_VAL to ANOMALY_NONE, which means the ANOMALY_EXCESS_STEER_VAL case is never executed.

So, it is critical that nothing changes gl_LastAnomalyCode between HandleAnomalousConditions() calls. As it turns out, the fix was simply to remove the call to HandleAnomalousConditions() at the top of loop(), resulting in the following in loop():

This works because all functions that contain a local loop (like TrackLeft/RightWall()) call UpdateAllEnvironmentParameters() each time through their loop, and exit when a non-NONE anomaly code is detected. With the removal of the call to UpdateAllEnvironmentParameters() in loop(), the next call to HandleAnomalousConditions() ‘sees’ the correct value in gl_LastAnomalyCode.

I made this change and made another run, with the following telemetry output:

In the above telemetry, the significant points are:

  • The first pass through HandleAnomalousConditions(NEITHER) ANOMALY_NONE CASE decides which wall to track and selects TrackLeftWallOffset()
  • Tracking starts at 0.7Sec and ends at 3.0Sec when an EXCESS_STEER_VAL anomaly is detected.
  • In HandleAnomalousConditions(LEFT) with last anomaly code = EXCESS_STEER_VAL shows that the gl_LastAnomalyCode has not been changed
  • In HandleAnomalousConditions(LEFT) ANOMALY_EXCESS_STEER_VAL case shows that the proper case section is executing.
  • HandleExcessSteervalCase(LEFT) open doorway block shows that the appropriate handler function is launched.
  • Ultimately this results in the right wall being captured and tracked (the run was terminated just before the robot actually started tracking the righthand wall).

WallE3 Doesn’t Like Reflective Surfaces

Leave a reply

Posted 08 December 2023

WallE3 went with us last month when we travelled to St. Louis for Thanksgiving with family, and I showed off his autonomous wall following skills. WallE3 actually did great for quite a while – that is until he found himself staring at the side of a floor-mounted wine cooler (wine ‘safe’?). As can be seen in the following short video, WallE3 fell in love with the cooler, and showed his love by repeatedly head-butting it – oops!

After looking at the telemetry data for the run, I saw that WallE3 was measuring much larger front distances – like several hundred centimeters – when it was only a few centimeters from the object. It appears he was backing up to a defined front distance in response to a ‘WALL_OFFSET_DISTANCE_AHEAD’ anomaly, but somehow convinced himself that instead of 20cm from the wall, he was actually more like 100cm away. Of course, since he wanted to be at 30cm (the desired wall offset distance), he drove forward to lessen the distance, thereby bonking into the wall. Then, when he hit the wall, the front LIDAR line of sight geometry changed enough to produce a true measurement of just a few centimeters, which then sent WallE3 running backwards to open up the distance. Lather, rinse, repeat. Here’s an Excel plot of a representative (but not exact – I somehow lost the actual telemetry data for this run).

Representative reflective surface ‘headbutt’ telemetry

As can be seen from the above the measured distance oscillates between the maximum measurable distance of 1000cm to nearly zero. Here’s a photo of the experimental setup that produced the above data.

WallE3 and a reflective surface

The ‘reflective surface’ is a piece of glossy black translucent plastic, oriented at an angle to reflect WallE3’s LIDAR beam upward to the ceiling and then the reflected signal from the ceiling back to WallE3.

I’ve been thinking about this issue ever since first seeing it in St. Louis, but hadn’t come up with any firm ideas about how to solve it. I tinkered with the idea of generating two running averages of the front distance when in the ‘MoveToDesiredFrontDistCm()’ function, with the two averages separated in time by some amount. When approaching a normal non-reflective surface, the two averages would closely track each other, but when approaching a reflective surface that produced the above dramatic distance shifts, then the two averages would be dramatically different around the transitions. This could then be detected, and something done to recover. Then last night while falling asleep, I wondered whether or not the STMicro VL53LXX infra-red LIDAR sensors would have the same problem – hmm, maybe not! If that were the case, then I could probably run one in parallel with the Garmin LIDAR unit, and use it instead of the Garmin for all ‘MoveToDesiredFront/RearDistCm()’ calls.

I tried this experiment using the currently installed rear distance sensors by calling ‘MoveToDesiredRearDistCm()’ with the same reflective surface setup as before, as shown in the following photo:

‘MoveToDesiredRearDistCm()’ setup with reflective surface

Here’s an excel plot of the rear distance run:

MoveToRearDistCm() with 30cm target and reflective surface

As can be seen in the above plot, the rear distance run was completely normal, so the VL53LXX infra-red LIDAR sensors don’t have the same problem – at least not with this translucent glossy plastic material.

10 December 2023 Update:

I got to thinking that maybe the reason the rear distance sensor worked so well with the shiny black material is that it could be IR transparent, meaning that while the front sensor (an LED LIDAR system) would see a ‘mirror’, the rear sensor would just see the toolbox. So, I jumped up on Amazon and got a cheap mirror square so I could answer that question. Here is a short video and Excel plot showing a ‘MoveToDesiredFrontDistCm()’ run with the new mirror square.

As can be seen from the video, WallE was perfectly happy to drive right through the mirror, but I had visions of mirror pieces all over the bench, the floor, and me, so I manually prevented that from happening. Here’s an Excel plot showing the same run:

MoveToFrontDist(cm) approaching tilted mirror

As the Excel plot shows, the robot did OK for the first 20 measurements (about 1sec) but immediately thereafter started a steady 200cm, and it stayed that way until I stopped it at about 2sec

Then I tried the same experiment, but this time utilizing the STMicro VL53LXX IR LIDAR sensor on the rear of the robot, as shown in the following short video:

As the video shows, the IR LIDAR behaved pretty much the same as the LED LIDAR (no real surprise, as they are both LIDAR technology, but still a bummer!

Here’s the Excel plot for this run:

As the Excel plot shows, the distance decreased monotonically for the first 30 points (about 1.5sec) but then shot up to 200 due to the mirror. I believe the lower distances after about point 40 (2sec) were due to me interfering with the IR beam.

So, unfortunately my theory about the rear IR sensor doing better with the shiny black plastic ‘mirror’ because it appeared transparent at that wavelength seems to be bolstered, so I now think that using the VL53LXX sensor instead of the Garmin LIDAR LED sensor for ‘MoveToFrontDistCm()’ operations is NOT going to work. Back to the drawing board :(.

23 December 2023 Update:

I’ve been thinking about this problem for a while now, and have not come up with a good answer; WallE3’s perception of the world around it depends entirely on LIDAR-type distance sensors, so if those sensors produce ‘false’ distance reports due to mirror or mirror-ish surfaces, WallE3 has no way to know the ‘true’ distance – bummer!

So, what I decided to do is to simply detect the symptoms of the ‘mirrored surface’ situation, halt the robot and yell for help. The detection algorithm uses the knowledge that when the robot is moving forward toward a ‘normal’ object, the front distance should decrease monotonically with time. Similarly when the robot is moving backwards toward a ‘normal’ object, the rear distance should decrease monotonically with time. Conversely, when approaching a ‘mirror-like’ object, the measured distance tends to be unstable, with distance increasing instead of decreasing.

The detection algorithm calculates a three-point average each time through the action loop, and compares the result to the last time through the loop. If the new average is greater than the old average, a ‘mirrored surface detection’ is declared and the robot calls ‘YellForHelp()’ which stops the motors and emits an audible Morse code ‘SOS’.

Here’s the full code for ‘DoOneMoveToFrontDistCm()’ function that actually moves the robot and checks for ‘mirrored surface’ error conditions

02 January 2024 Update:

I wasn’t really happy with my previous attempt at ‘mirrored surface’ detection as it seemed pretty to produce false positives. After thinking about the problem some more, I thought I might be able to use a distance variance calculation as a more robust detection method. The idea is that a normal monotonic increase or decrease in distance measurements would have a pretty low variance, while a distance reversal would generate a much larger value.

So, I ran some simulations in Excel using a 5-point running variance calculation, and the results were encouraging. Then, with the help of my lovely lab assistant, I set up an experiment in my office sandbox to see if I could capture a representative mirrored surface ‘screwup’, as shown below:

And here is an Excel plot showing the results (both the distance and 5-pt variance vertical scales have been truncated to show the smaller scale variations)

Front Distance and 5-pt Variance (truncated vertical scales for better visibility)

As can be see in the video and the Excel plot, the robot undergoes a number of distinct front/back oscillations, but then eventually settles down. It is clear from the plot that the 5-pt variance calculation is a good indicator of a ‘mirrored surface’ condition. Just looking at the plot, it appears that a variance threshold of 40-60 should provide for robust detection without much of a risk of false positives.

One other note about this experiment. To get multiple oscillations as shown in the video and plot, the mirrored surface had to be slanted slightly up toward the ceiling. If the surface was oriented vertically like a normal wall, the robot would often miss the first distance and hit the wall, but then would typically back off to the correct distance. I think this indicates that in the vertical configuration, there is enough backscatter from the floor and/or the robot itself to get a reasonable (if not entirely accurate) LIDAR distance measurement.

07 January 2024 Update:

After playing around some more with this issue, I think the above 5pt variance calculation for ‘mirrored surface’ detection will work, so I revised my current well-tested ‘CalcBruteFrontDistArrayVariance()’ function to take a integer argument denoting the number of elements to use, starting at the end (most recent data) and working backwards.

Then I ran another test in my lab, but this time on a non-mirrored surface, to verify that the 5pt variance calc would still work properly and to settle on a good threshold for ‘mirror’ detection. The Excel plot below shows the results of a run where the robot moved backwards (still using the front LIDAR sensor) to 90cm from the wall.

Running 5-point variance of front distances

As can be seen, the variance starts out in the 10 to 20 range during the initial ‘coarse’ distance movement, but then drops into the 0 to 5 range during the second ‘fine tuning’ movement. Comparing this to the previous ‘mirrored surface’ plot leads me to believe that a threshold of 40 would almost certainly (eventually) detect a ‘mirrored surface’ condition.

24 January 2024 Update:

I added the ‘CalcFrontNPointVar(uint16_t N)’ function to the robot code so I could obtain the front variance using just the last N front distances instead of the entire 100 point array. This turned out to work very well for detecting the ‘Mirrored Surface’ anomaly condition. Then I added ‘ANOMALY_MIRRORED_SFC’ to the list of anomaly codes, and added a ‘case’ block in ‘HandleAnomalousConditions()’ to deal with this condition. The handler function is:

At the moment, all it does is call the ‘RunToDaylightV2()’ function, which does a 360º search for the best direction in which to move next. Here’s the telemetry and a short video showing the action.

This experiment led to one of those “Well, DUH!! moments, as the robot’s preferred ‘RunToDaylight()’ heading was right back at the mirrored surface!

After recovering from my ‘face-palm’ moment, I realized I needed to modify the ‘RunToDaylight() function to take heading-start & heading-end parameters to exclude the mirrored surface sector from the ‘RunToDaylight()’ search for an appropriate recovery heading.

28 January 2024 Update:

I modified ‘RunToDaylightV2()’ to take two float parameters denoting the start and ending headings for the search. In the normal non-mirrored surface case, ‘RunToDaylightV2()’ is called with no arguments. The no-argument overload simply calls ‘RunToDaylightV2(startHdg, endHdg)’ with startHdg = endHdg. In the mirrored surface case, the two-parameter version of ‘RunToDaylightV2()’ is called from the ‘ANOMALY_MIRRORED_SFC’ case block of ‘HandleAnomalousConditions()’

Here’s a short video and the telemetry from a test run toward a mirrored surface.

Stay tuned!

Frank

PID Integral Windup Problem in MoveToDesired(Front|Back|Left|Right)Distance()

Leave a reply

Posted 14 November 2023

After getting the charging station (re)integrated with the rest of the system, I have been working on complete travel-charge-travel charge cycles, where WallE3 travels around the house, finds and connects to the charger, disconnects, travels around the house some more, and then finds its way back to the charger – lather, rinse, repeat.

However, in the process I have run into a problem with the MoveToDesired(Front|Back|Left|Right)Distance() function. On several occasions the robot has blown right by the desired distance and run headlong (or backlong?) into a wall. Investigating has led me to realize that the cause of this problem is the infamous ‘integral windup’ characteristic inherent in insufficiently sophisticated PID algorithms. Here’s the relevant telemetry from a recent MoveToDesiredFrontDistance() run, and an Excel plot showing the ‘integral windup’ issue.

MoveToDesiredFrontDistance run showing integral windup problem

In the above Excel plot, the initial error is -221, which causes an output of ~300. The motor speed is clamped to -75, so the distance and error start slowly heading toward the target of 30cm and an error of zero. However, the integral (I) term continues to increase from near zero to well over -1000, and the output term was completely dominated by the integral value, keeping the motor speed clamped at -75 even as the measured distance approached and passed the target. In this particular case, I got lucky as the actual distance and the target distance were within the +/- 1cm termination window and the loop terminated. In other cases where the actual distance went through the termination window too rapidly, the robot would basically continued forever – or at least as long as necessary to ‘unwind’ the integral term.

So, what to do? Reading up on ‘integral windup’, I found this article from around 1990 (judging by dates on the references), and I decided to try the method described there as the ‘back-calculation and tracking’ method. The idea is that when the output gets clamped to some maximum value (+/- 75 in this application), the integral value is recomputed to a value that would produce the output that would naturally produce the clamp value. For instance for an error value of -221, the Ival is -22.1 which results in an output of -309, which gets clamped to -75. For an output of -75 with an input error of -221 we have -221 + Ival*(-221) + (-221*1.5) = -75 –> Ival = (-75 + 331.5)/-221 = -1.16. Checking, -1.16*(-221) = 256.36, 1.5*(-221) = 331.5, 256.36 – 331.5 = -75.14.

After thinking about this some more, I wondered if instead of back-calculating a new I value, maybe I could simply zero out the retained ‘last_Ival’ parameter whenever the PID output value is high enough so that it would get clamped. Assuming the same clamping value of 75 and a ‘P’ value of 1.5, this would happen any time the absolute value of the error value is greater or equal to 75/1.5 = 50.

Using Excel to apply this algorithm to the telemetry above telemetry data, I get the following plot.

MoveToDesiredFrontDistance run showing integral windup problem, with modified Integral term algorithm

In the above plot, the integral term (yellow line) was modified to be exactly zero whenever the output would have been > 75 or < -75 (the motor speed clamping values), even with a zero I value. As can be seen above, it stays zero until point 75. After that it smoothly decreases to about -55.6 at point 99, and then smoothly increases to about 30.98 at point 135, at which point is gets clamped to zero again. The motor speed value (green line) stays at -75 to point 75, at which point it smoothly (and linearly it appears) increases to +75, where it is clamped again.

I believe this might just do the trick. It will certainly prevent infinite runaway if the error term goes through zero too quickly to cause the loop to exit, as the robot will stop and then back up to the target distance.

I *think* I can modify my ‘PIDCalcs()’ function, and then everything that uses it will get the benefit of the ‘non-winding integral term’ algorithm.

17 November 2023 Update:

One day to go until SpaceX makes its second attempt at getting the world’s largest rocket into space – yeah!

To work the ‘integral term windup’ issue, I ported the FrontBackMotionTest code from an earlier program into the WallE3_Quicksort_V4 project so I could iterate easier, and got that running. As a baseline, here is the telemetry and a short video from the first run:

As can be seen in the above telemetry, the ‘Ival’ column shows that the integral term increases monotonically from -19.7 to -998.4, forcing the speed to its maximum value (75) for the entire run. This causes the robot to badly overshoot the target distance, which is why I decided to have the robot perform the ‘second time through’ action shown in the telemetry and video.

I added the following code to MoveToDesiredFrontDistCm() to zero out the ‘lastIval’ value if the error term * Kp > MOTOR_SPEED_QTR

Here is the telemetry and the video from the run with the above change

As can be seen from the above, this change did not affect the robot’s behavior significantly – it still badly overshot the target distance and the ‘second try’ was still required to bring the robot back to nearer the target. However, from inspection of the ‘Ival’ values it is clear that the added code is doing its job of zeroing out the ‘lastIval’ term when [error_term]*Kp > Max speed. Here’s an Excel plot showing the ‘Ival’ and ‘output’ terms from both the above runs.

comparing Ival, output, speed vals before and after Ival clamp mod

In the above plot, the ‘after Ival term stays at a very low (negative) value for almost the entire run, due to the new clamping code. Consequently the ‘after’ output term decreases linearly with the decreasing error term until the resultant ‘after’ speed command comes off the -75 ‘stop’ as shown by the gray line in the above plot.

Unfortunately, the speed reduction from ‘lastIval’ clamping isn’t enough to prevent the robot from overshooting almost as much as it did before. This indicates (at least to me) that at least the Kp value is way too high. Just as a thought, the Kp value should be just high enough so that if the distance error is, say, 100cm, then the output would just clamp at MOTOR_SPEED_LOW –> 75. So, 100*Kp = 75 –> Kp = 0.75, or about half its current value.

It turned out that I not only needed to cut the Kp value in half, but the Ki value as well, so now PID = (0.75, 0.05, 0.2). Here’s the telemetry and video from a run using these values:

I wasn’t able to eliminate the ‘second try’ requirement, although I was able to reduce the overshoot to about half its previous value.

One possible fly in the ointment is the rapid drop of the front variance value during the second run through MoveToDesiredFrontDistCm(); this *should* have triggered an exit from the function with anomaly code = ANOMALY_STUCK_AHEAD, but didn’t – and I don’t know why.

Too tired to go down this rabbit hole tonight – try again tomorrow while waiting for Space X to make history with its second StarShip test flight!

18 November 2023 Update:

Wow! Wow! Wow! I had the pleasure of watching SpaceX’s literally historic second Starship test launch this morning, and I’m still psyched! I have now had the pleasure of watching both a Saturn V launch as part of the Apollo moon landing program and now the Starship and booster launch as part of (I hope) the Mars landing program.

OK, back to robots. Last night while drifting off to sleep, it occurred to me that the ANOMALY_STUCK_AHEAD alert is conditioned on the current motor configuration as well as the front variance value. Here is the actual code that determines this:

So in this case, the motors were running in reverse, so neither of the motor conditions were TRUE, and therefore IsStuckAhead() returned FALSE. Mystery solved!

At this point I think I’ve done as much as I can with the FrontBackMotionTest program. I will modify the other ‘MoveTo’ functions to clamp the Ival term as described above. However, it is also clear that the ‘second try’ part of the ‘MoveTo’ functions cannot be eliminated without having to accept significant target distance overshoot, and that the original PID values (1.5, 0.1, 0.2) actually work better than (0.75, 0.05, 0.2), as the lower PI values cause the second try operation to take significantly longer. Here’s an Excel plot comparing just the ‘second try’ results for both:

‘Second try’ results for (0.75,0.1,0.2) and (0.75, 0.05, 0.2)

In the above plot, the ‘2’ results are for (0.75, 0.05, 0.2). As can be seen, the ‘2’ configuration takes over twice as long to complete as the ‘normal’ (1.5, 0.1, 0.2) run, and the issue with the front variance value going to zero doesn’t exist for the ‘normal’ run.

This makes it clear that while clamping the Ival does help significantly, the change in PID values does not.

18 November 2023 8:43PM EST Update:

While cleaning up the code, I noticed I had missed one spot where the global OffsetDistKp value should have been replaced by ‘Kp’, the local value, and unfortunately it was in the section that determined whether or not the Ival value would be clamped, as shown below:

This had the effect of skewing the point at which the Ival stopped being clamped, so the previous results are a bit suspect. I ran a few more tests, and discovered the function actually performed better with a higher Kp value than the original 1.5. The reason for this is that a higher Kp value moves the point at which Ival can start accumulating later in the run (closer to the target distance thus less error), and so more quickly reduces motor speed. In addition, the Ival clamping action has no effect on the second pass through the function as the motor speed values never approach the max speed, so this part operates as designed as well. Here’s a run showing the result of using a PID of (2.0, 0.1, 0.2):

As the data shows, the overshoot is only 6cm (as opposed to the 18cm overshoot without the Ival clamp), and the recovery pass completes in about 2 sec instead of over 5. Here are Excel plots showing the primary and secondary passes.

Second pass correction with PID (2.0,0.1,0.2). Time scale is 20 units/sec, so total time ~ 1.5sec

So, it looks like I want to change the global OffsetKp value from 1.5 to 2.0, leaving Ki & Kd unchanged.

19 November 2023 Update:

Looking back at the results from the above tests, I realized that the second call to MoveToDesiredFrontDistCm(Offset, Kp, Ki, Kd) (the ‘second time though’ step) isn’t actually in the MoveToDesiredFrontDistCm() function – it is part of the test harness. So, I think I need to incorporate the second pass directly into the function so it will get performed everywhere MoveToDesiredFrontDistCm() is called. I wonder if I can do this recursively, by calling MoveToDesiredFrontDistCm() from inside MoveToDesiredFrontDistCm()?

So, I moved the ‘Second Time Through’ code from the test harness into MoveToDesiredFrontDistCm(), as shown below:

Here’s the telemetry from the run with recursive MoveToDesiredFrontDistCm() calls:

The first run through MoveToDesiredFrontDistCm() occurs normally, with the exit at the 30cm target distance as expected, and then the expected overshoot to 25cm. Then the second call to MoveToDesiredFrontDistCm() starts at 24cm and gets to 29cm in approximately 1.5sec, followed by two more 10-item distance displays. I guess the first distance display is from the one inside the MoveToDesiredFrontDistCm() call, and the second one is the one from the test harness. I need the delay caused by the 10-distance display, but I can probably replace it with just a 1-sec delay.

Here’s another run with the loop replaced by a 1-sec delay:

From the above, it looks like the function is doing what it should. It keeps iterating until the distance measured after a 1-sec delay fits within the +/- 1cm window.

MoveToDesiredRearDistCm():

Now that MoveToDesiredFrontDistCm() is working, I started on MoveToDesiredRearDistCm(). I started by copying MoveToDesiredFrontDistCm() and changing all relevant occurrences of ‘Front’ to ‘Rear’. This was about 99% of the required effort. However, there were two issues that surfaced during the port. The first was that the test program needed the ability to pass the PID values into the function, but all the mainline code uses just a single parameter (the desired offset). As a temporary fix I created a four-parameter version of the function with all the code, and had the single parameter version pass the global PID values to the four-parameter on. However a much better solution was to remove the single parameter version of MoveToDesiredFrontDistCm() entirely, and instead declare the four-parameter version with three default values right after the definition, as shown here:

The second issue was the sign of the speed variable in the RunBothMotorsBidirectional() function. For MoveToDesiredFrontDistCm() the parameters to this function had to both be negative, as in RunBothMotorsBidirectional(-speed, -speed), but in the MoveToDesiredRearDistCm() they have to be positive as in RunBothMotorsBidirectional(speed, speed). Once this was accomplished, the MoveToDesiredRearDistCm() operation was successful.

MoveToDesiredLeft/RightDistCm():

I started on these two functions, but soon realized that these two functions aren’t called anymore. They were originally used as part of wall offset capture operations, but were abandoned in favor of an algorithm that uses a perpendicular approach. Soooo, I will just remove these two functions entirely and call it good!

Stay tuned,

Frank

28 October 2023 ‘Field Test’

Leave a reply

Posted 28 October 2023

Field test starting in bedroom hallway, just before MBR door. I have elected to start here, as WallE3 has been consistently making it this far with no problems whatsoever. Everything wen OK except for a couple of strange occurrences where it appeared that WallE3 was simply going straight without tracking anything. One of these occurrences was right at the end of the run. Here’s the video of the run:

And here’s the telemetry:

The strange behavior occurs just after WallE3 changed from the left to right-hand wall when it hits the open doorway of the larger guest bedroom, and then makes the hard right-hand turn to follow the near wall when it exits the hallway into the dining room area.

From the video, this occurs at approximately 203 sec, so I have excerpted the telemetry starting at the point where the robot transitions from the left to right wall.

At 215.3 sec WallE3 experiences another EXCESS_STEER_VAL anomaly (as the robot exits the hallway into the dining room area). WallE3 makes a slight right turn, moves forward a bit, and stops; this is the normal ‘move ahead one skosh’ after an EXCESS_STEER_VAL anomaly detection.

Then at 216.6 sec with gl_Left/RightCenterCm = 219.7/77.2, Left/RightSteerVal = 1.00/-1.00 it makes another right turn to, presumably, follow the right-hand wall, but it doesn’t try to capture the 30 cm offset and track – it just goes straight, eventually running into the wall on the far side of the open doorway into the kitchen area bathroom. From the telemetry it looks like WallE3 called ‘MoveToDesiredFrontDistCm() with a target distance of 30 cm as part of ‘CaptureWallOffset()’ but started the process almost parallel to the wall rather than perpendicular.

Matching the code up with the telemetry and the video, it becomes apparent that ‘HandleExcessSteervalCase(RIGHT)’ is called by HandleAnomalousConditions(RIGHT) as soon as the EXCESS_STEER_VAL anomaly is detected.

The first thing that HandleExcessSteervalCase() does is move forward for 500 mSec – the ‘skosh’ intended to clear the point where the anomaly occurred to obtain valid distance measurements. In this case, however, the video shows that the ‘forward’ direction was slanted off to the right instead of straight ahead, which meant that instead of detecting an ‘open corner’ condition with no trackable wall in range, the robot saw a trackable wall to the right at 77.2 cm, as the following telemetry line shows:

216672: gl_Left/RightCenterCm = 219.7/77.2, Left/RightSteerVal = 1.00/-1.00  

So, instead of the ‘open corner’ block in HandleExcessSteervalCase() executing, the ‘else if (trkcase == TRACKING_RIGHT)’ block was run , as shown below:

Since HandleExcessSteervalCase was called with TRACKING_RIGHT, and because gl_RightCenterCm = 77.2 was less than MAX_TRACKING_DISTANCE_CM (100 cm), ‘TrackRightWallOffset(WALL_OFFSET_TRACK_Kp, WALL_OFFSET_TRACK_Ki, WALL_OFFSET_TRACK_Kd, WALL_OFFSET_TGTDIST_CM)’ was called without running ‘ChooseBetterTrackingSide()’.

In TrackRightWallOffset(), CaptureWallOffset(TRACKING_RIGHT, 77.2) was called because the starting offset was too high for immediate tracking. In CaptureWallOffset(), a 90º CW turn was performed to (supposedly) point WallE3 directly at the wall to be tracked, and then ‘MoveToDesiredFrontDistCm(tgt_offset_cm)’ was called to capture the wall offset of 30 cm.

Unfortunately, the 90º CW turn step assumes the robot is already oriented parallel to the wall to be tracked, but because the actual physical configuration at this point was an open corner instead of an open doorway, WallE3 wasn’t at all parallel to the wall. So, the turn just oriented the robot to about 40-45º to the wall rather than 90º. The next step in the offset capture routine is to call MoveToDesiredFrontDistCm() to move the robot to the desired offset distance (in this case, 30 cm). But the front distance measurement never got down to 30 cm, as the robot wound up running along the baseboard, parallel to the wall. This would have continued indefinitely except WallE3 ran into the far edge of the doorway into the kitchen area bathroom.

So, everything worked just like it was supposed to, except that when the robot exited the bedroom hallway it was turned just enough so that it saw what looked like a trackable wall to the right, instead of gl_LeftCenterCm & gl_RightCenterCm > MAX_TRACKING_DIST_CM, so the open corner case wasn’t executed.

To fix this problem (and hopefully not create other ones) I added the condition that to bypass the ‘open corner’ case, the absolute value of the ‘trackable side’ steering value must be less than 1/2 the max steerval of +/- 1.

Well, that didn’t work as well as I thought. I made another field run, this time starting just before the end of the bedroom hallway where it opens into the dining area. What should have happened is the robot should have detected the ‘open corner’ configuration, made a 90º CW turn and carried on tracking the right-hand wall. What actually happened wasn’t that. Here’s a short video showing the action, along with the telemetry from the run

So, it appears the same sort of thing happened again, only this time it was the left-side distance that was less than MAX_TRACKING_DIST_CM. Since both steervals were +/- 1 this should have caused the ‘open corner’ block to execute.

Fixed (again!). Running another test.

ALRIGHT!! This time things worked just like they are supposed to! Here’s a short video showing the action, and the telemetry from the run.

As can be seen from the video and telemetry, the ‘open corner’ configuration was properly detected even with gl_Left/RightCenterCm = 84.1/235.6, Left/RightSteerVal = 1.00/1.00 (left side within MAX_TRACKING_DIST_CM but abs(left steerval) > MAX_STEERVAL ). Yay!!

Stay Tuned,

Frank

The WALL_OFFSET_DIST_AHEAD Anomaly

Leave a reply

Posted 21 October 2023,

The WALL_OFFSET_DIST_AHEAD anomaly is triggered when WallE3 is about to run head-on into an upcoming wall. The idea behind handling this anomaly is to allow WallE3 to navigate around internal corners. Up until a few days ago, WALL_OFFSET_DIST_AHEAD anomaly just called ‘BackupAndTurn90Deg(bool bIsCCW)’. This function backed the robot up to achieve the desired wall offset, turned 90º in the direction away from the last tracked wall to parallel the upcoming wall and then exited, whereupon loop() was re-entered from the top and a new tracking assessment was made.

However, this treatment failed spectacularly in the guest bedroom, as the ‘new’ wall was too short to track properly. This problem led to the development of the ‘RunToDaylight’ algorithm to allow WallE3 to find the best direction in which to travel. I now have RunToDaylightV2() working very nicely, but now I have the opposite problem; now RunToDaylightV2() is more likely than not to simply turn WallE3 180º and go back the way he came – perfectly legitimate from RunToDaylight’s point of view, but boring and too simplistic.

So, how to mix the two approaches (BackupAndTurn90Deg & RunToDaylight) so that RunToDaylight is used in tight corners, but BackupAndTurn90Deg is used for ‘normal’ wall configurations where there is ample (or at least reasonable) room for travel on the perpendicular wall.

I’m going to try combining the two options. The idea is that BackupAndTurn90Deg would be the default response to a WALL_OFFSET_DIST_AHEAD anomaly, but if the available front distance after the 90º turn is less than something like 2 x WALL_OFFSET_DIST_AHEAD (currently set at 30cm), then the RunToDaylight will be called to find a better direction in which to move.

This turned out to be pretty easy. The changes necessary to the WALL_OFFSET_DIST_AHEAD anomaly case in HandleAnomalousConditions is shown below:

The following short video and telemetry shows the action when WallE3 encounters a corner with not enough room to track the next wall in the internal corner.

And the following video and telemetry shows the action when WallE3 does have room enough to follow the internal corner wall. Note that in these two conditions, the code was not changed – the only thing that changed between the two runs is the ‘third wall’ was moved away from the first one to give WallE3 sufficient room to follow the internal corner wall.

So it seems this problem is pretty much solved – YAY!!

Stay Tuned,

Frank

Using QuickSort on Multiple Associated Arrays

Leave a reply

Posted 06 October 2023,

At the very end of the 04 October 2023 Update to the “The Guest Bedroom Problem” post , I made the following statement:

“Also, I need to figure out how to sort through the 360º search results. I would like to wind up with an array of steervals sorted in decreasing magnitude order, with a companion array of headings so Hdg[i] <–> Steerval[i]. Not sure how to do this yet, but it sounds promising!“

After some research on array sorts in C++ I came across this post with a nice example of a quick sort program, which I shamelessly copied. After some fumbling around (including writing my own ‘swap’ routine to allow future porting to Arduino code) I got it to work in my Visual Studio 2022 Community Edition setup with a single int[] array as shown in the following image:

Now the challenge was to extend this algorithm to sort multiple same-sized companion arrays. Looking at the QuickSort code, it appeared all I had to do was duplicate the ‘swap’ operation on all the companion arrays using the same swap indices determined for the ‘master’ array. One additional fly in the ointment was the requirement to handle both int[] and float[] arrays.

First I modified my ‘swap’ routine to be a generic template function as shown below:

Then I renamed the ‘arr’ array from the example to ‘FrontD’ and defined a second ‘Hdg[]’ array of float values with the same length as the original example array as shown below:

Then, for each occurrence of a call to ‘mySwap’ for the ‘FrontD’ array, I added a second call for the ‘Hdg’ array as shown below:

When I ran this code, it *almost* worked right off the bat. Unfortunately the ‘Hdg’ slave array wound up being sorted slightly differently than the ‘FrontD’ master array. After closely examining the code, I finally found the problem. In one place the original programmer used the indexing syntax ‘[i++]’ and ‘[i–]’ as input to the ‘mySwap’ function. This worked fine for the single master array, but failed with the second array because on the second call to ‘mySwap’ the indices had been changed – oops! Here is the original and revised syntax:

Now both calls to mySwap() use the same index values, and life is good. Here’s a debug printout from VS2022 showing a successful program run with a master (int Frontd[]) array and one slave (float Hdg[]) array:

And here is the complete code that produced the above output:

09 October 2023 Update:

After convincing myself that this scheme for synchronizing the sorting of ‘master’ and ‘slave’ arrays, I decided to port the capability into my robot code. I created a new WallE3 program called ‘WallE3_Quicksort_V3’ as a copy of ‘WallE3_Complete_V5’, and then added the ‘quickSort’, ‘partition’, and ‘mySwap’ functions from ‘Quicksort_V3’.

Then I set up a test block in setup() as shown below:

With this setup I got the following output:

In this test, the FrontD[] is the ‘master’ and the ‘Hdg[] is the ‘slave’, and the algorithm is set up to sort the array in increasing order.As can be seen from the above output, the FrontD[] array after Quicksort is indeed sorted from smallest to largest value, and the Hdg[] array elements after Quicksort are ordered in such a way as to correspond to their original relationship to FrontD[].

In my application I want to sort the master array in descending order rather than ascending, so after some googling I found that making the following change:

causes the sort to run in the other direction, giving the following output:

As desired, the ‘FrontD’ master array is sorted in descending order, and the ‘Hdg’ slave array elements are still synchronized with their original FrontD companion elements.

So I changed the test to use real data using the initialization code below:

and got the following output:

Gee, that went well — The FrontD distance array isn’t ordered at all – yuk!

OK, so back to basic troubleshooting. The first thing I did was to replace the FrontD[6] test array in my QuickSortV3 C++ program with the FrontD[36] array of actual front distance values (edited in Notepad++ to be a single line of comma-separated values) to see if I could establish a working baseline – an absolute necessity for efficient troubleshooting.

I had to edit the QuickSortV3 program to remove the references to the second Hdg[](slave) array, as I didn’t want to complicate things, but after I did this, the program sorted FrontD[36] properly in both the forward and reverse direction. To get the reverse sort, I had to flip ‘<=’ to ‘>’ in two places, and ‘>’ to ‘<=’ in one place, as shown below:

The following Excel plot shows the result for both the forward and reverse sorts

Now that I have a working baseline with my QuickSortV3 C++ program, it became easy to see the problem with my WallE3_Quicksort_V3 Arduino program; there are three places that need to be changed for reverse sorts, and I only changed one – ooops! After fixing these problems, I got the following output:

And now, the slave sort seems to be working as well, as shown by the following Excel plot:

The above plot looks very confusing at first, but it seems to be accurate; the ‘before’ picture is straightforward – the robot rotated in 10º steps, so the smooth blue line is expected. The ‘after’ plot looks crazy, but remember it is synchronized with the reverse sorted front distance array, so there is no organizing principal. The relationship of heading values with front distance values after the reverse sort is easier to see with the text output from the program, shown below:

For instance, the largest front distance shown is 507cm (row 9 in the original listing). In the unsorted data, the heading associated with 507cm is 82.1º. In the reverse sorted listing, 507cm is of course on the first row, and so is 82.1º. The lowest front distance (last row of the reverse sorted list) is 24cm, and the heading on the same line is -106.1º. Looking through the original (unsorted) list, 24cm is found on line 26, and the heading on that line is -106.1º as expected.

At this point, it is clear that my plan to have WallE3 turn a full circle while recording front distances and associated headings, then reverse sort the distance data array as the ‘master’ array while maintaining each distance value’s associated heading (the ‘slave’ array) is going to work. Then I should be able to easily find the heading associated with the median front distance of the first(only) group of front distances greater than some threshold – say 1.5m. Looking at the reverse-sorted front distance data above, I see there are about 10 distance measurements above 1.5m as shown below:

The median heading value for this group of 11 distances is 71.4º, which is associated with the front distance value of 444cm.

11 October 2023 Update:

After figuring out how to change my quickSort() function from a forward (increasing values) to a reverse (decreasing values) sort, I decided that I should make it capable of performing either sort (fwd or rev), by adding a boolean parameter to the function signature. I started by going back to my C++ program and making the mods there, and I was able to make it work fairly quickly, as the following output shows:

Then I ported the changes to my Arduino program and got the same results, as shown in the following output:

And here is the test code that produced the above output:

13 October 2023 Update:

After getting this to work in my C++ project, I ported it over to WallE3_QuickSort_V3 and got it working there. Thinking about the overall ‘Guest Bedroom Problem’, it is clear to me that I will need six synchronized arrays – FrontD, Hdg, L/R Dist, L/R Steer. At first I thought I could do this with five calls to ‘QuickSort() – one each for (FrontD, Hdg) and then one each for the remaining four, each using FrontD as the ‘master’ array. However, when I tried this, it failed miserably – only the first sort (FrontD, Hdg) worked, and the remaining four calls did nothing. After thinking about this for a while, I eventually figured out that the first call – (FrontD, Hdg) worked because each time two FrontD array items got swapped in mySwap(), the Hdg array got swapped in the same way – preserving synchrony. However, when the sorted FrontD array was used in the second and subsequent calls, mySwap() never gets called because all FrontD items are already in order. This meant that the second and subsequent ‘slave’ arrays stayed in their original unsorted state – oops!

So the answer to this problem is either keep replacing the ‘master’ parameter to QuickSort() with the unsorted version of FontD[] so that it will get sorted again (causing the required mySwap() calls to the ‘slave’ array, or modify QuickSort to take all five ‘slave’ arrays as parameters. Either way is a real PITA, but I think the ‘all at once’ strategy is more straightforward.

After implementing the ‘all at once’ strategy, I got the following output from my test program:

It appears that both the FWD & REV sorts succeeded (at least with respect to the front distance values). Spot checking the other arrays, we see for front distance values of 23 & 449:

So it is clear that all six arrays are synchronized through both FWD & REV sorts – Yay!

Looking at the reverse sorted and synched data for FrontD values above 1.5m we see a number options for travel directions, as detailed by the following lines from the reverse sort output:

The first four lines above are all ‘left-side’ tracking options. The fifth line above (at FrontD = 240) could actually utilize either the left or right walls for tracking, and the last two are ‘right-side’ options.

The option with the largest ‘head room’ (515cm) is shown in the first line above; on a relative heading of 82.0º, there is 515cm of travel distance available, and the robot is 30.2cm away from the left wall and is oriented almost parallel to it (left steerval is -0.2).

So it looks like this ‘Run To Daylight’ scheme might actually work, but there were a LOT more options than I thought I would have for tracking side and tracking direction. This may have been caused by the fact that I was doing the testing on my lab bench, with lots of ‘clutter’ around. It may be that in a real situation there are very few (or even no) options – we’ll see!

I modified my test program to choose the first acceptable parameter set from the reverse-sorted data, then turn WallE3 to that heading and refresh all parameters. The following short video and the resulting telemetry output are shown below:

As can be seen from the above, the first set of parameters in the synchronized arrays met the criteria, and was chosen. When WallE3 turned to the selected heading and refreshed parameters, everything except the front distance matched very well. I believe the difference in front distances was due to a very slight change in heading which resulted in the distance to a desk chair being measured instead of the distance to the far wall.

Next I tried a test in my office with a simulated corner situation, to see if WallE3 could used his new superpowers for good. I removed the infinite loop at the end of the test program and let loop() run as normal, after setting gl_LastAnomalyCode to ANOMALY_NONE. The following short video and telemetry readout shows the action:

Test of WallE3’s new ‘Run To Daylight’ capability

From the telemetry output we can see that WallE3 found an acceptable tracking option at index 1 in the reverse-sorted FrontD array, at a relative heading of 141.4º from the start of rotation, with the following parameters:

Then the robot turned to the desired heading, and then dropped into the normal ‘top of loop’. This caused TrackLeftWallOffset(350.0, 0.0, 20.0, 30) to be called. WallE3 tracked the left wall nicely from 0.7 sec to 3.2 sec where it ran out of wall and detected an EXCESS_STEERVAL Anomaly. All in all, this test seemed to work perfectly.

Stay tuned,

Frank

The Guest Bedroom Problem

Leave a reply

Posted 01 October 2023

After solving the problems described at the bottom of my “Responding to Steerval out of Range” post, I was able to get WallE3 to successfully enter our small guest bedroom, as shown in the following short video:

Unfortunately the robot wasn’t really tracking the left ‘wall’ (door surface) – it was just going straight until it ran into the trashcan, and then couldn’t figure out what to do next. As I worked with the code I got WallE3 to track the door OK, but then it still had problems with the trashcan and cupboard. Rather than starting in the hallway for each test, I decided to emulate the configuration in my office for more rapid iterations. Here’s the actual guest bedroom configuration:

And here is the simulated guest room configuration in my office:

Simulation of guest room configuration

And here is a short video showing a mostly successful run in the above simulated guest room configuration:

The telemetry from the run is shown below:

The salient events from the telemetry:

  • 0.6 – 2.1 sec: tracks left wall
  • 2.1 sec: detects WALL_OFFSET_DIST_AHEAD anomaly.
  • 2.1 – 7.6 sec recovers from WALL_OFFSET_DIST_AHEAD and starts tracking left wall again
  • 7.7 – 9.2 sec tracks left wall (simulated doorway into guest br)
  • 9.2 sec detects WALL_OFFSET_DIST_AHEAD anomaly (end of simulated door?)
  • 9.3 – 10.7 sec: recovers from WALL_OFFSET_DIST_AHEAD and starts tracking left wall again
  • 10.7 – 12.1 sec: tracks left wall
  • 12.1 sec: detects WALL_OFFSET_DIST_AHEAD anomaly. I think this is caused by the box (the simulated near end of the cupboard). This time the anomaly recovery calls ChooseBetterTrackingSide() with an average heading value of 93.1º. The result of this algorithm is 9 votes for left side, 0 for right. The robot starts tracking the left wall with an initial offset of 17.9cm, which is too close, and this causes ‘CaptureWallOffset()’ to be called. CaptureWallOffset() makes a 90º CCW turn to face the left wall, and then calls ‘MoveToDesiredFrontDistCm()’ to move (backup) to a front distance of 30cm. Once this is accomplished, a 90º CW turn is made to re-parallel the left wall, and then ‘RotateToParallelOrientation(LEFT)’ is called to complete the orientation operation.
  • 29.9 – 31.9 sec: tracks left wall (the simulated cupboard face).
  • 31.9 sec: an EXCESS_STEER_VAL anomaly is detected after the robot runs off the end of the simulated cupboard face

02 October 2023 Update:

Having solved the problem of getting from the hallway into the guest bedroom, there remained the problem of negotiating a ‘cul-de-sac’ at the far end of the cupboard, which I simulated in my office as shown in the following video:

And here is the telemetry from the run:

The robot tracks the left wall properly for the first 2.5sec, but then gets trapped in the ‘cul-de-sac’, and eventually resorts to calling a ‘RunToDaylight()’ function to get itself unstuck. RunToDaylight() calls ‘RotateToMaxDistance()’ which performs a full 360º rotation, memorizing the front distance and heading at each step. After the full rotation, the robot turns to the heading associated with the peak front distance and heads that way. As can be seen from the video, this works very well, but over the course of several runs the robot hit the corner of the wall a couple of times because the ‘peak’ heading was recorded just as the front distance sensor (but not necessarily the robot itself) cleared that corner. Here’s an excel plot that shows the issue.

Looking at the plot, it is clear that any heading from about 83 to 150º would suffice, so maybe picking the peak value isn’t so smart. Maybe picking the median value above some distance threshold (like maybe 1.5-2.0 meters) would be a better bet.

04 October 2023 Update:

As I was drifting off to sleep last night, I was mentally reviewing the above ‘run to daylight’ experiment and it struck me that WallE3 had ignored a perfectly good trackable wall segment (at about 31sec, step 22, 94.4º, 215cm in the above 360º search). It occurred to me that I should be searching for the point(s) where WallE3 is parallel or mostly parallel to a wall, with a meter or more of front distance available.

I think I might be able to use a “Figure of Merit” (FoM) defined as the front distance divided by the left or right steering value to detect the optimum travel direction. Steering values vary from 0 to +/-1 so it would need to be abs(FrontD/SteerVal), and it would be maximum at the point where steerval is minimum and front distance is maximum.

One fly in the ointment is the case where steerval is == 0; that would cause FrontD/steerval to be undefined – oops! So I think I need to modify the function that retrieves steervals from the Teensy to limit steerval to something like .001, so the FoM wouldn’t go above a few thousand (10/07/24 note: This was done in ‘GetRequestedVL53l0xValues()’)

Also, I need to figure out how to sort through the 360º search results. I would like to wind up with an array of steervals sorted in decreasing magnitude order, with a companion array of headings so Hdg[i] <–> Steerval[i]. Not sure how to do this yet, but it sounds promising!

07 October 2023 Update:

After some web searching and experimentation, I now have a program that can sort a ‘master’ array and one or more ‘slave’ arrays such that the slave array is rearranged into the same order as the master. See this post for all the gory details.

Now I think I have the tools to address the ‘guest bedroom problem’, as follows:

  • Make the same 360º rotation as before, saving L/R steervals, headings, and front distances into separate 36-element arrays.
  • Sort the above arrays, using front distances as the master, and the others as slaves.
  • Find the element in the front distance array that is greater than a set threshold (say 150cm), has an associated steerval <= (MAX_STEERVAL / 2) and produces the highest ‘FoM’ as described above. Turn to the heading associated with this element, and start tracking that side.
  • If no front distance element / steerval combination satisfies the above criteria, then turn to the heading associated with the middle element of the first/only set of front distance values above the above distance threshold and move forward until another anomaly is detected.

11 October 2023 Update:

I think I’m going to need to save/sort the L/R distances in addition the values listed above, for a total of six parameters; L/R distances, L/R steervals, Hdg, and Front distance. The FrontD array would be the ‘master’ array, and the other five would be ‘slave’ arrays. I could modify my current QuickSort() routine to do all five slave arrays at once (by adding four more ‘swap’ lines in two places in ‘partition’) but this would also require modifying the signatures of QuickSort(), partition() – yuk! I could also simply call ‘QuickSort()’ five times (once for each ‘slave’ array) with the unsorted FrontD array as the master each time. I like this latter approach as it preserves more of the generality of the ‘QuickSort()’ routine.

To do this, I’ll have to define all six arrays at global scope with the following memory requirements:

  • uint16_t FrontD[36]: 2 bytes/value, 36 values –> 72 bytes
  • float Hdg[36]: 4 bytes/value, 36 values –> 144 bytes
  • float LeftSteer[36]: 4 bytes/value, 36 values –> 144 bytes
  • float RightSteer[36]: 4 bytes/value, 36 values –> 144 bytes
  • float LeftDist[36]: 4 bytes/value, 36 values –> 144 bytes
  • float RightDist[36]: 4 bytes/value, 36 values –> 144 bytes

for a total of 4*144 + 72 = 648 bytes. The current memory requirements for WallE3_QuickSort_V3 on a Teensy 3.5 is:

So another 648 bytes is not going to be a problem. After defining and initializing the additional four arrays and recompiling, I got:

So the increase from 115248 to 115312 in program size and from 7728 to 8304 bytes wasn’t enough to even move the percentages – woohoo!

16 October 2023 Update:

After getting the ‘one master and five slave array’ quicksort algorithm working, I did some more testing in my office. The test configurations that featured side walls within tracking distance (100cm) worked very well; however, when I simulated the configuration where no trackable side walls could be found, the robot consistently failed to orient to the most open direction – instead it wound up pointing to one edge or the other of the ‘open direction’ wedge. The algorithm I was using was to use the heading associated with the middle entry of the first N (reverse sorted) front distances.

Then I tried using the middle entry of the first N (reverse sorted) heading values, where the group of heading values to be sorted were the ones associated with the above first N reverse-sorted front distances. However, this didn’t work either, because when I designed the experiment, the maximum distances naturally occurred around 180º from the starting orientation, and this meant that the first N headings included both positive and negative values. When reverse-sorted, the positive values were at the top as expected, but the negative values got sorted to the bottom – oops!

The solution to the problem was to create a temporary heading array, fill it with the heading values associated with the first N front distances as before, but with the heading values modified to fit into the 0-359º range (by adding 360 to negative values). The result of this trick was to ‘homogenize’ the headings so they were all sequential, and then selecting the median value did indeed point the robot in the correct direction, as shown in the following short video and telemetry output.

In the above telemetry output, the first two values in the Hdg array associated with the reverse-sorted FrontD array are negative, so if the Hdg array is reverse sorted to find the best RunToDaylight heading, the two headings associated with the longest open run aren’t considered unless they are first modified such that all headings lie in the 0-360 range. The ‘tempHdg’ array accomplishes this.

After the above successful test of the new ‘RunToDaylight’ capability, it is time to integrate this into the rest of the program. At the moment the code is contained in ‘RunToDaylightV2’ and it is only called from ‘#pragma region QUICKSORT_TEST in setup(). In the mainline code, ‘RunToDaylight’ is called from ChooseBetterTrackingSide() when the ‘Better Side’ can’t be determined.

ChooseBetterTrackingSide() is called from HandleExcessSteervalCase() when a trackable wall is found on both sides, and a determination has to be made on how to proceed. So, it appears that all I have to do is comment out (or delete) the QUICKSORT_TEST #pragma region and RunToDaylightV2 will be called as necessary.

After making the necessary changes, I made a long ‘field test’, starting in our entrance hallway and going through the kitchen and down the bedroom hallway to the end, and then once again into the guest bedroom. This all worked perfectly, and I was pleased to see that ‘RunToDaylightV2()’ was called at the end. See the accompanying video and the telemetry for all the details:

17 October 2023 Update:

I tried another Guest Bedroom ‘field test’, with different results this time, as shown in the accompanying video and telemetry:

Stay tuned,

Frank

WallE3_Complete_V5 Print Timing Issue

Leave a reply

Posted 24 September 2023,

This post addresses the problem I have started to see regarding print buffer overflow due to the decreased time (now 50mSec) between environment updates. When running longer field tests, I have started to see many lines where telemetry printout has become garbled, with the next line of printout starting before the previous line has finished, as shown in the accompanying printout

In the above printout, you will notice that the horizontal scrollbar is much smaller than normal, indicating the line lengths are much longer than normal, and the portion that is visible is really really ugly!

So, what to do? I ran a search through my almost 7000 lines of code for ‘gl_pSerPort->printf(‘ to see how big of a problem I’m looking at, and the search came back with some 400+ hits – ouch! One thought would be to put an ‘if’ statement around each printf call and use an ‘EllapsedMillisec’ variable to limit calls to, say, every 100 or 200 msec, but doing that for 400+ calls would get pretty tedious. Another thought is to abstract all these print statements into a function, and have the time spacing internal to the function. So the function call would look something like:

and the function internals would look something like:

The problem with the function internals is the argument list would have to match the contents of ‘formatstr’, which means I would have to have an overload for every unique format, and changing anything would be a hassle, as I would have to find and edit the correct overload – yuck!

OK, more thought needed on this – maybe the ‘if’ statement is the way to go (maybe a macro?)

Another thought that just came up; if I filter all print statements by an elapsed time gate, that might actually filter out important one-time error information – ouch! This potential problem suggests adding a priority value to the ‘LogPrint()’ function, so the function might look like:

And good programming technique would demand that the PRIORITY_CODE have a default value, which (for Arduino programming) would require that every overload have a function declaration at the top of the program – yuck! yuck!

Maybe it would be easier to just put the

around each and every ‘low priority’ print statement (all 400+ of them)? yuck! yuck! yuck!

After thinking about this some more, it occurs to me that this problem may be a lot smaller than I think. The primary culprit is the long telemetry string printed out each WALL_TRACK_UPDATE_INTERVAL_MSEC (currently set to 50mSec). It may well be that I only need to guard this one print statement and 99% of the time that will do the trick. As I find other printout statements that are causing problems, I can treat them as well.

25 September 2023 Update:

I did a search for ‘gl_pSerPort->printf(‘ but bypassed everything in setup() and loop(). Starting in TrackLeftWallOffset() in the ‘while (gl_LastAnomalyCode == ANOMALY_NONE )’ block I put an ‘if(mSecSinceLastTelemetryUpdate > PRINT_INTERVAL_MSEC)’ guard around the call to ‘OutputTelemetryLine(TRACKING_LEFT)’, as shown below:

26 September 2023 Update:

After guarding the ‘OutputTelemetry()’ call in both TrackLeftWallOffset() and TrackRightWallOffset(), I ran another field test, and saw that telemetry output was clean, and now coming in 0.1sec intervals, as intended – yay!

Stay tuned,

Frank

Responding to ‘Steerval out of Range’ Anomalies

Leave a reply

Posted 02 September 2023

As described in this earlier post, I hatched a plan to use ‘out of range’ (>= 1) steering values to manage ‘open doorway’, ‘open corner’ and other navigation anomalies, but as usual, this has turned out to be ‘easier said than done’. The problem of deciding what to do after an excessive steerval anomaly detection is complicated by the robot’s behavior leading up to the anomaly. As the steering value rises toward the limit, the robot naturally changes its wheel motor speeds in a manner intended to bring the steering value back down again. In the case of an ‘open doorway’ event, this means the robot turns into the open doorway instead of continuing straight. Then, when the robot stops in response to the associated anomaly detection, it is oriented in a way that makes figuring out what to do next very difficult. The following short video, illustrates the problem. In the first part of the video the robot stops at the 45º break and executes the current anomaly response algorithm; it turns parallel to the left wall, turns 90º to the wall and goes forward to the desired offset distance, then makes another 90º turn and continues wall-following. At the end of the video at the next ‘excess steerval’ anomaly, the robot makes an abrupt turn into the open doorway and can’t recover.

So, what to do, what to do? The robot doesn’t have any idea where it is in the house; all it knows is the steering value for the tracked wall first increased sharply (causing the abrupt left turn shown in the video) and then went out of range (causing the robot to stop). What I want the robot to do is figure out that there is a trackable wall opposite the open doorway, and discovering/tracking it is the preferred behavior.

So, how does the robot ‘discover’ that the other wall is a better tracking candidate? The following diagram shows the situation just after the robot stops.

From the robot’s point of view, the nearest wall is still on its left side but the short left-side segment (the side of a bookcase in this particular configuration) isn’t long enough to actually perform a ‘RotateToParallelOrientation’ procedure. In the meantime, the perfectly good wall a meter or so away is totally ignored. Oops! May have found the problem; if the center distance to the other wall is more than 1m – it is outside the current value for ‘MAX_TRACKING_DISTANCE_CM’

07 September 2023 Update:

After further review, it appears the issue wasn’t so much distances > MAX_TRACKING_DISTANCE_CM as having the reported distance to both sides < MAX_TRACKING_DISTANCE_CM, even after the ‘excessive steerval’ anomaly trigger. This occurs, for instance, when the robot successfully navigates the ‘open door’ into my office, transitions to the right-hand wall (kitchen counter) and then runs off the end of the counter wall. This generates an ‘excessive steerval’ anomaly, but when the robot checks again to decide what to do, both side sensors report distance values < MAX_TRACKING_DISTANCE_CM, most likely due to a ‘false’ distance report from a kitchen eat-in counter bar stool, as shown in the following diagram.

In the above diagram, the robot is located at ‘Position B’. There is nothing to track on to its right (except the legs and cross-braces of the bar stool), and a nice wall to the left. However, because the robot can still ‘see’ the bar stool legs on the right, both sides report distances < MAX_TRACKING_DISTANCE_CM.

So, I’m thinking about introducing the ability for the robot to discern the ‘best’ way forward, but this begs the question of what ‘forward’ and ‘best’ mean in this context.

Forward:

I propose that ‘forward’ in this context means – the direction roughly in line with the previous average heading of the robot. In the above diagram, the average robot heading was toward the top of the diagram, so ‘forward’ would mean no more than 90º either side of this heading (note that WallE3 knows nothing about compass headings – so the baseline average heading at the point of decision can be anything).

Best:

I propose that ‘best’ in this context means “the wall that subtends the largest angle in the ‘forward’ (i.e. +/- 90º from average heading from last 2-3 sec before the anomaly) hemisphere.

In order to implement this scheme, I need to also implement a way of determining the average heading value for the last 2-3 sec. This implies an array of heading values that is updated at each measurement interval, with a length that covers the desired time interval. I already do this with front and side distances with aFrontDist, aLeftDistCM, aRightDistCM, and aRearDistCM, with various sizes. The aFrontDist and aRearDistCM sizes are supposed to be large enough so that the front and rear variance calculations will trigger a ‘stuck’ anomaly after about 5 sec of no movement; however, I see from the current code that while the front array size was bumped to 100 just last month to accommodate the new 50mSec cycle time, the rear one is still at 50. It occurs to me that the array size should automatically change based on the cycle time to maintain the desired ‘stuck’ detection time frame (nominally 5 sec). To do this I could create a ‘STUCK_TIME_FRAME_SEC’ constant, and then multiply this by the measurement time interval in Sec in the aFront and aRear array declarations. Here’s what I came up with:

This at least compiles, so I am optimistic that I haven’t created a huge disaster.

To start this implementation, I created a new global variable ‘gl_AvgHeadingDeg’, a new function to initialize a new ‘aAvgHdgDeg’ array, and ‘UpdateAvgHdgArray()’ to update the array and the average heading at each regular measurement interval as shown

Then I added code to UpdateAllEnvironmentalParameters() to update the average heading array, as shown below:

Then I created a ‘ChooseBetterTrackingSide()’ function to actually determine which side to ‘nominate’ for further wall tracking, as shown below:

10 September 2023 Update:

To test the new average heading idea, I modified my existing ‘#ifdef HDG_ONLY’ code block to add the average heading, and then manually rotated the robot back and forth to see if the average heading value was reasonable. After the usual amount of code debugging, I got what I think are reasonable results, as shown in the following Excel plot:

Robot very rapidly rotated from ~0deg to ~90deg

With the actual and average headings stabilized, I very rapidly rotated the robot from ~0º to ~90º and then allowed the average heading value to catch up. As predicted with the current update rate of 50mSec, the average value array of 60 points took approximately 3Sec to catch up.

Now that I have the average heading value feature working, I can start testing the ability to actually use the average value. Here’s a short video and telemetry output from a test in my office showing the ‘ChooseBetterSide’ algorithm in action:

Choose Better Side algorithm test

The salient points in the above video:

  • At 4sec (3.3sec in telemetry), the robot detects an ‘EXCESS_STEER_VAL’ anomaly
  • At about 4.5sec (3.8sec in telemetry) ChooseBetterSide() is called
  • 4.5-10.0sec, the robot makes nine 10º CCW turns to measure left/right distances in ‘forward’ direction, and then turns back to the initial average heading.
  • As shown in the telemetry, ChooseBetterSide found 7 left and 0 right trackable wall segments, and returned TRACKING_LEFT
  • At 12sec, the robot executes a ‘RotateToParallelOrientation(TRACKING_LEFT) to prepare to track the left side wall
  • At 13-15sec the robot tracks the left side wall. Note that because the robot was within the tracking distance window at the end of the RotateToParallelOrientation call, it started tracking immediately.

Here’s the telemetry from the above test run:

Next I ran a ‘field’ test in the hallway outside my office, as shown in the following short video:

Interestingly, the normal ‘open door’ algorithm was used for the open doorway into my office, at about 14sec into the video, causing WallE3 to transition to the ‘other’ (kitchen counter) wall. At the end of the kitchen counter (at about 31sec) the ‘ChooseBestSide’ algorithm was used to transition back to the left side wall. Unfortunately he robot failed to make the left turn into the bedroom hallway at the end of the run. More work to go!

Here’s the telemetry from the run:

Stay Tuned!

Frank

13 September 2023 Update:

Yesterday, at the end of the run the robot quit with the output “HandleExcessSteervalCase in ‘else //OpenCorner:’ Can’t decide what to do.” However, when I tried to duplicate this problem in my office with a simple ‘L’ (open corner) configuration, the robot instead chose the ‘ChooseBetterTrackingSide() option – rats.

Looking through the code and the results, I can see this happened because the steervals from both sides were < MAX. The distances from both sides were > MAX_TRACKING_DIST_CM, but I am currently not using this factor for decisions.

Looks like I need to use both distance and steerval to distinguish among the various error configurations (or maybe I need to use the previous steerval, the one that caused the anomaly?

So, just using current steerval, we get the following four possibilities

  • 1: Left < MAX, Right < MAX
  • 2: Left < MAX, Right > MAX
  • 3: Left > MAX, Right < MAX
  • 4: Left > MAX, Right > MAX

If we then add the previous steerval (the one that created the anomaly) into the mix we get eight – the above four with PrevLeft > MAX, and four with PrevRight > Max.

So, assuming that the anomaly was EXCESS_STEERVAL on the left side, then the first possibility above was true in the test case, so what should have happened is recognition of an ‘open corner’ case, with the previously tracked wall on the left side. This should produce a left 90º turn, a forward run of 0.5-1.0sec and then either track left or ‘choose best side’.

15 September 2023 Update:

After thinking about this problem for a while, I have come to the conclusion that I’m overthinking the ‘Excess Steerval’ and ‘Open Corner’ conditions. I now think that the ‘Open Corner’ condition can be uniquely defined as the condition where both left and right distances are > MAX_TRACKING_DIST_CM after an ‘Excess Steerval’ anomaly occurs. This means my earlier re-write of HandleExcessSteervalCase() to incorporate steerval information is incorrect – bummer!

Looking at the two wall diagrams above, it looks like there are only a few conditions that are possible after an ‘Excess Steerval’ anomaly:

  • An ‘open doorway’ configuration can be identified uniquely because it will have a tracked-side distance > MAX_TRACKING_DIST_CM and a non-tracked side < MAX_TRACKING_DIST_CM.
  • An ‘open corner’ configuration can be identified uniquely because it will have tracked-side and non-tracked side distances > MAX_TRACKING_DIST_CM.
  • The ‘Which Wall?’ (ChooseBestWall) configuration can be identified uniquely because it will have a tracked-side distance < MAX_TRACKING_DIST_CM.

So I think this all boils down to the following flow chart:

Based on the above, it appears that my original version of HandleExcessSteervalCase() was closer to the mark than my 09/04/23 re-write – oops! Fortunately I kept the old version as a comment block, so I may be able to fix my ‘oops’ without too much agony.

After modifying (once again) my HandleExcessSteervalCase() function, I got the ‘open corner’ case working – at least I think I did. The following short video and telemetry printout shows the action.

Successful ‘Open Corner’ Run

Here is the current ‘state of code’ for the ‘HandleExcessSteervalCase’ function:

25 September 2023 Update: Open Corner/Doorway Success!

After quite a bit of yelling, cursing, coding and re-coding, I now believe that WallE3, my autonomous wall-following robot has truly gained the ability to correctly handle ‘open doorway’ and ‘open corner situations. Here’s a short video showing the results of a ‘field test’:

Successful ‘open doorway’ and ‘open corner’ field test

Salient points from the video:

  • 0:12: WallE3 encounters an open doorway to the left, with a trackable wall (kitchen counter) to the right. WallE3 stops and transfers to the right wall.
  • 0:29: WallE3 encounters an ‘Excess Steerval’ situation, but doesn’t know which wall is better. It utilizes the ‘ChooseBetterTrackingSide()’ function to determine that the left side wall is a better bet, and transfers to it for further progress.
  • 0:49: WallE3 encounters an ‘open corner’ situation, where both left and right side distances are > MAX_TRACKING_DISTANCE_CM. Since the last-tracked wall was on the left side, WallE3 turns that way and continues to track around the corner.
  • 0:57: WallE3 encounters another ‘open doorway’ situation, and properly transfers to the right wall for onward tracking.
  • 1:09: WallE3 encounters another open doorway on the left (non-tracked) side, but because it is tracking the right wall, this is simply ignored.
  • 1:14: WallE3 encounters another ‘open doorway’ situation, and properly transfers to the left wall for onward tracking.
  • 1:28: WallE3 encounters a WALL_OFFSET_DIST_AHEAD anomaly and quits, because there is no handling code for this yet.

This is by far the ‘cleanest’ run WallE3 has made to date, and gives me some hope for the future. The handling code for the WALL_OFFSET_DIST_AHEAD anomaly has already been written and tested in earlier versions of the code – I just haven’t ported it into the new ‘HandleExcessSteervalCase()’ architecture.

26 September 2023 Update:

Looking at the ‘HandleAnomalousConditions() function in ‘WallE3_Complete_V4.ino’, I see the following:

For the ‘ANOMALY_OBSTACLE_AHEAD’ case, the response was to call ‘BackupAndTurn90Deg(trkcase == TRACKING_RIGHT, true, MOTOR_SPEED_FULL), so it backed up and turned right (CW) if it was tracking the left wall, and left (CCW) if it was tracking the right wall. The ‘BackupAndTurn90Deg()’ function turns all the LEDs ON, calls ‘MoveToDesiredFrontDistCm(WALL_OFFSET_TGTDIST_CM)’ to move backward, turns all the LEDs OFF, and then calls ‘SpinTurn()’ to do the 90deg turn. Note that this procedure now ignores the ‘uint16_t motor_speed’ parameter in the sig to ‘BackupAndTurn90Deg().

The ‘BackupAndTurn90Deg()’ function is called in only two places; the above HandleAnomalousConditions() block, and in ‘IRHomeToChgStn()’. Both calls specify a speed to use, which ‘BackupAndTurn90Deg()’ ignores because the ‘backup’ part of ‘BackupAndTurn90Deg()’ is controlled by a PID and is limited to MOTOR_SPD_QTR.

So, I have revised BackupAndTurn90Deg() to remove the speed parameter and the front/back boolean from the signature, and ported the code for ANOMALY_OBSTACLE_AHEAD, ANOMALY_STUCK_AHEAD, ANOMALY_STUCK_BEHIND and ANOMALY_WALL_OFFSET_DIST_AHEAD cases from WallE3_Complete_V4

28 September 2023 Update:

After porting the remaining anomaly code handlers to WallE3_Complete_V5, I made another ‘field test’. This time WallE3 travelled all the way down the hallway as before, but this time instead of stopping with an unhandled anomaly condition, the robot backed up from the door, turned around a few times, and then butted its head into the left-side wall – oops! It then backed up and did it again! If I hadn’t turned it off, I suspect it would have continued doing that forever. Here’s the video showing the action:

The telemetry from the above run is shown below:

As shown in the video and telemetry, everything goes swimmingly until about 89sec (90sec & change in the video) where the robot reports a WALL_OFFSET_DIST_AHEAD anomaly. This is expected, but WALLE3 doesn’t back up as far as it should, and then after doing the expected 90º CW turn, instead of starting to track the new left wall, it does another 180º CW turn and runs headlong into the old left wall (the left wall of the hallway, facing the closed door). Regarding the backup maneuver, it appears it started at 18cm and stopped at 19cm, well short of the expected 30cm.

From the code in HandleAnomalousConditions(), The ANOMALY_OBSTACLE_AHEAD case is shown below:

So BackupAndTurn90Deg(false) is called which should result in a backup to 30cm, then a 90º CW turn. Looking at BackupAndTurn90Deg(false), we see

So BackupAndTurn90Deg() calls MoveToDesiredFrontDistCm(WALL_OFFSET_TGTDIST_CM) which should back up to 30cm.

In MoveToDesiredFrontDistCm() we have:

This function starts with a measured distance of 18cm and a target of 30. The column header line gets printed OK, but then it immediately stops at a distance of 19cm. Obviously the

statement is returning a FALSE value and terminating the block, but I don’t see why

Oops! Using the wrong distance variable – gl_LeftCenterCm instead of gl_FrontCm. Apparently I never ported the correct code from 230918_WallE3_MoveTo_Test.ino to WallE3_Complete_V5.ino.

Stay Tuned,

Frank