Wall-E3 Multi-Lap Wall-Following Trial

Posted 06 March 2022

I’ve pretty much finished with transitioning my autonomous wall-following robot from the old Arduino MEGA 2560 main controller to the new Teensy 3.5 main controller, and now I am moving on to actually getting Wall-E3 to do the job it was created for – namely, to follow walls autonomously. This post describes the result of a multi-lap run through my little testing ‘sandbox’, consisting of a set of barriers forming a 2m X 2m rectangular ‘room’. Here’s a short video showing the run:

I captured the telemetry output from the run and went back through it pretty much line-by-line, trying to make sure I understood all the actions displayed in the video – particularly the little off-piste excursion at about 1:00 on the second lap, just before the end of the run (the run was cancelled when a portion of the wall fell over on Wall-E3).

First Leg: 36.0- 38.9 Sec (5-8 sec in video)

In the first leg, the robot starts off parallel to the wall on the left, but too close (16cm instead of 40cm). It makes a 27.1º CW turn away from the wall to achieve a cut angle of 30º, moves forward, and then turns back 30º CCW to end up parallel to the wall, and about 36cm away – almost perfectly spaced 40cm off the wall.

Next it tracks down the wall from 14.992 sec to 17.390sec , trying (and generally succeeding) to maintain the 40cm standoff distance.

At 17.390 sec it detects an upcoming obstacle (the obstacle detection distance was set to 20cm for this run), and stops (not quite getting all the way stopped before running into the wall – oops!).

First-to-Second Leg Transition:

The handling procedure for the ‘OBSTACLE_AHEAD’ case is to stop, back straight up to achieve the nominal wall offset distance (40cm here), then make a 90º turn (CW in this case) away from the wall to orient itself parallel to the wall again.

As can be seen from the above telemetry, that is exactly what happens, backing up to the point where the front LIDAR sensor shows 38cm and then making the required 90º turn with SpinTurn(CW, 90.00, 45.00). The ‘backup and turn’ evolution is completed in approximately 1.5 sec.

Second Leg: 47.4 – 49.0 Sec (16-19 sec in video)

Second-to-Third Leg Transition:

Third Leg: 57.0 – 59.8 Sec (25-29 sec in video)

Transition and Fourth Leg: 61.0 – 70.4 Sec (30-40 sec in video)

Transition and Fifth Leg: 71.6 – 80.9 Sec (40-50 sec in video)

Fifth-to-Sixth Leg Transition: 82.1 – 88.0 Sec (50-58 sec in video)

Sixth Leg up to Anomaly 88.9 – 89.8 Sec (57-59 sec in video)

On this leg an anomaly occurred. The robot detected a ‘Stuck Ahead’ condition, defined by the condition where the mathematical variance of the last N front LIDAR distance readings falls below a set threshold. This should never happen while the robot is actually moving, but it clearly did happen in this case (unfortunately I wasn’t recording the front distance measurement or the distance measurements to the other wall, so I can’t go back and see exactly what happened). The recovery procedure for a ‘stuck ahead’ condition is to make a 90 deg turn away from the nearest wall, move forward for 1 second, and then make another 90 deg turn to return to a parallel course, but offset 10-20 cm from the previous track. In this case, the robot turned toward the nearest wall, clearly a mistake (it was a mistake in TrackLeftWallOffset() – since fixed). However, it was not a disaster, as the robot made the second turn before hitting the wall, and from there on it returned to normal left wall tracking mode.

Summary:

This first test of left wall tracking performance was very encouraging. The robot was able to continue tracking operations over several laps, including an instance where it recovered from an inadvertent ‘Stuck’ detection. The test could easily have continued until the batteries died, but had to be aborted when one section of the foam-core wall fell in on top of the robot – oops!

Also, this run pointed out the need for more focused telemetry. For this test I was only reporting the left-side and rear distances, but now I know I need to add the right-side measurements as well as the front and rear variance numbers.

09 March 2023 Update:

After cleaning up some messy initialization code, and improving telemetry readouts, I ran another complete left-side wall-tracking lap in my sandbox, as shown in the following short video:

09 March 22 Left-side wall-tracking lap

The telemetry for this run is shown below:

From the telemetry, it takes about 8 sec for all sensor hardware initialization. After that the left/right/front/rear distance arrays are initialized, and the initial front/rear variances are calculated. All this is summarized on line 26-27.

First leg: 12.866 – 15sec (3-5 sec in video)

Left-side tracking starts on line 34. First (line 37) the robot turns to its initial offset capture heading, moves to the desired offset distance (not shown) and then turns back to parallel the wall (line 39). Actual tracking starts on line 43 at 12.866 sec elapsed time (this corresponds to about 3 sec into the video)

At line 65 (about 15 sec elapsed time) the robot ‘sees’ the upcoming wall, stops and then backs up to 38cm (16.4 – 17.3 sec, 5-6 sec in video). At line 78 it makes a 90deg CW turn to line up with the current wall section, and then navigates down the wall (18.6-21.3sec, 8-11sec in video)

Second leg: 18.6 – 21.3sec (8-11 sec in video)

The robot ‘sees’ the upcoming wall on line 119 (21.010 sec, 11sec in video), backs up (lines 123-129, 11-12 sec in video) and makes another 90deg CW turn to follow the 3rd wall

The third and 4th legs are very similar to the first two, with the robot ending back where it started at 32.574 sec (23 sec on video), ‘seeing’ the upcoming wall at line 221. At this point I transmitted the ‘C’ character over the wireless link to enter manual control to terminate the run.

Summary:

This 4-leg run was pretty much perfect. I adjusted the MIN_FRONT_OBSTACLE_DIST_CM from 20 to 30cm, and this stopped the robot from banging its head against the walls – yay! Also, the telemetry readout changes made for a much more understandable output. I was happy to see that the front variance stayed well above 10,000 the whole time, but unhappy to see that the rear variance was essentially zero the entire time. The low rear variance is due to the fact that the rear VL53L0X sensor range is only about 100cm, and after that it always reports ‘819’. This is not a real problem – it just means that I can’t use the rear variance number to detect a ‘rear stuck’ condition unless it happens within a meter or so from a wall. Hmm, maybe I could use the information from both the front & rear variance numbers to create a more robust detection system.

Stay tuned,

Frank

Wall-E3 Replacing Mega 2560 With Teensy 3.5 Part VIII

Posted 19 February 2022,

At this point in the evolution of Wall-E3, all the hardware seems to be working, so it’s time to get serious about wall tracking. Last fall I made another run at wall tracking with Wall-E2, and wound up with an algorithm that would first capture the desired wall offset, and then track it ‘forever’. This worked great, but the approach is at odds with the general processing architecture. The current tracking architecture is set up as a loop, where all pertinent parameters are updated every loop period, and the appropriate action is taken. In the case of wall tracking, the ‘action’ was one left/right motor speed update. This allows rapid recognition of, and adaptation to, various ‘error’ conditions, like being stuck or about to run into something. The algorithm developed last fall does none of this, so it can’t react properly (or at all, for that matter) to things like an upcoming wall.

I’m starting to think I can use a hybrid approach – use the current capture/tracking algorithm pretty much as it stands from last fall, but have it check for ‘error’ conditions each time through its own internal loop. If any unusual conditions are detected, then force an exit from the tracking routine and another pass through the main ‘traffic director’ function ‘GetOpMode()’. The updated mode assignment will then percolate back down through loop() and cause the appropriate handling function to be called.

22 February 2022 Update:

As a start, I ported the ‘TrackLeft/RightWallOffset() functions to Wall-E3 and, after the normal number of screwups and mistakes, I got the left side tracking algorithm working, as shown in the Excel plot and short movie clip below:

Here’s the complete code for ‘TrackLeftWallOffset()’:

The above algorithm works great, but it runs in an infinite loop once it has captured the wall offset. Based on my above comments about a hybrid approach, I could add tests for stuck and/or front or back obstacles to this loop (instead of the current ‘while(true)’). This would cause TrackLeftWallOffset() to exit, and the GetOpMode() function could assign the appropriate mode, which would then cause the proper function to execute.

Or, I could eliminate GetOpMode() entirely and put it’s logic in ‘loop()’? Actually, looking at the loop() function in FourWD_WallE2_V12.ino, my last iteration with the Arduino Mega2560, I see that GetOpMode() is called at the start of loop(), and then the OpMode switch statement comes pretty much immediately afterwards. Here’s the code:

The MODE_CHARGING and MODE_HOMING cases are self-contained, so no changes would be needed for them. The MODE_WALLFOLLOW case is sub-divided into TRACKING_LEFT and TRACKING_RIGHT cases. If all the inline code in TRACKING_LEFT was replaced with TrackLeftWallOffset() and that of TRACKING_RIGHT with TrackRightWallOffset(), with these two functions augmented by the current if (bIsStuck) , if(bObstacleAhead) and if(bObstacleBehind) guard code (pretty much as it now stands), then that should work. I think I’ll give that whirl and see what happens.

To start the process, I created yet another project – WallE3_WallTrack_V3 (to preserve the currently ‘working OK on left side’ status of WallE3_WallTrack_V2) and try porting the GetOpMode() and loop() code from FourWD_WallE2_V12.

02 March 2022 Update:

I now have a ‘loop() only’ version of WallE3 running that properly tracks the left side. Everything is basically the same as before, except the loop() function, shown below:

The ‘IR HOMING’ and ‘CHARGING’ blocks are essentially unchanged, and the ‘WALL TRACKING’ block is much simpler. All ‘anomaly’ (robot stuck either forward or backward, robot approaching an obstacle ahead or behind, dead battery, etc are all handled internally to the two ‘TrackLeft/RightWallOffset()’ functions. Here’s the (potentially infinite) ‘while()’ loop:

As the code above shows, the while loop will continue to execute as long as the ‘errcode’ value is ‘NO_ANOMALIES’. Internally the ‘CheckForErrorCondx()’ function surveys the inputs from all sensors and attempts to detect any anomalous behavior. Any return value except NO_ANOMALIES causes the while() loop to exit. Each potential anomaly condition has its own handling function, which exits back to ‘loop()’ and the process starts all over again. I believe this is a much cleaner approach than I had before with the ‘GetOpMode()’ function.

I also took the opportunity at this point to fix a long-standing problem with the code. The front-facing LIDAR unit returns distances in Cm, while all seven VL53L0X time-of-flight distance sensors report in mm. Not only did this torture me mentally (let’s see – is it Cm or mm here?), but it caused the new ‘CalcRearVariance()’ function to crater, because the 10x larger numbers, when squared, caused the ‘uint16_t’ type to overrun and produce crazy variance numbers. So, I changed the GetRequestedVL53L0XValues() function to convert mm to Cm, changed all the variable names from xxxxMM to xxxxCm and carefully combed through the entire codebase, correcting the inevitable wash of ’10x’ errors. In the end though, I made the codebase much more consistent and understandable (I hope).

Stay Tuned,

Frank

Wall-E3 Replacing Mega 2560 With Teensy 3.5 Part VII

Posted 02 February 2022,

After struggling through all the I2C craziness with the Wire library vs i2c_t3, and internal pullups vs no internal pullups, I think I’m back in business with a running Wall-E3 system

Wall-E3 with second deck installed and (mostly) tested

So far, I have been able to verify that the main processor can talk to the second deck VL53L0X array, the IR Homing subsystem, and that the red laser still works. Now I need to go through the rest of the robot’s subsystems:

  • Irun & Itot current sensors << CHECK
  • HC-05 Bluetooth link
  • Front LIDAR distance sensing << CHECK
  • Speaker << CHECK
  • Charging subsystem << CHECK
  • MPU6050 IMU response << CHECK
  • Charge/Turn Status LEDs << CHECK

I also still need to do something about porting the ‘tail-light’ assembly from Wall-E2 to Wall-E3. On Wall-E2, this assembly was part LED holder and part terminal strip mount – but the terminal strip portion is no longer needed. What to do? What I did was simply mount just the top half of the assembly at the rear center of the robot, and connect up the pins to the T3.5. Neat, clean, sweet!

Tail Light assembly and power strip cover on old WALL-E2
Top part of LED assembly mounted rear center on WALL-E3

13 February 2022 Update:

When I looked at the HC-05 Bluetooth module again I decided to see if I can replace it with a Pololu Wixel module, due to the problems I have been having with the 1-3 second delay associated with the HC-05 Bluetooth link. On several occasions I was trying to stop my robot remotely, only to have the delay that seems to be inherent in the BT link allow my robot to crash into (or dive over) something that it shouldn’t have. I have been using Wixels for years with my robot project, and find them quite reliable, and getting a single command through the link and into the robot seems to be instantaneous.

I started by simply switching the Teensy 3.5’s Rx0 & Tx0 jumpers from the HC-05 to a Wixel module, without doing anything else. I already had a companion Wixel connected to my PC via USB, so getting telemetry out of (and commands into) the robot was pretty painless.

Pololu Wixel wired into Teensy 3.5 Rx0 & Tx0 pins

The next step in the process was to figure out how to actually replace the HC-05 module with a Wixel. I figured I could just hack the current perfboard module that houses the 5V regulator and the HC-05, but I decided maybe I could make thinks a bit nicer by combining the Wixel module (mounted vertically like the HC-05), the Adafruit 1NA169 high-side current sensor, and the voltage regulator all into one PCB. IBased on my previous experience with the VL53L0X array PCB, I thought I could do this fairly easily (OK, so I was smoking dope – what can I say).

I did manage to work out a method for plugging the Wixel perfboard into the regulator board, without having to change its original wiring at all. I did have to add an ‘outboard’ 2-pin female header for Wixel Tx & Rx lines, as the two header pins I used for the HC-05 configuration were obstructed by the Wixel/perfboard module.

Wixel/perfboard module plugged into HC-05 socket

Anyway I did manage to get a PCB designed and sent off to JCLPCB without too much agony, and I gained some more skill working with DipTrace as well. I hadn’t done any real pattern and/or component editing before, and this project forced me to learn how to do that – neat!

Here’s the schematic for the combined circuits:

Schematic combining Adafruit 1NA169 current sensor, Wixel and 5V regulator

I decided to use the 1NA169 ‘as is’, and simply use a 5-pin header to connect the module to the PCB. This required that I construct a custom component for the module, along with a custom ‘attached pattern’, as shown:

Custom module and ‘attached pattern’ for Adafruit 1NA169

Then I used DipTrace’s very nice and intuitive PCB layout software modules to produce a reasonable layout, as shown:

PCB layout for the combined module

Then I literally copied the playbook from my VL53L0X Array PCB project and ordered 5 PCBs from JLCPCB. Within an hour of finalizing the above layout in DipTrace, I had uploaded the Gerber files to JLCPCB and ordered 5 boards (the default order number). Total cost for 5 boards, including shipping? – – – –

Put that in your pipe and smoke it! (oops – the PC SWAT squad will be after me now!)

09 March 2022 Update:

When I received the finished PCBs, I discovered that my little perfpoard Wixel module didn’t fit onto the PCB – I hadn’t taken into account the ‘overhang’ of the Wixel on the component side of the module, and the fact that the actual Wixel module sports a 5-pin header instead of a 4-pin one. So, back to the drawing board (literally) where I produced another PCB design and sent it off to JLCPCB (at $10/shot for 5 PCB’s, I can afford a lot of mistakes!). Here’s the new design schematic and PCB

Wixel and 5V Regulator Module. Note added ‘Power ON’ LED
Revised PCB design

11 March 2022:

So I got my new PCBs in, and had a chance today to check it against the schematic (yep – works), populate one PCB, and check it electrically (yep – works). Here’s the finished module:

populated and checked PCB. Note ‘Wixel’ installed on far side of perfboard.

The next step is to install the new module to replace the stand-along INA169 module and the 5V Regulator/Wixel module. Here’s a comparison shot:

New PCB on the left, stand-alone 1NA169 module and 5V Reg/Wixel on the right

I printed up a nice PLA mount for the new PCB (I forgot to design mounting holes into the PCB, so I had to add those manually), and installed it onto the robot, as shown below:

Much cleaner layout after combining current sensor, Wixel module, and 5V reg onto one PCB

02 April 2022 Update:

After using the Rev2 regulator/Wixel board for a while, I decided I needed to turn the vertical Wixel board around, so I could see the status LEDs when the board is mounted on the robot. So, in just a few minutes with DipTrace, I was able to spin the mounting configuration for the Wixel 180 degrees, as shown the comparison photos below:

Revised PCB design
Flipped the vertical Wixel, and added mounting holes
Latest PCB design mounted on Wall-E3

Stay Tuned,

Frank

Wall-E3 Replacing Mega 2560 With Teensy 3.5 Part VI

Posted 30 January 2022

This ‘Replacing Mega 2560 with Teensy 3.5’ series of posts deals with upgrading my autonomous wall-following robot with a new form factor for easier turning, and a Teensy 3.5 for the main processor instead of the older/slower Arduino Mega 2560. In my previous post on this subject I described my failed effort to interface the new Teensy 3.5 main processor with the existing second deck hardware, specifically the Teensy 3.5 processor that manages the seven-element array of VL53L0X ‘time-of-flight’ distance sensors. The failure was due to the lack of pullup resistors on the I2C SCL/SDA lines between the main processor and the Teensy 3.5 VL53L0X array manager. I2C pullups were not required when I was using the i2c_t3 library, but apparently are when using the Teensy variant of the Wire library

I have spent the week between that last post and this one trying to determine why the Wire library requires external pullups and the i2c_t3 library doesn’t, and how I can work around that requirement so that I don’t have to find a way of installing the external pullups. Although it wouldn’t be a disaster to use external pullups it would be a major PITA, because there is no convenient place to put them; the connection from the main T3.5 processor to the VL53L0X T3.5 processor is made with direct male-male pin jumpers. To install external jumpers I’d have to figure out an intermediate landing spot on either the main deck or the second deck, and add an auxiliary board with +3.3V (although +5V would probably be OK as T3.5 GPIO pins are all 5V tolerant), ground, and the 4 connections for the two sets of I2C lines, all to add two measly resistors to the circuit. And to add insult to injury, I would have to do all this while knowing for a fact that the resistors aren’t actually necessary – grrr!

Being a stubborn type, I spent the week trying to figure out how to enable the internal (~33KΩ) pullups on the I2C lines without screwing up the normal I2C activity on the pins, and I think I finally succeeded just this morning (see this post for all the gory details).

So now that I have eliminated the need for external pullup resistors (HAH!!) I can continue my effort to port the second deck functionality to Wall-E3. The following photo shows where I left off with the I2C pullup resistor investigation:

simulated main processor T3.5 on plugboard, with I2C connection to VL53L0X processor – and NO PULLUPS!

As can be seen in the above photo, I could successfully communicate between the two T3.5’s using I2C without external pullups with simple I2C demo example programs running on both processors.

Now the challenge is to install the main processing code on the plugboard T3.5 and the VL53L0X management code on the second deck T3.5 and verify that the main program can ‘see’ distance measurements from the seven VL53L0X modules tied to the second deck T3.5.

The first thing I did was to load up the unmodified sketches for both the VL53L0X array manager T3.5 and the main processor T3.5. When I started them up, the main processor sketch hung up at the VL53L0X array processor connection check:

Then I added in the physical 2.2KΩ pullups on the plugboard, and the system immediately came to life, displaying VL53L0X distances on the main processor telemetry screen – yay! This told me two things immediately; the first was, all I had to do way back when I first transitioned from the i2c_t3 library to the Wire library (but NO, I was too damned stubborn! . The second was, it is pretty much a dead certainty that adding the internal pullup activation code to both the main processor and VL53L0X array processor sketches will allow comms without pullups.

So, I modified both the sketches to do just that. The VL53L0X processor only requires internal pullup activation on Wire0 – the I2C buss connected to the main processor, as the VL53L0X breakout boards contain 10KΩ pullups (I did it for all three I2C busses anyway). Here’s the modified section:

Note in the above that the Wire2 code uses PORT_PCR_MUX(5) rather than PORT_PCR_MUX(2), as the I2C function is found on ALT5 for pins 3 & 4. Kurt E’s wonderful spreadsheet is a great reference for this sort of thing – any serious Teensy user should have this document in their reference library.

After making these changes, the I2C comms between the main processor and the VL53L0X array processor worked perfectly – job done!

The Wire1 I2C bus goes to the T3.5 VL53L0X array processor as discussed above. I enabled internal pullups on this bus as follows:

And then I added code to the Wire I2C bus going to the T3.2 IR Homing processor as follows:

To verify proper operation with the actual Wall-E3 robot Teensy 3.5 main processor, I loaded the main operating system on the main processor, and connected the second deck hardware via the multi-pin second deck connector. I already had the second deck firmware installed on the Teensy 3.5 VL53L0X array manager, and as soon as the power came up after loading the main processor firmware, I was seeing valid distances from all seven VL53L0X modules via the main processor’s ‘distances only’ debugging configuration. This validates I2C communications between two Teensy modules without the need for external pullup resistors by using the CORE_PIN37_CONFIG = PORT_PCR_PE | PORT_PCR_PS | PORT_PCR_MUX(2) – style commands to set each relevant I2C pin to enable the internal pullup feature

Hopefully this all spells the end of the I2C library compatibility goat-rope, and now I can get back to actual robot work! 😉

Stay tuned,

Frank

Wall-E3 Replacing Mega 2560 With Teensy 3.5 Part V

Posted 15 January 2022,

In my last post on this subject I described my effort to get IR Homing functional on my new Wall-E3 robot. This post is intended to document the process of getting the ‘second deck’ from Wall-E2 ported over to Wall-E3. The second deck from Wall-E2 houses the forward-looking PulsedLight (acquired by Garmin in 2015) LIDAR system, the two side-looking VL53L0X arrays, and a rear-looking VL53L0X, as shown in the photo below

Wall-E2’s ‘second deck’ module

The second deck connects to the main system via the 16-pin Amp connector visible in the bottom left-hand corner of the photo above.

Looking at the code, the only pin assignments associated with ‘second deck’ functionality are:

  • const int RED_LASER_DIODE_PIN = 5;//Laser pointer
  • const int LIDAR_MODE_PIN = 2; //LIDAR MODE pin (continuous mode)
  • const int VL53L0X_TEENSY_RESET_PIN = 4; //pulled low for 1 mSec in Setup()

To start the process, I ported the following sections from From FourWD_WAllE2_V12.ino::setup() to T35_WallE3::setup():

  • The code to reset the VL53L0X Teensy on the second deck
  • The code to initialize the above output pins.
  • The entire #pragma region VL53L0X_TEENSY section
  • The entire #ifdef DISTANCES_ONLY section
  • pragma region L/R/FRONT DISTANCE ARRAYS

That should take care of all new declarations and initializations associated with the second deck. And wonder of wonders, the T35_WallE3_V5 project compiled without errors! – time to quit for the night! 🙂

16 January 2022:

Well, of course when I connected up the second deck – nothing worked, so back to basic troubleshooting. First, I connected a USB cable directly to the T3.5 running the VL53L0X array, and determined that I can, in fact, see valid distance values from all seven sensors – yay! Now to determine why I can’t see them from the main processor.

Next, I loaded a basic I2C scanner program onto the main processor to see if the main processor could see the VL53L0X process on Wire1. The I2C scanner reported it could find the MPU6050 IMU module and the Teensy 3.2 IR homing beacon detection processor, but nothing else. After a few more seconds (and yet another face palm!) I realized that the I2C scanner program wasn’t finding the VL53L0X processor because it was only checking the Wire bus, not Wire1 or Wire2 – oops!

So, I modified the basic scanner so it would optionally check Wire1 & Wire2 in addition to Wire1, as shown below:

And here’s the output with all three I2C busses enabled.

So now that I know that the main processor can ‘see’ the VL53L0X Teensy on Wire1, I have to figure out why it’s not working properly. As usual, this turned out to be pretty simple once I knew what I was looking for. The entire solution was to change this line:

To this line:

Wall-E2 used a Mega 2560 processor with only one I2C bus, so everything had to be on that one bus. However, when I changed to the Teensy 3.5, I had more busses available, so I chose to move the VL53L0X array manager to Wire1, but forgot to change the initialization code to use Wire1 vs Wire – oops!

18 January 2022 Update:

Or,….. Maybe not. When I tried to compile the above changes I started running into ‘#include file hell’. I couldn’t figure out whether to use #include <i2c_t3.h> or #include <Wire.h> and every time I changed the include in one file, it seemed to conflict with an earlier change in another – argggghhhhh!

So, I took my troubles to the Teensy forum and asked what the difference was between the ‘i2c_t3.h’ and ‘Wire’ libraries, particularly with respect to multiple I2C bus support. The answer I got from ‘defragster’ (a very experienced Teensy forum contributor) was:

Wire.h is the base i2c supplied and supported by PJRC. It covers all WIRE#’s on various Teensy models: See {local install}\hardware\teensy\avr\libraries\Wire\WireKinetis.h

i2c_t3.h is an alternative library that can be used to instead of WIRE.h when it works or offers some added feature or alternate method.

I took his post to say “use Wire.h” unless you have some specific reason why the i2c_t3.h library offers a needed feature that Wire.h doesn’t offer.

So, now I have to go back through all the code that I have dicked with over the years to make work (like ‘I2C_Anything’, for instance) with multiple I2C buses, and see what needs to be done to, as much as possible, use ‘Wire.h’ in lieu of ‘i2c_t3.h’

The main file for this project has the following #include’s that may require work:

MPU6050_6Axis_MotionApps_V6_12.h: Right away I run into problems; the first thing this file does is #include “I2Cdev.h”, which has the following code:

Which I modified sometime in the distant past to use #include <i2c_t3.h> instead of #include <Wire.h>. At the time I think I was convinced that I had to use i2c_t3.h to get access to multiple I2C buses, but now I know that isn’t necessary. Fortunately this is a ‘local’ file, so changing this back shouldn’t (fingers crossed!) break other programs.

Aside: I did a search on MPU6050_6Axis_MotionApps_V6_12.h and I2Cdev.h in the Arduino folder, and got 48 different hits for MPU6050_6Axis_MotionApps_V6_12.h and 108 hits for ‘I2Cdev.h – ouch!! Not going to worry about this now, but I’m sure I’ll be dealing with this problem forever – if not longer 🙁

Aside2: The ‘I2Cdev.h’ file gets specialized to different hardware in I2CDevLib. For ‘Arduino’ hardware it is this file:

Where it can be seen that sometime in the distant past, I modified the original library file to select ‘I2CDEV_TEENSY_3X_WIRE’, which has the effect of #include <i2c_t3.h> instead of #include <Wire.h> – oops! So, any project that uses the library version of this file will always #include <i2c_t3.h> instead of #include <Wire.h> – double, triple, and quadruple oops! In addition, I discovered that the version of i2cdev.h I am using for this project is at least 6 years out of date – wonderful (at least it looks like the MPU6050_6Axis_MotionApps_V6_12.h is reasonably current (in fact, it is essentially identical to the library version).

To start with, I made sure T35_WallE3_V5 compiles for T3.5 ‘as is’, and then modified the includes to use the library version of MPU6050_6Axis_MotionApps_V6_12.h. This actually worked (yay!), although I discovered I had to use the “file.h” format rather than <file.h> – don’t know why.

23 January 2022 Update:

As usual, there were a few detours on the way to full ‘second-deck’ functionality. To start with, I discovered/realized that I was using the wrong I2C library (i2c_t3.h) for working with multiple I2C busses. The i2c_t3.h library does a lot of nice things, including multiple I2C bus support, but unfortunately isn’t compatible with a lot of the hardware driver libraries I use, most of which assume you are using the ‘Wire.h’ library. I had ‘sort-of’ solved this problem with many of my Teensy projects by hacking the needed hardware driver libraries to use i2c_t3.h instead of Wire.h, but this got old pretty quickly, and even older when I started trying to use multiple I2C bus enabled hardware driver libraries (like Kurt E’s wonderful multiple bus MPU6050 driver).

So, I wound up spending way too many hours tracking down the differences and similarities between i2c_t3.h and the Teensy version of Wire.h See this post for all the gory details. Along the way I learned how to simplify access to deeply buried library files using the Windows 10 flavor of symlinks, which was kinda cool, but I could probably have spent the time more wisely elsewhere. Also, I ran headlong into yet another multiple bus problem with one of my favorite libraries – Nick Gammon’s wonderful ‘I2C_Anything’ library that does just two things – it reads/rights ‘anything’ – ints, floats, unsigned long ints, doubles, whatever – across an I2C connection – no more worrying about how to do that, or being forced to transmit ASCII across the bus and construct/deconstruct as necessary on both ends. Unfortunately, this library is unabashedly single-bus – it expects to be used in a Wire.h environment and that is that. The good news is, by the time I was done the question of ‘why use i2c_t3.h as opposed to Wire.h?’, I was able to intelligently (I hope) modify I2C_Anything.h to accommodate multiple I2C busses (though it still assuming a ‘Wire.h’ environment). At the end, I made a pull request to the I2C_Anything github repo, so maybe others can use this as well.

After all this was done, I still hadn’t even started on getting the second-deck VL53L0X sensors talking to the main Teensy 3.5 processor, but at least now I (kinda) knew what I was doing.

Anyhoo, once I got back to trying to get connected to the VL53L0X array, I was able to reasonably quickly revise both the main Teensy 3.5 processor program and the VL53L0X array management program to use Wire.h vs i2c_t3.h, and to instantiate/initialize the multiple busses required for both processors (the main processor talks to the VL53L0X array manager via Wire1 on its end to Wire0 on the array end, and the array manager talks to the seven VL53L0X modules via its Wire1 & Wire2 busses). And with my newly modified I2C_Anything library, I was ready to get them talking to each other.

Here’s the working Array manager code (pardon the extra comment lines):

And here’s the working main processor code. Note that the code as it is presented here has the ‘DISTANCES_ONLY’ define enabled, and all non-existent hardware modules ‘#defined’ out so I could concentrate on just the VL53L0X array connection.

And here is the ‘multiple I2C bus’ version of I2C_Anything.h, just in case you come across this post before Nick updates the Github repo (assuming he likes what I have done):

Here’s the the test setup:

Main Teensy 3.5 processor on plugboard, connected via I2C to the Teensy 3.5 VL53L0X array manager

Using Wire1 & Wire2 with the I2CDevLib & MPU6050 Libraries

While working on porting Wall-E2’s ‘second deck’ hardware (VL53L0X array and LIDAR) to Wall_E3, I started running into problems associated with using Wire1 on the main Teensy 3.5 master processor to communicate with the Teensy 3.5 slave processor that manages the seven VL53L0X time-of-flight distance sensors. As I worked to troubleshoot the issue, it soon became evident that “I wasn’t in Kansas anymore”, and in fact had once again gone down a rabbit hole into Wonderland, with nary a bread-crumb in sight. Basically, I have been trying to use both the ‘i2c_t3.h’ and ‘Wire.h’ Teensy to utilize multiple I2C buses (Wire1, Wire2, etc) over the last few years, without really understanding what I was doing. In the process I have created a spaghetti mess of conflicting I2C library file locations and configurations.

So, this post is my account of what went wrong, and the steps taken to get things working properly again.

The problem – utilizing multiple I2C buses:

Teensy 3.x processors provide for multiple I2C buses, a feature I used originally to manage the 7-element VL53L0X array located on the ‘second deck’ of Wall_E2, my autonomous wall following robot. With the addition of a Teensy 3.5 for the main robot processor, the multiple I2C bus problem is now relevant to the main processor as well. The main processor now uses Wire to talk to the second-deck VL53L0X array manager (Teensy 3.5), and Wire1 for the IR homing beacon detector/demodulator and the MPU6050 IMU. Consequently, the main processor must utilize (multi-wire) capable library functions.

The 7-element VL53L0X array manager (Teensy 3.5)

This worked great, even though I had tried my best to screw it up. The main project file has already been changed to use <Wire.h> and although it uses a local copy of I2C_Anything.h, that copy uses <Wire.h> as well. Also, VL53L0X.h (local folder copy) also uses <Wire.h>. So, I made the following changes:

  • Removed VL53L0X.h/cpp and I2C_Anything.h from project references and deleted the local file copies
  • Changed #include “file.h” to <file.h> for both (not sure this is necessarsy
  • Deleted the vl53l0x-arduino folder from the \Libraries folder so there would be only one version of VL53L0X.h/cpp available.
  • Re-scanned for libraries, did a File->SaveAll, and recompiled OK – yay!

So now at least the Teensy_7VL53L0X_Slave_V3 project has been cleaned up

Main Wall-E3 Processor (Teensy 3.5)

This is where all the trouble with multiple I2C busses started. The main processor has to talk to the VL53L0X array manager via I2C on Wire1, which means that not only does the main processor code need to utilize multi-bus functionality, but I2C_Anything (which internally uses Wire for bit-wise comms) does as well. In addition, interfacing to the MPU6050 requires the use of Jeff Rowberg’s I2CDevLib stuff, specifically MPU6050_6Axis_MotionApps_V6_12.h, I2CDev.h, I2C_Anything.h, and a Wire1 capable version of I2C_Anything.h. To make all this work, I made the following changes:

The #include for MPU6050_6Axis_MotionApps_V6_12.h, I2CDev.h is aimed at \Libraries\MPU6050\, which is a (old) copy of C:\Users\paynt\Documents\Arduino\Libraries\i2cdevlib\Arduino\MPU6050\. A suggestion in Jeff Rowberg’s ReadME file regarding the use of symlinks instead of actual copies led me to this ‘how-to’ page on creating symlinks in Windows. So I deleted the C:\Users\paynt\Documents\Arduino\Libraries\MPU6050\ folder and instead created a ‘hard’ symlink from there to C:\Users\paynt\Documents\Arduino\Libraries\i2cdevlib\Arduino\MPU6050\. Then I similarly deleted the C:\Users\paynt\Documents\Arduino\Libraries\I2Cdev\ folder and created a ‘hard’ symlink from there to C:\Users\paynt\Documents\Arduino\Libraries\i2cdevlib\Arduino\I2Cdev\.

Here are the cmdline commands for both operations:

and here is the result of dir /A in the C:\Users\paynt\Documents\Arduino\Libraries\ folder

showing that the hard links were actually created properly.

So now when I right-click on include “MPU6050_6Axis_MotionApps_V6_12.h”, the file opens properly, and the location is shown as in the C:\Users\paynt\Documents\Arduino\Libraries\MPU6050 folder even though the file actually resides in C:\Users\paynt\Documents\Arduino\Libraries\i2cdevlib\Arduino\MPU6050\. Similarly, for include “I2Cdev.h” the file opens properly and the location is shown as C:\Users\paynt\Documents\Arduino\Libraries\I2Cdev\ even though it is actually in the C:\Users\paynt\Documents\Arduino\Libraries\i2cdevlib\Arduino\I2Cdev\ folder

This all worked, except now I’m getting errors that say that ‘I2C_PINS_18_19’ (and all the other Teensy I2C-specific enums) can’t be found – argggggghhhhh! They don’t seem to be defined anywhere in the C:\Program Files (x86)\Arduino\hardware\teensy\avr\ folder tree either – I’m at a loss

Well, maybe not. I’m beginning to think that the enums are <i2c_t3.h>-specific, and a more basic style of initialization is used with <Wire.h>. I sent off a plea to the Teensy forum – we’ll see.

In the meantime, I tried a couple of simple experiments that determined pretty conclusively that the <Wire.h> style does indeed work, but in a different (more constrained?) way than with <i2c_t3.h>. I created a ‘Wire_Slave_Sender’ VS2022 project by copying the ‘slave_sender.ino’ example and loaded it onto a T3.2. Then I created a ‘Wire_Master_Reader’ VS2022 project by copying the ‘master_reader.ino’ example, and loaded it onto a T3.5. With this setup I was able to demonstrate that ‘Wire.begin()’ facilitated I2C comms on pins 18 & 19 (the default pinouts for Wire0) on the T3.5 (pins 18 & 19 on the slave didn’t change), and ‘Wire1.begin()’ facilitated the same thing, but this time using pins 37 & 38 (the default pinouts for Wire1). Here are the programs:

and here’s a sampling of the output:

OK, now that I understand the <Wire.h> vs <i2c_t3.h> issues, I still have at least one more hurdle to clear. It appears that ” MPU6050_6Axis_MotionApps_V6_12.h ” is an older version of the DMP-enabled code for the MPU6050, and “MPU6050_6Axis_MotionApps612.h” (no ‘V’, no underscores in version number) is the latest and greatest. However, using this version also requires the correct version of i2cdev.h (I think). In any case, I was able to change the #includes on my T35_WallE3_V5 project to “MPU6050_6Axis_MotionApps612.h” and “I2Cdev.h” and get the program to compile for a T3.5 target. Whether or not it will actually behave as required is TBD.

To pursue this issue, I set up a simple T3.5 – MPU6050 plugboard configuration, and loaded an old MPU6050 test project – “Teensy_MPU6050_DMP6_V3”. This project uses the older “MPU6050_6Axis_MotionApps_V6_12.h” code (located in the project folder) along with I2Cdev.h/cpp, helper_3dmath.h, and MPU6050.h/cpp all in the project folder. It compiled right out of the box, and I was able to demonstrate successful interfacing with the MPU6050.

Next, I changed #include “MPU6050_6Axis_MotionApps_V6_12.h” to #include “MPU6050_6Axis_MotionApps612.h” whereupon it blew a whole bunch of compile errors. I was able to confirm that, due to the ‘hard’ symlink magic, the compiler thinks the “MPU6050_6Axis_MotionApps612.h” file is at …\Arduino\Libraries\MPU6050 rather than way down in the i2cdevlib tree – yay!

At this point, rather than continuing to modify the Teensy_MPU6050_DMP6_V3 project, I decided to create a Teensy_MPU6050_DMP6_V4 project and make all the changes there, so that when I screw up and get lost I can go back and start all over (ask me how I know to do this….). So I changed the include back to #include “MPU6050_6Axis_MotionApps_V6_12.h” and verified it still compiled OK, then I created a new project called Teensy_MPU6050_DMP6_V4 and copy/pasted the entire .ino file into it.

When I tried to compile the new project, it blew a bunch of errors about MPU6050_Base:: functions not being found. This sounds like the i2cdev.h/cpp that is being found is the later one, so I went ahead and changed #include “MPU6050_6Axis_MotionApps_V6_12.h” to #include “MPU6050_6Axis_MotionApps612.h” without doing anything else. This time it didn’t blow any errors about MPU6050_Base:: functions but did blow some siimilar to ‘I2C_PULLUP_EXT was not declared’. I think this is due to using <Wire.h> in the #include chain rather than <i2c_t3.h>, so I changed

With just that one change, the project now compiles for a Teensy 3.5 target – woohoo! In addition, I didn’t have to add any references to the project, which I have had to do in almost every other case – double woohoo!

Then I uploaded this project to the T3.5, and it actually worked – the MPU6050 is generating valid azimuth values – triple woohoo!

So now I have a working program using the latest/greatest DMP-enabled driver for the MPU6050, and a much simpler #include file/reference setup. Compare this:

to this:

Just for grins, I modified the _V4 project to see if I can move the MPU6050 from Wire (pins 19,18) to Wire1 (pins 37,38). This requires changing MPU6050 mpu to MPU6050((uint8_t) 0x68, &Wire1) and Wire.begin to Wire1.begin(). This worked like a champ, so at this point I think I have figured out all I need to know on this subject.

20 January 2022 – One last piece of the puzzle:

I use Nick Gammon’s wonderful I2C_Anything library (just two template functions, but…) a lot, but it doesn’t support multiple I2C buses. So, I decided to see if I could modify his template functions to accept a default argument that if present, specifies which I2C bus to use. Here’s the modified file, temporarily renamed to ‘I2C_AnythingMultiWire.h’

I used my previously constructed ‘Wire_Slave_Sender.ino’ and ‘Wire_Master_Reader.ino’ projects to test this, and it seems to work just as advertised. In ‘Wire_Slave_Sender.ino’ I use:

Which automagically uses ‘Wire’ because the 2nd argument is missing from the call, and in ‘Wire_Master_Reader.ino’ I use:

Which causes Wire1 to be used. Here are both demo programs in their entirety, along with the full ‘I2C_AnythingMultiWire.h’ file:

I made a ‘pull request’ to the I2C_Anything github repo so that everyone who uses it can benefit, but in the meantime feel free to use the I2C_AnythingMultiWire.h file.

25 January Update: Just one more ‘one last piece of the puzzle’

When I was using the ‘i2c_t3.h’ library, I noticed that I didn’t have to use external pullup resistors as long as I used the ‘I2C_PULLUP_INT’ treatment in the Wire.begin() call. This was somewhat contrary to the general run of the posts on the Teensy forum, so it was a bit disconcerting. However, I ran a series of experiments that clearly showed that I2C between two Teensy 3.5 processors didn’t need external pullups, and O’scope waveform analysis showed no difference between using external 2.2KΩ pullups and no external pullups with the ‘I2C_PULLUP_INT’ option.

However, when I switched to the ‘Wire.h’ library, all this changed. I had to use external 2.2KΩ pullups – and this was verified via O’scope analysis. This usually isn’t a big deal, but it turns out that in my case it’s going to be a real PITA to add the pullups. my hardware configuration uses all point-to-point jumpers and no PCB, so there just isn’t any easy way to do this.

You would think that, since there is obviously a way to enable the pullups on the I2C lines (obviously, because that is exactly what the i2c_t3.h library does), there must be a way to do the same thing with the Wire.h library. You would think that, but I’ll be darned if I can figure it out. I have posted this issue to the Teensy forum, but so far no luck finding a solution.

OK, I may have found a clue. Buried deep in i2c_t3.cpp is the following macro:

it is this macro that actually casts the magic spell over the currently defined SCL & SDA pins to enable (or disable) internal pullups.

29 January 2022 Update:

After a lot of forum and Google searching, I decided to try some small experiments regarding how to set an ‘open-drain’ or ‘input-pullup’ configuration on a Teensy 3.5 GPIO pin. Here’s the code:

And here’s some of the output:

In particular, I was able to indirectly measure the pin pullup resistor value by tying the pin to GND through a 10KΩ resistor. The voltage at the junction was about 0.78V, so the voltage divider equation Vr2 = V *R2/(R1+R2) when solved for Vr2 = 0.78V and R2 = 10K gives R1 ~33K, which agrees well with the known pullup resistor value for the Teensy 3.5.

From this it seems that I should be able to initialize the appropriate pins for I2C with ‘wirex.begin()’ and then follow that with the code to set the pins for input_pullup. We’ll see.

I uploaded a very basic I2C ‘master’ sketch to my T3.5, as shown:

And verified with an O’scope that the SDA & SCL pins were active, and that external pullup resistors were required.

When I added ‘pinMode(SCL1, INPUT_PULLUP);’ just after Wire1.begin(), the I2C activity was disabled – no signal at all on either line. I tried some other combinations of pinMode() and digitalWrite(), but nothing changed – clearly the use of pinMode() and/or digitalWrite() overwrites at least some of the required configuration for I2C output.

So, next I plan to try some direct port control and see if that will do the trick.

From Kurt E’s spreadsheet, SCL1 & SDA1 (pins 37/38) are PortC pin 10 & 11 respectively. So,

PORT_PCR_MUX(n): selects the ALTernate function for the pin in question. PORT_PCR_MUX(1) just selects the pin as a GPIO. For instance, to select T3.5 pin 37 as I2C1 SCL, we would use PORT_PCR_MUX(2). Thus, the code line might look like:

where ‘PORT_PCR_ODE’ is the defined constant that selects the ‘Open-Drain-Enable’ bit in the Port Control Register for Port C, bit 10, which is connected to T3.5 pin 37. ‘PORT_PCR_ODE’ is defined in Kinetis.h as:

The ‘0x00000020’ part, when translated to binary is: 0000 0000 0000 0000 0000 0000 0001 0000 << selects the 5th bit in the 32-bit Port Control Register.

PORT_PCR_MUX(n) is defined in Kinetis.h as:

so PORT_PCR_MUX(2) –> (uint32_t)(((2 & 7) << 8)) –> (uint32_t)(((0010 & 0111) << 8)) –> (uint32_t)(((0010) << 8)) –> (uint32_t)(0010 << 8) –> 0000 0000 0000 0000 0000 0000 0000 0010 << 8 –> 0000 0000 0000 0000 0000 0010 0000 0000 –> 0x200

Based on the above information, I thought that I might be able to accomplish my goal by using the following construct in setup():

Unfortunately, this did not work; I2C activity on pins 37/38 ‘flatlined’ and that was that. However, after letting my mind work on the problem while the rest of me slept, I had a new thought when I woke up this morning. Maybe the problem with the above construct is the ‘PORT_PCR_MUX(1)’ fragment. Maybe selecting the default GPIO port overwrites the prior I2C function selection?

So, I decided to try again this morning, using ‘PORT_PCR_MUX(2)’ (I2C function selected) instead. The code looks like this:

And, “son of a gun” – it worked! pins 37/38 activity continued, and physical pullups aren’t required – YES!

Here’s the full program:

I2C activity visible on scope with 2.2K pullup resistors (foreground under green wire jumper) disconnected

I went ahead and connected my plug-board ‘master’ with the Teensy 3.5 on my ‘second-deck’ plate to test end-end I2C comms without external 2.2KΩ pullups. I loaded the Wire library master_reader example on the plugboard Teensy 3.5, and the slave_sender example on the ‘second-deck’ Teensy 3.5. Then I connected Wire1 on the master to Wire (19/18) on the slave. I modified both the master and slave examples with the

On the affected lines (37/38 on the master, 19/18 on the slave). When I ran the examples, I got the following output:

I ran the above experiment with both ends modified for internal pullups, just one end modified, and one or both modified with external 2.2K pullups. the link worked perfectly for all these cases. Her’s a photo of the setup:

master/slave I2C connection with 6″ jumpers. Note 2.2K resistors NOT used

Frank

Wall-E3 Replacing Mega 2560 With Teensy 3.5 Part IV

Posted 14 January 2022

After finding and fixing the connector problem that prevented the IR Homing Teensy 3.2 module from communicating via I2C with the Teensy 3.5 main controller, I was ready to move on to some real IR homing tests.

For this effort, I decided to change the IR homing algorithm over from the PID library to my home-grown PID routine, discussed here. In the process, I realized that my PIDCalcs() function was overly complex; in particular the ‘sampletime’ calling argument is never used inside the function, as a regular interval is assumed. In all my PID implementations, this is a valid assumption, as I use an ‘elapsedMillisec’ object to enforce regular calls to PIDCalcs(). After eliminating this parameter, the PIDCalcs() function now looks like this:

And the calling routine for this round of testing (not using distance information) looks like this:

The ‘steering value’ output from the IR Homing beacon demodulator ranges from -1 to +1, and motor speeds range from 0 to 255 with a median value of 127. So, I started testing WALL-E3 with a PID of 100,0,0 – the idea being that a max output of +/- 1 from the demodulator would produce an output of +/- 100, which, when used to modulate motor speeds from the median value of 127 would produce motor speeds of 227/27 in one direction and 27/227 in the other. In other words, a Kp value of 100 should produce significant motor speed modulation without hitting the motor speed limits in either direction.

I set up a small test range in my office, and made a couple of runs. Here’s a short video showing on of the runs, and an Excel plot showing steering value and motor speeds vs time.

Homing beacon on the carpet patch. Note smooth turning behavior at start due to new wheel geometry

The next step is to increase the Kp value to the point where oscillations occur, as the starting point for the Zeigler-Nichols PID tuning method. For PID = (200,0,0) I got this behavior.

And for PID = (250,0,0),

And Kp = 300

Looks like Kp = 300 might be a good place to start. So, according to the Zeigler-Nichols PID tuning method, I should use Kp = 0.5Kc = 150, Ki = 0.45Kc = 135, Kd = 0.6Kc = 180. Using these values, I get:

Needless to say, this is NOT what I had in mind! Clearly I erred somewhere along the way. Abandoning the Z-N tuning algorithm, I turned to the ‘manual’ tuning procedure I have used in the past (also discussed in the Z-N article): With Ki, Kd set to 0, increase Kp until the system oscillates, but before it becomes unstable. Then increase Ki until the oscillations stop. Then increase D until “the system achieves an acceptable quick loop to its set-point” (whatever that means).

Starting with a Kd of 200, I increased Ki in increments of 50 to get:

This looks pretty good, even as a first try. However, when compared to the original PID = (200,0,0) plot (shown below), it looks like the Ki value of 50 didn’t actually do very much one way or the other.

Increasing Ki to 100 produces the following plot.

Not much, if any, improvement. Keeping Kp = 200 and Ki = 100, try increasing Kd in increments of 25. The first increment (Kd = 25) produced a significant change, as shown below:

Which leads me to believe the 25 increment is way too much. Trying 10, we get:

PID = 200,100,10 is much nicer than the Kd = 25 version, but there was an annoying little ‘jag’ off-course right at the end – don’t know why. Just for grins I tried PID = (150,100,10):

And this looks pretty good, actually. I think I may run with these values for a while and see how it works. Here’s the video of the above run:

One final note before leaving this subject. I added some code to IRHomeToChgStnNoPings() to measure the time required to run through each motor speed determination loop. The code simply turns ON a digital output at the start, and turns it OFF again at the end. The loop runs every 200 mSec (the current value of MSEC_PER_IR_HOMING_ADJ), but the time required to do everything is only about 1.5 mSec. In other words, the Teensy sits idle for about 99% of the time while homing to the charging station. Or to put it another way, I could run the loop 10 times faster (20mSec vs 200mSec) and the Teensy wouldn’t even break a sweat!

Stay tuned,

Frank

Wall-E3 Replacing Mega 2560 With Teensy 3.5 Part III

Posted 04 January 2022

At the conclusion of my last post on this subject, I had refurbished the charging station IR transmit subsytem with a new perfboard setup to replace the ugly terminal strip implementation. The next step in ‘bringing up Wall-E3’ is to verify IR Homing functionality. The IR transmitter is modulated by a very stable 520.8Hz square wave produced by the Teensy 3.2 waveform generator, and this signal is demodulated by a Teensy 3.2 on the robot (see this post for the details). This technique has been working for years with the Wall-E2 robot, and since I simply moved the entire IR homing subsystem from Wall-E2 to Wall-E3, I have some hopes that it will work correctly (fingers crossed!). Here’s the test setup:

The first step was just to verify that both phototransistors were receiving the IR signal, as shown in the scope photo below:

At this point I also verified that rotating the robot caused the left and right detector signals to vary appropriately, so that all works. The next step is to verify that the demodulator output still behaves properly. To do this, I ported the appropriate code from my Wall-E2 project into T35_WallE3_V4.

06 January 2022 Update:

When attempting to verify IR Demodulator output from the onboard Teensy 3.2, all I got was zeros across the board. After investigating some more, I came to the conclusion that the main Teensy 3.5 was for some reason unable to even talk to the IR Demod Teensy via I2C. After the usual amount of fumbling around and searching the Teensy forum, I began to think that the problem was caused by the lack of proper pullup resistors on the I2C bus connecting the two. In the case of Wall-E2, I had pullup resistors mounted on the Wixel shield board, but this did not get transferred over to Wall-E3. I was also misdirected a bit by the IC2_PULLUP_INT defined value for Wire.begin calls – this, at least in my mind, implied that the internal pullups available in the ARM processor would work with two Teensy processors – apparently not. So, in order to run this issue down, I created a small test bed consisting of a T3.5 ‘master’ and a T3.2 ‘slave on a plug-in board as shown below:

I2C Master/Slave Test Setup. Note 2.2K pullup resistors above ‘master’ T3.5

Then I loaded the ‘Basic_Master’ and ‘Basic_Slave’ i2c_t3 library sketches and verified that the master and slave were communicating properly. Then I removed the pullups and changed the ‘Wire.begin()’ statements to use I2C_PULLUP_INT to see if they would still communicate – and, contrary to some posts on the Teensy Forum they did! I grabbed some scope photos of the SDA & SCL lines with and without external 2.2K pullups, as follows:

With external 2.2K pullups
Without external pullups

As can be seen from the above photos, there is no perceivable difference between the waveforms for the ‘with’ and ‘without’ external pullups cases. So, for at least this very simple Teensy-to-Teensy case, pullups don’t appear to be required.

Next, I changed the configuration to replace the ‘Slave’ T3.2 with the T3.2 mounted on the IR detector housing, as shown below:

Slave T3.2 replaced by T3.2 on IR detector housing. MPU6050 not connected to I2C

Then I loaded T35_WallE3_V4 onto the ‘master’ T3.5 and re-ran the experiment. This time the T3.5 detected the T3.2 and produced valid output, as shown below:

plot of IR detector module output while moving IR xmt back and forth past detector hood

As can be seen from the above, the computed ‘IRHomingValTotalAvg’ value varies with the alignment of the IR transmitter module. Moreover, since this value is computed using the N-path bandpass filter algorithm implemented earlier, it also verifies that the code is running properly on the IR detection/homing module, as well as the corresponding code in the main controller Teensy 3.5. Yay!

While the above validates the T3.5 main controller and the I2C bus between it and the T3.2 IR Demodulator module, it doesn’t explain why it didn’t work on the actual robot. The only difference between the configuration tested above and the configuration on Wall-E4 is the presence of the MPU6050 module on the same I2C bus. So, I added this piece back into the circuit as shown below:

MPU6050 added to I2C bus

With this configuration, the main controller failed to connect to the IR demod module, just as it did on Wall-E3. So, at least the failure is consistent with previous work. To further investigate, I looked at the I2C bus waveforms with and without the MPU6050 module on the bus, as shown below:

With the MPU6050 added to the I2C bus. Note the horizontal time scale (5 uSec/div)
Without the MPU6050 added to the I2C bus. Note the horizontal time scale (5 uSec/div)

After some more head-scratching, I finally narrowed the issue down to the double-layer connector on the MPU6050 module, where I found not just one, but two bad solder connections – oops! After repairing the connections, I reran the test with the MPU6050 on the I2C bus, and this time I got the proper behavior; case closed, and it only cost me most of a day! 🙁

07 January 2022 Update:

I re-installed the IR Homing/MPU6050 subsystem on Wall-E3 and verified proper operation. Next step will be to try some homing experiments.

Stay tuned,

Frank

Wall-E3 Replacing Mega 2560 With Teensy 3.5 Part II

Posted 12 December 2021,

Just after getting to the point noted at the end of Part I of this saga, I discovered that I had managed to kill the Teensy 3.5 I was using at the time. This post (and maybe subsequent ones) describes my efforts to determine what happened, and how to keep it from happening again.

Well, there was a small detour on the way to the forum…. After installing a Teensy 3.5 on the robot chassis and wiring everything up, I did something that killed the Teensy. I was connecting the charger to the robot, and noticed that the Teensy rebooted (I was running a sketch that blinked the on-board LED 5 times/sec). That should never happen, so I disconnected and reconnected the charging plug a few more times. Sometimes (but not all the time) the Teensy would reboot – and then it stopped responding entirely; the LED was still blinking, but it was much dimmer, and the Teensy would no longer respond to programming inputs. I measured the 3.3V regulated output, and it was now down to about 1.9V – ouch!

I spent a LOT of time looking around for the cause of the problem without finding anything really credible. I finally decided that having a Teensy input connected directly to the pin on the charging jack that gets disconnected from GND when the charging plug is inserted must be problematic somehow. So I added an RC filter to the line so that when the disconnect occurred, the signal presented to the Teensy pin would be filtered through the RC filter, as shown below:

This didn’t work either – a brand-new Teensy rebooted after just a few connect/disconnect cycles – rats!

Next I abandoned the idea of using the normally closed switch portion of the charging jack entirely, and decided instead to look at a stepped down (4:1 divider) version of the +12V input line with one of the Teensy’s many analog inputs, as shown below:

This too failed to protect the Teensy, and after a few connect/disconnect cycles it too was completely unresponsive; now I’m a multiple Teensy 3.5 killer – yikes!

After killing not one, but two Teensy’s, it was time to realize I simply did not know what was happening, so as usual I went back to basics. First I connected yet another expensive Teensy 3.5 to my PC via USB, but this time I simply placed it near, but not on, my robot chassis. Then I performed 10-20 charging plug connect/disconnect cycles, and confirmed that the Teensy still lived. While this step seems pretty stupid, it at least eliminated some form of magic that killed all Teensy’s within some radius of the robot chassis. Next, I placed this Teensy on the robot chassis, but not connected electrically to anything – just the same USB connection back to my PC. Another 10-20 connect/disconnect cycles with no complaints, and now I’m convinced whatever is killing Teensy’s is a conducted signal, not radiated.

Next, I connected the Teensy’s ground pin to the ground side of the 5V LDO output, which is also the ground side of the charging connector. Another 10/20 cycles with nothing bad happened, and now I’m convinced that whatever is killing Teensy’s is a conducted signal, but not something inherently on the ground line only.

At this point I realized that since I now had Teensy ground, USB ground, and robot power ground all connected together, I could now look at the +12V side of the charger power supply with my Hanmatek DOS1102 100MHz digital O’scope, and compare/contrast it to my lab power supply output when using the latter as a substitute charger input. After a bunch more connect/disconnect cycles with both my charger power supply and my lab power supply acting as a charger input, I noticed that the charger power supply exhibited very pronounced ‘ringing’ on the +12V line when repeatedly connected and disconnected from the charging jack, as shown below:

Charging PS ‘ringing’ during connect/disconnect cycles. Note scale is 5V/div

In the above photo, the vertical scale is 5V/div, meaning that some of the excursions are upwards of 15V (plus AND minus!) – plenty enough to kill a Teensy.

So now I knew what was happening, but not why. After some more playing around I realized that the really big ringing pulses were caused by just tapping the front of the charging plug against the front of the circular jack opening — and then I realized that the front surface of the charging plug isn’t insulated – it’s actually the same surface as the +12V inner cylinder!

Plug on left from Wall-E2 charging cable. Plug on right doesn’t cause problems

So, I began to understand that the why was because I was occasionally shorting the +12V output of the power supply to ground, effectively shorting out the power supply. The power supply doesn’t die because it has short-circuit protection, but the process of shorting and then opening the circuit causes (I think) the ringing.

So, why didn’t I see this before? I’ve been using this same power supply and jack for (literally) years with my older 4WD Wall-E2 robot, with no problems. The answer (I think) is shown in the following photo:

Charger connection port on Wall-E2 robot

As can be seen in the above photo, the charger jack on Wall-E2 features a lead-in ‘registration’ cone so that slightly off-center alignments can be accommodated. An unintended (and unknown till now) side-effect of this arrangement was to prevent the front face of the old charging plug from touching the grounded inner outer circumference of the jack.

Assuming all this holds water as I move forward, I can go back to looking at the normally-closed grounding switch on the charging jack for physical charger plug insertion detection, or leave it the way it is now with the 4:1 voltage divider on the +12 line to the charger module.

13 December 2021 Update:

Well, maybe not. I removed the probe lead-in collar from my old robot and put it on the new one, then re-ran the experiment where I look at the output of the 4:1 voltage divider wrt robot system ground, and I still see very large +/- excursions when I connect and disconnect the charging power supply using the original plug. So, There may be something else going on, or it may be that these perturbations have existed all the time, but the Mega2560 wasn’t negatively affected. When I do the same experiment with my lab power supply, I see almost no excursions or ‘ringing’. The next experiment will be to swap the connectors (conductor-faced one on my lab supply, insulator-faced on on my charging supply) and see what happens.

Well, not so clear; when I swapped plugs I could still see significant, but much reduced, excursions when connecting/disconnecting the charging supply. However, the lab supply looked pretty clean, with either connector. Here are the screen grabs from my scope for all four conditions

Charging PS, Old (conductor-faced) Plug
Charging PS, New (insulator-faced) Plug
Lab PS, New (insulator-faced) Plug
Lab PS, Old (conductor-faced) Plug

From the above plots, it looks like the charging PS with the new (insulator-faced) plug should be OK. We’ll see.

16 December 2021 Update:

After thinking about the situation for a while, I realized there was simply no way to avoid potentially dangerous (to a Teensy, at least) +/- voltage excursions on anything connected to the +12V charging input line, so “hoping for the best” is probably not the best plan moving forward. I started thinking about optical coupler ideas, and after sleeping on this for a couple of nights, I realized that I already had half of an optical coupler available; the TP5100 charger typically comes with a two-color (red/green) LED for visual display of charge/end-of-charge states. This LED typically isn’t installed, but the PCB pads are there and ready to go. So, If I installed the LED and placed a photodetector of some sort nearby, I could implement a non-conductive connection between the charger +12V input and my poor defenseless Teensy 3.5 – woohoo!

I poked around in my parts bin for a while, and came up with some photo transistors and a couple of small, simple GL5537 photoresistor parts. According to the datasheet, it exhibits a dark resistance in the meg-ohms and a fully-illuminated resistance in the single-digit ohms – perfect! So I whipped up a small breadboard circuit to test this out, as shown in the following photo:

GL5537 Photoresistor nose-to-nose wth the green/red dual color LED shipped with each TP5100 charger

I used a 20K resistor pullup to 3.3V to simulate a Teensy GPIO pin set for a digital input with a pullup resistor, and this worked great; with the LED OFF, the scope showed very nearly 3.3V, and nearly zero with the LED ON (I did have to cover the photoresistor/LED combination with an opaque shade to keep my lab overhead lights from interfering though). Now the +12V supply and anything electrically connected to it can perturbate all it wants to – it won’t be able to kill any more Teensys because the only electrical connection now is through the battery itself to the +5V LDO regulator and then through the 5/3.3V regulator on the Teensy – yay!!

The next step was to integrate the photoresistor and LED onto the TP5100 charging module. I was able to replace the voltage divider resistors on the perfboard holding the TP5100 charging module with the photoresistor and I installed the 2-color LED on the PCB pads provided on the TP5100 module, with the leads arranged so that the LED boresight pointed toward the photoresistor, as shown below:

With this setup, I can connect/disconnect either style charging plug to my heart’s content without damaging anything. The only question remaining is whether or not I should install a light-shield around the LED/photoresistor pair. I could do that easily with a small section of heat-shrink tubing, but I’m not sure it’s necessary; in addition, if I do that I’d lose the ability to visually confirm that the charger is actually charging. Something to think about, anyways.

15 January 2022 Update:

While the above solution worked perfectly for decoupling any charging plug connection transients from the Teensy(s), it meant that the charge status LED on the TP5100 module wasn’t very (i.e. not at all!) visible from the outside of the chassis, so I had no way to visually confirm that plugging in the charging cable was actually doing something. I thought about this for a while, and had the idea that this might be the perfect place for a ‘light pipe’. After some Google searching, I found this part.

This light pipe worked great, and there was just enough length for me to attach the input end to the TP5100 charge status LED and the output end to the front panel of the robot, as shown below:

black 4″ light pipe attached with hot glue to charge status LED
Output end of light pipe installed on Wall-E3 front panel.
Red ‘charging’ status light visible from across the room
TP5100 charge status LED changes to green to indicate full charge

Stay tuned,

Frank

Teensy, MPU6050 and Rowberg’s I2CDev Library, Take Two

Posted 17 November 2021,

This post describes my efforts, once again, to figure out how to get a Teensy running with Jeff Rowberg’s I2CDev library for the purpose of interfacing to a MPU6050 6-axis IMU with on-board DMP.

After some initial teething problems, I have been using the MPU6050 for years on my various autonomous wall-following robots using Arduino Mega 2560 controllers with great success. However, when I tried switching over from Arduino UNO/Mega controllers to the Teensy 3.x controller, Jeff Rowberg’s library wouldn’t compile at all. At the time I got the library to compile by replacing #include Wire.h with #include i2c_t3.h in i2cdev.h/cpp. See this post from almost two years (December 2019) ago for the gory details.

Unfortunately, two years later when I tried to change out the Mega 2560 for a Teensy 3.5 in Wall-E3 my latest autonomous wall-following robot, I ran into compile problems again – rats! Looking at the code, I saw that the maintainers had accepted my pull request to add a ‘I2CDEV_TEENSY_3X_WIRE’ identifier to I2Cdev.h to switch from <Wire.h> to <i2c_t3.h> as I had done previously, but I still couldn’t get the library to compile with a Teensy 3.x target.

So, back to basics. Fortunately, I had kept all the old versions of the required i2cdevlib files, so my two-year old project (Teensy_MPU6050_DMP6_V2.ino) still compiled and ran properly – whew. So now to figure out why it doesn’t compile with the newest version of the libraries.

So I created a new VS2019 Arduino project called Tensy_MPU6050_DMP6_V3, copied my code and other files from two years ago, and trimmed the program down to just the code required to print out MPU6050 headings every 200 mSec.

And here is a sample of the output:

This program, using the i2cdevlib files from two years ago, compiles and runs fine. The things that have changed in i2cdevlib since then. Comparing i2cdev.h as used in this program with the current i2dev.h, I see

Changelog diffs
I2CDEV_TEENSY_3X_WIRE for convenience in switching from Wire.h to i2c_t3.h
I2CDEV_TEENSY_3X_WIRE used to switch to i2c_t3.h (as opposed to manually as before)
added ‘void *wireObj=0’ parameter to end of every I2Cdev method signature

And the difference in the i2Cdev.cpp files:

‘void wireObj’ added to all i2Cdev class method implementations
All instances of ‘wire.’ are replaced by ‘useWire->’ where *useWire is defined as a TwoWire object

In the i2Cdev class methods that actually talk to the MPU6050 over the i2c bus (readBytes, readWords, writeBytes, writeWords) the new version uses a pointer to a ‘TwoWire’ object called ‘useWire’ instead of Wire. The ‘useWire’ object is defined at the top of each of these methods with the lines

but the declaration (invocation?) of the ‘TwoWire’ class is guarded by an #ifdef as shown

Declaration of ‘Wire’ as a ‘TwoWire’ object

Which means it isn’t included if ‘I2CDEV_IMPLEMENTATION == I2CDEV_TEENSY_3X_WIRE’ is used. In my old version of I2Cdev.cpp, this didn’t matter, because all the class method declarations & definitions use the ‘Wire.’ object technique for invoking a ‘i2c_t3’ class method. However, the new code uses the ‘useWire->’ pointer technique which does require the ‘TwoWire’ declaration, only it’s not available for ‘I2CDEV_IMPLEMENTATION == I2CDEV_TEENSY_3X_WIRE’ – ouch!

Comparing the old and new MPU6050.h files, the only changes of significance are:

  • The name of the class was changed from MPU6050 in the old file to MPU6050_Base in the new one, with ‘, void * wireObj = 0’ added as the last parameter in the constructor
  • GetCurrentFIFOPacket, setFIFOTimeout, and getFIFOTimeout methods were added
  • The entire #ifdef MPU6050_INCLUDE_DMP_MOTIONAPPS20 section was removed
  • void *wireObj and uint_32t fifoTimeout objects were defined

The new GetCurrentFIFOPacket method added by Homer Creutz replaces the inline method I used in my older code, and does a nicer job to boot, and the wireObj declaration makes possible changing all the method calls from ‘Wire.’ to ‘useWire_>’ style.

Comparing the old and new MPU6050.cpp files, the only changes of significance are:

  • The names for all method implementations are changed from MPU6050:: to MPU6050_Base::, with a ‘,wireObj(wireObj)’ parameter added at the end of the constructor method and most other methods as well
  • The implementation code for GetCurrentFIFOPacket, setFIFOTimeout, and getFIFOTimeout methods were added

Comparing MPU6050_6Axis_MotionApps_V6_12.h (the old version) to MPU6050_6Axis_MotionApps612.h (the new version), there were a lot of changes:

  • The implementation code was split out from the header file into a new MPU6050_6Axis_MotionApps612.cpp file
  • A new MPU6050_6Axis_MotionApps612 : public MPU6050_Base class is declared and all the dmpxxx methods are now part of this class.
  • All the hard-coded MPU6050 DMP firmware image is gone (in the cpp file?)
  • At the very bottom is the line ‘typedef MPU6050_6Axis_MotionApps612 MPU6050;’ declaring ‘MPU6050’ to be a object of type MPU6050_6Axis_MotionApps612

The new MPU6050_6Axis_MotionApps612.cpp file now contains the DMP firmware image and the definitions for all the methods declared in the .h file.

So, it looks like the whole problem with changing over from the old setup to the new one may be just the fact that the declaration (invocation?) of the ‘TwoWire’ class is guarded by an #ifdef that disables the line for ‘I2CDEV_IMPLEMENTATION == I2CDEV_TEENSY_3X_WIRE’. So tomorrow (neeeeeeedddddd, sleeeeepppp!) I’ll try again with the new setup, with the additional condition added to the #ifdef line.

So, I created yet another VS2019 Arduino project called ‘Teensy_MPU6050_DMP6_V3’, identical in every way to ‘Teensy_MPU6050_DMP6_V2’ except this project includes MPU6050_6Axis_MotionApps612.h instead of MPU6050_6Axis_MotionApps_V6_12.h. To make things simpler, I copied all the reference files from the i2cdevlib folder into the project’s local folder, as shown below:

After editing I2Cdev.h as shown below

to utilize the i2c_t3.h version of the Wire library, I compiled it for Teensy 3.2 and got the following output:

As shown, all the compile errors reference a missing declaration for TwoWire, which I hope means that adding the ‘I2CDEV_TEENSY_3X_WIRE’ identifier to the guard around the code that declares TwoWire in i2Cdev.cpp.

I edited i2Cdev.cpp to add the ‘I2CDEV_TEENSY_3X_WIRE’ identifier to the guard around the code that declares TwoWire in I2Cdev.cpp, and edited I2CDev.h to do the same thing for the guard around the ‘class TwoWire’ declaration, and recompiled. This time the output was:

So I commented out the ‘

line and recompiled, still getting way too many errors – time to punt.

20 November 2021 Update:

As mentioned above, I decided to punt on the idea of incorporating the newest I2CDevlib files, as I keep getting lost trying to sort out the ‘Wire/TwoWire/useWire’ mess.

However, I did discover that the new library versions work just fine with an Arduino UNO or MEGA2560 as the target. So, I created an Arduino project called ‘Arduino_MPU6050_6Axis_MotionApps612’ and used it to tune the rate-controlled turn PID for the new robot. After the requisite number of false-starts and screwups, I was able to get pretty decent 45º/sec and 90º/sec turns with:

So, at this point I have an Arduino program ( Arduino_MPU6050_6Axis_MotionApps612) targeting the Arduino UNO that works fine with the new versions of the i2cdevlib libraries, and an older Arduino program (Teensy_MPU6050_DMP6_V3) targeting the Teensy 3.x that works fine with the 2-year old versions of the i2cdevlib libraries. The next step I think is to move ahead with the Teensy-targeted program using old libraries on the assumption that eventually Jeff Rowberg and company will either fix the current libraries to work properly with Teensy, or show me the error of my ways so that I can fix the Teensy compilation problem.