���ճԹ�51

Happy Pi Day! Not only is today Pi Day, but this also marks five years since the first public build of the LipSync. A lot has happened since then, so we thought we would take this opportunity to look back at the LipSync and introduce the new, updated design that we are finalizing.

The First Build

Five years ago, hosted our first public LipSync build during the 2017 BC Tech Summit at the Vancouver Convention Centre.

While our engineers had built a number of LipSyncs already, this was the first time that we had volunteer makers actually build the LipSync, giving the maker model a good test.

Over a four hour period, makers worked on 12 LipSyncs. The build went great, and we learned a lot of lessons that were incorporated into the builds — namely that makers might need more time to assemble a LipSync, and that the force-sensitive resistors (FSRs) that allowed the LipSync to sense the user’s movements were quite delicate and hard to build. (Something we’ll expand on as you continue reading on).

One of the makers who joined us for that first build in March 2017 was Jake McIvor, an engineer and volunteer maker, who had previously taken part in our Access Makeathon a few weeks before.

“I was really impressed by the detail and a quality of the instructions,” Jake says, reflecting on the buildathon. “It was incredibly rewarding to know that I could use my skills to create something that would really benefit someone.”

Since then, Jake has joined the Makers Making Change team, and is now part of the team working on the LipSync 4.0, the latest incarnation of the LipSync, informed in part by the lessons of that build five years ago.

Versions of the LipSync

Leading up to LipSync 4.0, we’ve created and released several different versions of the LipSync and have made incremental improvements to the assembly guide and 3D printed parts.

When we first released the LipSync, there was a single version of the software and the option to add an optional wireless hardware module. This allowed you to use it as a mouse through USB or wirelessly via Bluetooth. Now, using that same initial hardware but different versions of the software, the LipSync can be used to emulate a USB gamepad or joystick and a USB keyboard.

The LipSync Gaming provides a low-cost option for joystick control with PCs and devices like the Xbox Adaptive Controller. Gaming is fun and can help fight social isolation and low-cost, accessible devices like the LipSync Gaming can make platforms more accessible. When we first released the LipSync, many users indicated that they wanted to use it as a joystick for video games — it is now the second most requested version of the LipSync.

The LipSync Macro emulates a USB keyboard. This version was created to provide an alternative input option for Apple phones and tablets, which at the time did not support cursor control. Movements of the joystick and sip and puff inputs are mapped to keystrokes, which can then be used with the built-in switch accessibility features like scanning. Thankfully, due to software updates, newer Apple devices now provide some cursor support and are compatible with the LipSync and LipSync Wireless.

Time for an Upgrade

With five years of LipSync builds, these devices have found there way to users all over the world, and we’ve had the opportunity to gather feedback from many LipSync users. Two issues that are often raised are the amount of force required to move the joystick, and cursor drift — undesirable cursor movement when the mouthpiece isn’t being moved.

After an extensive testing and failure analysis, we decided to take on a two-phase upgrade to the LipSync.

The first phase was a software overhaul primarily directed towards mitigating cursor drift with the current hardware, and this resulted in the September 2021 release of the LipSync V3.0 software. We also incorporated numerous other changes, including making it easier to mount the LipSync from different angles, and incorporating an application programming interface (API) to make it easier to adjust settings. During this phase, we also created the LipSync Switch Input Module, enabling users who couldn’t use the sip and puff effectively to use switches as alternative input.

The second phase was a major overhaul of the design, which we have dubbed LipSync 4.0.

The primary goals for this version of the LipSync were to improve the user experience and reduce the operating force of the joystick. We wanted to take advantage of the updated hardware that is now available and maintain both the cost and the buildability of the device.

In the next few sections, we’ll dig a bit deeper into some of the technical details of this new version.

Hardware

Updates to the hardware are focused on improving the joystick. The original LipSync utilizes force sensitive resistors (FSRs) with a spring-loaded platform to measure the mouthpiece deflections for cursor movement. While this approach works, the FSRs have several inherent challenges. First, FSRs contribute to cursor drift. Second, the way the FSRs are assembled increases the force required to move the mouthpiece. And lastly, they are delicate and expensive and prone to damage during installation.

For the LipSync 4.0, we’ve decided to do away with the FSRs due to their finicky nature and the difficulty in soldering them, and settled on magnetic sensing to perform the cursor measurements. This opened the floor for many different joystick type designs to create the desired motion. Our team decided to try for a lower force option that also made it easier to source components.

The result is a three-bearing gimbal that offsets the mouthpiece above the rotational axes. The same type of magnet used for the sensing is also used for centering the mechanism, creating operating forces nearly 20 times lower than the original LipSync.

With these significant changes made to the hardware, the LipSync 4.0 also requires a new enclosure. To address issues such as seeing the feedback lights with mounting the LipSync at varying angles, the new enclosure will be rotatable in the mount so that it can always be in the upright orientation. This will help ensure users can see the visual feedback lights and that the bulk of the device is always below their line of vision.

Electronics

From an electronics perspective, we had a few driving goals. First, the microcontrollers that are currently available have more memory and features than the Arduino Micro on the original LipSync. Second, there are a few electronics components that have become challenging to source. The analog pressure sensor has been caught up in the global chip shortage, and the Bluetooth module is now discontinued. Lastly, we wanted to ensure the assembly difficulty was the same or easier than the original LipSync.

For this update we evaluated several options for microcontroller and eventually settled on the Adafruit Feather nRF52840. This chip is an FCC certified 2.4 GHz telecom module that provides Bluetooth Low Energy and native USB support. It also has 30 times more memory and over four times greater speed compared to the Arduino Micro in the original design.

The shift away from FSRs and the shortage of the original pressure sensor provided an opportunity to consider different options for sensors. As we started looking at options, it became clear that the recently developed ecosystem of STEMMA and QWIIC modules and cables would be a viable solution. There are a range of modules with standardized connectors and cables that provide a solder-free way of connecting different devices. This enables us to use sensors that would otherwise be very difficult to solder (e.g., surface mount pressure sensors). The modules can be daisy-chained together using commercially available cables, which makes for simpler assembly and provides more options for positioning.

While these modules are more expensive than the individual components, we feel that the gain in modularity and simplifying the assembly process is a worthwhile trade-off. These modules also have open-source licenses, so they can be produced if they are not available from Adafruit or one of its distributors.

Another benefit of utilizing the STEMMA ecosystem is that it provides the opportunity to easily add additional sensors and functions by connecting additional modules. This could include a tactile motor driver for providing vibratory feedback, or additional switch input or switch output.

Software

For this version of the LipSync, we started the code from scratch. We have utilized several techniques to make the code easy to maintain and expandable, including utilizing an Object-Oriented Programming structure and using state machines. Utilizing timers instead of delays will optimize performance and allow for simultaneous input actions, something that is not possible with the original code.

To provide greater control and customizability, we have kept and expanded upon the application programmer interface (API) that we started with LipSync V3.0. This should make it easier for users to adjust settings independently, instead of having to rely on help from others to activate physical buttons on the device. This serial-based API can open the door for the developers and makers to create or adapt applications to directly interact with the LipSync to customize the settings which can lead to a better user experience.

Cursor drift is one of the major challenges with the LipSync and we have been learning more and more about its causes and methods of reducing it in the last five years. The new LipSync joystick assembly has contributed to the reduction of drift at a hardware level, but we are implementing a drift reduction algorithm to further improve joystick performance. We have been performing a range of tests and have been analyzing the collected data to identify and implement a suitable drift reduction algorithm.

Next Steps

So far, we’ve built and tested two functional prototypes: Alpha I and Alpha II. We’re doing final preparation for more extended user testing as part of a closed beta and an open beta, then we’ll incorporate the design changes for a final release. As always, the design will be released open-source so anyone can build, use, and improve upon the design.

This post originally appeared on the ���ɱ�������ٱ�.