Updated firmware for openSip+Puff
In my previous post about openSip+Puff, I showed some working prototype PCBs and demonstrated how it can be used to hit keyboard and mouse buttons for you. The firmware I was using for the boards was very basic and was meant to just verify that everything was working the way it was supposed to.
In the last couple weeks I’ve developed some much more robust and fleshed-out firmware that abstracts keyboard and mouse actions into “commands” that are stored in EEPROM (which persist even without power to the board) so that complex actions can easily be assigned to simple triggers like sipping or puffing softly or hard.
In this post I’ll explain how this new firmware works, but if you’re not interested in the code or electronics side of the project, don’t worry! You won’t need to know any of this to use the openSip+Puff, I just like to document things :)
The most recent firmware for openSip+Puff can be found on it’s official Github repo here:
Detecting and acting on sips and puffs
openSip+Puff relies on readings from an on-board pressure transducer to detect when the user has “sipped” or “puffed” on a vinyl tube. These readings from the transducer are compared to pre-programmed thresholds that let the program determine if a “sip” or “puff” has occurred, and whether that action was “soft” or “hard”. These actions are associated with “commands” that represent what the device is supposed to do when it detects those actions (like press a keyboard button or move the mouse).
Commands
Commands are text representations of events or sequences of events that the device must do when a sip or puff has been detected. I created a simple format that is used internally by the firmware that can be used to represent just about any keyboard or mouse actions I could think of, including single key presses, multiple key presses (like Alt + F4), mouse button clicks, mouse movements and mouse wheel scrolling.
Commands are stored in EEPROM, then read and assigned to sip and puff actions as soon as the program starts. By storing the commands in EEPROM the end-user only has to configure the device once, and the device will remember their configuration every time it is turned on.
The end user will never need to know how these commands work, or even that they exist, but for the sake of documentation for the electronics geeks here is how the command format is currently laid out:
Command format:
Keyboard command: [P|S],[S|H],K,[number of keys],[key #1],...,[key n],255
Mouse command: [P|S],[S|H],M,[C|M|D],[parameter #1],...,[parameter n],255
Command parameter legend:
[P|S]= puff or sip[S|H]= soft or hard[K]= keyboard action[M]= mouse action[C|M|D]= click, move, drag[key]= ASCII value of key to press (keyboard modifiers are also valid: http://arduino.cc/en/Reference/KeyboardModifiers)[parameter]= interpretation depends on selected action type. Refers to mouse button(s), relative movement amount or combination of the two.255= indicates end of command
Example commands
Here are some examples of commands to perform different actions based on sips and puffs:
S,S,K,1,KEY_RETURN,255= press “return” key on soft sipP,H,K,2,KEY_GUI_LEFT,r,255= press “Win + R” on hard puffP,S,M,C,1,255= click left mouse button on soft puffS,H,M,M,0,5,0,255= move mouse position up 5 units on hard sipS,S,M,D,1,-5,0,0,255= click left mouse button and drag left 5 units on soft sip
Operation mode
The very first byte stored in EEPROM (before all the commands) is a single character that tells the firmware what “mode” to operate in. Currently, the firmware can operate in two modes:
B= basic mode = only two actions allowed; sipping and puffingF= fine mode = four actions allowed; soft sip, hard sip, soft puff and hard puff
In the future I’d like to add support for more complex modes like patterns (something like Morse code) and expressive MIDI output!
New firmware in action
I actually coded most of the firmware over the course of one day, then made this video that night. For this video I have configured the openSip+Puff prototype to hit the Enter key, Alt + F4, Alt + Tab and click the mouse and drag 10 pixels to the right depending on hard and soft sips and puffs. Notice that throughout the entire video I never touched the keyboard or trackpad of the laptop!
Next steps
Right now I’m working on creating a program that the end user can run on their PC or Mac that will allow them to easily reconfigure their openSip+Puff through a GUI. The program will let the user click on keys on a virtual keyboard or buttons on a mouse and assign their actions to whatever trigger they want (sips and puffs). The program will generate commands based on whatever the end-user does, and send those commands to a connected openSip+Puff via a serial connection so that they user will never need to know how it all works behind the scenes.
My go-to development environment these days tends to be Processing, but honestly all of the GUI libraries are not great to use. I’d love to create a browser-based program that uses jQuery and pretty graphics (would love to use Bootstrap for the GUI), but I still need to find a simple, distributable way to make a serial connection to the openSip+Puff for sending new commands.
NodeJS looked promising, but I’m not sure how easy it would be to distribute to end-users. I’m currently looking into using Python to connect to a browser-based app through a WebSocket connection, then communicate data through a serial connection to a connected openSip+Puff. We’ll see where it goes!
If you have any tips on simple ways to send serial data from a browser-based app to an Arduino-compatible board, I’m all ears!
Again, the most recent firmware (and all other source files) for openSip+Puff can be found at the official Github repo here:
You can also keep an eye on the overall project by watching it’s official wiki page.