Options for contactless rotary knob gesture recognition

[Georgy.Moshkin]

For my projects I need a touchless rotary knob gesture sensor. Idea is to tune value using pointing finger clockwise/counterclockwise movements about 3cm from the sensor while observing feedback on a small screen (digital value / computer graphics generated rotated knob image). And the second wearable project requires the same without visual feedback but with size and power consumption constraints. Currently I am testing IR based gesture sensors, but they do not include recognition functionality or only include very simplistic algorithm. From my perspective IR gesture sensors provide enough data to accomplish this task and I recorded a lot of gesture waveforms and developed pretty good algorithms for APDS9960, but maybe there is a better solution. Camera algorithms seems attractive, but probably will require much more processing power and would not work in low light conditions. There are also electromagnetic based sensors, but I haven't found a single video demonstrating good reaction to small amplitude movements.
To sum up, what I am trying to find / achieve is:
1) touchless recognition of pointing finger rotary gestures
2) small amplitude finger movements
3) realtime data on current angular position (rotation direction, speed) ~20fps
4) works in low-light conditions
5) low price, but may be up to 100 USD for something very robust and small (can fit in earphone-like enclosure)

 

[BradtheRad]

3 cm is suitable distance for a neodymium magnet to influence a compass. Attach a magnet to your fingernail with sticky gum. By moving your finger near the compass you spin it any way you wish.

The compass needs to be damped somehow, though not too much. Or perhaps fabricate something resembling a clock hand pivoting around at the center, while a magnet attached to the free end follows your finger magnet.

Led sensors or hall sensors, etc., are detection methods.

 

[Georgy.Moshkin]

Thanks for an idea. For a commercial product, there must be some kind of wearable thimble with a magnet, the thing I want to avoid. IR led sensor I've tested have 4 photoelements that are relatively well matched and have some optics that results in readings as if those elements was placed further apart (I think). There is similar sensor called PAJ7620 which includes simple recognition algorithms for clockwise and counter-clockwise recognition, but does not provide data before recognition.

 

[BradtheRad]

1.
There's the principle of the theremin... a musical instrument controlled by the operator's hands moving near proximity sensors (one to change pitch, another to change volume).
Probably three proximity sensors are sufficient to triangulate a finger's position and motion.

2.
Three opto-sensors detect a fingertip's position and motion. It's illuminated uniformly from below by an array of led's or bulbs. The finger needs to be bright enough, and in view of all 3 sensors, in order to triangulate its position.

 

[Georgy.Moshkin]

I overlooked theremin, thanks.

 

[barry]

A Theremin uses the capacitance of your hand to de-tune an RF circuit. I'm not sure how you can detect two-dimensional motion of a finger using that technology. Problems of RF leakage, etc. sound too daunting to me. Optical might be a more practical approach.

 

[Georgy.Moshkin]

A Theremin uses the capacitance of your hand to de-tune an RF circuit. I'm not sure how you can detect two-dimensional motion of a finger using that technology. Problems of RF leakage, etc. sound too daunting to me. Optical might be a more practical approach.

Yes, I'll probably focus on commercial IR based sensor.
How do you think, is there a demand for a robust gesture sensing module with recognition? I have achieved some pretty good results, but always wanted to find a better solution. Now I think that what I already made may be the best thing on the market. I am thinking of creating small recognition module with configurable outputs, which provides classic up/down/left/right gestures, and much more information, such as speed, angular position, fine tuning. Control of different things using touchless gestures, maybe passwords. I think it may be applied in many areas, from hobby and music, to laboratory and industrial equipment. I really hope I can start selling this stuff, self-fund my bigger project, and start work for myself.

 

[Georgy.Moshkin]

Here are my current results using APDS9960

I aimed for more, but there are some hardware sensing area constraints that difficult to overcome algorithmically without adding more sensors. It may be used for robotics, especially robotic toys, by DIYers and entertainment. My next step is to add two more sensors and process data simultaneously to add very nice proximity features. For best user experience gesture sensor must be as responsive and reliable as physical rotary button or switch.

 

[Georgy.Moshkin]

Seems IR gesture sensing technology have bad PR. Technology have some limitations, but not as bad as described below (!):

It is from review of generic apds9960 based module. Maybe was defective one or problem with the code, too simplistic or does not work at all. I think I can do much better and putting a lot of efforts into this project.

Meanwhile, first prototype is assembled and tested. Going to do some work on the sensor, make it more prototyping-friendly, add an USB-C port, remove dummy pins.

Upper sensor is VL53L0X (TOF),
APDS9960 in the middle and regulator at the bottom.
On the left is USB-to-TTL converter, it provides 5v power and RS232 port for data readout.
I haven't made a new video yet, demo-2 currently is too simplistic (only added TOF distance readout).

 

[stenzer]

Hi,

I overlooked theremin, thanks.

if you are still considering other sensing techniques, like the mentioned capacitve one, the 3D gesture sensors from Microchip might be of interest [1]. But with a unit price of about 5€ (e.g. MGC3130 [2]) it is pricier, and of course a dedicated sensor is needed as well.

[1] https://www.microchip.com/en-us/products/touch-and-gesture/solutions-for-3d-gestures
[2] https://www.microchip.com/en-us/product/MGC3130#document-table

BR

 

[Georgy.Moshkin]

Thank you. I've looked into this sensor datasheet, and it seems like a good candidate, mainly because it mentions "raw data streaming" feature, which is the only way to try custom, possibly much better processing algorithms

 

[Georgy.Moshkin]

A small update (less than 2 minutes):

currently working on Arduino compatibility and usage examples.

 

[D.A.(Tony)Stewart]

I can't imagine a better solution than IR optical mouse wheel with detents for fine incremental step control, variable software step gain and fast sweep up/dn for $1 in high volume.

I may not appreciate the reasons for this HMI.

There are dust issues that may block some wheel slots over time or speed quality issues or alignment issues that separate the best from the rest.

Now Logitech has an upgraded magnetic version.

Logitech MX Master 3 teardown offers an intriguing look at its new scroll wheel

The best mouse teardown you’ll read all day, probably

www.theverge.com

Also, there is the fingerprint swipe strip solution for control.
The laser mouse can be used for remote sensing but is focused for surface depth.
Try your finger on the bottom of your mouse with sensitivity reduced significantly to get the idea at different gaps.
It may be much harder to control unless you change mouse sensitivity, but you only need 1 axis.
That may be the ticket and cheapest touchless solution.