Doublepoint integrates gesture recognition into consumer gadgets for better AR

Doublepoint has raised $3.3 million to integrate its gesture recognition technology into consumer electronics gear and other hardware.

The idea behind the Helsinki, Finland-based company’s tech is to make it simpler for people to use hardware devices for things like augmented reality and other purposes with zero learning-curve issues, said CEO Ohto Pentikäinen in an interview with GamesBeat.

The tech can detect whether you are pinching your fingers together simply by using the sensors already present in Android smartwatches. The company said it can very accurately detect such “microgestures” and do so much more cheaply than cameras, which are used in devices such as virtual reality headsets. (In fact, there are 10 cameras to detect movement in the Meta Quest Pro VR headset).

“People can create devices that are significantly less expensive,” said Pentikäinen.

Doublepoint will official show off its technology at Augmented World Expo USA 2023, which runs from May 31 to June 2 in Santa Clara, California. The company plans to license the technology to others.

After three years in stealth, Doublepoint has pending patents for its software platform, which works with a range of Google Android-based hardware devices such as AR headsets.

The funding came from a range of angel investors and established VCs from across Finland and Europe. Doublepoint investors include FOV Ventures, Superangel VC, Superhero Capital, Sune Alstrup, Timo Rein, Business Finland, and others. Doublepoint currently has a total of 11 employees situated in Finland, Sweden, Germany, and the United Kingdom.

How it works

Doublepoint currently offers a highly-advanced machine learning algorithm that accurately detects touch-based microgestures, surface interactions and more, said CTO Jamin Hu in an interview with GamesBeat.

“When I tap my fingers together, my index finger and thumb together, it registers a tap,” Hu said. “But the key point here is that it only registers when I actually touch my fingers together, not when I try to trick it without actually touching. Iit detects moments of touch from different fingers on various different objects. And the fidelity depends on whether we have custom hardware built for it.”

The tech supports the simplest and most universal gesture the creators could think of — the pinch. They designed it to not need any effort for it to work in any hand orientation. This means there is no need for memorizing a large number of gestures, and you don’t need to pay any attention to it.

This is all accomplished using generic smartwatch sensors and minimal compute cycles from an embedded central processing unit (CPU). The platform uses an IMU (Inertial Measurement Unit) to detect movement. Such sensors are pretty cheap.

“So for the smartwatch, we just need the inertial measurement unit, which means any smartwatch will do because all of them have initial measurement units,” Hu said.

On top of that, it uses bio-acoustic sensors. Overall, it can detect your wrist’s topography and how that changes as you move.

“So, for example, when I press on a table, you’ll see that the tendons at the bottom of the wrist deform,” Hu said.

Hand position and movement in devices can be measured in many different ways, including wristbands, rings, radar, LIDAR, and cameras. Initially focused on gesture-enabling off-the-shelf smartwatches as well as XR devices, Doublepoint aims to solve the most vexing input problems in the most simple way, Hu said.

This is accomplished by using an extremely sensitive, mechanically-based sensor in a smart wristband to detect movement, position and microvibrations.

They found it is hard to design an algorithm to detect only touch and nothing else. It must not trigger accidentally when you touch a table, grab objects around you and otherwise just go on about your everyday life. It should only react when you are specifically pinching your fingers together.

With Doublepoint, no expensive optical cameras or LIDAR sensors are required. Doublepoint’s wristband-based technology instead focuses on the vibrations that occur only when you touch something — a far simpler, far more inexpensive and far more robust offering than other, optically-based solutions, Hu said.

In addition to algorithmic programming solutions, the company also plans on providing custom hardware designs that are dedicated to – and optimized for – touch detection. Doublepoint offers a free Touch SDK, which turns existing smartwatches into actual input devices and provides it as a download on the company’s website.

Initially focused on gesture-enabling Google Android Wear OS-powered smartwatches as well as XR devices, Doublepoint uses physical muscle-monitoring to operate and does not require any computer vision sensors. Doublepoint started with the input problem of Android-based AR, since there is currently no discrete and/or tactile input method in production.

Partners can license the company’s algorithm and include it in their smartwatches. Once embedded, the algorithm will run in the watch OS continuously and transmit the required gesture events via Bluetooth to the appropriate headsets.

Origins

Pentikäinen previously founded a real estate technology company and he has known Hu since middle school. Hu is a classically-trained pianist and a biomedical engineer. The company got started thanks to his invention of how to type messages in mid-air with a wristband.

Hu started studying this technology when he was trying to figure how how to read his own subconscious finger twitches. Then it turned into the notion of measuring humans as they touched real physical objects.

“We first founded Doublepoint under the stealth name of ‘Port 6’ to keep our competitors guessing and under the sole focus of gesture-enabling AR devices,” said Pentikäinen. “We have since found a ready and eager market in smart watches as well as XR and we have finally revealed our production name, full funding and full mission. We are extremely grateful to our VCs for the most recent influx of capital and their supreme confidence in our team, technology and vision.”

Other leaders include Eemil Visakorpi, developer. He has previously worked as a researcher at the Quantum Computing Group at Aalto University. And there’s Lauri Tuominen, ML Engineer. He has previously worked at Nokia Bell Labs, specializing in machine learning.

Advisers include Sune Alstrup. Co-founded Eye Tribe, an eye tracking company that was acquired by Meta. Another is Miklu Silvanto, chief design officer at Bang & Olufsen. And Markku Mäkeläinen, previously senior director of engineering partnerships at Meta.

The road ahead

Doublepoint is most often compared to Israeli-based Wearable Devices and Taiwanese-based Coolsotech. Asked if Apple’s rumored mixed-reality headset might use such technology, Doublepoint said nothing is confirmed yet, but based on the most reliable public sources so far, gesture recognition appears to be of high interest to Apple for its upcoming headset.

Doublepoint isn’t talking about any customers yet, as it is really just coming out. Some rival companies are doing their own custom hardware, and Apple itself makes “assistive touch,” which is a recognition tech for smartwatches.

“We’re currently the only one who will provide a software-based solution for this gesture recognition and will license to everyone,” Pentikäinen said.

For gaming, the applications could get pretty interesting. You could create homebrew controllers that you can 3D print, which no electronics inside, where you could create a sticker with the sensors that could detect your finger touches. And so maybe for $5 you could create your own game controller. It’s reminiscent of Nintendo’s Labo cardboard devices.

As for VR devices, “We just need a smartwatch and we don’t need to add anything extra on a headset,” Hu said.