Back pain? This app uses your webcam to detect bad posture – TechCrunch
Are you slouching at your desk again? There’s an app for that… Zen uses posture mirroring software to help information workers stop hunching over their desks — sending alerts when it detects you’re not sitting up straight so you can correct your posture and, hopefully, avoid a lifetime of back problems.
The catch? It uses your webcam to check on your posture. So, er, you have to be comfortable with Zen’s software ‘watching’ through the lens as you work.
Given how many people routinely tape over their webcam just to be sure — y’know, that the NSA isn’t watching — that’s quite the ask. So we asked Zen’s co-founder and CEO, Daniel James, how the San Francisco-based startup works around that privacy concern.
As well as offering the tool direct to consumers (it currently has 1,000+ users of its subscription service), the startup sells a version of Zen to employers and has signed up around 30 companies (including over a dozen enterprises) since it launched the offering back in October 2020. And given rocketing growth in worker surveillance tools since the pandemic-triggered boom in remote working there’s plenty of reason for privacy concerns.
For example, could an employer that’s signed up to Zen use the tool, if not literally to spy on staff sitting at their desks (which would probably be pretty boring tbh), then to log how many hours they are physically sat in front of the screen, say — and use those data-points to pressure employees to shorten any desk breaks they might wish to take?
Dystopian uses of webcam-based tools aren’t hard to imagine because such stuff, sadly, isn’t science fiction. Take Amazon’s launch in the US last year of AI-powered cameras in its Prime delivery vans, which it said would be used to assess driver “safety” — but which critics instantly dubbed Orwellian surveillance…
In short, ‘AIs with eyes’ can just feel, er, function creepy.
Zen says it’s taken a “privacy centric” approach to building this webcam-based posture correcting tech — meaning its taken some specific steps to try to reassure users they’re not being watched by it or anyone else as the AI watches them.
Firstly, its posture correction software is open source (the code is here on Github). “We use open source software for the entire app, except our exercise and educational content, which we custom make,” notes James when as asked about that.
The AI also processes data locally, on device — which means it does not require an internet connection to function — so he says users can verify for themselves that it’s not uploading/streaming any data to the cloud by testing it with their wi-fi/internet connection disabled.
“The posture correction software feature runs offline, without internet, without recording or storing visuals,” he emphasizes. “Since data, like photos or videos, can’t be passed to the cloud without internet connection, which is the only way that employers could spy on employees, it’s technically impossible for us (Zen) or employers to record or store any visuals and ultimately spy on people.”
He also confirms that employers who use Zen’s software only receive “aggregated” (entire company) and “anonymous” (no individual names) information on how many employees sign up to the app and how many use it on a weekly basis.
“With these two data points they can see if employees are engaged with Zen or not, which is usually the determining factor for them when deciding to extend their contract with us,” he adds.
So — to be clear — Zen’s claim is that neither employers who pay it for the software (or Zen itself) can access the camera feed of users to record or store any visuals.
“The posture correction feature of the app, which is the only feature that accesses individuals’ cameras doesn’t run on the cloud, which technically means that no one can tap into the app and access data, including visuals,” says James, adding: “Surprisingly, no employer has ever asked for ‘spy’ type data. They really just want to know if employees are using the solution that they are paying for.”
Even so, privacy conscious desk workers may still not like the idea of sitting in front of a naked camera lens all day.
After all, there is little in tech so blissfully out-of-sight-out-of-mind as a webcam with a sticker stuck firmly over it.
Posture check-ins
On this, James suggests Zen users have come up with their own way to feel comfortable with the tool as he says they tend to use the app to do short posture checks in — say of 30 minutes or an hour with the feature enabled, a couple of times a day — rather than keeping it on all the time.
The developers have smartly leaned into that — recommending users do just a short daily check in to stay posture aware.
“We noticed that the majority of our users don’t leave Zen on all day. Instead they do a short 30-60 minute posture session during their first work session in the morning and another one around Noon or 4 PM. This correlates to how most people meditate,” James tells TechCrunch. “They spend a short time actively being aware of their thoughts which strengthens their passive awareness throughout the day. This works for posture as well.
“Doing short small sessions daily ultimately increases your posture awareness and behavior change. Furthermore, in user interviews people often tell us that ‘Zen seems to be in the back of my head even when I’m not at my desktop. I find myself slouching at the dinner table, and I’m starting to naturally notice it and move back into an upright position.’ From this anecdotal and analytical data, we decided to recommend people to start using the app for just thirty minutes a day during what we call the ‘7-Day Posture Challenge’ and people are seeing amazing results both in increased posture awareness and decreased back and joint pain.”
“We’ve found that people aren’t as concerned about any privacy invasions when they realize that Zen doesn’t have to be on all day. You can turn it on and off as you wish,” he adds.
James also says users often combine use of the posture monitoring feature with use of other apps which require the webcam to be on, such as when they’re making a videoconferencing call.
“They can show posture awareness and confidence during their video calls and they have their camera on for Zoom and G Meets regardless so they aren’t as concerned about any privacy invasions,” he notes.
Zen integrates with the user’s general computer workflow — running in the background and mirroring the user’s posture via a stick-man icon displayed in the menu bar which lets users keep discreet tabs without being interrupted with alert messages. Blue and upright is good; bent and red is bad. (James says it never sends distracting/pop-up message alerts; but users can choose from a few options how they want to be alerted.)
How Zen alerts users to poor posture: A larger graphic can be pinned to the screen (with an optional alarm feature) or users can rely on watching a discreet stick-man in the menu bar can (Credit: Zen)
The posture correcting AI works off a user-defined baseline — meaning the user needs to demonstrate their upright posture on set up. The app then uses that to build a user-specific model made of vectors that record key posture points/indictors (joints, nose, ears etc) so the AI can detect posture changes in real-time (i.e. when the camera monitoring is enabled) and determine whether or not the person is slouching.
“These posture points are fed to a mathematical model that constantly compares your current posture position to the original baseline posture position that you set as your ‘upright’ position,” he explains. “In addition, the app applies geometrical formulas to vectors formed by your current posture position and your original baseline upright posture position to determine if you’re slouching.”
James has a personal reason to be keen on keeping good posture, having been a “very active” NCAA Division-One college football player at Yale University who then went on to working at Adobe in San Francisco — and “living the typical sedentary corporate lifestyle of sitting in front of a computer for over eight hours a day” — which eventually led to him developing serious low back pain and carpal tunnel.
“Adobe offered great ergonomic resources like a free ergonomic consultation and a stand-up desk and purchased different devices that claimed to help with posture but my pain just continued to increase,” he says, fleshing out the reasons that led to founding Zen.
There was a lucky strike too: His co-founder, Alex Secara — who was his housemate at the time and is now Zen’s CTO — had already developed a posture correction software for himself in college to help with a spine-related condition he has (kyphosis) which had also been exacerbated after long hours of coding during tech internships.
“We ultimately decided to join forces to build Zen alongside top ergonomists and physical therapists,” adds James.
Zen is disclosing $3.5M in pre-seed funding raised from investors including Y Combinator, Valor Equity Partners, Goodwater Capital, Samsung Next, Softbank and others which it says it will be using to invest in expanding its team for growth and product dev — with plans in the works for key workday integrations (Slack, G Cal, Microsoft Teams), and for versions of the software for different devices/platforms (mobile, tablet, etc.).
The startup also tells us says it’s exploring partnerships with larger companies and “further proving out the efficacy of our solutions through clinical studies”. (Current enterprise customers include Brex, Alation and Cedar.)
Zen also plans to switch the consumer product to a freemium version — saying it’s aiming for a model akin to the meditation app Calm with premium paid features.
Expanding into selling physical products (more ergonomic chairs, mice, keyboards etc) is also on its roadmap, per James, who says it’s also looking to explore whether it can make use of existing movement sensor hardware in devices like more high end headphones, wearables and mobiles to see if it could repurpose those signals for determining if a person is slouching or not.
If it can devise AI models to figure that out, it might end up being possible for users to get real-time, back-saving posture-mirroring tips without ever needing to switch the camera on. Bliss!
