What Is On-Device AI? How to Tell If Your Notetaker Is

The short answer: “on-device” only means something if both transcription and AI analysis run on your hardware. Most assistants that use the phrase mean only that audio capture is local. Hedy is the clearest example of the full version: speech recognition always runs on your device, and with Local AI on supported hardware the summaries, notes, chat and live coaching do too, so nothing has to leave your machine. The fastest check for any tool is to switch off your internet and record a meeting.

“On-device” has become the most stretched phrase in this category. It appears on the marketing pages of tools that stream every second of your audio to a cloud server, and buyers read it as a privacy guarantee.

The phrase is not usually a lie. It is usually true about one stage of a pipeline that has four, and the vendor lets you assume it covers all four. This guide is about learning to ask which stage.

The four stages

Every meeting assistant does roughly the same four things, in order.

Capture. Getting audio off the microphone or out of the meeting application.

Transcription. Turning that audio into text.

Analysis. Producing a summary, action items, decisions, and whatever else the tool sells.

Storage. Keeping the results so you can find them later.

A tool can perform any of these locally or remotely, and the combinations matter. When a vendor says “on-device,” they are almost always talking about stage one.

Why the hardware got good enough

For the first years of AI meeting assistants the architecture was a given: send audio to the cloud, process it with server-side models, send the results back. Laptops could not run the models.

That changed with the chips. Apple’s M-series Neural Engine runs speech recognition models faster than real time, and Qualcomm’s Snapdragon X and Intel’s Meteor Lake bring dedicated AI accelerators to Windows laptops. Recent phones and tablets can now run small language models as well. Running speech recognition and a language model on the device you are holding is practical on current hardware, not a lab demo.

The shift also brings benefits beyond privacy. Local processing has no network round trip, so live transcripts and coaching suggestions appear as people speak. It keeps working on a plane or in a conference center with bad Wi-Fi, and it does not go down when a vendor’s servers do.

Why capture-only is the common case

Local capture is useful, and it is the reason bot-free tools exist. Instead of sending a bot into your Zoom call as a visible participant, the app records from your machine’s own audio. Nothing joins the meeting and nothing has to be admitted from a lobby, which matters more now that Google Meet and Microsoft Teams both restrict third-party bots.

But local capture says nothing about where the audio goes next. In most of these tools, it goes straight to the vendor’s servers for transcription and analysis.

Granola captures on your Mac and processes in the cloud. Jamie captures on your device and processes in Germany, which is good for EU residency and is not on-device. Circleback captures locally and analyzes in its cloud. Those are decent products with reasonable privacy postures. But if you bought one believing your meeting content never leaves your laptop, you bought something other than what you thought. For the bot-free angle, see our comparison of bot-free notetakers.

The test that settles it

Turn off your internet and record a meeting.

If you get a transcript and a summary, the processing is local. If you get an error, a spinning indicator, or a note saying it will finish later, it is not. Everything else (the marketing copy, the architecture diagram, the privacy page) is downstream of that one result.

If you cannot easily test, ask the vendor in writing: “does transcription run locally, and does summarization run locally?” Treat them as two questions. Vendors who do it locally answer immediately and specifically. Vendors who do not tend to answer a different question, usually about encryption.

Encryption is not the same thing. It protects data in transit and at rest on someone else’s computer. Local processing means there is no someone else. Encryption fails when a vendor is breached, subpoenaed, or acquired by someone with different values. Local processing does not have that failure mode, because there is nothing at the other end to breach.

Which tools actually pass

Of the 28 tools we rank, the honest list of full on-device options is short.

Hedy is the only tool in our rankings with a built-in option to run the whole pipeline (transcription, summaries, notes, chat and its live coaching layer) on-device. Speech recognition always runs locally: NVIDIA’s Nemotron model since June 2026, with Whisper still available. Local AI shipped in May 2026 and is opt-in. It runs on Apple Silicon Macs, Windows PCs with a capable GPU, iPhone 15 Pro and newer, M-series or A17 Pro iPads, and Android phones with a supported GPU and roughly 8 GB of memory. On a device outside that list, transcription still stays on the device and the AI step can use Hedy’s EU or US cloud instead (or Apple Private Cloud Compute on iOS 27). On supported hardware it passes the airplane test with live coaching intact, which nothing else in the category does out of the box. See our best offline AI meeting assistants for the full shortlist.

MacWhisper runs Whisper and Parakeet models entirely on your Mac, with no account and no server. It is a transcription tool more than a full meeting assistant: no calendar integration, no team features. Summaries are local only if you install and configure a local model through Ollama or LM Studio. The developer’s Pro license is a one-time €65 (about $72); the App Store edition is sold separately from $6.99 a month. For a journalist transcribing a sensitive interview, it is close to unimprovable.

Krisp processes noise cancellation and English transcription on-device, which earned it a strong privacy score. Other languages and its meeting notes run in the cloud.

Plaud.ai, TwinMind and Notta (whose desktop Privacy Mode, in beta, transcribes offline) have meaningful on-device components without being fully local.

Everything else in our rankings processes in the cloud, from Otter and Fireflies to the built-in assistants from Microsoft, Google and Zoom.

When cloud is the right answer

Local processing is not always better. Cloud processing gets you larger models than some devices can run, cross-device sync, team-wide search, CRM integration, and analytics over hundreds of calls. If you want to know which objections correlate with lost deals, that computation happens centrally. Gong cannot be an on-device product.

Local models also have gaps. Smaller models can trail cloud models on difficult audio, language coverage can vary, and cross-team analytics are easier to build where data is pooled. Those gaps are narrowing with each hardware generation.

The reasonable position: local processing is the right default for sensitive conversations, and cloud is the right default for collaborative ones. A client consultation, a board discussion, a performance review and a privileged legal conversation belong in the first category. A weekly sales standup belongs in the second. Many organizations need both. See our privacy-first guide for the shortlist.

What to check before you buy

Run the airplane test. Everything else is inference.

Ask about transcription and analysis separately. They are frequently split, with transcription local and analysis remote.

Ask what happens on unsupported hardware. Tools with an on-device mode usually have a cloud fallback. Find out when it triggers and where the data goes.

Ask whether local mode is the default or an option. A local mode nobody enables protects nobody.

Check the training default separately. Local processing and no-training are different guarantees, and after the August Otter.ai ruling the training question carries its own legal weight. Our privacy comparison covers the defaults tool by tool.

The phrase “on-device” is not going to become more precise on its own. But the airplane test is not something marketing can argue with, and it costs you one meeting to run.