Local AI ChatBeta

Download an open AI model straight from Hugging Face and chat with it inside your browser — on your GPU or your CPU, offline after the first load, with nothing ever sent to a server. The same tab also transcribes audio, reads text aloud and writes music.

This runs a real AI model on your own machine. Pick a model, press Download & load, and your browser fetches the weights once — a few hundred megabytes to several gigabytes, depending on the model. After that the chat works with no internet at all, and nothing you type is ever sent anywhere.

The Speech & music tab does the same trick with other kinds of model: transcribe an audio file, have text read aloud, or generate a short piece of music — each a separate small download, none of it uploaded anywhere.

You need a recent browser, a few GB of free disk space, and patience on the first load. A GPU (WebGPU) makes it many times faster, and Run on: Automatic uses it whenever this device passes a real GPU test — if it doesn't, or if the GPU gives up part way, everything falls back to the CPU on its own and tells you why.

Choose a model and press Download & load to begin.

The first load downloads the weights into this browser's storage. Later visits reuse them and start in seconds.

Resource usage no model loaded
CPU load 0% busy
CPU threads —
Memory —
GPU —
Model storage —
Output speed —

Browsers do not report CPU or GPU load to a web page, so nothing here is invented. CPU load is measured two ways and shows the higher: how much of each second the engine spends blocking its own worker, and how much slower a fixed piece of arithmetic runs right now than on an idle machine — which is what a multi-threaded engine looks like from here. Memory is the JavaScript and WebAssembly heap, which only Chromium-based browsers expose. GPU utilisation and video memory are not available in any browser.

Generation settings

Three more models, all running on this device: one turns speech into text, one reads text aloud, one writes music from a description. They share a slot, so loading one replaces the previous one — your chat model stays where it is.

Transcribe audio

No model loaded yet.
No audio chosen yet.

Read text aloud

No model loaded yet.

Make music

No model loaded yet.

MusicGen is slow without a GPU — measured on a plain CPU, about 50 seconds of computing for 2 seconds of music. Start with a short clip.

Load any model from Hugging Face

Search the Hub, or paste a repository id. Each weight variant is listed with its real download size and whether it is likely to fit on this device.

Downloaded on this device

Models are stored by your browser and count towards its storage quota. Delete any of them here at any time.

This device

Graphics drivers and browsers get updated. If GPU inference was switched off after a failure, you can ask for it to be tested again.

Log


  


🔒 Runs entirely in your browser — nothing you type is uploaded or stored on a server.
Local AI Chat runs a real large language model on your own computer, inside this tab. There is no server doing the thinking, no API key, no account and no usage limit. You pick an open model, your browser downloads its weights once from Hugging Face, and every token after that is computed on your own hardware. Close the network connection and it keeps working.
That matters most for text you cannot paste into a hosted chatbot: a contract, an internal report, medical notes, customer records, unreleased copy, source code under NDA. A remote assistant means shipping that text to somebody else's machine and trusting a privacy policy. Here the prompt never leaves the device — you can watch the network panel and see it for yourself.
It runs two ways. With WebGPU — Chrome, Edge and recent Safari, plus Firefox on Windows — models up to roughly 7–8 billion parameters run on your graphics card at conversational speed. Without WebGPU there is a CPU path built on llama.cpp compiled to WebAssembly, which runs anywhere and handles small models (135M to about 1.5B) perfectly well. The tool measures your device on load and tells you honestly what will fit before you spend the download.
You do not have to know which of those you have. On load, the page runs a real GPU test — it compiles a compute shader, runs it and checks the answer — because plenty of browsers advertise WebGPU and then fail on the first actual workload. If the test passes, the GPU is used; if it fails, or if a model that started on the GPU falls over mid-answer, the tool switches to the CPU by itself, says so plainly, and carries on. It never leaves you staring at an error because your graphics driver is old.
It is not only chat. The Speech & music tab runs three more kinds of model on the same device and under the same rule that nothing is uploaded: transcription (Whisper or Moonshine — drop in an audio file, in 99 languages, with optional timestamps and translation to English), text to speech (a 38 MB voice, in English, Spanish, French, German or Russian), and music (MusicGen writes a short instrumental clip from a description). Each one downloads once and then works offline, and both the speech and the music can be saved as a WAV file.
You are not limited to a shortlist. The Models tab searches the Hugging Face Hub directly, reads any repository's real file sizes, and reports which of the three engines can run it: Transformers.js for ONNX exports, wllama for GGUF quantisations, and WebLLM for MLC-compiled builds. Every model you download is listed with its size and can be deleted in one click.
Some real numbers, measured rather than estimated, on a plain Linux container with no GPU at all: SmolLM2-135M in Q4_K_M is a 105 MB download and loads in about 19 seconds across 9 CPU threads, and pressing Stop mid-answer takes effect in roughly half a second rather than running to the end of the paragraph. A counter-intuitive one worth knowing before you pick a quantisation: on the WebAssembly CPU path, 8-bit beat 4-bit by about 3× on output speed and was the smaller download (348 MB against 370 MB), because ONNX Runtime's 4-bit WASM kernels are far weaker than its 8-bit ones. So the tool picks q8 first on CPU and q4f16 first on GPU, instead of assuming smaller is faster.
This is a beta. In-browser inference is young, models are large, and browsers differ enormously in how much memory they will hand to a web page — so expect the occasional failed load on modest hardware, and tell us when a model misbehaves. What is not experimental is the privacy guarantee: the architecture has no server component to send anything to.
How to use
  1. Read the capability chips at the top — they show whether GPU inference passed its test on this device, how many CPU threads are available, and how much storage your browser will allow.
  2. Pick a model. If you have no GPU, start with SmolLM2 135M (about 105 MB) to see it working, then move up.
  3. Press Download & load. The progress bar shows the real download; the first load of a large model can take several minutes.
  4. Type a question and press Enter. Tokens stream in as they are generated, and Stop interrupts a long answer immediately.
  5. Open Generation settings to set a system prompt, temperature, response length or context size.
  6. Watch Resource usage under the chat to see how hard the engine is working, how many threads and how much memory it is using, and how fast tokens are coming out.
  7. Open Speech & music to transcribe an audio or video file, hear text read aloud, or generate a short piece of music — each has its own small model to download, and they share one slot, so loading one replaces the previous one and leaves your chat model alone.
  8. Use the Models & storage tab to load anything else from Hugging Face — start typing a name and it suggests matching repositories, so you never have to spell an id exactly — and to delete downloaded weights when you want the disk space back.
Examples
Summarise something confidential
Paste a contract or an internal report and ask for the key points, knowing the text never leaves your laptop.
Work offline
Load a model before a flight; the chat keeps working at 35,000 feet with the Wi-Fi off.
Try a model before committing
Search the Hub for a new release, see its true download size and whether it fits, and chat with it in a minute.
Draft and rewrite
Ask for a shorter version, a different tone or a plain-English rewrite without a subscription or a message quota.
See how small models really behave
Compare a 135M model against a 1.5B one on the same prompt — an honest demonstration of what model size buys you.
Transcribe a private recording
Turn an interview, a voice memo or a lecture into text without uploading the audio to a transcription service.
Hear a draft read back
Paste what you wrote and listen to it — mistakes and clumsy sentences are much easier to catch by ear.
Frequently asked questions
Does my text get sent to a server?
No. The model runs inside your browser, so prompts and answers are computed on your own device and never transmitted. The only network traffic is the one-off download of the model weights from Hugging Face — after that you can disconnect entirely and the chat still works.
Do I need a GPU?
No, but it helps a great deal. With WebGPU a model runs on your graphics card and replies at a readable speed. Without it, the CPU engine (llama.cpp compiled to WebAssembly) still works everywhere — just slower, so stick to models under about 1.5 billion parameters. Leave “Run on” at Automatic and the page works out which you have, tests it for real, and picks for you.
It says GPU inference isn't available — why, and what now?
Because your browser either has no WebGPU, or has it but failed a basic GPU test we run before trusting it with a model: a small compute shader that has to return the right numbers. Old or blocklisted graphics drivers, virtual machines, Linux without a working Vulkan driver and some locked-down browsers all fail it. Nothing is broken — the tool simply runs everything on the CPU instead, which works everywhere. Drivers and browsers do get updated, so Diagnostics has a Test GPU again button that re-runs the check.
What happens if the GPU fails while a model is loading?
It falls back on its own. If a GPU load or a GPU answer fails — out of graphics memory, a lost device, a driver bug — the tool remembers that this device's GPU can't be trusted, reloads the model on the CPU (picking the weight format that is fastest there, which is not the same one), and replays your message. You are told what happened; you are not left with an error. The remembered verdict expires after a month, or you can clear it from Diagnostics.
Can I see how much CPU, memory and GPU it is using?
Partly, and the panel is honest about which parts. Open Resource usage under the chat. CPU load is measured, not guessed: we time how much of each second the engine spends blocking its own worker, and separately how much slower a fixed piece of arithmetic runs right now than it does on an idle machine — the second one is what catches a multi-threaded engine, whose work happens on other threads entirely. CPU threads shows how many of your cores the model was given. Memory is the JavaScript and WebAssembly heap, which only Chromium-based browsers report — in Firefox and Safari the row says so rather than guessing. Output speed is tokens per second as they arrive. There is no GPU figure because no browser exposes GPU load or video memory to a web page; we would have to invent it, so we don't.
Why is an engine or a device greyed out instead of hidden?
Because a missing option looks like a missing feature. WebLLM only runs GPU-compiled weights, and wllama only runs on the CPU, so on any given device some combinations genuinely cannot work — those stay in the list, greyed, with the reason written into the label, and a line under the row says what is being used instead. If GPU inference was switched off after a failure you can still select it deliberately, because drivers get fixed and the load falls back on its own if it hasn't.
Can it transcribe audio, read text aloud or make music?
Yes — all three are in the Speech & music tab, and all three run on this device like the chat does. Transcription uses Whisper (English-only Tiny at about 96 MB, or the multilingual Base at 142 MB, which also translates to English) or Moonshine; you give it an audio or video file and it reads the sound out of it in your browser. Text to speech is a 38 MB model that starts speaking within seconds, with a separate small download per language (English, Spanish, French, German, Russian). Music is MusicGen Small, which writes a short instrumental clip from a description — it needs about 650 MB and is by far the slowest of the three: on a plain CPU we measured roughly 50 seconds of computing for 2 seconds of music, so ask for a short clip first. Speech and music can both be downloaded as WAV files.
Is my audio uploaded when I transcribe it?
No. The audio is decoded by the browser and handed straight to the model running in this tab; it is never sent anywhere, and neither is the transcript. The only network traffic is the one-off model download. That is the whole reason to use it over a hosted transcription service for anything sensitive.
How big is the download?
It depends entirely on the model. The smallest useful chat model here is about 105 MB; a 1B model is roughly 0.8 GB; a 7–8B model at 4-bit is 4–5 GB. Every model in the list and every repository in the Hub browser shows its real size before you commit to it.
Where are the models stored, and how do I delete them?
In your browser's own storage — the Cache API and OPFS, the same places a web app keeps offline data. Nothing is written to your Downloads folder. The “Models & storage” tab lists everything cached with its size and a Delete button, and “Delete everything” clears the lot.
Is it as good as ChatGPT?
No, and it would be dishonest to pretend otherwise. Models that fit in a browser are far smaller than the hosted frontier models. A 7–8B model is genuinely useful for summarising, rewriting, drafting and answering questions about text you give it; it is weaker at reasoning, long context and factual recall. What you gain is privacy, offline use, no account and no limits.
What is the biggest model I can run?
On a discrete GPU with 8 GB or more, a 7–8B model at 4-bit is the realistic ceiling and works well. Larger 12B-class models need 12–16 GB of graphics memory and a browser willing to allocate it. Laptops with integrated graphics comfortably manage 1–3B. Phones top out at around 1B. There is no VRAM API on the web, so the fit estimate is an informed guess — it warns rather than blocks, and you can always override it.
Which browsers work?
Chrome and Edge 113+, Safari 26+, and Firefox 141+ on Windows have WebGPU. Any modern browser can use the CPU engine. Cross-origin isolation is enabled on this page so multi-threaded WebAssembly is available, which makes CPU inference several times faster.
Can I use any model from Hugging Face?
Almost. It needs to be in a format one of the three engines can read: an ONNX export for Transformers.js, a GGUF quantisation for wllama, or an MLC build for WebLLM. Paste any repository id into the Models tab and it will tell you which engines can run it, listing every weight variant with its size. Gated repositories that require a token are flagged and cannot be loaded.
Why is the first answer slow?
Two things happen before the first token: the weights are read into memory, and your prompt is processed. On the CPU path both are noticeably slow. Subsequent messages in the same conversation skip the load, and reloading the model on a later visit comes from the browser cache in seconds rather than minutes.
Is it free?
Yes, entirely, with no account. The models are open-weight releases published by their authors on Hugging Face under their own licences, and the compute is your own hardware — so there is nothing to meter.
Why is it labelled beta?
Because in-browser inference is still new. The CPU path is verified end to end, and the GPU path is guarded by a real test plus an automatic fallback — but the range of graphics hardware on the web is enormous and we have not seen all of it. Image generation is not shipped yet at all. Expect rough edges on unusual devices, and expect the model list and defaults to change as better small models are released.
Related tools
AI Text CleanerStrip smart quotes, em dashes, invisible Unicode and odd spaces from AI-generated or pasted text.Image to Text (OCR)Extract the text from an image, screenshot or photo — drag, drop or paste a picture and copy the recognised words.Language DetectorPaste any text and find out what language it is — 176 languages, with confidence scores, detected entirely in your browser.Readability CheckerScore your writing with Flesch Reading Ease, grade level and Gunning Fog — live as you type.Online NotepadA clean, distraction-free notepad that autosaves in your browser — with a live word and character count.