Frequently asked questions

No. Prompts, responses and model activations never leave your machine. Inference is fully local: the chat, the OpenAI-compatible API and the admin panel are all served on localhost. You can run it on an air-gapped machine.

Two narrow, disclosed cases, neither of which ever includes your prompts or data:

• Update check: the server periodically fetches a small version manifest so it can tell you a new build exists. Disable it with auto_update = false in config.toml.
• License activation (optional, commercial licenses only): if you enter a license key AND enable activation_consent, one activation record is sent. Off by default.

Both code paths are source available, so you can read exactly what is sent, or block the domains entirely: inference does not depend on them.

Yes, that is the point. The compression engine, the CUDA kernel and the Rust runtime are source available at github.com/neuralboot/trapetum. No Python at runtime, a small dependency surface, and every benchmark number, apart from the November 2026 update, is reproducible with one command.

Yes, and it is expected: this beta is not code-signed yet. Click More info, then Run anyway. If you prefer not to trust a warning, audit the source on GitHub first, that is what it is there for.

Either an NVIDIA GPU (Ampere, Ada or Hopper: RTX 30/40 series, A-series, H100) with the CUDA runtime, or an Apple Silicon Mac (M1 to M4, Metal). A 7B model at 4-bit needs about 4.7 GB of VRAM, so an 8 GB card is comfortable; on a Mac the model lives in unified memory. A 70B at 2-bit fits a single 24 GB card. An AMD (ROCm) backend is in active development.

Two paths: download ready-compressed models from the hub (checksums verified), or pick a model from Hugging Face in the app and Trapetum compresses it on your own GPU. Gated models that require a Hugging Face login are filtered out of search since they cannot be downloaded anonymously.

Measured, not estimated: a real Llama-2-7B at 4-bit decodes at 135 tok/s on one RTX 4090 in the pure-Rust runtime (June 2026), with 2.1x less energy per token than fp16. The November 2026 decode kernels, not yet in an app release, reach 194.5 tok/s on the same GPU with CUDA graphs, over a 512-token generation, and exact speculative decoding reaches 2.51x on Llama-3.1-8B-Instruct over four prompt sets (details). The fused kernel decodes 2.0 to 2.45x faster than fp16 cuBLAS on bandwidth-limited GPUs. Full details and raw numbers on the benchmarks page.

Measurably, and we publish the numbers instead of hiding them. Read back from the files on wikitext-2 (the repository's 30-window harness), Llama-2-7B goes from 5.83 perplexity in fp16 to 6.79 with the 4-bit file written by the runtime's legacy exporter, the format of the app's downloadable models, and to 6.10 with the new exporter's file (November 2026, not yet in the app). Correction: this answer used to quote 5.92, a figure computed with the LM head kept in fp16 rather than measured on a runtime file. Separately, the runtime itself is exact: given the same prompt, it reproduces the Hugging Face forward pass of the same 4-bit weights token for token, 16/16.

2-bit at the same model size costs quality, we agree, and the hub does not serve it. How much depends on the checkpoint: for Llama-2-7B, the PV-tuned AQLM 2-bit checkpoint the runtime can load scores 6.67 perplexity in the repository's 30-window harness (its exported file 6.665), 0.57 behind the new exporter's 4-bit file (6.10) but slightly better than the 4-bit file written by the legacy exporter (6.79). 2-bit exists for one purpose: fitting a model 4x bigger in the same memory. A 70B at 2-bit (perplexity 4.06) comfortably beats a 7B at fp16 (5.47) on the same 24 GB card. Rule of thumb: 4-bit is the default for quality, 2-bit is capacity mode, and every model in the hub is labeled with which one it is.

Honest answer: those are excellent projects and they run quantized models too. Trapetum differs on three measurable points:

• A fused codebook decode kernel that beats fp16 cuBLAS on consumer GPUs, with the methodology published in a paper including the negative results.
• Verifiability: 16/16 token-for-token reproduction of the reference model, checksummed model files, reproducible benchmarks.
• An installable product: one line, background service, chat, API, admin panel and update flow out of the box.

The beta is free, on macOS, Linux and Windows. The code is source available under the Business Source License 1.1: you can read, audit, build and use it; production/commercial terms are defined by the license, with commercial licensing arriving with the company launch. BUSL is not OSI open source and we do not claim otherwise.

Yes. Once a model is on disk, inference needs no network at all. Download models on a connected machine, move the files, and run air-gapped if that is your policy.

One line on Windows: irm get.neuralboot.com/uninstall.ps1 | iex (or Programs and Features). It removes the service, the binaries and the shortcuts. Your downloaded models stay unless you delete the models directory.

Bugs: open an issue on GitHub. Security reports: email contact@neuralboot.com, we read everything.