Scaffold tsjetpiti: on-device Qwen3.5-2B chat app

Android app that runs an uncensored Qwen3.5-2B GGUF fully on-device via an
embedded llama.cpp (pinned b10333, built through the NDK) behind a barebones
WebView chat UI. Model is downloaded on first launch. "new tsjet" wipes the
conversation and resets the KV cache.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-08-09 20:24:14 +02:00
commit ac8490b452
20 changed files with 1416 additions and 0 deletions

79
README.md Normal file
View File

@@ -0,0 +1,79 @@
# tsjetpiti
A dead-simple Android chat app that runs an **uncensored Qwen3.5-2B** model
fully **on-device** via an embedded [llama.cpp](https://github.com/ggml-org/llama.cpp),
wrapped in an **extremely barebones WebView UI**.
Talk to the tsjet. Hit **new tsjet** to wipe the conversation and start fresh.
---
## How it works
```
┌─────────────────────────────────────────┐
│ MainActivity (Kotlin) │
│ • full-screen WebView ── UI ──────────┼──> assets/web/{index.html,app.js,styles.css}
│ • JS bridge "TsjetNative" │ (the whole chat UI, ~200 lines)
│ • model download + prompt formatting │
└──────────────┬──────────────────────────┘
│ JNI (LlamaBridge)
┌─────────────────────────────────────────┐
│ cpp/llama-jni.cpp → libtsjet.so │
│ talks to llama.cpp C API (pinned) │
└─────────────────────────────────────────┘
```
- **Inference:** llama.cpp compiled from source (pinned tag `b10333`) through the
NDK. Pulled automatically at build time via CMake `FetchContent` — no submodule
to init.
- **Model:** `Qwen3.5-2B-Uncensored-HauhauCS-Aggressive-Q4_K_M.gguf` (~1.2 GB) is
**downloaded on first launch** from Hugging Face into the app's private storage.
It is *not* bundled in the APK.
- **Conversation:** the web layer holds the full history and sends it each turn;
native rebuilds the ChatML prompt and clears the KV cache before every reply, so
"new tsjet" is just: clear JS state + reset cache.
## Requirements
- **A build machine** with the Android SDK + NDK. Easiest path: open the project
in **Android Studio** (it will offer to install the matching NDK `27.2.12479018`
and CMake `3.22.1`).
- **A phone:** 64-bit ARM (`arm64-v8a`), Android 8.0+ (minSdk 26), and enough free
RAM to hold a ~1.2 GB Q4 model (~34 GB RAM device recommended).
- Network on first launch to download the model.
## Build
```bash
./gradlew assembleDebug
```
The APK lands in `app/build/outputs/apk/debug/`. Or just Run ▶ from Android Studio.
> First build compiles llama.cpp from source, so it takes a while and needs
> network (CMake fetches the pinned llama.cpp).
## Where things live
| What | Where |
|------|-------|
| Chat UI (HTML/CSS/JS) | `app/src/main/assets/web/` |
| Android glue + model download | `app/src/main/java/monster/autisme/tsjetpiti/` |
| Native llama.cpp bridge | `app/src/main/cpp/llama-jni.cpp` |
| Pinned llama.cpp version | `app/src/main/cpp/CMakeLists.txt` (`GIT_TAG b10333`) |
| Model URL / filename | `ModelDownloader.kt` |
| Generation params (ctx, temp, tokens) | `MainActivity.kt` + `llama-jni.cpp` |
## TODO / notes
- **Logo:** juli is on it. The header logo is a placeholder `🐟` in
`assets/web/index.html` (`#logo`), and there's no launcher icon yet — add an
`android:icon` in `AndroidManifest.xml` + a `mipmap` when the artwork is ready.
- The model may emit `<think>…</think>` blocks; the UI strips them from the
display and from history (`stripThink` in `app.js`).
- Bumping the llama.cpp tag? Re-check the C API calls in `llama-jni.cpp` against
that tag's `include/llama.h` — it uses the raw C API directly.
- Not yet compiled end-to-end in CI; first real build is the acid test for the
native layer.