Scaffold tsjetpiti: on-device Qwen3.5-2B chat app
Android app that runs an uncensored Qwen3.5-2B GGUF fully on-device via an embedded llama.cpp (pinned b10333, built through the NDK) behind a barebones WebView chat UI. Model is downloaded on first launch. "new tsjet" wipes the conversation and resets the KV cache. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
79
README.md
Normal file
79
README.md
Normal file
@@ -0,0 +1,79 @@
|
||||
# tsjetpiti
|
||||
|
||||
A dead-simple Android chat app that runs an **uncensored Qwen3.5-2B** model
|
||||
fully **on-device** via an embedded [llama.cpp](https://github.com/ggml-org/llama.cpp),
|
||||
wrapped in an **extremely barebones WebView UI**.
|
||||
|
||||
Talk to the tsjet. Hit **new tsjet** to wipe the conversation and start fresh.
|
||||
|
||||
---
|
||||
|
||||
## How it works
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────┐
|
||||
│ MainActivity (Kotlin) │
|
||||
│ • full-screen WebView ── UI ──────────┼──> assets/web/{index.html,app.js,styles.css}
|
||||
│ • JS bridge "TsjetNative" │ (the whole chat UI, ~200 lines)
|
||||
│ • model download + prompt formatting │
|
||||
└──────────────┬──────────────────────────┘
|
||||
│ JNI (LlamaBridge)
|
||||
▼
|
||||
┌─────────────────────────────────────────┐
|
||||
│ cpp/llama-jni.cpp → libtsjet.so │
|
||||
│ talks to llama.cpp C API (pinned) │
|
||||
└─────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
- **Inference:** llama.cpp compiled from source (pinned tag `b10333`) through the
|
||||
NDK. Pulled automatically at build time via CMake `FetchContent` — no submodule
|
||||
to init.
|
||||
- **Model:** `Qwen3.5-2B-Uncensored-HauhauCS-Aggressive-Q4_K_M.gguf` (~1.2 GB) is
|
||||
**downloaded on first launch** from Hugging Face into the app's private storage.
|
||||
It is *not* bundled in the APK.
|
||||
- **Conversation:** the web layer holds the full history and sends it each turn;
|
||||
native rebuilds the ChatML prompt and clears the KV cache before every reply, so
|
||||
"new tsjet" is just: clear JS state + reset cache.
|
||||
|
||||
## Requirements
|
||||
|
||||
- **A build machine** with the Android SDK + NDK. Easiest path: open the project
|
||||
in **Android Studio** (it will offer to install the matching NDK `27.2.12479018`
|
||||
and CMake `3.22.1`).
|
||||
- **A phone:** 64-bit ARM (`arm64-v8a`), Android 8.0+ (minSdk 26), and enough free
|
||||
RAM to hold a ~1.2 GB Q4 model (~3–4 GB RAM device recommended).
|
||||
- Network on first launch to download the model.
|
||||
|
||||
## Build
|
||||
|
||||
```bash
|
||||
./gradlew assembleDebug
|
||||
```
|
||||
|
||||
The APK lands in `app/build/outputs/apk/debug/`. Or just Run ▶ from Android Studio.
|
||||
|
||||
> First build compiles llama.cpp from source, so it takes a while and needs
|
||||
> network (CMake fetches the pinned llama.cpp).
|
||||
|
||||
## Where things live
|
||||
|
||||
| What | Where |
|
||||
|------|-------|
|
||||
| Chat UI (HTML/CSS/JS) | `app/src/main/assets/web/` |
|
||||
| Android glue + model download | `app/src/main/java/monster/autisme/tsjetpiti/` |
|
||||
| Native llama.cpp bridge | `app/src/main/cpp/llama-jni.cpp` |
|
||||
| Pinned llama.cpp version | `app/src/main/cpp/CMakeLists.txt` (`GIT_TAG b10333`) |
|
||||
| Model URL / filename | `ModelDownloader.kt` |
|
||||
| Generation params (ctx, temp, tokens) | `MainActivity.kt` + `llama-jni.cpp` |
|
||||
|
||||
## TODO / notes
|
||||
|
||||
- **Logo:** juli is on it. The header logo is a placeholder `🐟` in
|
||||
`assets/web/index.html` (`#logo`), and there's no launcher icon yet — add an
|
||||
`android:icon` in `AndroidManifest.xml` + a `mipmap` when the artwork is ready.
|
||||
- The model may emit `<think>…</think>` blocks; the UI strips them from the
|
||||
display and from history (`stripThink` in `app.js`).
|
||||
- Bumping the llama.cpp tag? Re-check the C API calls in `llama-jni.cpp` against
|
||||
that tag's `include/llama.h` — it uses the raw C API directly.
|
||||
- Not yet compiled end-to-end in CI; first real build is the acid test for the
|
||||
native layer.
|
||||
Reference in New Issue
Block a user