Files
OrcaSlicer/scripts/build_preset_cache.bat
ExPikaPaka 5ed56eb876 Cache system presets to eliminate startup and wizard load times (#14217)
* Add caching system for presets

* Removing user\bundle serialization and keeping it only for system presets

* Integrate caching into WebGuideDialog which speeds up time of SetupWizzard and PrinterSelection dialog

* Add CI\CD step to prepare cache file in ahead of time so user does not need to wait

* Add partial cache generation when only one of the vendros is changed to speed up recalculation time

* Handle corrupted files

* Add cache to GuideDialog as previos version didn't work as expected

* Add inspecting tool and fix CI cache generation

* Generate cache per vendor

* Simplify code by mergin it in PresetBundle

* Simplify code a bit more

* Add cereal serialize() to VendorProfile, PrinterModel, Preset, and Semver

* Remove CachedPrinterModel/VendorProfile/Preset mirror structs from VendorCache

* Fix use-after-free in CallAfter lambda; replace raw thread pointer with unique_ptr

* Use get_vendor_cache_key() to match cache keys written by the app

* Remove BOM added by VSC

* Skip invalid vendors

* Remove leftover cache file

* Fix build for windows arm64

* Revert json cache back

* Update check for stale cache

* Serealize all value fields for Preset class to minimize regression later

* Minimize field duplication by moving Cache thing into PresetBundle

* Add tests for Cache system

* Add a bit more tests

* Merge branch 'main' into feature/cache_profiles_and_optimize_loading_speed

* Rvert from per-verndor to single cache file

Replace N per-vendor .cache files with a single system_presets.cache
that holds all vendors and presets in one serialized blob.

Cache load is now all-or-nothing: on hit all vendors are applied from
the bundle (sub-second); on miss all vendors are parsed from JSON and
a fresh bundle is written to the user cache dir.

Invalidation is driven by bundle_key - a sorted concatenation of all
vendor JSON version strings. Any vendor update invalidates the whole
cache and triggers re-parse on next launch.

Guide wizard (WebGuideDialog) loads the bundled cache into a plain
PresetBundle instead of a separate VendorGuideData struct, removing
the duplicate data model.

generate_system_cache simplified from a per-vendor loop to a single
save_system_presets_cache() call producing one output file.

* Transfer all Preset fields from cache via move assignmet

apply_vendor_preset_group was copying fields manually and missed
bundle_id, user_id, base_id, sync_info, updated_time, key_values,
ini_str. Replace field-by-field copy with move assignment of the
fully-deserialized Preset, then restore the vendor pointer which
is excluded from serialization.

* Ignore cache for future

* Remove not used files

* Ship one preset cache per vendor in place of the profile JSONs

Each vendor's system presets serialize into a single <vendor>.opc built at
package time, and a shipped build carries that file alone — the profile JSON
and its sub-file tree are pruned. The vendor loader, the setup wizard's profile
list and the resource installer all read a vendor through its cache, falling
back to parsing whenever one is absent, stale or unreadable, so the cache stays
an optimization and never a source of truth. Caches hold presets in source form
and resolve inheritance at load, through the same code the JSON path uses.

* Make the preset cache self-describing and load each vendor from the system folder alone

The cached DynamicPrintConfig is keyed by name, through a per-file dictionary of the
distinct opt_keys, the type each was written as, and the distinct enum value names,
instead of by serialization_key_ordinal — a position assigned by declaration order at
static init, where inserting one option shifts every later ordinal and the lookup then
succeeds on the wrong option. Because a name-keyed payload drops the options this build
cannot place rather than being rejected wholesale, the schema fingerprint goes, and with
it the two fallbacks that existed only because an installed cache died on every app
upgrade: the second lookup tier into resources/profiles and the parse fallback to the
same place. A vendor is loaded from <data_dir>/system/ and nowhere else, as on main —
which is what makes the app write its .opc files there again.

* Simplify the preset cache internals after review

* Use the shared temp-dir helper in the preset bundle loading test

* Bound stamp string reads in the preset cache

* Speed up the setup wizard with a profile-data cache

The wizard's per-vendor fast path threw on vendors present only in
resources, falling back to a ~29 s raw JSON scan on every open. Each
vendor now loads from the directory it was found in, and the derived
model/machine/filament/process catalog is cached whole in
<data_dir>/cache/wizard_profile_data.json, stamped by each vendor's
name and version - a fresh cache makes an open one file read, with no
bundle built and no presets installed (~0.2 s vs ~2 s).

* Remove debug SVG dump from a geometry test

* Move the per-vendor cache file format into PresetCacheFormat

* Move the vendor install helpers from PresetBundle into Utils

* rename

* fix flatpak

* change cache version to 1

---------

Co-authored-by: SoftFever <softfeverever@gmail.com>
2026-08-21 16:56:52 +08:00

142 lines
5.2 KiB
Batchfile

@echo off
rem Build the per-vendor system preset caches (one <vendor>.opc per vendor) by
rem running the generate_system_cache.exe dev tool against a profiles directory,
rem and make every profiles directory named on the command line ship-ready:
rem install the caches into it and delete the preset JSONs they replace, so a
rem build ships one copy of its presets instead of two.
rem
rem scripts\build_preset_cache.bat [build_dir] [target_dir ...]
rem
rem build_dir defaults to "build"
rem target_dir profiles directories to ship into. Caches are generated into
rem the source tree's resources\profiles, which is what every
rem packaging step copies from; a target may be that same
rem directory, which then only gets pruned.
rem --prune-source
rem allow a target that is the directory the caches were generated
rem into (resources\profiles). Pruning it deletes the checkout's
rem own preset JSONs, which is a packaging step - not something a
rem build should do to a working tree by surprise. CI passes it.
rem
rem Shipping deletes, so it is a CI packaging step. A vendor's own <vendor>.json
rem goes along with its preset JSONs: the cache carries the vendor profile and
rem the version it was built at, so discovery, version checks and installing all
rem read it there. Only a vendor that has a cache is pruned, so non-vendor JSONs
rem (blacklist.json) are left alone, as are the vendor directories themselves -
rem thumbnails, covers and bed models still live there.
rem
rem set CONFIG=<cfg> to pin the build config for multi-config generators
rem (default: the config of the tool already in the build tree, else Release)
setlocal enabledelayedexpansion
set "REPO_ROOT=%~dp0.."
set "PRUNE_SOURCE="
:parse_flags
if /i "%~1"=="--prune-source" (
set "PRUNE_SOURCE=1"
shift
goto :parse_flags
)
set "BUILD_DIR=%~1"
if "%BUILD_DIR%"=="" set "BUILD_DIR=build"
if not exist "%BUILD_DIR%\" (
echo ERROR: build tree not found: %BUILD_DIR% 1>&2
exit /b 1
)
if not "%~1"=="" shift
rem Newest match wins: a stale binary silently produces a stale cache layout.
call :find_tool
if not defined CONFIG (
for %%c in (Debug Release RelWithDebInfo MinSizeRel) do (
echo !TOOL! | findstr /i "\\%%c\\" >nul && set "CONFIG=%%c"
)
)
if not defined CONFIG set "CONFIG=Release"
echo Building generate_system_cache in %BUILD_DIR% (%CONFIG%)
cmake --build "%BUILD_DIR%" --config %CONFIG% --target generate_system_cache
if errorlevel 1 (
echo ERROR: could not build generate_system_cache - configure the build tree with -DORCA_TOOLS=ON: 1>&2
echo cmake -S "%REPO_ROOT%" -B "%BUILD_DIR%" -DORCA_TOOLS=ON 1>&2
exit /b 1
)
call :find_tool
if not defined TOOL (
echo ERROR: generate_system_cache.exe not found under %BUILD_DIR% - build with -DORCA_TOOLS=ON 1>&2
exit /b 1
)
set "PROFILES=%REPO_ROOT%\resources\profiles"
if not exist "%PROFILES%\" (
echo ERROR: profiles directory not found: %PROFILES% 1>&2
exit /b 1
)
for %%d in ("%PROFILES%") do set "PROFILES=%%~fd"
rem Add the slicer's runtime DLL directory to PATH so generate_system_cache.exe
rem can resolve its dependencies (TKernel.dll etc.) without a full install step.
set "DLL_DIR="
for /f "delims=" %%f in ('dir /s /b "%BUILD_DIR%\TKernel.dll" 2^>nul') do (
if not defined DLL_DIR set "DLL_DIR=%%~dpf"
)
if defined DLL_DIR set "PATH=%DLL_DIR%;%PATH%"
echo Generating per-vendor preset caches in %PROFILES%
rem Start clean so vendors that went away - and caches written by older tool
rem versions - don't linger next to the freshly generated ones.
del /q "%PROFILES%\*.opc" 2>nul
del /q "%PROFILES%\*.cache" 2>nul
"%TOOL%" --path "%PROFILES%" --log_level 2
if errorlevel 1 exit /b %errorlevel%
:next_target
if "%~1"=="" exit /b 0
call :ship "%~1"
if errorlevel 1 exit /b 1
shift
goto :next_target
:ship
set "TARGET=%~1"
if not exist "%TARGET%\" (
echo ERROR: profiles directory not found: %TARGET% 1>&2
exit /b 1
)
for %%d in ("%TARGET%") do set "TARGET=%%~fd"
if /i "%TARGET%"=="%PROFILES%" if not defined PRUNE_SOURCE (
echo %TARGET%: skipped - this is where the caches were generated.
echo Pass --prune-source to prune it; that deletes this checkout's preset JSONs.
exit /b 0
)
if /i not "%TARGET%"=="%PROFILES%" copy /y "%PROFILES%\*.opc" "%TARGET%\" >nul
set /a SHIPPED=0
set /a PRUNED=0
for %%c in ("%PROFILES%\*.opc") do (
set /a SHIPPED+=1
set "VENDOR=%%~nc"
if exist "%TARGET%\!VENDOR!.json" (
del /q "%TARGET%\!VENDOR!.json"
set /a PRUNED+=1
)
if exist "%TARGET%\!VENDOR!\" (
for /f %%n in ('dir /s /b "%TARGET%\!VENDOR!\*.json" 2^>nul ^| find /c /v ""') do set /a PRUNED+=%%n
del /s /q "%TARGET%\!VENDOR!\*.json" >nul 2>&1
rem Deepest first, so a directory the delete above emptied goes too; rd
rem refuses the ones still holding covers or meshes.
for /f "delims=" %%d in ('dir /s /b /ad "%TARGET%\!VENDOR!" 2^>nul ^| sort /r') do rd "%%d" 2>nul
)
)
echo %TARGET%: !SHIPPED! caches, dropped !PRUNED! preset JSONs
exit /b 0
:find_tool
set "TOOL="
for /f "delims=" %%f in ('dir /s /b /o-d "%BUILD_DIR%\generate_system_cache.exe" 2^>nul') do (
if not defined TOOL set "TOOL=%%f"
)
exit /b 0