Cache system presets to eliminate startup and wizard load times (#14217)

* Add caching system for presets

* Removing user\bundle serialization and keeping it only for system presets

* Integrate caching into WebGuideDialog which speeds up time of SetupWizzard and PrinterSelection dialog

* Add CI\CD step to prepare cache file in ahead of time so user does not need to wait

* Add partial cache generation when only one of the vendros is changed to speed up recalculation time

* Handle corrupted files

* Add cache to GuideDialog as previos version didn't work as expected

* Add inspecting tool and fix CI cache generation

* Generate cache per vendor

* Simplify code by mergin it in PresetBundle

* Simplify code a bit more

* Add cereal serialize() to VendorProfile, PrinterModel, Preset, and Semver

* Remove CachedPrinterModel/VendorProfile/Preset mirror structs from VendorCache

* Fix use-after-free in CallAfter lambda; replace raw thread pointer with unique_ptr

* Use get_vendor_cache_key() to match cache keys written by the app

* Remove BOM added by VSC

* Skip invalid vendors

* Remove leftover cache file

* Fix build for windows arm64

* Revert json cache back

* Update check for stale cache

* Serealize all value fields for Preset class to minimize regression later

* Minimize field duplication by moving Cache thing into PresetBundle

* Add tests for Cache system

* Add a bit more tests

* Merge branch 'main' into feature/cache_profiles_and_optimize_loading_speed

* Rvert from per-verndor to single cache file

Replace N per-vendor .cache files with a single system_presets.cache
that holds all vendors and presets in one serialized blob.

Cache load is now all-or-nothing: on hit all vendors are applied from
the bundle (sub-second); on miss all vendors are parsed from JSON and
a fresh bundle is written to the user cache dir.

Invalidation is driven by bundle_key - a sorted concatenation of all
vendor JSON version strings. Any vendor update invalidates the whole
cache and triggers re-parse on next launch.

Guide wizard (WebGuideDialog) loads the bundled cache into a plain
PresetBundle instead of a separate VendorGuideData struct, removing
the duplicate data model.

generate_system_cache simplified from a per-vendor loop to a single
save_system_presets_cache() call producing one output file.

* Transfer all Preset fields from cache via move assignmet

apply_vendor_preset_group was copying fields manually and missed
bundle_id, user_id, base_id, sync_info, updated_time, key_values,
ini_str. Replace field-by-field copy with move assignment of the
fully-deserialized Preset, then restore the vendor pointer which
is excluded from serialization.

* Ignore cache for future

* Remove not used files

* Ship one preset cache per vendor in place of the profile JSONs

Each vendor's system presets serialize into a single <vendor>.opc built at
package time, and a shipped build carries that file alone — the profile JSON
and its sub-file tree are pruned. The vendor loader, the setup wizard's profile
list and the resource installer all read a vendor through its cache, falling
back to parsing whenever one is absent, stale or unreadable, so the cache stays
an optimization and never a source of truth. Caches hold presets in source form
and resolve inheritance at load, through the same code the JSON path uses.

* Make the preset cache self-describing and load each vendor from the system folder alone

The cached DynamicPrintConfig is keyed by name, through a per-file dictionary of the
distinct opt_keys, the type each was written as, and the distinct enum value names,
instead of by serialization_key_ordinal — a position assigned by declaration order at
static init, where inserting one option shifts every later ordinal and the lookup then
succeeds on the wrong option. Because a name-keyed payload drops the options this build
cannot place rather than being rejected wholesale, the schema fingerprint goes, and with
it the two fallbacks that existed only because an installed cache died on every app
upgrade: the second lookup tier into resources/profiles and the parse fallback to the
same place. A vendor is loaded from <data_dir>/system/ and nowhere else, as on main —
which is what makes the app write its .opc files there again.

* Simplify the preset cache internals after review

* Use the shared temp-dir helper in the preset bundle loading test

* Bound stamp string reads in the preset cache

* Speed up the setup wizard with a profile-data cache

The wizard's per-vendor fast path threw on vendors present only in
resources, falling back to a ~29 s raw JSON scan on every open. Each
vendor now loads from the directory it was found in, and the derived
model/machine/filament/process catalog is cached whole in
<data_dir>/cache/wizard_profile_data.json, stamped by each vendor's
name and version - a fresh cache makes an open one file read, with no
bundle built and no presets installed (~0.2 s vs ~2 s).

* Remove debug SVG dump from a geometry test

* Move the per-vendor cache file format into PresetCacheFormat

* Move the vendor install helpers from PresetBundle into Utils

* rename

* fix flatpak

* change cache version to 1

---------

Co-authored-by: SoftFever <softfeverever@gmail.com>

src/libslic3r/Utils.hpp

@@ -3,6 +3,7 @@
 
 #include <iomanip>
 #include <locale>
+#include <set>
 #include <utility>
 #include <functional>
 #include <type_traits>
@@ -18,6 +19,7 @@
 #include <openssl/md5.h>
 
 #include "libslic3r.h"
+#include "Semver.hpp"
 
 //define CLI errors
 
@@ -722,11 +724,42 @@ void copy_directory_recursively(const boost::filesystem::path& source,
                                 std::function<bool(const std::string)> filter = nullptr,
                                 bool merge_mode                               = false);
 
-// Install vendor bundles from resources directory to data directory
-// bundle_names: vector of vendor bundle names (without .json extension)
-// resource_subdir: subdirectory under resources_dir() (default: "profiles")
-// data_subdir: subdirectory under data_dir() (default: "system")
-// Returns: true if all bundles installed successfully, false otherwise
+// ---- Vendor installation on disk ------------------------------------------
+// How a vendor bundle is installed from resources into data_dir()/system: as
+// its profile and preset JSONs or, in a build that ships preset caches, as its
+// .opc preset cache alone. Loading what is installed is PresetBundle's business;
+// the cache file format itself is VendorCacheFile's (PresetCacheFormat.hpp).
+
+// True if `vendor` is installed in data_dir()/system. A build that ships preset
+// caches installs the cache alone, so it — not the profile — marks a vendor
+// installed; a cache this build cannot read marks nothing.
+bool is_vendor_installed(const std::string& vendor);
+
+// The version the installed vendor would be loaded at: its cache's stamp while
+// that covers the profile beside it, the profile's own version once it does not.
+// Invalid Semver if neither form is installed.
+Semver installed_vendor_version(const std::string& vendor);
+
+// Remove every form `vendor` can be installed as from data_dir()/system: its
+// profile, its preset cache, and its preset directory.
+void remove_installed_vendor(const std::string& vendor);
+
+// The vendors `dir` holds, sorted: one is named by its profile or, in a build that
+// ships preset caches instead of the raw profile JSONs, by its cache alone.
+std::set<std::string> vendor_names_in(const boost::filesystem::path& dir);
+
+// The version a build ships `vendor` at: whichever of its preset cache and its
+// profile is newer, that being the one installing lays down. Invalid Semver if the
+// build ships neither.
+Semver resource_vendor_version(const std::string& vendor);
+
+// Install vendors from the resources directory into the data directory, each as
+// its preset cache or as its profile and preset JSONs — whichever of the two the
+// build ships at the newer version. Anything the previous install of that vendor
+// left behind goes, so only the form just installed is there to be loaded.
+// bundle_names: vendor names, without extension.
+// Every bundle that can be installed is, whatever the others do. Returns false
+// if any named bundle could not be installed.
 bool install_vendor_bundles_from_resources(const std::vector<std::string>& bundle_names,
                                            const std::string& resource_subdir = "profiles",
                                            const std::string& data_subdir     = "system");