mirror of
https://github.com/OrcaSlicer/OrcaSlicer.git
synced 2026-09-30 20:31:09 +00:00
Cache system presets to eliminate startup and wizard load times (#14217)
* Add caching system for presets * Removing user\bundle serialization and keeping it only for system presets * Integrate caching into WebGuideDialog which speeds up time of SetupWizzard and PrinterSelection dialog * Add CI\CD step to prepare cache file in ahead of time so user does not need to wait * Add partial cache generation when only one of the vendros is changed to speed up recalculation time * Handle corrupted files * Add cache to GuideDialog as previos version didn't work as expected * Add inspecting tool and fix CI cache generation * Generate cache per vendor * Simplify code by mergin it in PresetBundle * Simplify code a bit more * Add cereal serialize() to VendorProfile, PrinterModel, Preset, and Semver * Remove CachedPrinterModel/VendorProfile/Preset mirror structs from VendorCache * Fix use-after-free in CallAfter lambda; replace raw thread pointer with unique_ptr * Use get_vendor_cache_key() to match cache keys written by the app * Remove BOM added by VSC * Skip invalid vendors * Remove leftover cache file * Fix build for windows arm64 * Revert json cache back * Update check for stale cache * Serealize all value fields for Preset class to minimize regression later * Minimize field duplication by moving Cache thing into PresetBundle * Add tests for Cache system * Add a bit more tests * Merge branch 'main' into feature/cache_profiles_and_optimize_loading_speed * Rvert from per-verndor to single cache file Replace N per-vendor .cache files with a single system_presets.cache that holds all vendors and presets in one serialized blob. Cache load is now all-or-nothing: on hit all vendors are applied from the bundle (sub-second); on miss all vendors are parsed from JSON and a fresh bundle is written to the user cache dir. Invalidation is driven by bundle_key - a sorted concatenation of all vendor JSON version strings. Any vendor update invalidates the whole cache and triggers re-parse on next launch. Guide wizard (WebGuideDialog) loads the bundled cache into a plain PresetBundle instead of a separate VendorGuideData struct, removing the duplicate data model. generate_system_cache simplified from a per-vendor loop to a single save_system_presets_cache() call producing one output file. * Transfer all Preset fields from cache via move assignmet apply_vendor_preset_group was copying fields manually and missed bundle_id, user_id, base_id, sync_info, updated_time, key_values, ini_str. Replace field-by-field copy with move assignment of the fully-deserialized Preset, then restore the vendor pointer which is excluded from serialization. * Ignore cache for future * Remove not used files * Ship one preset cache per vendor in place of the profile JSONs Each vendor's system presets serialize into a single <vendor>.opc built at package time, and a shipped build carries that file alone — the profile JSON and its sub-file tree are pruned. The vendor loader, the setup wizard's profile list and the resource installer all read a vendor through its cache, falling back to parsing whenever one is absent, stale or unreadable, so the cache stays an optimization and never a source of truth. Caches hold presets in source form and resolve inheritance at load, through the same code the JSON path uses. * Make the preset cache self-describing and load each vendor from the system folder alone The cached DynamicPrintConfig is keyed by name, through a per-file dictionary of the distinct opt_keys, the type each was written as, and the distinct enum value names, instead of by serialization_key_ordinal — a position assigned by declaration order at static init, where inserting one option shifts every later ordinal and the lookup then succeeds on the wrong option. Because a name-keyed payload drops the options this build cannot place rather than being rejected wholesale, the schema fingerprint goes, and with it the two fallbacks that existed only because an installed cache died on every app upgrade: the second lookup tier into resources/profiles and the parse fallback to the same place. A vendor is loaded from <data_dir>/system/ and nowhere else, as on main — which is what makes the app write its .opc files there again. * Simplify the preset cache internals after review * Use the shared temp-dir helper in the preset bundle loading test * Bound stamp string reads in the preset cache * Speed up the setup wizard with a profile-data cache The wizard's per-vendor fast path threw on vendors present only in resources, falling back to a ~29 s raw JSON scan on every open. Each vendor now loads from the directory it was found in, and the derived model/machine/filament/process catalog is cached whole in <data_dir>/cache/wizard_profile_data.json, stamped by each vendor's name and version - a fresh cache makes an open one file read, with no bundle built and no presets installed (~0.2 s vs ~2 s). * Remove debug SVG dump from a geometry test * Move the per-vendor cache file format into PresetCacheFormat * Move the vendor install helpers from PresetBundle into Utils * rename * fix flatpak * change cache version to 1 --------- Co-authored-by: SoftFever <softfeverever@gmail.com>
This commit is contained in:
ExPikaPaka
@@ -1,2 +1,7 @@
|
||||
# Set the default behavior, in case people don't have core.autocrlf set.
|
||||
* text=auto
|
||||
|
||||
# Shell scripts are run by Git Bash on Windows CI, which cannot read a script
|
||||
# with CRLF line endings: it fails on the first line. Windows checkouts default
|
||||
# to core.autocrlf=true, so keep these LF whatever the platform.
|
||||
*.sh text eol=lf
|
||||
|
||||
@@ -14,6 +14,7 @@ on:
|
||||
- 'localization/**'
|
||||
- 'resources/**'
|
||||
- ".github/workflows/build_*.yml"
|
||||
- 'scripts/build_preset_cache.*'
|
||||
- 'scripts/flatpak/**'
|
||||
- 'scripts/msix/**'
|
||||
- 'tests/**'
|
||||
@@ -33,6 +34,7 @@ on:
|
||||
- 'build_release_vs.bat'
|
||||
- 'build_release_vs2022.bat'
|
||||
- 'build_release_macos.sh'
|
||||
- 'scripts/build_preset_cache.*'
|
||||
- 'scripts/flatpak/**'
|
||||
- 'scripts/msix/**'
|
||||
- 'tests/**'
|
||||
|
||||
@@ -162,6 +162,14 @@ jobs:
|
||||
retention-days: 5
|
||||
if-no-files-found: error
|
||||
|
||||
- name: Build system preset cache (macOS)
|
||||
if: runner.os == 'macOS' && !inputs.macos-combine-only
|
||||
working-directory: ${{ github.workspace }}
|
||||
shell: bash
|
||||
# The bundle was already packed from resources/, so the caches have to be
|
||||
# installed into it here; the source tree keeps its JSONs for later jobs.
|
||||
run: ./scripts/build_preset_cache.sh -b build/${{ inputs.arch }} build/${{ inputs.arch }}/OrcaSlicer/OrcaSlicer.app/Contents/Resources/profiles
|
||||
|
||||
- name: Pack macOS app bundle ${{ inputs.arch }}
|
||||
if: runner.os == 'macOS' && !inputs.macos-combine-only
|
||||
working-directory: ${{ github.workspace }}
|
||||
@@ -390,6 +398,13 @@ jobs:
|
||||
if ($arch -eq "arm64") { .\build_release_vs.bat slicer arm64 tests } else { .\build_release_vs.bat slicer tests }
|
||||
shell: pwsh
|
||||
|
||||
- name: Build system preset cache (Windows)
|
||||
if: runner.os == 'Windows'
|
||||
shell: cmd
|
||||
# Shipped into both the already-installed tree (portable zip, MSIX) and
|
||||
# the checkout cpack re-installs from when it builds the NSIS installer.
|
||||
run: scripts\build_preset_cache.bat --prune-source "%BUILD_DIR%" "resources\profiles" "%BUILD_DIR%\OrcaSlicer\resources\profiles"
|
||||
|
||||
- name: Pack unit tests Win
|
||||
if: runner.os == 'Windows'
|
||||
working-directory: ${{ github.workspace }}
|
||||
@@ -539,6 +554,20 @@ jobs:
|
||||
retention-days: 5
|
||||
if-no-files-found: error
|
||||
|
||||
- name: Build system preset cache (Linux)
|
||||
if: runner.os == 'Linux'
|
||||
shell: bash
|
||||
run: |
|
||||
# Both were packed from resources/ before the caches existed, so the
|
||||
# AppImage is unpacked first and the caches shipped into it and into
|
||||
# the package tree; the source tree keeps its JSONs for later steps.
|
||||
appimage=$(find build -maxdepth 1 -name "OrcaSlicer_Linux_AppImage*.AppImage" | head -1)
|
||||
chmod +x "$appimage"
|
||||
"$appimage" --appimage-extract
|
||||
./scripts/build_preset_cache.sh -b build build/package/resources/profiles squashfs-root/resources/profiles
|
||||
appimagetool=$(find build -name "appimagetool.AppImage" | head -1)
|
||||
ARCH=$(uname -m) "$appimagetool" --appimage-extract-and-run squashfs-root "$appimage"
|
||||
rm -rf squashfs-root
|
||||
# Ship the freshly-built validator so slice_check_linux (build_all.yml)
|
||||
# can slice-sweep the shipped profiles with this PR's engine. Taken from
|
||||
# the aarch64 leg so the sweep also exercises the arm build; x86_64 on
|
||||
|
||||
@@ -49,3 +49,4 @@ internal_docs/
|
||||
# Python bytecode
|
||||
__pycache__/
|
||||
*.pyc
|
||||
*.opc
|
||||
|
||||
@@ -567,6 +567,8 @@ if [[ -n "${BUILD_ORCA}" ]] || [[ -n "${BUILD_TESTS}" ]] ; then
|
||||
print_and_run cmake --build $BUILD_DIR --config "${BUILD_CONFIG}" --target OrcaSlicer
|
||||
echo "Building OrcaSlicer_profile_validator .."
|
||||
print_and_run cmake --build $BUILD_DIR --config "${BUILD_CONFIG}" --target OrcaSlicer_profile_validator
|
||||
echo "Building generate_system_cache ..."
|
||||
print_and_run cmake --build $BUILD_DIR --config "${BUILD_CONFIG}" --target generate_system_cache
|
||||
./scripts/run_gettext.sh
|
||||
fi
|
||||
if [[ -n "${BUILD_TESTS}" ]] ; then
|
||||
|
||||
@@ -0,0 +1,402 @@
|
||||
# System Preset Cache — High Level Design
|
||||
|
||||
## Why it exists
|
||||
|
||||
OrcaSlicer ships tens of thousands of system preset JSON files. Every launch used to
|
||||
parse all of them: read each vendor profile, walk its machine, process and filament
|
||||
sub-files, resolve inheritance, and build the preset collections from scratch. That
|
||||
parse dominated startup, and it produced the same result every time, because system
|
||||
presets only change when the app is updated or a profile update is installed.
|
||||
|
||||
The preset cache replaces that parse with a read. Each vendor's presets are serialized
|
||||
once — at build time, in CI — into a single binary file the app reads in one pass. The
|
||||
read replaces the file walk and the JSON parsing, which is where the time went;
|
||||
resolving inheritance and registering the presets still runs at load, through the same
|
||||
code the JSON path uses, so the result is the parse's result without the parse.
|
||||
|
||||
The cache is **only ever an optimization**. Every rule below exists to guarantee that a
|
||||
cache is either provably equivalent to parsing the JSONs, or rejected. There is no
|
||||
"mostly right" cache.
|
||||
|
||||
## The unit is one vendor
|
||||
|
||||
A cache covers exactly one vendor. `BBL.opc` sits beside `BBL.json` and holds
|
||||
everything `BBL.json` and the `BBL/` sub-file tree would have produced.
|
||||
|
||||
Per-vendor granularity is what makes the system practical:
|
||||
|
||||
- A vendor whose profile is bumped invalidates only its own cache. The other 60-odd
|
||||
vendors keep theirs — even when the bumped vendor is the shared Orca filament
|
||||
library everyone else inherits from.
|
||||
- The setup wizard, which loads vendors one at a time, gets the same speedup as
|
||||
startup without a second code path.
|
||||
- A vendor with no cache, or a broken one, costs only that vendor a parse.
|
||||
|
||||
A cache holds *system* presets only. User presets, project settings and modified
|
||||
presets are never serialized — they have their own storage and their own lifecycle.
|
||||
|
||||
## Where the files live
|
||||
|
||||
| Location | Contents on a shipped build | Role |
|
||||
|---|---|---|
|
||||
| `resources/profiles/` | `<vendor>.opc` alone — the profile and its preset JSONs both pruned | What the app ships with; what installing copies from, and the only thing it is read for |
|
||||
| `<data_dir>/system/` | `<vendor>.opc` alone, or `<vendor>.json` + `<vendor>/` after an update | What the user has installed |
|
||||
| `<data_dir>/system/` (dev build) | `<vendor>.json` + `<vendor>/` + `<vendor>.opc` written at runtime | A developer tree caches as it parses |
|
||||
| `<data_dir>/cache/wizard_profile_data.json` | The wizard's derived vendor catalog plus the stamps it was built from | Written and read by the setup wizard only; never shipped (see "The wizard's profile-data cache") |
|
||||
|
||||
Two forms of the same vendor therefore exist, and the system's central rule is that
|
||||
**a vendor's cache is the whole of it**. Where a cache ships or is installed, no profile
|
||||
and no preset JSONs sit beside it: the cache carries the presets, the vendor profile,
|
||||
and the version stamp that says which release it came from. A vendor is "installed" if
|
||||
either form is present *and usable*, and its installed version is read from whichever
|
||||
form a load would serve.
|
||||
|
||||
What stays beside the caches in `resources/profiles/` is everything that is not a
|
||||
preset: each vendor's directory of printer thumbnails, cover images, bed models and
|
||||
hotend meshes, which are read from disk by path and were never part of the cache. Files
|
||||
that are not vendors at all, `blacklist.json` chief among them, are untouched.
|
||||
|
||||
The alternative — shipping both and treating the cache as a sidecar — was rejected. It
|
||||
doubles the installed size, and it creates a class of bug where the two disagree and
|
||||
the app's behavior depends on which one a given code path happened to read.
|
||||
|
||||
## What a cache file is
|
||||
|
||||
A fixed-size header followed by one binary stream.
|
||||
|
||||
The header carries a magic number, the cache format version, the payload size and a
|
||||
CRC32 of the payload. It exists so that a truncated download, a half-written file or a
|
||||
file from an entirely different program is rejected in microseconds, before anything
|
||||
tries to interpret it.
|
||||
|
||||
The payload opens with the stamps that decide whether the cache may be used at all —
|
||||
format version, vendor name, vendor version — then a dictionary, and then the vendor's
|
||||
data: its vendor profile, three lists of preset entries (process, filament, machine),
|
||||
and the count of errors the original parse hit.
|
||||
|
||||
Each entry is one preset **in source form**: what its JSON sub-file states and nothing
|
||||
that resolving it derives — the preset's own config diff, the name of the preset it
|
||||
inherits, and the parse metadata (name, sub-path, description, instantiation, setting
|
||||
and filament ids, renames). Non-instantiated base presets are stored too; the children
|
||||
that inherit from them cannot resolve without them.
|
||||
|
||||
**The payload names its own keys.** The dictionary holds the distinct `opt_key`s the
|
||||
file uses, the `ConfigOptionType` each was written as, and the distinct enum *value
|
||||
names*; an option in an entry's config is then a `uint16` index into that dictionary
|
||||
plus its value. Names are written once per file rather than once per occurrence, and a
|
||||
reader resolves the dictionary against this build's `print_config_def` once, after
|
||||
which reading an option is a vector index.
|
||||
|
||||
This is what makes the cache survive config-schema drift. The alternative — keying an
|
||||
option by its `serialization_key_ordinal`, the position `ConfigDef::add` assigns by
|
||||
declaration order at static init — cannot: inserting one option into the middle of
|
||||
`PrintConfig.cpp` shifts every later ordinal, and the lookup on the way back in then
|
||||
*succeeds on the wrong option*, silently, wherever the two share a type. Because a
|
||||
name-keyed payload instead drops the individual options this build cannot place, the
|
||||
file as a whole stays readable, and there is no schema fingerprint — no checksum over
|
||||
the option schema that would reject every cache on every release. An option this build
|
||||
no longer defines, or now defines with a different type, gets exactly what it gets from
|
||||
a JSON profile: read, dropped, and the rest of the preset loads.
|
||||
|
||||
The ordinal-keyed cereal hooks in `PrintConfig.hpp` are untouched — they are also the
|
||||
undo/redo wire format, where the process cannot change underneath them. The cache has
|
||||
its own serialization in `PresetCacheFormat.{hpp,cpp}`.
|
||||
|
||||
Three deliberate choices in the layout:
|
||||
|
||||
- **Stamps come first**, so the question "what version is this vendor installed at?"
|
||||
can be answered by reading the first kilobyte. The updater asks that question for
|
||||
every vendor on every launch; reading tens of megabytes to answer it would give back
|
||||
the startup time the cache saved. The dictionary sits behind them, ahead of the
|
||||
entries, so a reader that does go on resolves it once and then indexes.
|
||||
- **Nothing inherited is baked in.** A filament preset that inherits from the shared
|
||||
library is stored as its own diff plus its parent's name, and the parent is looked up
|
||||
when the entry is installed, against whatever library is loaded then. A cache
|
||||
therefore carries no other vendor's values, and no other vendor's update — the
|
||||
library's included — can make it stale.
|
||||
- **Nothing derived is stored.** Default presets, flattened configs, aliases and
|
||||
lookup maps are all reconstructed at load by the same code the JSON path runs, and
|
||||
state that path never fills (obsolete-preset lists) is not stored either. This keeps
|
||||
the cache a record of the vendor's data, not a memory image of the program's state.
|
||||
|
||||
## When a cache may be used
|
||||
|
||||
A cache is accepted only if every gate below passes. Any failure means "parse the
|
||||
JSONs instead" — never a hard error, never a partial load.
|
||||
|
||||
**1. Integrity.** Magic number, a declared body size that is exactly the rest of the
|
||||
file, CRC32 over the payload. The size is checked against the file's real length before
|
||||
anything is allocated on the strength of it, so an eight-byte field in an unauthenticated
|
||||
file cannot ask for a gigabyte.
|
||||
|
||||
**2. Cache format version.** A single integer bumped by hand whenever the binary layout
|
||||
changes in a way nothing else would catch: reordering or retyping a hand-written
|
||||
serialized field, or changing what the cache's own stamps mean. Config-schema drift is
|
||||
explicitly *not* such a change — the dictionary handles it — so this no longer moves
|
||||
every release.
|
||||
|
||||
**3. Vendor identity and version.** The cache names the vendor it holds and the profile
|
||||
version it was built from. It is accepted only if that version is at least as new as
|
||||
the profile now on disk. Where no profile sits beside the cache — the shipped,
|
||||
cache-only form — the comparison is skipped, because nothing on disk can be newer than
|
||||
a cache that is the installation.
|
||||
|
||||
**4. Every entry installs.** Entries are installed as they are read, and an entry that
|
||||
cannot be — typically one that inherits a parent the currently loaded filament library
|
||||
no longer provides — rejects the whole cache, never just the entry. A partial vendor is
|
||||
not a vendor.
|
||||
|
||||
There is deliberately no stamp for the shared filament library. A cache stores its
|
||||
filaments' inheritance by name and resolves it at load, so a library update changes
|
||||
what a cache load *produces*, never whether the cache is *valid* — the same file yields
|
||||
the updated result. This matters most on a shipped build, where a vendor is its cache
|
||||
and nothing else: a profile update that delivered only the library would otherwise have
|
||||
stranded every other vendor with a cache it invalidated and no JSONs to fall back on.
|
||||
|
||||
A vendor profile with no parsable version is never cached and never served from a
|
||||
cache. There would be no way to tell later whether the cache had gone stale, and a
|
||||
cache nothing can invalidate is worse than no cache.
|
||||
|
||||
## How a vendor is loaded
|
||||
|
||||
Vendors load in a fixed order, because filament inheritance crosses exactly one
|
||||
boundary: any vendor's filament may inherit from the shared Orca filament library,
|
||||
and nothing else reaches across vendors. The library therefore goes first, alone;
|
||||
every other vendor follows in parallel, resolving against it; and the results are
|
||||
merged in a stable order:
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
lib["1 · OrcaFilamentLibrary<br/>loaded first, synchronously"] --> par["2 · every other vendor in parallel,<br/>each into its own bundle, filaments<br/>resolving against the loaded library"] --> merge["3 · bundles merged into one,<br/>sequentially, in stable vendor order"]
|
||||
```
|
||||
|
||||
Whether a vendor comes from its cache or from a parse changes nothing in that
|
||||
order — both produce the same bundle, so cached and parsed vendors mix freely in
|
||||
one startup.
|
||||
|
||||
**A vendor is loaded from where it is installed and nowhere else.** For startup that
|
||||
is `<data_dir>/system/`; resources reaches the app by being *installed* into that
|
||||
directory first, never by being loaded from. (The setup wizard is the one caller with
|
||||
a different notion of "where": it also shows vendors the user has not installed, and
|
||||
loads those from `resources/profiles` — see "The wizard's profile-data cache".) There
|
||||
is one lookup tier and one parse source:
|
||||
|
||||
```
|
||||
load vendor V from <data_dir>/system:
|
||||
system/V.opc passes CACHE_VERSION + size + CRC + vendor name + version gate?
|
||||
yes -> serve from it
|
||||
no -> parse system/V.json, then write system/V.opc back
|
||||
```
|
||||
|
||||
The same decision drawn out — "the gates" are the four acceptance checks above:
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
start["load vendor V from a directory dir<br/>— normally <data_dir>/system/"]
|
||||
start --> stamp["installed version = version of dir/V.json<br/>— or ∞ with no profile there,<br/>the cache then being the installation"]
|
||||
stamp --> g1{"dir/V.opc<br/>passes all four gates?"}
|
||||
g1 -- "yes" --> hit(["served from the<br/>installed cache"])
|
||||
g1 -- "no" --> pd["parse the JSONs in dir"]
|
||||
pd --> ver{"profile version<br/>parsable?"}
|
||||
ver -- "yes" --> save(["loaded; dir/V.opc written back —<br/>the next load takes the top path"])
|
||||
ver -- "no" --> raw(["loaded, never cached"])
|
||||
```
|
||||
|
||||
A second tier into `resources/profiles/` used to sit between those two, and a parse
|
||||
fallback to the same place behind them. Both existed only because an installed cache
|
||||
died on every app upgrade, when the schema fingerprint rejected it; with the fingerprint
|
||||
gone there is nothing for them to rescue. They also had a cost: on a developer tree the
|
||||
shipped cache answered first, so the profile in `<data_dir>/system/` was never parsed
|
||||
and its cache was never written back.
|
||||
|
||||
Serving from a cache is not a memory-image restore. The entries are deserialized and
|
||||
then installed one by one — inheritance resolved against the presets installed before
|
||||
them and the currently loaded filament library, configs flattened onto the collection
|
||||
defaults, validated and registered — by the same function the JSON path calls straight
|
||||
after parsing a sub-file. The two paths share everything below the parse, which is what
|
||||
makes a cache-loaded bundle indistinguishable from a JSON-loaded one by construction
|
||||
rather than by test coverage. Installation also rebuilds each preset's file path from
|
||||
the local data directory, so a shipped cache never carries the generating machine's
|
||||
paths.
|
||||
|
||||
App upgrades work because a cache normally survives one. Only a deliberate
|
||||
`CACHE_VERSION` bump makes an installed cache unreadable, and that is handled at
|
||||
install time rather than at load: a vendor whose cache this build cannot read counts
|
||||
as **not installed**, so the updater lays down a working copy on the next launch (see
|
||||
below). A vendor that still has its profile JSONs beside the cache is simply parsed
|
||||
and re-cached.
|
||||
|
||||
If a parse does happen and the vendor's profile carries a version, the app writes the
|
||||
cache back beside where it looked for the vendor. That is how a developer build warms
|
||||
itself up on second launch, and how a vendor delivered by a profile update becomes
|
||||
cached without waiting for the next release.
|
||||
|
||||
## The wizard's profile-data cache
|
||||
|
||||
The setup wizard's printer and filament pages want every vendor in one bundle — the
|
||||
installed ones *and* the shipped ones the user has not installed yet, because the
|
||||
wizard is where installing is chosen. Its set therefore spans two directories:
|
||||
`<data_dir>/system/` for installed vendors (shadowing resources on a name collision),
|
||||
`resources/profiles` for the rest, each vendor loaded from its own directory.
|
||||
|
||||
What the wizard actually consumes from that bundle is one derived JSON — the model /
|
||||
machine / filament / process catalog its web pages render — and that JSON is a pure
|
||||
function of the vendor set: each vendor's name and version, in load order. A profile
|
||||
change requires a version bump, so name and version determine a vendor's content
|
||||
wherever its copy sits; which directory served it is deliberately **not** stamped,
|
||||
and installing or removing a copy at an unchanged version leaves the cache valid. So
|
||||
the wizard caches the *derived JSON*, not another form of the inputs:
|
||||
`<data_dir>/cache/wizard_profile_data.json` holds the stamp list and the catalog. On
|
||||
open, the wizard computes the current stamps (one version peek per vendor) and, when
|
||||
they match, serves the catalog from the file — no bundle built, no preset installed.
|
||||
Caching bundle inputs instead was tried and measured: rebuilding the bundle from
|
||||
per-vendor caches costs ~2 s of preset installation whatever feeds it, so only
|
||||
skipping the rebuild entirely wins.
|
||||
|
||||
Any change to the set — a vendor added, removed or updated, or its cache-only
|
||||
`.opc` replaced by a newer one — changes the stamps and retires the whole file;
|
||||
the wizard then rebuilds the bundle vendor by vendor (per-vendor caches serving where
|
||||
they cover) and writes the catalog back. Selections, region and per-open decorations
|
||||
are applied downstream of the cache either way, so a served catalog is
|
||||
indistinguishable from a rebuilt one. Nothing ships this file and the updater never
|
||||
touches it; it is a locally written artifact, re-derived whenever stale, written
|
||||
through a temp file and rename so half a cache is never readable.
|
||||
|
||||
The cache lives under `<data_dir>/cache/`, not beside the vendors: everything that
|
||||
scans `<data_dir>/system/` treats any `.opc` there as a vendor, so a non-vendor
|
||||
cache file must not sit in that directory. Relatedly, the stamp reader is hardened:
|
||||
`read_cache_stamps` validates the cache version before reading anything
|
||||
variable-length and bounds the stamp strings' lengths, so a reader pointed at a
|
||||
foreign or damaged `.opc` rejects it cleanly instead of aborting on a garbage
|
||||
64-bit allocation.
|
||||
|
||||
## How a vendor is installed
|
||||
|
||||
Installing copies from `resources/profiles/` into `<data_dir>/system/`. A shipped build
|
||||
offers only a cache and a source tree only JSONs, but a partially-generated tree can
|
||||
have both, at different versions, so the installer picks the form that ships at the
|
||||
**newer version** and installs only that one:
|
||||
|
||||
- Cache newer or equal, and readable → copy the `.opc`, verify the *copy* is one this
|
||||
build can read, and only then delete any profile and vendor directory a previous
|
||||
install left behind, so nothing can shadow it.
|
||||
- Profile newer, or the cache unreadable or absent → copy the profile and the vendor's
|
||||
preset JSONs exactly as the app did before caches existed, and delete any stale `.opc`
|
||||
once the profile is safely in place.
|
||||
|
||||
One vendor that cannot be installed is one vendor missing, not a reason to leave the
|
||||
rest uninstalled: the installer skips it, records the failure, and carries on with the
|
||||
batch. A vendor whose cache arrives unreadable falls back to installing its profile,
|
||||
which is decided by reading the copy rather than by the kilobyte peek that chose the
|
||||
form.
|
||||
|
||||
**"Installed" means present and usable.** Where the cache is the whole of a vendor's
|
||||
installation, a `.opc` this build cannot read is not an installation — counted as one,
|
||||
the vendor would be stranded with nothing to load and the updater would never repair
|
||||
it. The installed version is likewise whichever form a load would actually serve: the
|
||||
cache's stamp while it covers the profile beside it, the profile's own version once it
|
||||
does not.
|
||||
|
||||
The result is that only one form of a vendor is ever present, and it is the newest one
|
||||
the build has. This matters most for the update check, which compares what is installed
|
||||
against what installing *would* lay down: if those two disagreed about which form
|
||||
counts, a vendor could reinstall on every launch forever, or silently never update.
|
||||
|
||||
Profile updates delivered over the air always arrive as JSONs, and they win — an
|
||||
updated vendor's real profile lands in the data directory, the installed cache beside it
|
||||
is older and gets rejected, and the vendor is parsed and re-cached. An update that touches only
|
||||
the filament library needs nothing more: every other vendor's cache stays valid and
|
||||
simply resolves against the new library on its next load.
|
||||
|
||||
## How the caches are produced
|
||||
|
||||
Cache generation is a build step, not something a user ever runs.
|
||||
|
||||
One script per platform does the whole job, and CI calls it once on each. It builds a
|
||||
small dev-utility that loads a profiles directory exactly as the app would, with cache
|
||||
writing enabled, dropping a `<vendor>.opc` beside every vendor profile it parses; then
|
||||
it copies those caches into each packaged application it was pointed at and deletes
|
||||
every preset JSON they replace — the vendor's own profile included. Only a vendor that
|
||||
actually has a cache is pruned, so a vendor the generator skipped keeps its JSONs and is
|
||||
simply parsed at startup.
|
||||
|
||||
Caches are generated into the checkout's own `resources/profiles`, because that is what
|
||||
cpack re-installs from when it builds the NSIS installer — so that directory is also a
|
||||
prune target in CI. Pruning it deletes the checkout's preset JSONs, which is a packaging
|
||||
step, not something a build should do to a working tree by surprise: the Windows script
|
||||
refuses that target unless given `--prune-source`, and CI passes it.
|
||||
|
||||
Generation runs after the build, in the same job, so the caches ship with a build that
|
||||
can read them.
|
||||
|
||||
The flatpak differs only in where the script is called from. Nothing outside
|
||||
flatpak-builder ever builds it, so there is no packaged tree for the workflow to point
|
||||
the script at afterwards: the manifest runs it as a build step instead, against the
|
||||
profiles the install has already copied into `/app`.
|
||||
|
||||
## Behavior when things go wrong
|
||||
|
||||
The system is designed so that no cache problem is fatal:
|
||||
|
||||
- **Corrupt, truncated or foreign file** — rejected at the header, vendor parsed. A
|
||||
cache is written to a temp file beside its target and moved into place, so a write
|
||||
that dies partway leaves the previous cache intact rather than a truncated one.
|
||||
- **An option this build no longer has, or now types differently** — that option alone
|
||||
is dropped, exactly as a JSON profile's would be. The preset and the file load.
|
||||
- **Cache from a build with a different cache layout** — rejected on `CACHE_VERSION`.
|
||||
A vendor with JSONs beside it is parsed and re-cached; a cache-only vendor reads as
|
||||
not installed and the updater reinstalls it.
|
||||
- **Stale cache** — rejected on the vendor version stamp, vendor parsed and re-cached.
|
||||
- **Failure part-way through loading** — a deserialization error, or any entry that
|
||||
fails to install — rejects the whole cache, and the bundle is reset to a clean state
|
||||
before falling back, so a half-loaded cache can never leak into the parsed result.
|
||||
- **A vendor that can be neither read nor parsed** — logged, and left out. The setup
|
||||
wizard drops that vendor from its list and opens with the rest; startup records the
|
||||
error alongside the vendors that did load. One broken vendor never takes the app down.
|
||||
|
||||
The one genuine limit: on a shipped build a vendor is its cache and nothing else, so a
|
||||
rejected cache has nothing to fall back to for that vendor. This is by design — the
|
||||
alternative is shipping every preset twice — and it is why the acceptance gates are
|
||||
conservative and why CI generates the caches with the same build that ships them. The
|
||||
recovery path is a profile update, which delivers real JSONs.
|
||||
|
||||
It also means nothing may quietly assume a `<vendor>.json` exists. Discovery, version
|
||||
checks and the update decision all read whichever form is present, and a code path that
|
||||
enumerates only `*.json` will find no vendors at all in a packaged build.
|
||||
|
||||
## Maintenance rules
|
||||
|
||||
- **Adding, removing, retyping or reordering a config option** needs nothing. The
|
||||
payload names its keys and its enum values, so an option a cache carries and this
|
||||
build does not is dropped; one this build has and the cache does not is simply
|
||||
absent, as it would be from a JSON that predates it.
|
||||
- **Changing a hand-written `serialize()`** — `VendorProfile` or its nested types — or
|
||||
the `CachedPreset` field list — written and read by `visit_entry` in
|
||||
`PresetCacheFormat.cpp`, one list for the save, the load and the name peek alike — or
|
||||
the cache's own layout or stamps, requires bumping `CACHE_VERSION` by hand.
|
||||
- **The dictionary indexes with a `uint16`**, so `print_config_def` may hold at most
|
||||
65535 options and one cache at most 65535 distinct enum value names.
|
||||
`CacheDictionary::save` throws past that, which surfaces when CI generates the
|
||||
caches rather than on a user's machine.
|
||||
- **Bumping `CACHE_VERSION` is safe without a resources fallback** because
|
||||
`is_vendor_installed` means *present and usable*: cache-only vendors read as not
|
||||
installed after a bump, and the updater reinstalls them from resources.
|
||||
- **Bumping a vendor profile's version** invalidates that vendor's cache and nothing
|
||||
else — the filament library's included. Other vendors' caches resolve against the
|
||||
new library the next time they load.
|
||||
- **Caches are never committed.** They are build artifacts, generated per build,
|
||||
ignored by git.
|
||||
|
||||
## Where this lives in the tree
|
||||
|
||||
| Area | Files |
|
||||
|---|---|
|
||||
| Everything about the bytes on disk — the dictionary, one config's wire format, the file framing and stamps, entry serialization, `VendorCacheFile` save/load/peeks | `src/libslic3r/PresetCacheFormat.{hpp,cpp}` |
|
||||
| Serve-or-parse decision, installing cache entries into a bundle, cache write-back | `src/libslic3r/PresetBundle.{hpp,cpp}` |
|
||||
| Vendor profile serialization | `src/libslic3r/Preset.hpp` |
|
||||
| Vendor discovery, installed/shipped versions, installation | `src/libslic3r/utils.cpp` (declared in `Utils.hpp`) |
|
||||
| Update and reinstall decisions | `src/slic3r/Utils/PresetUpdater.cpp` |
|
||||
| Setup wizard and printer-selection dialog | `src/slic3r/GUI/ConfigWizard.cpp`, `src/slic3r/GUI/WebGuideDialog.cpp` |
|
||||
| Generator tool | `src/dev-utils/generate_system_cache.cpp` |
|
||||
| Build and packaging script | `scripts/build_preset_cache.{sh,bat}` |
|
||||
| Tests | `tests/libslic3r/test_vendor_cache.cpp` |
|
||||
@@ -0,0 +1,141 @@
|
||||
@echo off
|
||||
rem Build the per-vendor system preset caches (one <vendor>.opc per vendor) by
|
||||
rem running the generate_system_cache.exe dev tool against a profiles directory,
|
||||
rem and make every profiles directory named on the command line ship-ready:
|
||||
rem install the caches into it and delete the preset JSONs they replace, so a
|
||||
rem build ships one copy of its presets instead of two.
|
||||
rem
|
||||
rem scripts\build_preset_cache.bat [build_dir] [target_dir ...]
|
||||
rem
|
||||
rem build_dir defaults to "build"
|
||||
rem target_dir profiles directories to ship into. Caches are generated into
|
||||
rem the source tree's resources\profiles, which is what every
|
||||
rem packaging step copies from; a target may be that same
|
||||
rem directory, which then only gets pruned.
|
||||
rem --prune-source
|
||||
rem allow a target that is the directory the caches were generated
|
||||
rem into (resources\profiles). Pruning it deletes the checkout's
|
||||
rem own preset JSONs, which is a packaging step - not something a
|
||||
rem build should do to a working tree by surprise. CI passes it.
|
||||
rem
|
||||
rem Shipping deletes, so it is a CI packaging step. A vendor's own <vendor>.json
|
||||
rem goes along with its preset JSONs: the cache carries the vendor profile and
|
||||
rem the version it was built at, so discovery, version checks and installing all
|
||||
rem read it there. Only a vendor that has a cache is pruned, so non-vendor JSONs
|
||||
rem (blacklist.json) are left alone, as are the vendor directories themselves -
|
||||
rem thumbnails, covers and bed models still live there.
|
||||
rem
|
||||
rem set CONFIG=<cfg> to pin the build config for multi-config generators
|
||||
rem (default: the config of the tool already in the build tree, else Release)
|
||||
setlocal enabledelayedexpansion
|
||||
|
||||
set "REPO_ROOT=%~dp0.."
|
||||
|
||||
set "PRUNE_SOURCE="
|
||||
:parse_flags
|
||||
if /i "%~1"=="--prune-source" (
|
||||
set "PRUNE_SOURCE=1"
|
||||
shift
|
||||
goto :parse_flags
|
||||
)
|
||||
|
||||
set "BUILD_DIR=%~1"
|
||||
if "%BUILD_DIR%"=="" set "BUILD_DIR=build"
|
||||
if not exist "%BUILD_DIR%\" (
|
||||
echo ERROR: build tree not found: %BUILD_DIR% 1>&2
|
||||
exit /b 1
|
||||
)
|
||||
if not "%~1"=="" shift
|
||||
|
||||
rem Newest match wins: a stale binary silently produces a stale cache layout.
|
||||
call :find_tool
|
||||
if not defined CONFIG (
|
||||
for %%c in (Debug Release RelWithDebInfo MinSizeRel) do (
|
||||
echo !TOOL! | findstr /i "\\%%c\\" >nul && set "CONFIG=%%c"
|
||||
)
|
||||
)
|
||||
if not defined CONFIG set "CONFIG=Release"
|
||||
|
||||
echo Building generate_system_cache in %BUILD_DIR% (%CONFIG%)
|
||||
cmake --build "%BUILD_DIR%" --config %CONFIG% --target generate_system_cache
|
||||
if errorlevel 1 (
|
||||
echo ERROR: could not build generate_system_cache - configure the build tree with -DORCA_TOOLS=ON: 1>&2
|
||||
echo cmake -S "%REPO_ROOT%" -B "%BUILD_DIR%" -DORCA_TOOLS=ON 1>&2
|
||||
exit /b 1
|
||||
)
|
||||
call :find_tool
|
||||
if not defined TOOL (
|
||||
echo ERROR: generate_system_cache.exe not found under %BUILD_DIR% - build with -DORCA_TOOLS=ON 1>&2
|
||||
exit /b 1
|
||||
)
|
||||
|
||||
set "PROFILES=%REPO_ROOT%\resources\profiles"
|
||||
if not exist "%PROFILES%\" (
|
||||
echo ERROR: profiles directory not found: %PROFILES% 1>&2
|
||||
exit /b 1
|
||||
)
|
||||
for %%d in ("%PROFILES%") do set "PROFILES=%%~fd"
|
||||
|
||||
rem Add the slicer's runtime DLL directory to PATH so generate_system_cache.exe
|
||||
rem can resolve its dependencies (TKernel.dll etc.) without a full install step.
|
||||
set "DLL_DIR="
|
||||
for /f "delims=" %%f in ('dir /s /b "%BUILD_DIR%\TKernel.dll" 2^>nul') do (
|
||||
if not defined DLL_DIR set "DLL_DIR=%%~dpf"
|
||||
)
|
||||
if defined DLL_DIR set "PATH=%DLL_DIR%;%PATH%"
|
||||
|
||||
echo Generating per-vendor preset caches in %PROFILES%
|
||||
rem Start clean so vendors that went away - and caches written by older tool
|
||||
rem versions - don't linger next to the freshly generated ones.
|
||||
del /q "%PROFILES%\*.opc" 2>nul
|
||||
del /q "%PROFILES%\*.cache" 2>nul
|
||||
"%TOOL%" --path "%PROFILES%" --log_level 2
|
||||
if errorlevel 1 exit /b %errorlevel%
|
||||
|
||||
:next_target
|
||||
if "%~1"=="" exit /b 0
|
||||
call :ship "%~1"
|
||||
if errorlevel 1 exit /b 1
|
||||
shift
|
||||
goto :next_target
|
||||
|
||||
:ship
|
||||
set "TARGET=%~1"
|
||||
if not exist "%TARGET%\" (
|
||||
echo ERROR: profiles directory not found: %TARGET% 1>&2
|
||||
exit /b 1
|
||||
)
|
||||
for %%d in ("%TARGET%") do set "TARGET=%%~fd"
|
||||
if /i "%TARGET%"=="%PROFILES%" if not defined PRUNE_SOURCE (
|
||||
echo %TARGET%: skipped - this is where the caches were generated.
|
||||
echo Pass --prune-source to prune it; that deletes this checkout's preset JSONs.
|
||||
exit /b 0
|
||||
)
|
||||
if /i not "%TARGET%"=="%PROFILES%" copy /y "%PROFILES%\*.opc" "%TARGET%\" >nul
|
||||
|
||||
set /a SHIPPED=0
|
||||
set /a PRUNED=0
|
||||
for %%c in ("%PROFILES%\*.opc") do (
|
||||
set /a SHIPPED+=1
|
||||
set "VENDOR=%%~nc"
|
||||
if exist "%TARGET%\!VENDOR!.json" (
|
||||
del /q "%TARGET%\!VENDOR!.json"
|
||||
set /a PRUNED+=1
|
||||
)
|
||||
if exist "%TARGET%\!VENDOR!\" (
|
||||
for /f %%n in ('dir /s /b "%TARGET%\!VENDOR!\*.json" 2^>nul ^| find /c /v ""') do set /a PRUNED+=%%n
|
||||
del /s /q "%TARGET%\!VENDOR!\*.json" >nul 2>&1
|
||||
rem Deepest first, so a directory the delete above emptied goes too; rd
|
||||
rem refuses the ones still holding covers or meshes.
|
||||
for /f "delims=" %%d in ('dir /s /b /ad "%TARGET%\!VENDOR!" 2^>nul ^| sort /r') do rd "%%d" 2>nul
|
||||
)
|
||||
)
|
||||
echo %TARGET%: !SHIPPED! caches, dropped !PRUNED! preset JSONs
|
||||
exit /b 0
|
||||
|
||||
:find_tool
|
||||
set "TOOL="
|
||||
for /f "delims=" %%f in ('dir /s /b /o-d "%BUILD_DIR%\generate_system_cache.exe" 2^>nul') do (
|
||||
if not defined TOOL set "TOOL=%%f"
|
||||
)
|
||||
exit /b 0
|
||||
Executable
+161
@@ -0,0 +1,161 @@
|
||||
#!/usr/bin/env bash
|
||||
# Build the per-vendor system preset caches (one <vendor>.opc per vendor) by
|
||||
# running the generate_system_cache dev tool against a profiles directory, and
|
||||
# make every profiles directory named on the command line ship-ready: install
|
||||
# the caches into it and delete the preset JSONs they replace, so a build ships
|
||||
# one copy of its presets instead of two.
|
||||
#
|
||||
# ./scripts/build_preset_cache.sh # caches into resources/profiles
|
||||
# ./scripts/build_preset_cache.sh -b build/arm64 # search this build tree for the tool
|
||||
# ./scripts/build_preset_cache.sh <dir> [<dir> ...] # and ship into these profiles dirs
|
||||
#
|
||||
# Caches are generated into the source tree's resources/profiles, which is what
|
||||
# every packaging step copies from. Shipping deletes, so it is a CI packaging
|
||||
# step: pass packaged output directories, or the checkout of a build that is
|
||||
# about to be packaged from it.
|
||||
#
|
||||
# A vendor's own <vendor>.json goes along with its preset JSONs: the cache
|
||||
# carries the vendor profile and the version it was built at, so discovery,
|
||||
# version checks and installing all read it there. A shipped vendor is its cache
|
||||
# and nothing else. Only a vendor that has a cache is pruned, so an ungenerated
|
||||
# vendor keeps its JSONs and is simply parsed at startup; non-vendor JSONs
|
||||
# (blacklist.json) are left alone, as are the vendor directories themselves —
|
||||
# thumbnails, covers and bed models still live there.
|
||||
#
|
||||
# -b <dir> build tree holding the tool
|
||||
# (default: build/arm64, build/x86_64, or build — first that exists)
|
||||
# -p <dir> profiles directory to generate caches into
|
||||
# (default: <repo>/resources/profiles)
|
||||
# -c <cfg> build config for multi-config generators
|
||||
# (default: the config of the tool already in the build tree, else
|
||||
# the build tree's CMAKE_BUILD_TYPE)
|
||||
# -n skip the rebuild and run the tool already in the build tree
|
||||
# -l <level> tool log level (default: 2)
|
||||
# --prune-source
|
||||
# allow a target that is the directory the caches were generated
|
||||
# into (resources/profiles). Pruning it deletes the checkout's own
|
||||
# preset JSONs, which is a packaging step - not something a build
|
||||
# should do to a working tree by surprise.
|
||||
set -euo pipefail
|
||||
|
||||
repo_root="$(cd "$(dirname "$0")/.." && pwd -P)"
|
||||
build_dir=""
|
||||
profiles_dir=""
|
||||
config=""
|
||||
build_tool=1
|
||||
log_level=2
|
||||
prune_source=0
|
||||
|
||||
# getopts does not do long options; pull this one out first.
|
||||
args=()
|
||||
for arg in "$@"; do
|
||||
if [ "$arg" = "--prune-source" ]; then prune_source=1; else args+=("$arg"); fi
|
||||
done
|
||||
set -- ${args+"${args[@]}"}
|
||||
|
||||
while getopts "b:p:c:l:nh" opt; do
|
||||
case $opt in
|
||||
b) build_dir="$OPTARG" ;;
|
||||
p) profiles_dir="$OPTARG" ;;
|
||||
c) config="$OPTARG" ;;
|
||||
n) build_tool=0 ;;
|
||||
l) log_level="$OPTARG" ;;
|
||||
h) sed -n '2,${/^#/!q;p;}' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
|
||||
*) exit 1 ;;
|
||||
esac
|
||||
done
|
||||
shift $((OPTIND - 1))
|
||||
|
||||
if [ -z "$build_dir" ]; then
|
||||
for candidate in "$repo_root/build/arm64" "$repo_root/build/x86_64" "$repo_root/build"; do
|
||||
if [ -d "$candidate" ]; then build_dir="$candidate"; break; fi
|
||||
done
|
||||
fi
|
||||
if [ -z "$build_dir" ] || [ ! -d "$build_dir" ]; then
|
||||
echo "ERROR: build tree not found (pass -b <build_dir>)" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Newest match wins: multi-config trees keep one binary per config, and a stale
|
||||
# one silently produces a stale cache layout.
|
||||
find_tool() {
|
||||
local best="" f
|
||||
while IFS= read -r f; do
|
||||
[ -n "$f" ] || continue
|
||||
if [ -z "$best" ] || [ "$f" -nt "$best" ]; then best="$f"; fi
|
||||
done < <(find "$build_dir" -name generate_system_cache -type f 2>/dev/null)
|
||||
printf '%s' "$best"
|
||||
}
|
||||
|
||||
tool=$(find_tool)
|
||||
if [ -z "$config" ]; then
|
||||
case "$tool" in
|
||||
*/Debug/*) config=Debug ;;
|
||||
*/Release/*) config=Release ;;
|
||||
*/RelWithDebInfo/*) config=RelWithDebInfo ;;
|
||||
*/MinSizeRel/*) config=MinSizeRel ;;
|
||||
*) config=$(sed -n 's/^CMAKE_BUILD_TYPE:[A-Z]*=\(.\+\)$/\1/p' "$build_dir/CMakeCache.txt" 2>/dev/null | head -1 || true) ;;
|
||||
esac
|
||||
fi
|
||||
|
||||
if [ "$build_tool" = 1 ]; then
|
||||
echo "Building generate_system_cache in $build_dir${config:+ ($config)}"
|
||||
build_args=(--build "$build_dir" --target generate_system_cache)
|
||||
if [ -n "$config" ]; then build_args+=(--config "$config"); fi
|
||||
if ! cmake "${build_args[@]}"; then
|
||||
echo "ERROR: could not build generate_system_cache — configure the build tree with -DORCA_TOOLS=ON:" >&2
|
||||
echo " cmake -S \"$repo_root\" -B \"$build_dir\" -DORCA_TOOLS=ON" >&2
|
||||
exit 1
|
||||
fi
|
||||
tool=$(find_tool)
|
||||
fi
|
||||
|
||||
if [ -z "$tool" ]; then
|
||||
echo "ERROR: generate_system_cache not found under $build_dir — build with -DORCA_TOOLS=ON" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ -z "$profiles_dir" ]; then profiles_dir="$repo_root/resources/profiles"; fi
|
||||
if [ ! -d "$profiles_dir" ]; then
|
||||
echo "ERROR: profiles directory not found: $profiles_dir" >&2
|
||||
exit 1
|
||||
fi
|
||||
profiles_dir=$(cd "$profiles_dir" && pwd -P)
|
||||
|
||||
# Start clean so vendors that went away — and caches written by older tool
|
||||
# versions — don't linger next to the freshly generated ones.
|
||||
echo "Generating per-vendor preset caches in $profiles_dir"
|
||||
rm -f "$profiles_dir"/*.opc "$profiles_dir"/*.cache
|
||||
"$tool" --path "$profiles_dir" --log_level "$log_level"
|
||||
|
||||
for target in "$@"; do
|
||||
resolved=$(cd "$target" 2>/dev/null && pwd -P) || {
|
||||
echo "ERROR: profiles directory not found: $target" >&2
|
||||
exit 1
|
||||
}
|
||||
if [ "$resolved" = "$profiles_dir" ] && [ "$prune_source" -eq 0 ]; then
|
||||
echo "$resolved: skipped - this is where the caches were generated."
|
||||
echo " Pass --prune-source to prune it; that deletes this checkout's preset JSONs."
|
||||
continue
|
||||
fi
|
||||
if [ "$resolved" != "$profiles_dir" ]; then
|
||||
cp "$profiles_dir"/*.opc "$resolved"/
|
||||
fi
|
||||
|
||||
pruned=0
|
||||
shipped=0
|
||||
for cache in "$profiles_dir"/*.opc; do
|
||||
vendor=$(basename "$cache" .opc)
|
||||
shipped=$(( shipped + 1 ))
|
||||
if [ -f "$resolved/$vendor.json" ]; then
|
||||
rm -f "$resolved/$vendor.json"
|
||||
pruned=$(( pruned + 1 ))
|
||||
fi
|
||||
[ -d "$resolved/$vendor" ] || continue
|
||||
n=$(find "$resolved/$vendor" -name '*.json' | wc -l)
|
||||
find "$resolved/$vendor" -name '*.json' -delete
|
||||
find "$resolved/$vendor" -type d -empty -delete
|
||||
pruned=$(( pruned + n ))
|
||||
done
|
||||
echo "$resolved: $shipped caches, dropped $pruned preset JSONs"
|
||||
done
|
||||
@@ -347,6 +347,7 @@ modules:
|
||||
- |
|
||||
cmake . -B build_flatpak \
|
||||
-DFLATPAK=ON \
|
||||
-DORCA_TOOLS=ON \
|
||||
-DCMAKE_BUILD_TYPE=Release \
|
||||
-DCMAKE_PREFIX_PATH=/app \
|
||||
-DCMAKE_INSTALL_PREFIX=/app \
|
||||
@@ -357,6 +358,13 @@ modules:
|
||||
- ./scripts/run_gettext.sh
|
||||
- cmake --build build_flatpak --target install -j$FLATPAK_BUILDER_N_JOBS
|
||||
|
||||
# Per-vendor preset caches. On the other platforms CI runs this script
|
||||
# itself; the flatpak is built inside flatpak-builder and the generator
|
||||
# only exists in here, so the swap is a build step instead, against the
|
||||
# profiles the install above copied into /app.
|
||||
- cmake --build build_flatpak --target generate_system_cache -j$FLATPAK_BUILDER_N_JOBS
|
||||
- ./scripts/build_preset_cache.sh -n -b build_flatpak /app/share/OrcaSlicer/profiles
|
||||
|
||||
cleanup:
|
||||
- /include
|
||||
|
||||
@@ -403,6 +411,9 @@ modules:
|
||||
- type: file
|
||||
path: ../run_gettext.sh
|
||||
dest: scripts
|
||||
- type: file
|
||||
path: ../build_preset_cache.sh
|
||||
dest: scripts
|
||||
|
||||
# AppData metainfo for GNOME Software & Co.
|
||||
- type: file
|
||||
|
||||
@@ -20,6 +20,16 @@ if (SLIC3R_ENC_CHECK)
|
||||
)
|
||||
endif()
|
||||
|
||||
if (ORCA_TOOLS)
|
||||
set(_DEV_DEFS -DBOOST_ALL_NO_LIB -DBOOST_USE_WINAPI_VERSION=0x602 -DBOOST_SYSTEM_USE_UTF8)
|
||||
|
||||
# generate_system_cache: pre-generates per-vendor <vendor>.opc files under resources/profiles for CI bundling.
|
||||
add_executable(generate_system_cache generate_system_cache.cpp)
|
||||
target_link_libraries(generate_system_cache libslic3r boost_headeronly)
|
||||
target_compile_definitions(generate_system_cache PRIVATE ${_DEV_DEFS})
|
||||
|
||||
endif()
|
||||
|
||||
# Function that adds source file encoding check to a target
|
||||
# using the above encoding-check binary
|
||||
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
#include "libslic3r/PresetBundle.hpp"
|
||||
#include "libslic3r/Preset.hpp"
|
||||
#include "libslic3r/Utils.hpp"
|
||||
|
||||
#include <boost/algorithm/string/predicate.hpp>
|
||||
#include <boost/filesystem.hpp>
|
||||
#include <boost/log/trivial.hpp>
|
||||
#include <boost/program_options.hpp>
|
||||
#include <iostream>
|
||||
|
||||
using namespace Slic3r;
|
||||
namespace fs = boost::filesystem;
|
||||
namespace po = boost::program_options;
|
||||
|
||||
int main(int argc, char* argv[])
|
||||
{
|
||||
po::options_description desc("OrcaSlicer System Cache Generator\nUsage");
|
||||
// clang-format off
|
||||
desc.add_options()
|
||||
("help,h", "Show help")
|
||||
#ifdef __APPLE__
|
||||
("path,p", po::value<std::string>()->default_value("../../../../../../../resources/profiles"), "Path to profiles directory")
|
||||
#else
|
||||
("path,p", po::value<std::string>()->default_value("../../../resources/profiles"), "Path to profiles directory")
|
||||
#endif
|
||||
("log_level,l", po::value<int>()->default_value(2), "Log level (0=trace, 2=info, 4=error)");
|
||||
// clang-format on
|
||||
|
||||
po::variables_map vm;
|
||||
try {
|
||||
po::store(po::parse_command_line(argc, argv, desc), vm);
|
||||
if (vm.count("help")) { std::cout << desc << "\n"; return 0; }
|
||||
po::notify(vm);
|
||||
} catch (const po::error& e) {
|
||||
std::cerr << "Error: " << e.what() << "\n" << desc << "\n";
|
||||
return 1;
|
||||
}
|
||||
|
||||
const std::string profiles_path = vm["path"].as<std::string>();
|
||||
const int log_level = vm["log_level"].as<int>();
|
||||
|
||||
if (!fs::exists(profiles_path) || !fs::is_directory(profiles_path)) {
|
||||
std::cerr << "Error: '" << profiles_path << "' is not a valid directory\n";
|
||||
return 1;
|
||||
}
|
||||
|
||||
set_logging_level(log_level);
|
||||
set_data_dir(profiles_path);
|
||||
set_resources_dir(fs::path(profiles_path).parent_path().make_preferred().string());
|
||||
|
||||
const fs::path user_dir = fs::path(data_dir()) / PRESET_USER_DIR;
|
||||
if (!fs::exists(user_dir))
|
||||
fs::create_directories(user_dir);
|
||||
|
||||
AppConfig app_config;
|
||||
app_config.set("preset_folder", "default");
|
||||
|
||||
auto preset_bundle = std::make_unique<PresetBundle>();
|
||||
preset_bundle->set_is_validation_mode(true);
|
||||
preset_bundle->set_default_suppressed(true);
|
||||
preset_bundle->set_generate_vendor_caches(true);
|
||||
|
||||
std::cout << "Loading system presets from: " << profiles_path << "\n";
|
||||
|
||||
try {
|
||||
// In validation mode data_dir() is the profiles directory set above, so the
|
||||
// loader writes each <vendor>.opc next to its <vendor>.json as it parses it.
|
||||
preset_bundle->load_presets(app_config, ForwardCompatibilitySubstitutionRule::EnableSilent);
|
||||
} catch (const std::exception& ex) {
|
||||
std::cerr << "Failed to load presets: " << ex.what() << "\n";
|
||||
return 1;
|
||||
}
|
||||
|
||||
size_t cache_count = 0;
|
||||
for (auto& entry : fs::directory_iterator(profiles_path))
|
||||
if (boost::iends_with(entry.path().string(), ".opc"))
|
||||
++ cache_count;
|
||||
if (cache_count == 0) {
|
||||
std::cerr << "No vendor cache files were generated under " << profiles_path << "\n";
|
||||
return 1;
|
||||
}
|
||||
std::cout << "Generated " << cache_count << " vendor cache file(s) under " << profiles_path << "\n";
|
||||
return 0;
|
||||
}
|
||||
@@ -348,6 +348,8 @@ set(lisbslic3r_sources
|
||||
Polyline.hpp
|
||||
PresetBundle.cpp
|
||||
PresetBundle.hpp
|
||||
PresetCacheFormat.cpp
|
||||
PresetCacheFormat.hpp
|
||||
Preset.cpp
|
||||
Preset.hpp
|
||||
PrincipalComponents2D.cpp
|
||||
|
||||
@@ -28,6 +28,9 @@
|
||||
|
||||
#include <cereal/access.hpp>
|
||||
#include <cereal/types/base_class.hpp>
|
||||
// The serialize() members below archive ConfigOption hierarchies through
|
||||
// cereal::base_class, whose registration machinery lives in polymorphic.hpp.
|
||||
#include <cereal/types/polymorphic.hpp>
|
||||
|
||||
namespace Slic3r {
|
||||
struct FloatOrPercent
|
||||
|
||||
+24
-11
@@ -147,6 +147,9 @@ Semver get_version_from_json(std::string file_path)
|
||||
return Semver();
|
||||
//throw ConfigurationError(format("Failed loading configuration file \"%1%\": %2%", file_path, err.what()));
|
||||
}
|
||||
catch(...) {
|
||||
return Semver();
|
||||
}
|
||||
}
|
||||
|
||||
//BBS: add a function to load the key-values from xxx.json
|
||||
@@ -261,18 +264,28 @@ void extend_default_config_length(DynamicPrintConfig& config, const bool set_nil
|
||||
}
|
||||
};
|
||||
|
||||
// The four variant sets are immutable after static init and probed for every
|
||||
// key of every preset loaded; one merged map makes that a single lookup.
|
||||
// emplace keeps the first insertion, preserving the first-set-wins priority
|
||||
// of the else-if chain this replaces.
|
||||
static const std::unordered_map<std::string, int> variant_class = [] {
|
||||
std::unordered_map<std::string, int> m;
|
||||
for (const std::string& k : print_options_with_variant) m.emplace(k, 0);
|
||||
for (const std::string& k : filament_options_with_variant) m.emplace(k, 1);
|
||||
for (const std::string& k : printer_options_with_variant_1) m.emplace(k, 2);
|
||||
for (const std::string& k : printer_options_with_variant_2) m.emplace(k, 3);
|
||||
return m;
|
||||
}();
|
||||
|
||||
for(auto& key :config.keys()){
|
||||
if(auto iter = print_options_with_variant.find(key); iter != print_options_with_variant.end()){
|
||||
replace_nil_and_resize(key, process_variant_length);
|
||||
}
|
||||
else if(auto iter = filament_options_with_variant.find(key); iter != filament_options_with_variant.end()){
|
||||
replace_nil_and_resize(key, filament_variant_length);
|
||||
}
|
||||
else if(auto iter = printer_options_with_variant_1.find(key); iter != printer_options_with_variant_1.end()){
|
||||
replace_nil_and_resize(key, machine_variant_length);
|
||||
}
|
||||
else if(auto iter = printer_options_with_variant_2.find(key); iter != printer_options_with_variant_2.end()){
|
||||
replace_nil_and_resize(key, machine_variant_length * 2);
|
||||
auto iter = variant_class.find(key);
|
||||
if (iter == variant_class.end())
|
||||
continue;
|
||||
switch (iter->second) {
|
||||
case 0: replace_nil_and_resize(key, process_variant_length); break;
|
||||
case 1: replace_nil_and_resize(key, filament_variant_length); break;
|
||||
case 2: replace_nil_and_resize(key, machine_variant_length); break;
|
||||
case 3: replace_nil_and_resize(key, machine_variant_length * 2); break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -131,6 +131,10 @@ public:
|
||||
PrinterVariant() {}
|
||||
PrinterVariant(const std::string &name) : name(name) {}
|
||||
std::string name;
|
||||
|
||||
// All fields, declaration order — keep in sync; bump CACHE_VERSION on change.
|
||||
template<class Archive>
|
||||
void serialize(Archive& ar) { ar(name); } // PrinterVariant
|
||||
};
|
||||
|
||||
struct PrinterModel {
|
||||
@@ -139,7 +143,7 @@ public:
|
||||
std::string name;
|
||||
//BBS: this is internal id for the printer. Currently only used for searching in database
|
||||
std::string model_id;
|
||||
PrinterTechnology technology;
|
||||
PrinterTechnology technology = ptFFF;
|
||||
std::string family;
|
||||
std::vector<PrinterVariant> variants;
|
||||
std::vector<std::string> default_materials;
|
||||
@@ -162,6 +166,17 @@ public:
|
||||
}
|
||||
|
||||
const PrinterVariant* variant(const std::string &name) const { return const_cast<PrinterModel*>(this)->variant(name); }
|
||||
|
||||
// All fields, declaration order — keep in sync; bump CACHE_VERSION on change.
|
||||
template<class Archive>
|
||||
void serialize(Archive& ar) // PrinterModel
|
||||
{
|
||||
ar(id, name, model_id, technology, family, variants, default_materials,
|
||||
not_support_bed_types, bed_model, bed_texture, image_bed_type,
|
||||
bottom_texture_end_name, use_double_extruder_default_texture,
|
||||
bottom_texture_rect, bottom_texture_rect_longer, middle_texture_rect,
|
||||
hotend_model);
|
||||
}
|
||||
};
|
||||
std::vector<PrinterModel> models;
|
||||
|
||||
@@ -173,6 +188,14 @@ public:
|
||||
|
||||
bool valid() const { return ! name.empty() && ! id.empty() && config_version.valid(); }
|
||||
|
||||
// All fields, declaration order — keep in sync; bump CACHE_VERSION on change.
|
||||
template<class Archive>
|
||||
void serialize(Archive& ar) // VendorProfile
|
||||
{
|
||||
ar(name, id, config_version, config_update_url, changelog_url,
|
||||
models, default_filaments, default_sla_materials);
|
||||
}
|
||||
|
||||
// Load VendorProfile from an ini file.
|
||||
// If `load_all` is false, only the header with basic info (name, version, URLs) is loaded.
|
||||
static VendorProfile from_ini(const boost::filesystem::path &path, bool load_all=true);
|
||||
@@ -427,10 +450,10 @@ public:
|
||||
Preset(Type type, const std::string &name, bool is_default = false) : type(type), is_default(is_default), name(name) {}
|
||||
|
||||
protected:
|
||||
Preset() = default;
|
||||
|
||||
friend class PresetCollection;
|
||||
friend class PresetBundle;
|
||||
|
||||
Preset() = default;
|
||||
};
|
||||
|
||||
bool is_compatible_with_print (const PresetWithVendorProfile &preset, const PresetWithVendorProfile &active_print, const PresetWithVendorProfile &active_printer);
|
||||
|
||||
+531
-340
File diff suppressed because it is too large
Load Diff
@@ -2,10 +2,12 @@
|
||||
#define slic3r_PresetBundle_hpp_
|
||||
|
||||
#include "Preset.hpp"
|
||||
#include "PresetCacheFormat.hpp"
|
||||
#include "AppConfig.hpp"
|
||||
#include "enum_bitmask.hpp"
|
||||
|
||||
#include <memory>
|
||||
#include <set>
|
||||
#include <shared_mutex>
|
||||
#include <unordered_map>
|
||||
#include <optional>
|
||||
@@ -170,6 +172,31 @@ struct PresetBundleMetadata
|
||||
class PresetBundle
|
||||
{
|
||||
public:
|
||||
// ---- Per-vendor preset cache --------------------------------------------
|
||||
// One cache file per vendor (plus the Orca filament library), stamped with
|
||||
// the vendor's own profile version rather than a directory scan. The bytes
|
||||
// on disk are VendorCacheFile's business (PresetCacheFormat.hpp); what
|
||||
// lives here is how a cache's contents install into a bundle.
|
||||
|
||||
// The cache is not something a caller loads from: a vendor is loaded with
|
||||
// load_vendor_configs_from_json, which comes from the cache whenever one covers
|
||||
// it. What is public here is what the cache's own tests drive directly.
|
||||
|
||||
// Load a per-vendor cache into this bundle by installing its entries, with
|
||||
// base_bundle's filament library as the inheritance base. Rejects (returns
|
||||
// false, with this bundle left clean) unless VendorCacheFile::load accepts
|
||||
// the file — see its contract for the version and identity checks — and
|
||||
// every entry installs. Options this build no longer defines are dropped,
|
||||
// not fatal — the payload names its own keys.
|
||||
bool load_vendor_cache(const std::string& cache_path, const std::string& expected_vendor_name,
|
||||
const Semver& expected_vendor_version, const PresetBundle* base_bundle = nullptr);
|
||||
|
||||
// Enable writing a per-vendor cache after a JSON parse (off by default). Cache
|
||||
// content is pure parse output, so the guard is policy, not correctness: only
|
||||
// the deliberate generators (load_system_presets_from_json, the cache build
|
||||
// tool) write files, not every incidental load a dialog performs.
|
||||
void set_generate_vendor_caches(bool enable) { m_generate_vendor_caches = enable; }
|
||||
|
||||
static DynamicPrintConfig construct_full_config(Preset &in_printer_preset,
|
||||
Preset &in_print_preset,
|
||||
const DynamicPrintConfig &project_config,
|
||||
@@ -444,8 +471,12 @@ public:
|
||||
/*std::pair<PresetsConfigSubstitutions, size_t> load_configbundle(
|
||||
const std::string &path, LoadConfigBundleAttributes flags, ForwardCompatibilitySubstitutionRule compatibility_rule);*/
|
||||
//Orca: load config bundle from json, pass the base bundle to support cross vendor inheritance
|
||||
// Orca: `dir` is where the vendor is looked for — its own directory, whether or
|
||||
// not the profile JSONs are still there. A whole-vendor load comes from the
|
||||
// vendor's preset cache whenever one covers the profile on disk, and is parsed
|
||||
// from the JSONs in `dir` only when none does. Nothing here reads resources.
|
||||
std::pair<PresetsConfigSubstitutions, size_t> load_vendor_configs_from_json(
|
||||
const std::string &path, const std::string &vendor_name, LoadConfigBundleAttributes flags, ForwardCompatibilitySubstitutionRule compatibility_rule, const PresetBundle* base_bundle = nullptr);
|
||||
const std::string &dir, const std::string &vendor_name, LoadConfigBundleAttributes flags, ForwardCompatibilitySubstitutionRule compatibility_rule, const PresetBundle* base_bundle = nullptr);
|
||||
|
||||
// Export a config bundle file containing all the presets and the names of the active presets.
|
||||
//void export_configbundle(const std::string &path, bool export_system_settings = false, bool export_physical_printers = false);
|
||||
@@ -517,11 +548,49 @@ public:
|
||||
// Orca: for validation only.
|
||||
bool has_errors(bool check_duplicate_filament_subtypes = false) const;
|
||||
|
||||
// Errors the last load recorded. What the cache's error accounting promises —
|
||||
// a cache-served vendor reports what its parse would — is pinned against this.
|
||||
int error_count() const { return m_errors; }
|
||||
|
||||
// Orca: for validation only. Flag any system preset whose inherits / compatible_printers /
|
||||
// compatible_prints references a deleted (unknown) or renamed (old) preset name.
|
||||
bool check_preset_references() const;
|
||||
|
||||
// Merge one vendor's presets with the other vendor's presets, report duplicates.
|
||||
// Public so per-vendor-cache consumers (e.g. the setup wizard) can assemble a
|
||||
// bundle out of several per-vendor caches loaded into separate PresetBundle instances.
|
||||
std::vector<std::string> merge_presets(PresetBundle &&other);
|
||||
|
||||
private:
|
||||
// Load one vendor from the preset cache installed in `dir`, judged against
|
||||
// the vendor profile there. False, with this bundle left clean, when there
|
||||
// is no usable cache and the vendor has to be parsed. This is how
|
||||
// load_vendor_configs_from_json reads a cache.
|
||||
bool load_vendor_cache(const boost::filesystem::path& dir, const std::string& vendor_name, const PresetBundle* base_bundle);
|
||||
|
||||
// Load one source-form preset entry into this bundle: resolve `inherits`,
|
||||
// flatten, validate and register the preset. Returns the reason loading
|
||||
// failed, empty on success. See the definition for the sharing contract
|
||||
// between the JSON parse and the cache load.
|
||||
// retain_configs, when non-null, names the only presets registered into
|
||||
// config_maps (a full config copy each). The cache load passes the names its
|
||||
// entries inherit — the only ones ever looked up again; the JSON parse
|
||||
// retains all, not knowing what later subfiles inherit.
|
||||
std::string load_vendor_preset(const CachedPreset& entry,
|
||||
const std::string& path, const std::string& vendor_name,
|
||||
const PresetBundle* base_bundle,
|
||||
LoadConfigBundleAttributes flags,
|
||||
ConfigSubstitutionContext& substitution_context, PresetsConfigSubstitutions& substitutions,
|
||||
std::map<std::string, DynamicPrintConfig>& config_maps, std::map<std::string, std::string>& filament_id_maps,
|
||||
PresetCollection* presets_collection, size_t& count, bool is_from_lib,
|
||||
const std::set<std::string>* retain_configs = nullptr);
|
||||
|
||||
// Clear every collection's m_printer_hold_alias, which reset() leaves alone.
|
||||
void clear_printer_hold_aliases();
|
||||
|
||||
// Whether to (re)write a per-vendor cache after a JSON parse.
|
||||
bool m_generate_vendor_caches { false };
|
||||
|
||||
// Orca: validation only - flag any printer with two or more compatible
|
||||
// filament presets sharing one filament_id (ambiguous AMS subtype match).
|
||||
bool check_duplicate_filament_subtypes() const;
|
||||
@@ -529,8 +598,6 @@ private:
|
||||
//std::pair<PresetsConfigSubstitutions, std::string> load_system_presets(ForwardCompatibilitySubstitutionRule compatibility_rule);
|
||||
//BBS: add json related logic
|
||||
std::pair<PresetsConfigSubstitutions, std::string> load_system_presets_from_json(ForwardCompatibilitySubstitutionRule compatibility_rule);
|
||||
// Merge one vendor's presets with the other vendor's presets, report duplicates.
|
||||
std::vector<std::string> merge_presets(PresetBundle &&other);
|
||||
// Update the multicolor information for filaments.
|
||||
void update_filament_multi_color();
|
||||
// Update renamed_from and alias maps of system profiles.
|
||||
|
||||
@@ -0,0 +1,588 @@
|
||||
#include "libslic3r/PresetCacheFormat.hpp"
|
||||
|
||||
#include <algorithm>
|
||||
#include <memory>
|
||||
#include <sstream>
|
||||
#include <stdexcept>
|
||||
#include <utility>
|
||||
|
||||
#include <boost/crc.hpp>
|
||||
#include <boost/filesystem.hpp>
|
||||
#include <boost/iostreams/device/array.hpp>
|
||||
#include <boost/iostreams/stream.hpp>
|
||||
#include <boost/log/trivial.hpp>
|
||||
#include <boost/nowide/fstream.hpp>
|
||||
#include <cereal/types/map.hpp>
|
||||
#include <cereal/types/set.hpp>
|
||||
|
||||
#include "libslic3r/Utils.hpp"
|
||||
|
||||
namespace Slic3r {
|
||||
|
||||
CacheDictionary::CacheDictionary()
|
||||
{
|
||||
// ENUM_UNNAMED is index 0 and always the empty name.
|
||||
m_enum_values.emplace_back();
|
||||
}
|
||||
|
||||
// The ints an enum option holds — one for a coEnum, the whole vector for coEnums.
|
||||
static std::vector<int> enum_ints(const ConfigOptionDef& def, const ConfigOption* opt)
|
||||
{
|
||||
if (def.type == coEnum)
|
||||
return { opt->getInt() };
|
||||
return static_cast<const ConfigOptionInts*>(opt)->values;
|
||||
}
|
||||
|
||||
// The name this build gives one of those ints, empty where it has none — a
|
||||
// nullable option's nil, or a definition carrying no enum_keys_map. Enums are
|
||||
// written by name so a build that reorders an enum's values still reads it right.
|
||||
static std::string enum_name_of(const ConfigOptionDef& def, int value)
|
||||
{
|
||||
if (def.enum_keys_map != nullptr)
|
||||
for (const auto& kvp : *def.enum_keys_map)
|
||||
if (kvp.second == value)
|
||||
return kvp.first;
|
||||
return {};
|
||||
}
|
||||
|
||||
void CacheDictionary::collect(const DynamicPrintConfig& config)
|
||||
{
|
||||
for (auto it = config.cbegin(); it != config.cend(); ++ it) {
|
||||
const ConfigOptionDef* def = print_config_def.get(it->first);
|
||||
if (def == nullptr)
|
||||
continue; // save_config does not write it either
|
||||
if (m_key_index.try_emplace(it->first, uint16_t(m_keys.size())).second) {
|
||||
m_keys.push_back(it->first);
|
||||
m_types.push_back(uint16_t(def->type));
|
||||
}
|
||||
if (def->type != coEnum && def->type != coEnums)
|
||||
continue;
|
||||
for (int value : enum_ints(*def, it->second.get())) {
|
||||
std::string name = enum_name_of(*def, value);
|
||||
if (! name.empty() && m_enum_index.try_emplace(name, uint16_t(m_enum_values.size())).second)
|
||||
m_enum_values.push_back(std::move(name));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
uint16_t CacheDictionary::key_index(const t_config_option_key& key) const
|
||||
{
|
||||
auto it = m_key_index.find(key);
|
||||
if (it == m_key_index.end())
|
||||
throw std::runtime_error("preset cache: option " + key + " was never collected into the dictionary");
|
||||
return it->second;
|
||||
}
|
||||
|
||||
uint16_t CacheDictionary::enum_index(const std::string& name) const
|
||||
{
|
||||
if (name.empty())
|
||||
return ENUM_UNNAMED;
|
||||
auto it = m_enum_index.find(name);
|
||||
return it == m_enum_index.end() ? ENUM_UNNAMED : it->second;
|
||||
}
|
||||
|
||||
void CacheDictionary::save(cereal::BinaryOutputArchive& ar) const
|
||||
{
|
||||
// Checked here rather than left to the caller: an index that wrapped would
|
||||
// be written silently, and nothing downstream could tell.
|
||||
if (m_keys.size() > MAX_ENTRIES || m_enum_values.size() > MAX_ENTRIES)
|
||||
throw std::runtime_error("preset cache: the option dictionary outgrew the uint16 it is indexed with");
|
||||
ar(m_keys, m_types, m_enum_values);
|
||||
}
|
||||
|
||||
void CacheDictionary::load(cereal::BinaryInputArchive& ar)
|
||||
{
|
||||
ar(m_keys, m_types, m_enum_values);
|
||||
if (m_keys.size() != m_types.size())
|
||||
throw std::runtime_error("preset cache: dictionary key and type tables differ in length");
|
||||
if (m_keys.size() > MAX_ENTRIES || m_enum_values.size() > MAX_ENTRIES)
|
||||
throw std::runtime_error("preset cache: dictionary is larger than the uint16 it is indexed with");
|
||||
if (m_enum_values.empty() || ! m_enum_values.front().empty())
|
||||
throw std::runtime_error("preset cache: dictionary is missing its unnamed-enum slot");
|
||||
// Resolved once per file: every option read after this is a vector index.
|
||||
m_defs.resize(m_keys.size());
|
||||
for (size_t i = 0; i < m_keys.size(); ++ i) {
|
||||
const ConfigOptionDef* def = print_config_def.get(m_keys[i]);
|
||||
m_defs[i] = (def != nullptr && uint16_t(def->type) == m_types[i]) ? def : nullptr;
|
||||
}
|
||||
}
|
||||
|
||||
// ---- one config -----------------------------------------------------------
|
||||
|
||||
static void save_enum_option(cereal::BinaryOutputArchive& ar, const ConfigOptionDef& def,
|
||||
const ConfigOption* opt, const CacheDictionary& dict)
|
||||
{
|
||||
const std::vector<int> values = enum_ints(def, opt);
|
||||
ar(uint32_t(values.size()));
|
||||
for (int value : values) {
|
||||
const uint16_t idx = dict.enum_index(enum_name_of(def, value));
|
||||
ar(idx);
|
||||
if (idx == CacheDictionary::ENUM_UNNAMED)
|
||||
ar(int32_t(value));
|
||||
}
|
||||
}
|
||||
|
||||
// `config` may be null, in which case the option is read and dropped.
|
||||
static void load_enum_option(cereal::BinaryInputArchive& ar, ConfigOptionType type,
|
||||
const ConfigOptionDef* def, DynamicPrintConfig* config,
|
||||
const CacheDictionary& dict)
|
||||
{
|
||||
uint32_t cnt = 0;
|
||||
ar(cnt);
|
||||
if (type == coEnum && cnt != 1)
|
||||
throw std::runtime_error("preset cache: a scalar enum carrying more than one value");
|
||||
// Every element is read whatever happens, so the stream stays in sync and
|
||||
// whatever follows this option still loads.
|
||||
bool usable = def != nullptr && config != nullptr;
|
||||
std::vector<int> values;
|
||||
values.reserve(cnt);
|
||||
for (uint32_t i = 0; i < cnt; ++ i) {
|
||||
uint16_t idx = 0;
|
||||
ar(idx);
|
||||
if (! dict.valid_enum_index(idx))
|
||||
throw std::runtime_error("preset cache: enum value index past the end of the dictionary");
|
||||
if (idx == CacheDictionary::ENUM_UNNAMED) {
|
||||
// An int the writer could not name — a nil, or an option whose
|
||||
// definition carried no enum_keys_map. It travels verbatim.
|
||||
int32_t raw = 0;
|
||||
ar(raw);
|
||||
values.push_back(int(raw));
|
||||
continue;
|
||||
}
|
||||
if (! usable)
|
||||
continue; // the index above was this element's whole payload
|
||||
if (def->enum_keys_map == nullptr) {
|
||||
usable = false; // this build no longer maps this option's names
|
||||
continue;
|
||||
}
|
||||
const auto it = def->enum_keys_map->find(dict.enum_name_at(idx));
|
||||
if (it == def->enum_keys_map->end()) {
|
||||
usable = false; // a value this build dropped: the option goes with it
|
||||
continue;
|
||||
}
|
||||
values.push_back(it->second);
|
||||
}
|
||||
if (! usable)
|
||||
return;
|
||||
if (type == coEnum) {
|
||||
config->set_key_value(def->opt_key, new ConfigOptionEnumGeneric(def->enum_keys_map, values.front()));
|
||||
} else {
|
||||
auto* opt = def->nullable ? static_cast<ConfigOptionInts*>(new ConfigOptionEnumsGenericNullable(def->enum_keys_map))
|
||||
: static_cast<ConfigOptionInts*>(new ConfigOptionEnumsGeneric(def->enum_keys_map));
|
||||
opt->values = std::move(values);
|
||||
config->set_key_value(def->opt_key, opt);
|
||||
}
|
||||
}
|
||||
|
||||
void save_config(cereal::BinaryOutputArchive& ar, const DynamicPrintConfig& config, const CacheDictionary& dict)
|
||||
{
|
||||
struct Written { uint16_t idx; const ConfigOptionDef* def; const ConfigOption* opt; };
|
||||
std::vector<Written> written;
|
||||
written.reserve(config.size());
|
||||
for (auto it = config.cbegin(); it != config.cend(); ++ it)
|
||||
if (const ConfigOptionDef* def = print_config_def.get(it->first))
|
||||
written.push_back({ dict.key_index(it->first), def, it->second.get() });
|
||||
|
||||
ar(uint32_t(written.size()));
|
||||
for (const Written& w : written) {
|
||||
ar(w.idx);
|
||||
if (w.def->type == coEnum || w.def->type == coEnums)
|
||||
save_enum_option(ar, *w.def, w.opt, dict);
|
||||
else
|
||||
w.def->save_option_to_archive(ar, w.opt);
|
||||
}
|
||||
}
|
||||
|
||||
// `config` null means: read everything, keep nothing.
|
||||
static void read_config(cereal::BinaryInputArchive& ar, DynamicPrintConfig* config, const CacheDictionary& dict)
|
||||
{
|
||||
uint32_t cnt = 0;
|
||||
ar(cnt);
|
||||
if (config != nullptr)
|
||||
config->clear();
|
||||
// Reused across the loop: constructing a ConfigOptionDef per dropped option
|
||||
// would allocate its strings and vectors for nothing.
|
||||
ConfigOptionDef scratch;
|
||||
for (uint32_t i = 0; i < cnt; ++ i) {
|
||||
uint16_t idx = 0;
|
||||
ar(idx);
|
||||
if (! dict.valid_key_index(idx))
|
||||
throw std::runtime_error("preset cache: option index past the end of the dictionary");
|
||||
const ConfigOptionType type = dict.type_at(idx);
|
||||
const ConfigOptionDef* def = dict.def_at(idx);
|
||||
if (type == coEnum || type == coEnums) {
|
||||
load_enum_option(ar, type, def, config, dict);
|
||||
} else if (def != nullptr && config != nullptr) {
|
||||
config->set_key_value(def->opt_key, def->load_option_from_archive(ar));
|
||||
} else {
|
||||
// Read by the type the writer recorded, then drop: the same outcome
|
||||
// a JSON profile gets for an option this build no longer has.
|
||||
scratch.type = type;
|
||||
std::unique_ptr<ConfigOption> discard(scratch.load_option_from_archive(ar));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void load_config(cereal::BinaryInputArchive& ar, DynamicPrintConfig& config, const CacheDictionary& dict)
|
||||
{
|
||||
read_config(ar, &config, dict);
|
||||
}
|
||||
|
||||
void skip_config(cereal::BinaryInputArchive& ar, const CacheDictionary& dict)
|
||||
{
|
||||
read_config(ar, nullptr, dict);
|
||||
}
|
||||
|
||||
// ---- The per-vendor cache file (<vendor>.opc) -----------------------------
|
||||
|
||||
namespace {
|
||||
|
||||
#pragma pack(push, 1)
|
||||
struct CacheFileHeader {
|
||||
uint32_t magic;
|
||||
uint32_t version;
|
||||
uint64_t data_size;
|
||||
uint32_t crc32;
|
||||
};
|
||||
#pragma pack(pop)
|
||||
static_assert(sizeof(CacheFileHeader) == 20, "CacheFileHeader must be 20 bytes");
|
||||
|
||||
constexpr uint32_t CACHE_MAGIC = 0x4F52435A; // "ORCZ"
|
||||
// Bump when the wire format changes in a way the payload cannot describe
|
||||
// itself out of: reordering, removing or retyping a field of a hand-written
|
||||
// serialize() (VendorProfile and its nested types, CachedPreset via
|
||||
// save_entries below), or a change to the cache's own layout or the
|
||||
// meaning of its stamps. Option-schema drift is NOT such a change — the
|
||||
// dictionary handles it, which is why this no longer moves every release.
|
||||
constexpr uint32_t CACHE_VERSION = 1;
|
||||
|
||||
// A stamp-string read that refuses an absurd length before allocating anything.
|
||||
// The stamps are read from files named from the outside (peek_version is
|
||||
// pointed at whatever <vendor>.opc a directory holds), so the length word may
|
||||
// be arbitrary bytes — and a resize to a garbage 64-bit length does not fail as
|
||||
// a catchable bad_alloc here, it takes the app down through the out-of-memory
|
||||
// handler. A vendor name or profile version is a short token; anything longer
|
||||
// is not a cache this build wrote.
|
||||
std::string read_bounded_string(cereal::BinaryInputArchive& ar)
|
||||
{
|
||||
constexpr uint64_t MAX_STAMP_LEN = 1024;
|
||||
cereal::size_type len = 0;
|
||||
ar(cereal::make_size_tag(len));
|
||||
if (uint64_t(len) > MAX_STAMP_LEN)
|
||||
throw std::runtime_error("preset cache: string length out of bounds");
|
||||
std::string s(size_t(len), '\0');
|
||||
ar(cereal::binary_data(s.data(), size_t(len)));
|
||||
return s;
|
||||
}
|
||||
|
||||
// The prologue every cache reader starts with: the format version, then the
|
||||
// vendor's identity. Returns the vendor version stamped on a body this build can
|
||||
// read, empty on anything else — which is the same answer as "not this vendor".
|
||||
std::string read_cache_stamps(cereal::BinaryInputArchive& ar, const std::string& expected_vendor_name)
|
||||
{
|
||||
// The version is judged before anything variable-length is read: on a body
|
||||
// that is not a per-vendor cache of this version, the bytes where a string
|
||||
// length would sit may be arbitrary framing.
|
||||
uint32_t cache_version = 0;
|
||||
ar(cache_version);
|
||||
if (cache_version != CACHE_VERSION)
|
||||
return {};
|
||||
const std::string vendor_name = read_bounded_string(ar);
|
||||
const std::string vendor_version = read_bounded_string(ar);
|
||||
if (vendor_name != expected_vendor_name)
|
||||
return {};
|
||||
return vendor_version;
|
||||
}
|
||||
|
||||
// A cache stays usable as long as it was built from a vendor profile at least
|
||||
// as new as the one now on disk. Profiles whose version is invalid cannot be
|
||||
// judged this way and are never served from cache; where no profile sits
|
||||
// beside the cache at all, nothing can be newer than it — that state is passed
|
||||
// as Semver::inf(), which no real profile can carry (an invalid version could
|
||||
// not say it apart from "profile there but unjudgeable", and zero would
|
||||
// collide with a genuine "0.0.0"). This is the serve rule; the install rule
|
||||
// (cache_covers in PresetBundle.cpp) deliberately reads an unjudgeable profile
|
||||
// the other way, so the two are not one function.
|
||||
bool cache_covers_version(const std::string& cached, const Semver& on_disk)
|
||||
{
|
||||
if (on_disk == Semver::inf())
|
||||
return true; // before parsing `cached`: nothing exists that the stamp must cover
|
||||
if (! on_disk.valid())
|
||||
return false;
|
||||
const auto cached_ver = Semver::parse(cached);
|
||||
return cached_ver && *cached_ver >= on_disk;
|
||||
}
|
||||
|
||||
// CachedPreset on the wire: all fields, declaration order, in one place.
|
||||
// `config` writes, reads or skips the config sitting in the middle of that
|
||||
// order — the three things a reader can want to do with it — so save, load and
|
||||
// the name peek below cannot drift apart. Keep in sync with the struct in
|
||||
// PresetCacheFormat.hpp and bump CACHE_VERSION on change. Written here rather
|
||||
// than as a serialize() member because the config needs the file's dictionary,
|
||||
// which cereal cannot thread through one.
|
||||
template<class Archive, class Entry, class ConfigFn>
|
||||
void visit_entry(Archive& ar, Entry& e, ConfigFn&& config)
|
||||
{
|
||||
ar(e.name, e.sub_path);
|
||||
config();
|
||||
ar(e.inherits, e.description, e.instantiation, e.setting_id, e.filament_id, e.renamed_from);
|
||||
}
|
||||
|
||||
// The count comes from a file that has already passed magic and CRC, but a
|
||||
// reserve is a promise to allocate: cap it and let push_back grow the rest.
|
||||
constexpr uint32_t MAX_RESERVED_ENTRIES = 4096;
|
||||
|
||||
void save_entries(cereal::BinaryOutputArchive& ar,
|
||||
const std::vector<CachedPreset>& entries,
|
||||
const CacheDictionary& dict)
|
||||
{
|
||||
ar(uint32_t(entries.size()));
|
||||
for (const CachedPreset& e : entries)
|
||||
visit_entry(ar, e, [&] { save_config(ar, e.config_src, dict); });
|
||||
}
|
||||
|
||||
void load_entries(cereal::BinaryInputArchive& ar,
|
||||
std::vector<CachedPreset>& entries,
|
||||
const CacheDictionary& dict)
|
||||
{
|
||||
uint32_t cnt = 0;
|
||||
ar(cnt);
|
||||
entries.clear();
|
||||
entries.reserve(std::min(cnt, MAX_RESERVED_ENTRIES));
|
||||
for (uint32_t i = 0; i < cnt; ++ i) {
|
||||
CachedPreset e;
|
||||
visit_entry(ar, e, [&] { load_config(ar, e.config_src, dict); });
|
||||
entries.push_back(std::move(e));
|
||||
}
|
||||
}
|
||||
|
||||
// Read a raw cache body: verify magic, size, CRC.
|
||||
bool read_cache_blob(const std::string& path, std::string& out_blob)
|
||||
{
|
||||
try {
|
||||
boost::nowide::ifstream ifs(path, std::ios::binary);
|
||||
if (!ifs.is_open())
|
||||
return false;
|
||||
CacheFileHeader fhdr;
|
||||
if (!ifs.read(reinterpret_cast<char*>(&fhdr), sizeof(fhdr)))
|
||||
return false;
|
||||
if (fhdr.magic != CACHE_MAGIC)
|
||||
return false;
|
||||
// data_size is 8 bytes from a file nothing has authenticated yet, and
|
||||
// it is about to size an allocation. The body is the whole of the file
|
||||
// behind the header — anything else is not a cache this build wrote.
|
||||
ifs.seekg(0, std::ios::end);
|
||||
const std::streamoff file_size = ifs.tellg();
|
||||
if (file_size < std::streamoff(sizeof(fhdr)) ||
|
||||
fhdr.data_size == 0 ||
|
||||
fhdr.data_size != uint64_t(file_size) - sizeof(fhdr))
|
||||
return false;
|
||||
ifs.seekg(sizeof(fhdr), std::ios::beg);
|
||||
out_blob.assign(fhdr.data_size, '\0');
|
||||
if (!ifs.read(&out_blob[0], static_cast<std::streamsize>(fhdr.data_size)))
|
||||
return false;
|
||||
boost::crc_32_type crc;
|
||||
crc.process_bytes(out_blob.data(), out_blob.size());
|
||||
if (crc.checksum() != fhdr.crc32) {
|
||||
BOOST_LOG_TRIVIAL(warning) << "VendorCacheFile: CRC mismatch: " << path;
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
} catch (const std::exception& e) {
|
||||
BOOST_LOG_TRIVIAL(warning) << "VendorCacheFile: read failed (" << path << "): " << e.what();
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
// Write a cache body behind the standard 20-byte file header. False when the
|
||||
// file could not be opened or written whole.
|
||||
bool write_cache_blob(const std::string& path, const std::string& blob)
|
||||
{
|
||||
boost::crc_32_type crc;
|
||||
crc.process_bytes(blob.data(), blob.size());
|
||||
// Written beside the target and moved into place, as AppConfig::save does:
|
||||
// a cache is truncated and rewritten in full, so a write that dies partway
|
||||
// would otherwise leave a header claiming more body than the file holds.
|
||||
// The PID suffix also keeps two instances writing the same vendor from
|
||||
// interleaving.
|
||||
const std::string tmp_path = path + "." + std::to_string(get_current_pid()) + ".tmp";
|
||||
try {
|
||||
boost::filesystem::create_directories(boost::filesystem::path(path).parent_path());
|
||||
{
|
||||
boost::nowide::ofstream ofs(tmp_path, std::ios::binary | std::ios::trunc);
|
||||
if (!ofs.is_open()) {
|
||||
BOOST_LOG_TRIVIAL(warning) << "VendorCacheFile: cannot open for writing: " << tmp_path;
|
||||
return false;
|
||||
}
|
||||
CacheFileHeader fhdr;
|
||||
fhdr.magic = CACHE_MAGIC;
|
||||
fhdr.version = CACHE_VERSION;
|
||||
fhdr.data_size = static_cast<uint64_t>(blob.size());
|
||||
fhdr.crc32 = crc.checksum();
|
||||
ofs.write(reinterpret_cast<const char*>(&fhdr), sizeof(fhdr));
|
||||
ofs.write(blob.data(), static_cast<std::streamsize>(blob.size()));
|
||||
ofs.close(); // flush; close() raises failbit on error
|
||||
if (! ofs.good()) {
|
||||
BOOST_LOG_TRIVIAL(warning) << "VendorCacheFile: write failed (" << tmp_path << ")";
|
||||
boost::system::error_code ec;
|
||||
boost::filesystem::remove(tmp_path, ec);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if (const std::error_code ec = rename_file(tmp_path, path)) {
|
||||
BOOST_LOG_TRIVIAL(warning) << "VendorCacheFile: could not move " << tmp_path << " into place: " << ec.message();
|
||||
boost::system::error_code rm;
|
||||
boost::filesystem::remove(tmp_path, rm);
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
} catch (const std::exception& e) {
|
||||
BOOST_LOG_TRIVIAL(warning) << "VendorCacheFile: write failed (" << path << "): " << e.what();
|
||||
boost::system::error_code ec;
|
||||
boost::filesystem::remove(tmp_path, ec);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
} // anonymous namespace
|
||||
|
||||
// static
|
||||
bool VendorCacheFile::save(const std::string& path, const std::string& vendor_name,
|
||||
const std::string& vendor_version, const VendorCacheData& data)
|
||||
{
|
||||
try {
|
||||
// Collected before anything is written: the dictionary sits ahead of the
|
||||
// entries so a reader resolves it once and then indexes.
|
||||
CacheDictionary dict;
|
||||
for (const std::vector<CachedPreset>* entries : { &data.process_entries, &data.filament_entries, &data.machine_entries })
|
||||
for (const CachedPreset& e : *entries)
|
||||
dict.collect(e.config_src);
|
||||
|
||||
std::ostringstream body(std::ios::binary);
|
||||
{
|
||||
cereal::BinaryOutputArchive ar(body);
|
||||
ar(CACHE_VERSION);
|
||||
ar(vendor_name, vendor_version);
|
||||
dict.save(ar);
|
||||
ar(data.vendors);
|
||||
save_entries(ar, data.process_entries, dict);
|
||||
save_entries(ar, data.filament_entries, dict);
|
||||
save_entries(ar, data.machine_entries, dict);
|
||||
ar(data.parse_errors);
|
||||
}
|
||||
return write_cache_blob(path, body.str());
|
||||
} catch (const std::exception& e) {
|
||||
BOOST_LOG_TRIVIAL(warning) << "VendorCacheFile: failed to save vendor cache " << path << ": " << e.what();
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
// static
|
||||
bool VendorCacheFile::load(const std::string& path, const std::string& expected_vendor_name,
|
||||
const Semver& expected_vendor_version, VendorCacheData& data)
|
||||
{
|
||||
std::string blob;
|
||||
if (! read_cache_blob(path, blob))
|
||||
return false;
|
||||
try {
|
||||
// Read in place: an istringstream would copy the blob once more just to
|
||||
// stream over it.
|
||||
boost::iostreams::stream<boost::iostreams::array_source> body(blob.data(), blob.size());
|
||||
cereal::BinaryInputArchive ar(body);
|
||||
const std::string vendor_version = read_cache_stamps(ar, expected_vendor_name);
|
||||
if (vendor_version.empty() || ! cache_covers_version(vendor_version, expected_vendor_version))
|
||||
return false;
|
||||
CacheDictionary dict;
|
||||
dict.load(ar);
|
||||
ar(data.vendors);
|
||||
load_entries(ar, data.process_entries, dict);
|
||||
load_entries(ar, data.filament_entries, dict);
|
||||
load_entries(ar, data.machine_entries, dict);
|
||||
ar(data.parse_errors);
|
||||
if (data.vendors.find(expected_vendor_name) == data.vendors.end())
|
||||
throw std::runtime_error("vendor cache does not carry its own vendor profile");
|
||||
return true;
|
||||
} catch (const std::exception& e) {
|
||||
BOOST_LOG_TRIVIAL(warning) << "VendorCacheFile: rejecting vendor cache " << path << ": " << e.what();
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
// static
|
||||
std::string VendorCacheFile::peek_version(const std::string& path, const std::string& expected_vendor_name)
|
||||
{
|
||||
try {
|
||||
boost::nowide::ifstream ifs(path, std::ios::binary);
|
||||
CacheFileHeader fhdr;
|
||||
if (! ifs.read(reinterpret_cast<char*>(&fhdr), sizeof(fhdr)) || fhdr.magic != CACHE_MAGIC)
|
||||
return {};
|
||||
// Only the head of the body is read, and its CRC left unverified: the
|
||||
// stamps sit at the front, and this answers "what version is this?"
|
||||
// without paying for tens of megabytes. Callers that need to know the
|
||||
// file is whole use usable_version instead.
|
||||
std::string head(static_cast<size_t>(std::min<uint64_t>(fhdr.data_size, 1024)), '\0');
|
||||
if (! ifs.read(&head[0], static_cast<std::streamsize>(head.size())))
|
||||
return {};
|
||||
std::istringstream body(head, std::ios::binary);
|
||||
cereal::BinaryInputArchive ar(body);
|
||||
return read_cache_stamps(ar, expected_vendor_name);
|
||||
} catch (const std::exception&) {
|
||||
return {};
|
||||
}
|
||||
}
|
||||
|
||||
// static
|
||||
Semver VendorCacheFile::usable_version(const std::string& path, const std::string& expected_vendor_name)
|
||||
{
|
||||
std::string blob;
|
||||
if (! read_cache_blob(path, blob))
|
||||
return Semver::invalid();
|
||||
try {
|
||||
boost::iostreams::stream<boost::iostreams::array_source> body(blob.data(), blob.size());
|
||||
cereal::BinaryInputArchive ar(body);
|
||||
const auto ver = Semver::parse(read_cache_stamps(ar, expected_vendor_name));
|
||||
return ver ? *ver : Semver::invalid();
|
||||
} catch (const std::exception&) {
|
||||
return Semver::invalid();
|
||||
}
|
||||
}
|
||||
|
||||
// static
|
||||
bool VendorCacheFile::carries_preset(const std::string& path, const std::string& vendor_name,
|
||||
Preset::Type type, const std::string& preset_name)
|
||||
{
|
||||
std::string blob;
|
||||
if (! read_cache_blob(path, blob))
|
||||
return false;
|
||||
try {
|
||||
boost::iostreams::stream<boost::iostreams::array_source> body(blob.data(), blob.size());
|
||||
cereal::BinaryInputArchive ar(body);
|
||||
if (read_cache_stamps(ar, vendor_name).empty())
|
||||
return false;
|
||||
CacheDictionary dict;
|
||||
dict.load(ar);
|
||||
VendorMap vendors;
|
||||
ar(vendors);
|
||||
// Reused: every entry overwrites it, and only its name is ever looked at.
|
||||
CachedPreset entry;
|
||||
// Written in this order by save. The list that could carry the preset
|
||||
// is the last one worth reading.
|
||||
for (Preset::Type kind : { Preset::TYPE_PRINT, Preset::TYPE_FILAMENT, Preset::TYPE_PRINTER }) {
|
||||
uint32_t cnt = 0;
|
||||
ar(cnt);
|
||||
for (uint32_t i = 0; i < cnt; ++ i) {
|
||||
visit_entry(ar, entry, [&] { skip_config(ar, dict); });
|
||||
if (kind == type && entry.name == preset_name)
|
||||
return true;
|
||||
}
|
||||
if (kind == type)
|
||||
return false;
|
||||
}
|
||||
return false;
|
||||
} catch (const std::exception& e) {
|
||||
BOOST_LOG_TRIVIAL(warning) << "VendorCacheFile: could not read preset names from " << path << ": " << e.what();
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
} // namespace Slic3r
|
||||
@@ -0,0 +1,192 @@
|
||||
#ifndef slic3r_PresetCacheFormat_hpp_
|
||||
#define slic3r_PresetCacheFormat_hpp_
|
||||
|
||||
#include <cstdint>
|
||||
#include <string>
|
||||
#include <unordered_map>
|
||||
#include <vector>
|
||||
|
||||
#include <cereal/archives/binary.hpp>
|
||||
#include <cereal/types/string.hpp>
|
||||
#include <cereal/types/vector.hpp>
|
||||
|
||||
#include "libslic3r/Config.hpp"
|
||||
#include "libslic3r/Preset.hpp"
|
||||
#include "libslic3r/PrintConfig.hpp"
|
||||
#include "libslic3r/Semver.hpp"
|
||||
|
||||
namespace Slic3r {
|
||||
|
||||
// How the preset cache writes a DynamicPrintConfig.
|
||||
//
|
||||
// Not through the global cereal hooks in PrintConfig.hpp: those key an option by
|
||||
// its serialization_key_ordinal, which ConfigDef::add assigns by declaration
|
||||
// order at static-init time. Inserting one option into the middle of
|
||||
// PrintConfig.cpp shifts every later ordinal, and the lookup on the way back in
|
||||
// then SUCCEEDS on the wrong option — where the two share a type, and hundreds
|
||||
// of coFloat/coBool/coInt options do, the bytes deserialize cleanly into the
|
||||
// wrong key. Silently wrong print settings, no error. Those hooks are also the
|
||||
// undo/redo wire format, where the process cannot change underneath them, so
|
||||
// they stay as they are and the cache keys by name instead.
|
||||
//
|
||||
// Names are not repeated per preset. Each cache file carries one dictionary of
|
||||
// the distinct opt_keys it uses, the type each was written as, and the distinct
|
||||
// enum value names; an option on the wire is then a uint16 index into it plus
|
||||
// its value. The dictionary is resolved to this build's option definitions once
|
||||
// per file, after which reading an option is a vector index.
|
||||
class CacheDictionary
|
||||
{
|
||||
public:
|
||||
CacheDictionary();
|
||||
|
||||
// Index reserved in the enum table for an int the writing build could not
|
||||
// name — a nullable option's nil, or a definition carrying no
|
||||
// enum_keys_map. The raw int32 follows it on the wire and is loaded
|
||||
// verbatim, so those values survive too.
|
||||
static constexpr uint16_t ENUM_UNNAMED = 0;
|
||||
|
||||
// ---- writing ----
|
||||
|
||||
// Record every key and enum value `config` uses. Call for every config that
|
||||
// will be written, before writing the dictionary.
|
||||
void collect(const DynamicPrintConfig& config);
|
||||
|
||||
uint16_t key_index(const t_config_option_key& key) const;
|
||||
// ENUM_UNNAMED for an empty name or one that was never collected.
|
||||
uint16_t enum_index(const std::string& name) const;
|
||||
|
||||
// ---- reading ----
|
||||
|
||||
// The definition an index resolves to in THIS build, or nullptr where the
|
||||
// key is unknown here or is now defined with a different type. A nullptr
|
||||
// entry's value is still read — using type_at(idx), the type the writer
|
||||
// recorded — and then dropped, which is what a JSON profile gets for an
|
||||
// option this build no longer has.
|
||||
const ConfigOptionDef* def_at(uint16_t idx) const { return m_defs[idx]; }
|
||||
ConfigOptionType type_at(uint16_t idx) const { return ConfigOptionType(m_types[idx]); }
|
||||
const std::string& enum_name_at(uint16_t idx) const { return m_enum_values[idx]; }
|
||||
// m_defs, not m_keys: only load() sizes it, so this is false for every index
|
||||
// on a dictionary that was collected rather than read.
|
||||
bool valid_key_index(uint16_t idx) const { return size_t(idx) < m_defs.size(); }
|
||||
bool valid_enum_index(uint16_t idx) const { return size_t(idx) < m_enum_values.size(); }
|
||||
|
||||
// The layout these two agree on is covered by CACHE_VERSION (PresetCacheFormat.cpp);
|
||||
// bump it when they change.
|
||||
// Throws when either table outgrew the uint16 the wire format indexes it
|
||||
// with. Both are bounded by the option count (912 at the time of writing), so
|
||||
// that is a build-time failure in CI, not a runtime one.
|
||||
void save(cereal::BinaryOutputArchive& ar) const;
|
||||
// Throws on a dictionary that cannot be indexed as written.
|
||||
void load(cereal::BinaryInputArchive& ar);
|
||||
|
||||
private:
|
||||
// Indices are uint16, so a table may hold at most this many entries.
|
||||
static constexpr size_t MAX_ENTRIES = 0xFFFF;
|
||||
|
||||
std::vector<std::string> m_keys;
|
||||
// ConfigOptionType, as written. Sixteen bits, not eight: coVectorType is
|
||||
// 0x4000, so every vector type — coFloats, coEnums, coStrings — is above
|
||||
// 255, and a byte would fold each one onto its scalar counterpart.
|
||||
std::vector<uint16_t> m_types;
|
||||
std::vector<std::string> m_enum_values; // [ENUM_UNNAMED] is always empty
|
||||
|
||||
// Writing.
|
||||
std::unordered_map<std::string, uint16_t> m_key_index;
|
||||
std::unordered_map<std::string, uint16_t> m_enum_index;
|
||||
// Reading, resolved once by load().
|
||||
std::vector<const ConfigOptionDef*> m_defs;
|
||||
};
|
||||
|
||||
// One config, keyed through `dict`. Options print_config_def does not know are
|
||||
// not written: nothing could give them a type on the way back in.
|
||||
void save_config(cereal::BinaryOutputArchive& ar, const DynamicPrintConfig& config, const CacheDictionary& dict);
|
||||
// Throws only on a payload that cannot be indexed; an option this build cannot
|
||||
// place is dropped, not fatal.
|
||||
void load_config(cereal::BinaryInputArchive& ar, DynamicPrintConfig& config, const CacheDictionary& dict);
|
||||
// Consume one config without building it, for a reader that only wants what
|
||||
// comes after.
|
||||
void skip_config(cereal::BinaryInputArchive& ar, const CacheDictionary& dict);
|
||||
|
||||
// One preset as its JSON subfile states it: the config diff, the name of the
|
||||
// preset it inherits, and the parse metadata — everything the parse phase of
|
||||
// load_vendor_configs_from_json extracts and nothing it derives. Inheritance
|
||||
// is resolved when the entry is installed, against whatever filament library
|
||||
// is loaded then, so a cache carries no other vendor's values and no other
|
||||
// vendor's update can make it stale.
|
||||
// Written and read by visit_entry in PresetCacheFormat.cpp, which lists every
|
||||
// field below in this order — once, for the save, the load and the name peek alike.
|
||||
struct CachedPreset
|
||||
{
|
||||
std::string name;
|
||||
std::string sub_path; // path under the vendor's directory
|
||||
DynamicPrintConfig config_src; // the preset's own diff, nothing inherited
|
||||
std::string inherits;
|
||||
std::string description;
|
||||
std::string instantiation; // "true"/"false" as stated; anything else was already counted as a parse error
|
||||
std::string setting_id;
|
||||
std::string filament_id;
|
||||
std::vector<std::string> renamed_from;
|
||||
};
|
||||
|
||||
// What one per-vendor cache file carries besides its stamps: the vendor profile
|
||||
// map, the presets in source form, and how many errors their parse counted.
|
||||
struct VendorCacheData
|
||||
{
|
||||
VendorMap vendors;
|
||||
std::vector<CachedPreset> process_entries;
|
||||
std::vector<CachedPreset> filament_entries;
|
||||
std::vector<CachedPreset> machine_entries;
|
||||
uint64_t parse_errors = 0;
|
||||
};
|
||||
|
||||
// A per-vendor preset cache file (<vendor>.opc): a 20-byte header (magic, format
|
||||
// version, body size, CRC) framing one cereal body — stamps (format version,
|
||||
// vendor name, vendor profile version), the option dictionary, then the
|
||||
// VendorCacheData. Everything about those bytes lives here; when a vendor is
|
||||
// served from its cache, and how entries install into a bundle, is
|
||||
// PresetBundle's business.
|
||||
class VendorCacheFile
|
||||
{
|
||||
public:
|
||||
// Save one vendor (vendor_name at vendor_version). False when the file
|
||||
// could not be written whole.
|
||||
static bool save(const std::string& path, const std::string& vendor_name,
|
||||
const std::string& vendor_version, const VendorCacheData& data);
|
||||
|
||||
// Read a whole cache into `data`. False — with `data` in an unspecified
|
||||
// state — unless the file is a cache this build wrote, its CRC holds, it
|
||||
// names this vendor, it was built from a vendor profile at least as new as
|
||||
// `expected_vendor_version`, and it carries its own vendor profile. An
|
||||
// invalid expected version (a profile whose version
|
||||
// cannot be judged) is never served from cache; Semver::inf() (no profile
|
||||
// beside the cache at all) accepts whatever is cached.
|
||||
static bool load(const std::string& path, const std::string& expected_vendor_name,
|
||||
const Semver& expected_vendor_version, VendorCacheData& data);
|
||||
|
||||
// Read the profile version a cache was stamped with, without deserializing
|
||||
// its presets. Empty if the file is unreadable, not a cache this build
|
||||
// understands, or not this vendor's. This is how an installed vendor's
|
||||
// version is known when only its cache is installed.
|
||||
static std::string peek_version(const std::string& path, const std::string& expected_vendor_name);
|
||||
|
||||
// The profile version an installed cache can actually be served at, or an
|
||||
// invalid Semver when the file is not a cache this build can read. Unlike
|
||||
// peek_version this verifies the body's CRC, at the cost of reading the
|
||||
// whole file: where the cache is the vendor's whole installation, "a file
|
||||
// is there" is not enough to call it installed, and a vendor wrongly
|
||||
// believed installed is never repaired.
|
||||
static Semver usable_version(const std::string& path, const std::string& expected_vendor_name);
|
||||
|
||||
// Whether a cache carries a preset of `type` under `preset_name`, without
|
||||
// installing any of them. False when the file is not a cache this build can
|
||||
// read. The three kinds are written in one stream, so reaching the machines
|
||||
// means reading past the processes and filaments — their configs are consumed
|
||||
// and dropped rather than built. This is how a build that ships caches instead
|
||||
// of preset JSONs answers "which vendor carries this preset?".
|
||||
static bool carries_preset(const std::string& path, const std::string& vendor_name,
|
||||
Preset::Type type, const std::string& preset_name);
|
||||
};
|
||||
|
||||
} // namespace Slic3r
|
||||
|
||||
#endif // slic3r_PresetCacheFormat_hpp_
|
||||
@@ -2488,7 +2488,8 @@ namespace cereal {
|
||||
archive(serialization_key_ordinal);
|
||||
assert(serialization_key_ordinal > 0);
|
||||
auto it = Slic3r::print_config_def.by_serialization_key_ordinal.find(serialization_key_ordinal);
|
||||
assert(it != Slic3r::print_config_def.by_serialization_key_ordinal.end());
|
||||
if (it == Slic3r::print_config_def.by_serialization_key_ordinal.end())
|
||||
throw std::runtime_error("VendorCache: unknown serialization_key_ordinal " + std::to_string(serialization_key_ordinal) + " - cache is stale");
|
||||
config.set_key_value(it->second->opt_key, it->second->load_option_from_archive(archive));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -190,6 +190,19 @@ public:
|
||||
os << self.to_string();
|
||||
return os;
|
||||
}
|
||||
|
||||
// cereal: round-trip through the standard 3-part string (major.minor.patch).
|
||||
// to_string() uses a BBS 4-part format that semver_parse() cannot read back.
|
||||
template<class Archive>
|
||||
std::string save_minimal(const Archive&) const { return to_string_sf(); }
|
||||
template<class Archive>
|
||||
void load_minimal(const Archive&, const std::string& s) {
|
||||
auto v = Semver::parse(s);
|
||||
if (! v)
|
||||
throw std::runtime_error("Semver: cannot parse serialized version: " + s);
|
||||
*this = std::move(*v);
|
||||
}
|
||||
|
||||
private:
|
||||
semver_t ver;
|
||||
|
||||
|
||||
+38
-5
@@ -3,6 +3,7 @@
|
||||
|
||||
#include <iomanip>
|
||||
#include <locale>
|
||||
#include <set>
|
||||
#include <utility>
|
||||
#include <functional>
|
||||
#include <type_traits>
|
||||
@@ -18,6 +19,7 @@
|
||||
#include <openssl/md5.h>
|
||||
|
||||
#include "libslic3r.h"
|
||||
#include "Semver.hpp"
|
||||
|
||||
//define CLI errors
|
||||
|
||||
@@ -722,11 +724,42 @@ void copy_directory_recursively(const boost::filesystem::path& source,
|
||||
std::function<bool(const std::string)> filter = nullptr,
|
||||
bool merge_mode = false);
|
||||
|
||||
// Install vendor bundles from resources directory to data directory
|
||||
// bundle_names: vector of vendor bundle names (without .json extension)
|
||||
// resource_subdir: subdirectory under resources_dir() (default: "profiles")
|
||||
// data_subdir: subdirectory under data_dir() (default: "system")
|
||||
// Returns: true if all bundles installed successfully, false otherwise
|
||||
// ---- Vendor installation on disk ------------------------------------------
|
||||
// How a vendor bundle is installed from resources into data_dir()/system: as
|
||||
// its profile and preset JSONs or, in a build that ships preset caches, as its
|
||||
// .opc preset cache alone. Loading what is installed is PresetBundle's business;
|
||||
// the cache file format itself is VendorCacheFile's (PresetCacheFormat.hpp).
|
||||
|
||||
// True if `vendor` is installed in data_dir()/system. A build that ships preset
|
||||
// caches installs the cache alone, so it — not the profile — marks a vendor
|
||||
// installed; a cache this build cannot read marks nothing.
|
||||
bool is_vendor_installed(const std::string& vendor);
|
||||
|
||||
// The version the installed vendor would be loaded at: its cache's stamp while
|
||||
// that covers the profile beside it, the profile's own version once it does not.
|
||||
// Invalid Semver if neither form is installed.
|
||||
Semver installed_vendor_version(const std::string& vendor);
|
||||
|
||||
// Remove every form `vendor` can be installed as from data_dir()/system: its
|
||||
// profile, its preset cache, and its preset directory.
|
||||
void remove_installed_vendor(const std::string& vendor);
|
||||
|
||||
// The vendors `dir` holds, sorted: one is named by its profile or, in a build that
|
||||
// ships preset caches instead of the raw profile JSONs, by its cache alone.
|
||||
std::set<std::string> vendor_names_in(const boost::filesystem::path& dir);
|
||||
|
||||
// The version a build ships `vendor` at: whichever of its preset cache and its
|
||||
// profile is newer, that being the one installing lays down. Invalid Semver if the
|
||||
// build ships neither.
|
||||
Semver resource_vendor_version(const std::string& vendor);
|
||||
|
||||
// Install vendors from the resources directory into the data directory, each as
|
||||
// its preset cache or as its profile and preset JSONs — whichever of the two the
|
||||
// build ships at the newer version. Anything the previous install of that vendor
|
||||
// left behind goes, so only the form just installed is there to be loaded.
|
||||
// bundle_names: vendor names, without extension.
|
||||
// Every bundle that can be installed is, whatever the others do. Returns false
|
||||
// if any named bundle could not be installed.
|
||||
bool install_vendor_bundles_from_resources(const std::vector<std::string>& bundle_names,
|
||||
const std::string& resource_subdir = "profiles",
|
||||
const std::string& data_subdir = "system");
|
||||
|
||||
+141
-13
@@ -17,6 +17,10 @@
|
||||
#include "Platform.hpp"
|
||||
#include "Time.hpp"
|
||||
#include "libslic3r.h"
|
||||
// For the vendor-installation helpers: the vendor profile version
|
||||
// (get_version_from_json) and the preset cache stamp (VendorCacheFile).
|
||||
#include "Preset.hpp"
|
||||
#include "PresetCacheFormat.hpp"
|
||||
|
||||
#ifdef __APPLE__
|
||||
#include "MacUtils.hpp"
|
||||
@@ -1724,6 +1728,85 @@ void copy_directory_recursively(const boost::filesystem::path& source,
|
||||
return;
|
||||
}
|
||||
|
||||
// ---- Vendor installation on disk ------------------------------------------
|
||||
|
||||
// Whether a cache stamped `cache_ver` still speaks for a vendor whose profile on
|
||||
// disk claims `profile_ver`: it does unless the profile has moved ahead of it. A
|
||||
// profile that is missing or carries no judgeable version cannot be ahead of
|
||||
// anything. The one rule behind both "which form gets installed" and "which form
|
||||
// is installed"; they must not drift apart. Deliberately NOT the serve rule
|
||||
// (VendorCacheFile::load), which refuses an unjudgeable profile instead.
|
||||
static bool cache_covers(const Semver& cache_ver, const Semver& profile_ver)
|
||||
{
|
||||
return cache_ver.valid() && (! profile_ver.valid() || cache_ver >= profile_ver);
|
||||
}
|
||||
|
||||
bool is_vendor_installed(const std::string& vendor)
|
||||
{
|
||||
const boost::filesystem::path dir = boost::filesystem::path(data_dir()) / PRESET_SYSTEM_DIR;
|
||||
// A cache is the whole of a cache-only installation, so a file this build
|
||||
// cannot serve the vendor from is not an installation. Left counted as one,
|
||||
// the updater would never lay a working copy down.
|
||||
return boost::filesystem::exists(dir / (vendor + ".json"))
|
||||
|| VendorCacheFile::usable_version((dir / (vendor + ".opc")).string(), vendor).valid();
|
||||
}
|
||||
|
||||
Semver installed_vendor_version(const std::string& vendor)
|
||||
{
|
||||
const boost::filesystem::path dir = boost::filesystem::path(data_dir()) / PRESET_SYSTEM_DIR;
|
||||
const boost::filesystem::path json = dir / (vendor + ".json");
|
||||
// Guarded: get_version_from_json logs an error and throws-and-catches its way
|
||||
// to an invalid version on a file that is not there, and a cache-only vendor
|
||||
// never has one.
|
||||
const Semver from_json = boost::filesystem::exists(json) ? get_version_from_json(json.string()) : Semver();
|
||||
const Semver from_cache = VendorCacheFile::usable_version((dir / (vendor + ".opc")).string(), vendor);
|
||||
// Whichever form a load would serve.
|
||||
return cache_covers(from_cache, from_json) ? from_cache : from_json;
|
||||
}
|
||||
|
||||
void remove_installed_vendor(const std::string& vendor)
|
||||
{
|
||||
const boost::filesystem::path dir = boost::filesystem::path(data_dir()) / PRESET_SYSTEM_DIR;
|
||||
boost::filesystem::remove(dir / (vendor + ".json"));
|
||||
boost::filesystem::remove(dir / (vendor + ".opc"));
|
||||
if (boost::filesystem::exists(dir / vendor))
|
||||
boost::filesystem::remove_all(dir / vendor);
|
||||
}
|
||||
|
||||
std::set<std::string> vendor_names_in(const boost::filesystem::path& dir)
|
||||
{
|
||||
std::set<std::string> names;
|
||||
for (auto& dir_entry : boost::filesystem::directory_iterator(dir)) {
|
||||
const auto& path = dir_entry.path();
|
||||
if (Slic3r::is_json_file(path.string()) || path.extension() == ".opc")
|
||||
names.insert(path.stem().string());
|
||||
}
|
||||
return names;
|
||||
}
|
||||
|
||||
// A vendor's preset cache is the whole of its installation: it carries the presets,
|
||||
// the vendor profile and the version they were built at, so where one ships nothing
|
||||
// else needs copying. Unless the profile beside it claims a newer version — a cache
|
||||
// generated before that profile was bumped is out of date, and a cache that cannot
|
||||
// be read is no installation at all — and the vendor is installed the way it was
|
||||
// before caches existed, as its profile and the preset JSONs it points at. Returns
|
||||
// the version the cache is stamped with, invalid when it is not the form to install.
|
||||
static Semver installable_cache_version(const boost::filesystem::path& dir, const std::string& vendor)
|
||||
{
|
||||
const auto cache_ver = Semver::parse(VendorCacheFile::peek_version((dir / (vendor + ".opc")).string(), vendor));
|
||||
if (! cache_ver)
|
||||
return Semver::invalid();
|
||||
const Semver profile_ver = get_version_from_json((dir / (vendor + ".json")).string());
|
||||
return cache_covers(*cache_ver, profile_ver) ? *cache_ver : Semver::invalid();
|
||||
}
|
||||
|
||||
Semver resource_vendor_version(const std::string& vendor)
|
||||
{
|
||||
const boost::filesystem::path dir = boost::filesystem::path(resources_dir()) / "profiles";
|
||||
const Semver ver = installable_cache_version(dir, vendor);
|
||||
return ver.valid() ? ver : get_version_from_json((dir / (vendor + ".json")).string());
|
||||
}
|
||||
|
||||
bool install_vendor_bundles_from_resources(
|
||||
const std::vector<std::string>& bundle_names,
|
||||
const std::string& resource_subdir,
|
||||
@@ -1736,37 +1819,82 @@ bool install_vendor_bundles_from_resources(
|
||||
|
||||
BOOST_LOG_TRIVIAL(info) << "Installing " << bundle_names.size() << " bundles from resources...";
|
||||
|
||||
// One vendor that cannot be installed is one vendor missing, not a reason to
|
||||
// leave the rest uninstalled. The caller is told, and every bundle that can
|
||||
// be laid down is.
|
||||
bool all_installed = true;
|
||||
|
||||
for (const auto &bundle : bundle_names) {
|
||||
try {
|
||||
if (bundle.empty()) {
|
||||
BOOST_LOG_TRIVIAL(warning) << "Refusing to install a bundle with no name";
|
||||
all_installed = false;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Install the JSON file
|
||||
auto path_in_rsrc = (rsrc_path / bundle).replace_extension(".json");
|
||||
auto path_in_vendors = (vendor_path / bundle).replace_extension(".json");
|
||||
auto cache_in_rsrc = (rsrc_path / bundle).replace_extension(".opc");
|
||||
auto cache_in_vendors = (vendor_path / bundle).replace_extension(".opc");
|
||||
|
||||
if (!fs::exists(path_in_rsrc)) {
|
||||
// Either form of the vendor will do: a build may ship it as a cache alone.
|
||||
if (!fs::exists(path_in_rsrc) && !fs::exists(cache_in_rsrc)) {
|
||||
BOOST_LOG_TRIVIAL(warning) << "Bundle not found in resources: " << bundle;
|
||||
return false;
|
||||
all_installed = false;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Create target directory if needed
|
||||
if (!fs::exists(vendor_path))
|
||||
fs::create_directories(vendor_path);
|
||||
|
||||
// Copy JSON file
|
||||
std::string error_message;
|
||||
CopyFileResult cfr = copy_file(path_in_rsrc.string(), path_in_vendors.string(), error_message, false);
|
||||
if (cfr != CopyFileResult::SUCCESS) {
|
||||
BOOST_LOG_TRIVIAL(error) << "Failed to copy " << bundle << ".json: " << error_message;
|
||||
return false;
|
||||
bool installed_cache = false;
|
||||
if (installable_cache_version(rsrc_path, bundle).valid()) {
|
||||
installed_cache = copy_file(cache_in_rsrc.string(), cache_in_vendors.string(), error_message, false) == CopyFileResult::SUCCESS;
|
||||
if (! installed_cache) {
|
||||
BOOST_LOG_TRIVIAL(warning) << "Failed to copy " << bundle << ".opc: " << error_message;
|
||||
} else if (! VendorCacheFile::usable_version(cache_in_vendors.string(), bundle).valid()) {
|
||||
// The copy is what will be loaded, so it — not the kilobyte
|
||||
// peek that chose this form — decides whether the profile
|
||||
// beside it can go.
|
||||
BOOST_LOG_TRIVIAL(warning) << "Installed cache for " << bundle << " cannot be read; installing its profile instead";
|
||||
boost::system::error_code ec;
|
||||
fs::remove(cache_in_vendors, ec);
|
||||
installed_cache = false;
|
||||
}
|
||||
}
|
||||
|
||||
if (! installed_cache) {
|
||||
CopyFileResult cfr = copy_file(path_in_rsrc.string(), path_in_vendors.string(), error_message, false);
|
||||
if (cfr != CopyFileResult::SUCCESS) {
|
||||
BOOST_LOG_TRIVIAL(error) << "Failed to copy " << bundle << ".json: " << error_message;
|
||||
all_installed = false;
|
||||
continue;
|
||||
}
|
||||
// Only now: an earlier install's cache would shadow this profile,
|
||||
// but removing it before the profile lands would leave neither.
|
||||
boost::system::error_code ec;
|
||||
fs::remove(cache_in_vendors, ec);
|
||||
} else {
|
||||
// Left in place, an earlier install's profile would shadow the cache.
|
||||
boost::system::error_code ec;
|
||||
fs::remove(path_in_vendors, ec);
|
||||
if (ec)
|
||||
BOOST_LOG_TRIVIAL(warning) << "Could not remove the superseded profile " << path_in_vendors.string() << ": " << ec.message();
|
||||
}
|
||||
|
||||
// Copy the vendor directory (if it exists)
|
||||
auto dir_in_rsrc = rsrc_path / bundle;
|
||||
auto dir_in_vendors = vendor_path / bundle;
|
||||
|
||||
if (fs::exists(dir_in_rsrc) && fs::is_directory(dir_in_rsrc)) {
|
||||
// Remove existing directory
|
||||
if (fs::exists(dir_in_vendors))
|
||||
fs::remove_all(dir_in_vendors);
|
||||
// Whatever is installed came from an earlier version of this vendor and
|
||||
// would be parsed in place of the one being installed now.
|
||||
if (fs::exists(dir_in_vendors))
|
||||
fs::remove_all(dir_in_vendors);
|
||||
|
||||
if (! installed_cache && fs::exists(dir_in_rsrc) && fs::is_directory(dir_in_rsrc)) {
|
||||
fs::create_directories(dir_in_vendors);
|
||||
|
||||
// Copy with file filter (same as PresetUpdater::install_bundles_rsrc)
|
||||
@@ -1787,11 +1915,11 @@ bool install_vendor_bundles_from_resources(
|
||||
|
||||
} catch (const std::exception& e) {
|
||||
BOOST_LOG_TRIVIAL(error) << "Exception installing bundle " << bundle << ": " << e.what();
|
||||
return false;
|
||||
all_installed = false;
|
||||
}
|
||||
}
|
||||
|
||||
return true;
|
||||
return all_installed;
|
||||
}
|
||||
|
||||
void save_string_file(const boost::filesystem::path& p, const std::string& str)
|
||||
|
||||
@@ -432,14 +432,9 @@ const Snapshot& SnapshotDB::take_snapshot(const AppConfig &app_config, Snapshot:
|
||||
cfg.models_variants_installed.erase(it ++);
|
||||
else
|
||||
++ it;
|
||||
// Read the active config bundle, parse the config version.
|
||||
PresetBundle bundle;
|
||||
//BBS: change directoties by design
|
||||
//bundle.load_configbundle((data_dir / PRESET_SYSTEM_DIR / (cfg.name + ".ini")).string(), PresetBundle::LoadConfigBundleAttribute::LoadVendorOnly, ForwardCompatibilitySubstitutionRule::EnableSilent);
|
||||
bundle.load_vendor_configs_from_json((data_dir/PRESET_SYSTEM_DIR).string(), cfg.name, PresetBundle::LoadConfigBundleAttribute::LoadVendorOnly, ForwardCompatibilitySubstitutionRule::EnableSilent);
|
||||
for (const auto &vp : bundle.vendors)
|
||||
if (vp.second.id == cfg.name)
|
||||
cfg.version.config_version = vp.second.config_version;
|
||||
// Orca: the version the vendor is installed at, read from its profile or —
|
||||
// where the cache is the whole installation — from the cache's own stamp.
|
||||
cfg.version.config_version = installed_vendor_version(cfg.name);
|
||||
snapshot.vendor_configs.emplace_back(std::move(cfg));
|
||||
}
|
||||
|
||||
|
||||
@@ -66,41 +66,41 @@ using Config::SnapshotDB;
|
||||
|
||||
// Configuration data structures extensions needed for the wizard
|
||||
//BBS: set BBL as default
|
||||
bool Bundle::load(fs::path source_path, bool ais_in_resources, bool ais_bbl_bundle)
|
||||
bool Bundle::load(fs::path dir, const std::string &vendor_name, bool ais_in_resources, bool ais_bbl_bundle)
|
||||
{
|
||||
this->preset_bundle = std::make_unique<PresetBundle>();
|
||||
this->is_in_resources = ais_in_resources;
|
||||
this->is_bbl_bundle = ais_bbl_bundle;
|
||||
|
||||
std::string path_string = source_path.string();
|
||||
std::string parent_path = source_path.parent_path().string();
|
||||
//BBS: add json logic for vendor bundles
|
||||
std::string vendor_name = source_path.filename().string();
|
||||
if (Slic3r::is_json_file(path_string)) {
|
||||
// Remove the .json suffix.
|
||||
vendor_name.erase(vendor_name.size() - 5);
|
||||
}
|
||||
else
|
||||
// Orca: served from the vendor's preset cache where one covers it — which is
|
||||
// how a shipped build carries its vendors — and parsed from the JSONs otherwise.
|
||||
// A vendor that can be neither read nor parsed — a cache the build cannot use
|
||||
// with the preset JSONs behind it pruned, say — is one the wizard cannot offer.
|
||||
// Every other vendor still can be, so it is left out rather than thrown over.
|
||||
size_t presets_loaded = 0;
|
||||
try {
|
||||
auto [config_substitutions, loaded] = preset_bundle->load_vendor_configs_from_json(
|
||||
dir.string(), vendor_name, PresetBundle::LoadConfigBundleAttribute::LoadSystem, ForwardCompatibilitySubstitutionRule::Disable);
|
||||
UNUSED(config_substitutions);
|
||||
// No substitutions shall be reported when loading a system config bundle, no substitutions are allowed.
|
||||
assert(config_substitutions.empty());
|
||||
presets_loaded = loaded;
|
||||
} catch (const std::exception &e) {
|
||||
BOOST_LOG_TRIVIAL(fatal) << boost::format("Vendor bundle: `%1%`: cannot be loaded, leaving it out: %2%") % vendor_name % e.what();
|
||||
return false;
|
||||
|
||||
// Throw when parsing invalid configuration. Only valid configuration is supposed to be provided over the air.
|
||||
//BBS: add json logic for vendor bundles
|
||||
auto [config_substitutions, presets_loaded] = preset_bundle->load_vendor_configs_from_json(
|
||||
parent_path, vendor_name, PresetBundle::LoadConfigBundleAttribute::LoadSystem, ForwardCompatibilitySubstitutionRule::Disable);
|
||||
UNUSED(config_substitutions);
|
||||
// No substitutions shall be reported when loading a system config bundle, no substitutions are allowed.
|
||||
assert(config_substitutions.empty());
|
||||
}
|
||||
auto first_vendor = preset_bundle->vendors.begin();
|
||||
if (first_vendor == preset_bundle->vendors.end()) {
|
||||
BOOST_LOG_TRIVIAL(error) << boost::format("Vendor bundle: `%1%`: No vendor information defined, cannot install.") % path_string;
|
||||
BOOST_LOG_TRIVIAL(error) << boost::format("Vendor bundle: `%1%`: No vendor information defined, cannot install.") % vendor_name;
|
||||
return false;
|
||||
}
|
||||
if (presets_loaded == 0) {
|
||||
BOOST_LOG_TRIVIAL(error) << boost::format("Vendor bundle: `%1%`: No profile loaded.") % path_string;
|
||||
BOOST_LOG_TRIVIAL(error) << boost::format("Vendor bundle: `%1%`: No profile loaded.") % vendor_name;
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
BOOST_LOG_TRIVIAL(trace) << boost::format("Vendor bundle: `%1%`: %2% profiles loaded.") % path_string % presets_loaded;
|
||||
BOOST_LOG_TRIVIAL(trace) << boost::format("Vendor bundle: `%1%`: %2% profiles loaded.") % vendor_name % presets_loaded;
|
||||
this->vendor_profile = &first_vendor->second;
|
||||
return true;
|
||||
}
|
||||
@@ -125,15 +125,10 @@ BundleMap BundleMap::load()
|
||||
|
||||
//Orca: add custom as default
|
||||
//Orca: add json logic for vendor bundle
|
||||
auto orca_bundle_path = (vendor_dir / PresetBundle::ORCA_DEFAULT_BUNDLE).replace_extension(".json");
|
||||
auto orca_bundle_rsrc = false;
|
||||
if (!boost::filesystem::exists(orca_bundle_path)) {
|
||||
orca_bundle_path = (rsrc_vendor_dir / PresetBundle::ORCA_DEFAULT_BUNDLE).replace_extension(".json");
|
||||
orca_bundle_rsrc = true;
|
||||
}
|
||||
{
|
||||
const bool from_rsrc = ! is_vendor_installed(PresetBundle::ORCA_DEFAULT_BUNDLE);
|
||||
Bundle bbl_bundle;
|
||||
if (bbl_bundle.load(std::move(orca_bundle_path), orca_bundle_rsrc, true))
|
||||
if (bbl_bundle.load(from_rsrc ? rsrc_vendor_dir : vendor_dir, PresetBundle::ORCA_DEFAULT_BUNDLE, from_rsrc, true))
|
||||
res.emplace(PresetBundle::ORCA_DEFAULT_BUNDLE, std::move(bbl_bundle));
|
||||
}
|
||||
|
||||
@@ -141,18 +136,13 @@ BundleMap BundleMap::load()
|
||||
// and then additionally from resources/profiles.
|
||||
bool is_in_resources = false;
|
||||
for (auto dir : { &vendor_dir, &rsrc_vendor_dir }) {
|
||||
for (const auto &dir_entry : boost::filesystem::directory_iterator(*dir)) {
|
||||
//BBS: add json logic for vendor bundle
|
||||
if (Slic3r::is_json_file(dir_entry.path().string())) {
|
||||
std::string id = dir_entry.path().stem().string(); // stem() = filename() without the trailing ".json" part
|
||||
for (const std::string &id : vendor_names_in(*dir)) {
|
||||
// Don't load this bundle if we've already loaded it.
|
||||
if (res.find(id) != res.end()) { continue; }
|
||||
|
||||
// Don't load this bundle if we've already loaded it.
|
||||
if (res.find(id) != res.end()) { continue; }
|
||||
|
||||
Bundle bundle;
|
||||
if (bundle.load(dir_entry.path(), is_in_resources))
|
||||
res.emplace(std::move(id), std::move(bundle));
|
||||
}
|
||||
Bundle bundle;
|
||||
if (bundle.load(*dir, id, is_in_resources))
|
||||
res.emplace(id, std::move(bundle));
|
||||
}
|
||||
|
||||
is_in_resources = true;
|
||||
|
||||
@@ -71,9 +71,11 @@ struct Bundle
|
||||
Bundle() = default;
|
||||
Bundle(Bundle&& other);
|
||||
|
||||
// Load the vendor `vendor_name` as it is installed in `dir`, from its preset
|
||||
// cache or its profile JSONs, whichever is usable.
|
||||
// Returns false if not loaded. Reason for that is logged as boost::log error.
|
||||
//BBS: set BBL as default
|
||||
bool load(fs::path source_path, bool is_in_resources, bool is_bbl_bundle = false);
|
||||
bool load(fs::path dir, const std::string &vendor_name, bool is_in_resources, bool is_bbl_bundle = false);
|
||||
|
||||
const std::string& vendor_id() const { return vendor_profile->id; }
|
||||
};
|
||||
|
||||
@@ -2201,25 +2201,14 @@ bool CreatePrinterPresetDialog::load_system_and_user_presets_with_curr_model(Pre
|
||||
} else {
|
||||
selected_vendor_id = m_printer_preset_vendor_selected.id;
|
||||
|
||||
if (boost::filesystem::exists(boost::filesystem::path(Slic3r::data_dir()) / PRESET_SYSTEM_DIR / selected_vendor_id)) {
|
||||
preset_path = (boost::filesystem::path(Slic3r::data_dir()) / PRESET_SYSTEM_DIR).string();
|
||||
} else if (boost::filesystem::exists(boost::filesystem::path(Slic3r::resources_dir()) / "profiles" / selected_vendor_id)) {
|
||||
preset_path = (boost::filesystem::path(Slic3r::resources_dir()) / "profiles").string();
|
||||
}
|
||||
|
||||
if (preset_path.empty()) {
|
||||
BOOST_LOG_TRIVIAL(info) << "Preset path was not found";
|
||||
MessageDialog dlg(this, _L("Preset path was not found; please reselect vendor."), wxString(SLIC3R_APP_FULL_NAME) + " - " + _L("Info"),
|
||||
wxYES_NO | wxYES_DEFAULT | wxCENTRE);
|
||||
dlg.ShowModal();
|
||||
return false;
|
||||
}
|
||||
|
||||
try {
|
||||
// Pass the app's preset bundle (which already holds OrcaFilamentLibrary) as the base
|
||||
// bundle so vendor filaments that inherit OFL bases resolve via the existing
|
||||
// cross-vendor inheritance path.
|
||||
temp_preset_bundle.load_vendor_configs_from_json(preset_path, selected_vendor_id,
|
||||
// Orca: served from the vendor's preset cache where one covers it — a shipped
|
||||
// build carries that instead of the raw preset JSONs — and parsed otherwise.
|
||||
temp_preset_bundle.load_vendor_configs_from_json((boost::filesystem::path(Slic3r::data_dir()) / PRESET_SYSTEM_DIR).string(),
|
||||
selected_vendor_id,
|
||||
PresetBundle::LoadConfigBundleAttribute::LoadSystem,
|
||||
ForwardCompatibilitySubstitutionRule::EnableSilent,
|
||||
wxGetApp().preset_bundle);
|
||||
|
||||
@@ -6817,6 +6817,12 @@ void GUI_App::add_pending_vendor_preset(const std::pair<std::string, std::map<st
|
||||
|
||||
// Add the corresponding vendor
|
||||
std::string vendor_name = PresetBundle::find_preset_vendor(inherits_name, type);
|
||||
if (vendor_name.empty()) {
|
||||
// No vendor ships this preset's parent. An unnamed entry here becomes an
|
||||
// unnamed bundle at install time, which nothing can install.
|
||||
BOOST_LOG_TRIVIAL(warning) << __FUNCTION__ << ": no vendor carries " << inherits_name << ", skipping";
|
||||
return;
|
||||
}
|
||||
if (need_add_vendors.find(vendor_name) == need_add_vendors.end())
|
||||
need_add_vendors[vendor_name] = std::map<std::string, std::set<std::string>>();
|
||||
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
#include "WebGuideDialog.hpp"
|
||||
#include "ConfigWizard.hpp"
|
||||
|
||||
#include <boost/algorithm/string/join.hpp>
|
||||
#include <boost/filesystem/operations.hpp>
|
||||
#include <boost/nowide/fstream.hpp>
|
||||
#include <boost/filesystem/path.hpp>
|
||||
#include <boost/iostreams/detail/select.hpp>
|
||||
#include <boost/log/trivial.hpp>
|
||||
@@ -9,7 +11,9 @@
|
||||
#include "I18N.hpp"
|
||||
#include "libslic3r/AppConfig.hpp"
|
||||
#include "libslic3r/Config.hpp"
|
||||
#include "libslic3r/Preset.hpp"
|
||||
#include "libslic3r/PresetBundle.hpp"
|
||||
#include "libslic3r/PresetCacheFormat.hpp"
|
||||
#include "slic3r/GUI/wxExtensions.hpp"
|
||||
#include "slic3r/GUI/GUI_App.hpp"
|
||||
#include "libslic3r_version.h"
|
||||
@@ -41,8 +45,6 @@ using namespace nlohmann;
|
||||
|
||||
namespace Slic3r { namespace GUI {
|
||||
|
||||
json m_ProfileJson;
|
||||
|
||||
static wxString update_custom_filaments()
|
||||
{
|
||||
json m_Res = json::object();
|
||||
@@ -190,12 +192,10 @@ GuideFrame::GuideFrame(GUI_App *pGUI, long style)
|
||||
|
||||
GuideFrame::~GuideFrame()
|
||||
{
|
||||
m_destroy = true;
|
||||
if (m_load_task && m_load_task->joinable()) {
|
||||
*m_cancel_token = true; // stop the loading thread and any queued CallAfter lambdas before join
|
||||
if (m_load_task && m_load_task->joinable())
|
||||
m_load_task->join();
|
||||
delete m_load_task;
|
||||
m_load_task = nullptr;
|
||||
}
|
||||
m_load_task.reset();
|
||||
if (m_browser) {
|
||||
delete m_browser;
|
||||
m_browser = nullptr;
|
||||
@@ -301,15 +301,71 @@ void GuideFrame::OnNavigationRequest(wxWebViewEvent &evt)
|
||||
/**
|
||||
* Callback invoked when a navigation request was accepted
|
||||
*/
|
||||
// The empty shape every profile-loading path starts from or falls back to.
|
||||
void GuideFrame::reset_profile_json()
|
||||
{
|
||||
m_ProfileJson["model"] = json::array();
|
||||
m_ProfileJson["machine"] = json::object();
|
||||
m_ProfileJson["filament"] = json::object();
|
||||
m_ProfileJson["process"] = json::array();
|
||||
}
|
||||
|
||||
void GuideFrame::init_guide_paths()
|
||||
{
|
||||
m_ProfileJson = json::parse("{}");
|
||||
reset_profile_json();
|
||||
|
||||
vendor_dir = (boost::filesystem::path(Slic3r::data_dir()) / PRESET_SYSTEM_DIR).make_preferred();
|
||||
rsrc_vendor_dir = (boost::filesystem::path(resources_dir()) / "profiles").make_preferred();
|
||||
orca_bundle_rsrc = true;
|
||||
|
||||
if (boost::filesystem::exists(vendor_dir)) {
|
||||
for (const auto& entry : boost::filesystem::directory_iterator(vendor_dir)) {
|
||||
if (!boost::filesystem::is_directory(entry) &&
|
||||
boost::iequals(entry.path().extension().string(), ".json") &&
|
||||
!boost::iequals(entry.path().stem().string(), PresetBundle::ORCA_FILAMENT_LIBRARY)) {
|
||||
orca_bundle_rsrc = false;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
auto lib_json = boost::filesystem::path(PresetBundle::ORCA_FILAMENT_LIBRARY).replace_extension(".json");
|
||||
m_OrcaFilaLibPath = boost::filesystem::exists(vendor_dir / lib_json)
|
||||
? (vendor_dir / PresetBundle::ORCA_FILAMENT_LIBRARY).string()
|
||||
: (rsrc_vendor_dir / PresetBundle::ORCA_FILAMENT_LIBRARY).string();
|
||||
}
|
||||
|
||||
void GuideFrame::on_profile_loaded()
|
||||
{
|
||||
// Must be called on the main thread.
|
||||
SaveProfileData();
|
||||
const std::string strAll = m_ProfileJson.dump(-1, ' ', false, json::error_handler_t::ignore);
|
||||
BOOST_LOG_TRIVIAL(info) << __FUNCTION__ << ", finished, json contents:\n" << strAll;
|
||||
json res;
|
||||
res["command"] = "userguide_profile_load_finish";
|
||||
res["sequence_id"] = "10001";
|
||||
RunScript(wxString::Format("HandleStudio(%s)", res.dump(-1, ' ', true)));
|
||||
}
|
||||
|
||||
void GuideFrame::OnNavigationComplete(wxWebViewEvent &evt)
|
||||
{
|
||||
//wxLogMessage("%s", "Navigation complete; url='" + evt.GetURL() + "'");
|
||||
if (!bFirstComplete) {
|
||||
m_load_task = new boost::thread(boost::bind(&GuideFrame::LoadProfileData, this));
|
||||
// boost::thread LoadProfileThread(boost::bind(&GuideFrame::LoadProfileData, this));
|
||||
//LoadProfileThread.detach();
|
||||
|
||||
bFirstComplete = true;
|
||||
try {
|
||||
init_guide_paths();
|
||||
if (BuildProfileDataFromPresetBundle()) {
|
||||
if (!*m_cancel_token)
|
||||
on_profile_loaded();
|
||||
} else {
|
||||
// Presets not yet in memory — delegate to background thread.
|
||||
m_load_task = std::make_unique<boost::thread>(boost::bind(&GuideFrame::LoadProfileData, this));
|
||||
}
|
||||
} catch (const std::exception& e) {
|
||||
BOOST_LOG_TRIVIAL(error) << __FUNCTION__ << ", init error: " << e.what();
|
||||
m_load_task = std::make_unique<boost::thread>(boost::bind(&GuideFrame::LoadProfileData, this));
|
||||
}
|
||||
}
|
||||
|
||||
m_browser->Show();
|
||||
@@ -762,11 +818,9 @@ bool GuideFrame::apply_config(AppConfig *app_config, PresetBundle *preset_bundle
|
||||
bool check_unsaved_preset_changes = false;
|
||||
std::vector<std::string> install_bundles;
|
||||
std::vector<std::string> remove_bundles;
|
||||
const auto vendor_dir = (boost::filesystem::path(Slic3r::data_dir()) / PRESET_SYSTEM_DIR).make_preferred();
|
||||
for (const auto &it : enabled_vendors) {
|
||||
if (it.second.size() > 0) {
|
||||
auto vendor_file = vendor_dir/(it.first + ".json");
|
||||
if (!fs::exists(vendor_file)) {
|
||||
if (!is_vendor_installed(it.first)) {
|
||||
install_bundles.emplace_back(it.first);
|
||||
}
|
||||
}
|
||||
@@ -777,8 +831,7 @@ bool GuideFrame::apply_config(AppConfig *app_config, PresetBundle *preset_bundle
|
||||
if (it.second.size() > 0) {
|
||||
if (enabled_vendors.find(it.first) != enabled_vendors.end())
|
||||
continue;
|
||||
auto vendor_file = vendor_dir/(it.first + ".json");
|
||||
if (fs::exists(vendor_file)) {
|
||||
if (is_vendor_installed(it.first)) {
|
||||
remove_bundles.emplace_back(it.first);
|
||||
}
|
||||
}
|
||||
@@ -1127,99 +1180,324 @@ int GuideFrame::GetFilamentInfo( std::string VendorDirectory, json & pFilaList,
|
||||
return status;
|
||||
}
|
||||
|
||||
int GuideFrame::LoadProfileData()
|
||||
bool GuideFrame::BuildProfileJson(const PresetBundle& bundle, bool require_all_resource_vendors)
|
||||
{
|
||||
try {
|
||||
m_ProfileJson = json::parse("{}");
|
||||
m_ProfileJson["model"] = json::array();
|
||||
m_ProfileJson["machine"] = json::object();
|
||||
m_ProfileJson["filament"] = json::object();
|
||||
m_ProfileJson["process"] = json::array();
|
||||
// Models from vendor profiles
|
||||
for (const auto& [vendor_id, vp] : bundle.vendors) {
|
||||
for (const auto& model : vp.models) {
|
||||
std::string nozzle_str;
|
||||
for (const auto& v : model.variants) {
|
||||
if (!nozzle_str.empty()) nozzle_str += ";";
|
||||
nozzle_str += v.name;
|
||||
}
|
||||
const std::string materials_str = boost::algorithm::join(model.default_materials, ";");
|
||||
boost::filesystem::path cover_path =
|
||||
(boost::filesystem::path(resources_dir()) / "profiles" / vp.id / (model.id + "_cover.png"))
|
||||
.make_preferred();
|
||||
if (!boost::filesystem::exists(cover_path))
|
||||
cover_path =
|
||||
(boost::filesystem::path(resources_dir()) / "web/image/printer" / (model.id + "_cover.png"))
|
||||
.make_preferred();
|
||||
|
||||
vendor_dir = (boost::filesystem::path(Slic3r::data_dir()) / PRESET_SYSTEM_DIR).make_preferred();
|
||||
rsrc_vendor_dir = (boost::filesystem::path(resources_dir()) / "profiles").make_preferred();
|
||||
|
||||
// Orca: add custom as default
|
||||
// Orca: add json logic for vendor bundle
|
||||
orca_bundle_rsrc = true;
|
||||
|
||||
// search if there exists a .json file in vendor_dir folder, if exists, set orca_bundle_rsrc to false
|
||||
for (const auto& entry : boost::filesystem::directory_iterator(vendor_dir)) {
|
||||
if (!boost::filesystem::is_directory(entry) && boost::iequals(entry.path().extension().string(), ".json") && !boost::iequals(entry.path().stem().string(), PresetBundle::ORCA_FILAMENT_LIBRARY)) {
|
||||
orca_bundle_rsrc = false;
|
||||
break;
|
||||
json entry;
|
||||
entry["model"] = model.id;
|
||||
entry["name"] = model.name;
|
||||
entry["vendor"] = vp.id;
|
||||
entry["nozzle_diameter"] = nozzle_str;
|
||||
entry["materials"] = materials_str;
|
||||
entry["cover"] = cover_path.string();
|
||||
entry["nozzle_selected"] = "";
|
||||
entry["sub_path"] = "";
|
||||
m_ProfileJson["model"].push_back(entry);
|
||||
}
|
||||
}
|
||||
|
||||
// load the default filament library first
|
||||
std::set<std::string> loaded_vendors;
|
||||
auto filament_library_name = boost::filesystem::path(PresetBundle::ORCA_FILAMENT_LIBRARY).replace_extension(".json");
|
||||
if (boost::filesystem::exists(vendor_dir / filament_library_name)) {
|
||||
m_OrcaFilaLibPath = (vendor_dir / PresetBundle::ORCA_FILAMENT_LIBRARY).string();
|
||||
LoadProfileFamily(PresetBundle::ORCA_FILAMENT_LIBRARY, (vendor_dir / filament_library_name).string());
|
||||
} else {
|
||||
m_OrcaFilaLibPath = (rsrc_vendor_dir / PresetBundle::ORCA_FILAMENT_LIBRARY).string();
|
||||
LoadProfileFamily(PresetBundle::ORCA_FILAMENT_LIBRARY, (rsrc_vendor_dir / filament_library_name).string());
|
||||
}
|
||||
loaded_vendors.insert(PresetBundle::ORCA_FILAMENT_LIBRARY);
|
||||
// Machine map: preset name -> {model, nozzle variant}
|
||||
for (const Preset& p : bundle.printers()) {
|
||||
if (!p.is_system || !p.vendor) continue;
|
||||
const auto* printer_model = p.config.option<ConfigOptionString>("printer_model");
|
||||
const auto* printer_variant = p.config.option<ConfigOptionString>("printer_variant");
|
||||
if (!printer_model || printer_model->value.empty() || !printer_variant) continue;
|
||||
|
||||
//load custom bundle from user data path
|
||||
boost::filesystem::directory_iterator endIter;
|
||||
for (boost::filesystem::directory_iterator iter(vendor_dir); iter != endIter; iter++) {
|
||||
if (!boost::filesystem::is_directory(*iter)) {
|
||||
wxString strVendor = from_u8(iter->path().string()).BeforeLast('.');
|
||||
strVendor = strVendor.AfterLast('\\');
|
||||
strVendor = strVendor.AfterLast('/');
|
||||
|
||||
wxString strExtension = from_u8(iter->path().string()).AfterLast('.').Lower();
|
||||
if(strExtension.CmpNoCase("json") != 0 || loaded_vendors.find(w2s(strVendor)) != loaded_vendors.end())
|
||||
continue;
|
||||
|
||||
LoadProfileFamily(w2s(strVendor), iter->path().string());
|
||||
loaded_vendors.insert(w2s(strVendor));
|
||||
}
|
||||
if (m_destroy)
|
||||
return 0;
|
||||
json mach;
|
||||
mach["model"] = printer_model->value;
|
||||
mach["nozzle"] = printer_variant->value;
|
||||
m_ProfileJson["machine"][p.name] = mach;
|
||||
}
|
||||
|
||||
boost::filesystem::directory_iterator others_endIter;
|
||||
for (boost::filesystem::directory_iterator iter(rsrc_vendor_dir); iter != others_endIter; iter++) {
|
||||
if (!boost::filesystem::is_directory(*iter)) {
|
||||
wxString strVendor = from_u8(iter->path().string()).BeforeLast('.');
|
||||
strVendor = strVendor.AfterLast('\\');
|
||||
strVendor = strVendor.AfterLast('/');
|
||||
wxString strExtension = from_u8(iter->path().string()).AfterLast('.').Lower();
|
||||
if (strExtension.CmpNoCase("json") != 0 || loaded_vendors.find(w2s(strVendor)) != loaded_vendors.end())
|
||||
continue;
|
||||
// Filament map from system filament presets (vendor/type already resolved in config)
|
||||
const json& machines = m_ProfileJson["machine"];
|
||||
for (const Preset& p : bundle.filaments()) {
|
||||
if (!p.is_system || !p.vendor) continue;
|
||||
const auto* fila_vendor = p.config.option<ConfigOptionStrings>("filament_vendor");
|
||||
const auto* fila_type = p.config.option<ConfigOptionStrings>("filament_type");
|
||||
const auto* compat_printers = p.config.option<ConfigOptionStrings>("compatible_printers");
|
||||
|
||||
LoadProfileFamily(w2s(strVendor), iter->path().string());
|
||||
loaded_vendors.insert(w2s(strVendor));
|
||||
std::string vendor = (fila_vendor && !fila_vendor->values.empty()) ? fila_vendor->values[0] : "";
|
||||
std::string type = (fila_type && !fila_type->values.empty()) ? fila_type->values[0] : "";
|
||||
|
||||
std::string model_list;
|
||||
if (compat_printers) {
|
||||
for (const std::string& pname : compat_printers->values) {
|
||||
auto it = machines.find(pname);
|
||||
if (it != machines.end()) {
|
||||
const std::string m = (*it)["model"];
|
||||
const std::string n = (*it)["nozzle"];
|
||||
model_list += "[" + m + "++" + n + "]";
|
||||
}
|
||||
}
|
||||
}
|
||||
if (m_destroy)
|
||||
return 0;
|
||||
|
||||
json ff;
|
||||
ff["name"] = p.name;
|
||||
ff["sub_path"] = p.file;
|
||||
ff["vendor"] = vendor;
|
||||
ff["type"] = type;
|
||||
ff["models"] = model_list;
|
||||
ff["selected"] = 0;
|
||||
m_ProfileJson["filament"][p.name] = ff;
|
||||
}
|
||||
|
||||
wxGetApp().CallAfter([this] {
|
||||
if (!m_destroy) {
|
||||
//sync to appconfig first to populate current selections
|
||||
SaveProfileData();
|
||||
// Process list from visible system print presets
|
||||
for (const Preset& p : bundle.prints()) {
|
||||
if (!p.is_system || !p.vendor || !p.is_visible) continue;
|
||||
json entry;
|
||||
entry["name"] = p.name;
|
||||
entry["sub_path"] = p.file;
|
||||
m_ProfileJson["process"].push_back(entry);
|
||||
}
|
||||
|
||||
//sync to web after selections are populated
|
||||
std::string strAll = m_ProfileJson.dump(-1, ' ', false, json::error_handler_t::ignore);
|
||||
if (require_all_resource_vendors) {
|
||||
// If rsrc_vendor_dir has vendors (profile JSONs, or the preset caches a
|
||||
// packaged build ships instead) not covered by the current bundle, the
|
||||
// bundle is incomplete (e.g. dev env where data_dir/system only has
|
||||
// OrcaFilamentLibrary+Custom). Fall back so the slow path reads both dirs.
|
||||
try {
|
||||
for (const std::string& name : vendor_names_in(rsrc_vendor_dir)) {
|
||||
if (bundle.vendors.find(name) == bundle.vendors.end()) {
|
||||
BOOST_LOG_TRIVIAL(info) << "GuideFrame: vendor '" << name
|
||||
<< "' in resources but not in preset_bundle — falling back to JSON loading";
|
||||
reset_profile_json();
|
||||
return false;
|
||||
}
|
||||
}
|
||||
} catch (const std::exception&) {}
|
||||
}
|
||||
|
||||
BOOST_LOG_TRIVIAL(info) << __FUNCTION__ << ", finished, json contents: " << std::endl << strAll;
|
||||
json m_Res = json::object();
|
||||
m_Res["command"] = "userguide_profile_load_finish";
|
||||
m_Res["sequence_id"] = "10001";
|
||||
wxString strJS = wxString::Format("HandleStudio(%s)", m_Res.dump(-1, ' ', true));
|
||||
BOOST_LOG_TRIVIAL(info) << "GuideFrame: built profile data ("
|
||||
<< m_ProfileJson["model"].size() << " models, "
|
||||
<< m_ProfileJson["machine"].size() << " machines, "
|
||||
<< m_ProfileJson["filament"].size() << " filaments)";
|
||||
return !m_ProfileJson["machine"].empty();
|
||||
} catch (const std::exception& e) {
|
||||
BOOST_LOG_TRIVIAL(warning) << "GuideFrame::BuildProfileJson failed: " << e.what()
|
||||
<< " — falling back to JSON loading";
|
||||
reset_profile_json();
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
RunScript(strJS);
|
||||
bool GuideFrame::BuildProfileDataFromPresetBundle()
|
||||
{
|
||||
PresetBundle* pb = wxGetApp().preset_bundle;
|
||||
if (!pb || pb->vendors.empty())
|
||||
return false;
|
||||
return BuildProfileJson(*pb, /*require_all_resource_vendors=*/true);
|
||||
}
|
||||
|
||||
bool GuideFrame::BuildProfileDataFromVendors()
|
||||
{
|
||||
try {
|
||||
// Same vendor set and precedence as the JSON scan in LoadProfileData: a
|
||||
// vendor in the user's system dir shadows the bundled one of that name.
|
||||
// vendor_names_in names a vendor by its profile or, where a build ships
|
||||
// preset caches instead, by its cache alone.
|
||||
std::map<std::string, boost::filesystem::path> vendor_sources;
|
||||
for (const boost::filesystem::path& dir : { vendor_dir, rsrc_vendor_dir }) {
|
||||
boost::system::error_code ec;
|
||||
if (boost::filesystem::exists(dir, ec))
|
||||
for (const std::string& name : vendor_names_in(dir))
|
||||
vendor_sources.emplace(name, dir); // first dir wins
|
||||
}
|
||||
|
||||
// The load order: the filament library first, because the others'
|
||||
// filaments inherit from it, then every versioned vendor — each loaded
|
||||
// from the directory it was found in, so a vendor that is not installed
|
||||
// is served from the shipped profiles. Each is stamped by name and
|
||||
// version alone: a profile change requires a version bump, so those two
|
||||
// determine content wherever the vendor's copy sits.
|
||||
struct VendorSource { std::string name; boost::filesystem::path dir; std::string version; };
|
||||
std::vector<VendorSource> ordered;
|
||||
auto add_vendor = [&ordered](const std::string& name, const boost::filesystem::path& dir) {
|
||||
// The version a load from `dir` would serve: the profile's where one
|
||||
// exists (a cache is only served while it covers the profile beside
|
||||
// it), the cache's own stamp where the cache is the whole vendor.
|
||||
// A profile without a version (blacklist.json) carries no presets
|
||||
// and is passed over.
|
||||
const boost::filesystem::path profile = dir / (name + ".json");
|
||||
if (boost::filesystem::exists(profile)) {
|
||||
const Semver v = get_version_from_json(profile.string());
|
||||
if (v.valid())
|
||||
ordered.push_back({name, dir, v.to_string()});
|
||||
} else {
|
||||
ordered.push_back({name, dir,
|
||||
VendorCacheFile::peek_version((dir / (name + ".opc")).string(), name)});
|
||||
}
|
||||
};
|
||||
const std::string filament_library(PresetBundle::ORCA_FILAMENT_LIBRARY);
|
||||
if (auto it = vendor_sources.find(filament_library); it != vendor_sources.end())
|
||||
add_vendor(filament_library, it->second);
|
||||
for (const auto& [name, dir] : vendor_sources)
|
||||
if (name != filament_library)
|
||||
add_vendor(name, dir);
|
||||
if (ordered.empty())
|
||||
return false;
|
||||
json stamps = json::array();
|
||||
for (const VendorSource& v : ordered)
|
||||
stamps.push_back({v.name, v.version});
|
||||
|
||||
// What this function derives is a pure function of that stamped set, so
|
||||
// the derived JSON is cached whole: a fresh cache makes an open one
|
||||
// file read, with no bundle built and no preset installed. Stale or
|
||||
// absent, the bundle is rebuilt below and the result written back.
|
||||
const boost::filesystem::path cache_file =
|
||||
boost::filesystem::path(Slic3r::data_dir()) / "cache" / "wizard_profile_data.json";
|
||||
try {
|
||||
// Slurped whole and parsed from the buffer — nlohmann's fastest
|
||||
// input path; a stream adapter costs real time on a multi-MB file.
|
||||
boost::nowide::ifstream ifs(cache_file.string(), std::ios::binary);
|
||||
if (ifs.is_open()) {
|
||||
const std::string text{std::istreambuf_iterator<char>(ifs), std::istreambuf_iterator<char>()};
|
||||
json cached = json::parse(text);
|
||||
if (cached.value("format", 0) == 1 && cached["vendors"] == stamps &&
|
||||
! cached["profile"]["machine"].empty()) {
|
||||
for (const char* key : { "model", "machine", "filament", "process" })
|
||||
m_ProfileJson[key] = std::move(cached["profile"][key]);
|
||||
BOOST_LOG_TRIVIAL(info) << "GuideFrame: profile data served from " << cache_file;
|
||||
return true;
|
||||
}
|
||||
}
|
||||
} catch (const std::exception& e) {
|
||||
BOOST_LOG_TRIVIAL(info) << "GuideFrame: rejecting cached profile data: " << e.what();
|
||||
}
|
||||
|
||||
// Each vendor comes from its preset cache where one covers it, which is
|
||||
// what makes this worth doing instead of the scan below; loading into a
|
||||
// bundle per vendor keeps the install order the startup path has.
|
||||
PresetBundle bundle;
|
||||
auto load_vendor = [](PresetBundle& into, const std::string& vendor,
|
||||
const boost::filesystem::path& dir, const PresetBundle* base) {
|
||||
into.load_vendor_configs_from_json(dir.string(), vendor, PresetBundle::LoadSystem,
|
||||
ForwardCompatibilitySubstitutionRule::EnableSilent, base);
|
||||
};
|
||||
for (const VendorSource& v : ordered) {
|
||||
if (*m_cancel_token)
|
||||
return false; // as in the scan below: a vendor without a cache is parsed, and that takes time
|
||||
if (v.name == filament_library) {
|
||||
load_vendor(bundle, v.name, v.dir, nullptr);
|
||||
} else {
|
||||
PresetBundle tmp;
|
||||
load_vendor(tmp, v.name, v.dir, &bundle);
|
||||
bundle.merge_presets(std::move(tmp));
|
||||
}
|
||||
}
|
||||
if (bundle.vendors.empty())
|
||||
return false;
|
||||
if (! BuildProfileJson(bundle, /*require_all_resource_vendors=*/false))
|
||||
return false;
|
||||
|
||||
// Written through a temp file and moved into place, as the preset caches
|
||||
// are: half a cache must never be readable, and the PID suffix keeps two
|
||||
// instances from interleaving on one temp file.
|
||||
const std::string tmp_path = cache_file.string() + "." + std::to_string(get_current_pid()) + ".tmp";
|
||||
try {
|
||||
json out;
|
||||
out["format"] = 1;
|
||||
out["vendors"] = std::move(stamps);
|
||||
json& profile = out["profile"];
|
||||
for (const char* key : { "model", "machine", "filament", "process" })
|
||||
profile[key] = m_ProfileJson[key];
|
||||
boost::filesystem::create_directories(cache_file.parent_path());
|
||||
{
|
||||
boost::nowide::ofstream ofs(tmp_path, std::ios::binary | std::ios::trunc);
|
||||
ofs << out.dump(-1, ' ', false, json::error_handler_t::ignore);
|
||||
ofs.close();
|
||||
if (! ofs.good())
|
||||
throw std::runtime_error("write failed");
|
||||
}
|
||||
if (const std::error_code ec = rename_file(tmp_path, cache_file.string()))
|
||||
throw std::runtime_error(ec.message());
|
||||
} catch (const std::exception& e) {
|
||||
boost::system::error_code rm;
|
||||
boost::filesystem::remove(tmp_path, rm);
|
||||
BOOST_LOG_TRIVIAL(warning) << "GuideFrame: could not write the profile data cache: " << e.what();
|
||||
}
|
||||
return true;
|
||||
} catch (const std::exception& e) {
|
||||
BOOST_LOG_TRIVIAL(error) << __FUNCTION__ << " failed: " << e.what();
|
||||
reset_profile_json();
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
int GuideFrame::LoadProfileData()
|
||||
{
|
||||
// Background thread: the fast path in OnNavigationComplete failed (presets not yet loaded).
|
||||
// Loading order (fastest to slowest):
|
||||
// 1. Load every vendor, from its preset cache wherever one covers it
|
||||
// 2. Read all vendor JSONs by hand
|
||||
try {
|
||||
if (!BuildProfileDataFromVendors()) {
|
||||
// Last resort — read all vendor JSONs
|
||||
std::set<std::string> loaded_vendors;
|
||||
auto filament_library_name = boost::filesystem::path(PresetBundle::ORCA_FILAMENT_LIBRARY).replace_extension(".json");
|
||||
if (boost::filesystem::exists(vendor_dir / filament_library_name))
|
||||
LoadProfileFamily(PresetBundle::ORCA_FILAMENT_LIBRARY, (vendor_dir / filament_library_name).string());
|
||||
else
|
||||
LoadProfileFamily(PresetBundle::ORCA_FILAMENT_LIBRARY, (rsrc_vendor_dir / filament_library_name).string());
|
||||
loaded_vendors.insert(PresetBundle::ORCA_FILAMENT_LIBRARY);
|
||||
|
||||
boost::filesystem::directory_iterator endIter;
|
||||
for (boost::filesystem::directory_iterator iter(vendor_dir); iter != endIter; iter++) {
|
||||
if (!boost::filesystem::is_directory(*iter)) {
|
||||
wxString strVendor = from_u8(iter->path().string()).BeforeLast('.');
|
||||
strVendor = strVendor.AfterLast('\\');
|
||||
strVendor = strVendor.AfterLast('/');
|
||||
wxString strExtension = from_u8(iter->path().string()).AfterLast('.').Lower();
|
||||
if (strExtension.CmpNoCase("json") != 0 || loaded_vendors.find(w2s(strVendor)) != loaded_vendors.end())
|
||||
continue;
|
||||
LoadProfileFamily(w2s(strVendor), iter->path().string());
|
||||
loaded_vendors.insert(w2s(strVendor));
|
||||
}
|
||||
if (*m_cancel_token) return 0;
|
||||
}
|
||||
|
||||
boost::filesystem::directory_iterator others_endIter;
|
||||
for (boost::filesystem::directory_iterator iter(rsrc_vendor_dir); iter != others_endIter; iter++) {
|
||||
if (!boost::filesystem::is_directory(*iter)) {
|
||||
wxString strVendor = from_u8(iter->path().string()).BeforeLast('.');
|
||||
strVendor = strVendor.AfterLast('\\');
|
||||
strVendor = strVendor.AfterLast('/');
|
||||
wxString strExtension = from_u8(iter->path().string()).AfterLast('.').Lower();
|
||||
if (strExtension.CmpNoCase("json") != 0 || loaded_vendors.find(w2s(strVendor)) != loaded_vendors.end())
|
||||
continue;
|
||||
LoadProfileFamily(w2s(strVendor), iter->path().string());
|
||||
loaded_vendors.insert(w2s(strVendor));
|
||||
}
|
||||
if (*m_cancel_token) return 0;
|
||||
}
|
||||
}
|
||||
|
||||
// Capture the cancel token by value (shared_ptr) so the lambda doesn't
|
||||
// touch `this` if GuideFrame is destroyed before the event fires.
|
||||
auto tok = m_cancel_token;
|
||||
wxGetApp().CallAfter([this, tok] {
|
||||
if (!*tok)
|
||||
on_profile_loaded();
|
||||
});
|
||||
} catch (std::exception& e) {
|
||||
// wxLogMessage("GUIDE: load_profile_error %s ", e.what());
|
||||
// wxMessageBox(e.what(), "", MB_OK);
|
||||
BOOST_LOG_TRIVIAL(error) << __FUNCTION__ << ", error: " << e.what() << std::endl;
|
||||
} catch (const std::exception& e) {
|
||||
BOOST_LOG_TRIVIAL(error) << __FUNCTION__ << ", error: " << e.what();
|
||||
}
|
||||
|
||||
filament_info_cache.clear();
|
||||
|
||||
@@ -30,10 +30,14 @@
|
||||
#include "libslic3r/PresetBundle.hpp"
|
||||
#include "slic3r/Utils/PresetUpdater.hpp"
|
||||
|
||||
#include <atomic>
|
||||
#include <memory>
|
||||
#include <unordered_map>
|
||||
|
||||
#include <nlohmann/json.hpp>
|
||||
|
||||
#include <boost/thread.hpp>
|
||||
|
||||
namespace Slic3r { namespace GUI {
|
||||
|
||||
class GuideFrame : public DPIDialog
|
||||
@@ -78,6 +82,12 @@ public:
|
||||
int LoadProfileData();
|
||||
int SaveProfileData();
|
||||
int LoadProfileFamily(std::string strVendor, std::string strFilePath);
|
||||
void init_guide_paths();
|
||||
void on_profile_loaded();
|
||||
bool BuildProfileJson(const PresetBundle& bundle, bool require_all_resource_vendors);
|
||||
bool BuildProfileDataFromPresetBundle();
|
||||
bool BuildProfileDataFromVendors();
|
||||
void reset_profile_json();
|
||||
int SaveProfile();
|
||||
int GetFilamentInfo( std::string VendorDirectory,json & pFilaList, std::string filepath, std::string &sVendor, std::string &sType);
|
||||
|
||||
@@ -112,8 +122,11 @@ private:
|
||||
|
||||
//First Load
|
||||
bool bFirstComplete{false};
|
||||
bool m_destroy{false};
|
||||
boost::thread* m_load_task{ nullptr };
|
||||
// Set once in the destructor. Read through `this` by the loading thread
|
||||
// (joined before `this` dies) and captured as the shared_ptr by CallAfter
|
||||
// lambdas so they don't touch `this` after the object is freed.
|
||||
std::shared_ptr<std::atomic<bool>> m_cancel_token{std::make_shared<std::atomic<bool>>(false)};
|
||||
std::unique_ptr<boost::thread> m_load_task;
|
||||
|
||||
// User Config
|
||||
bool PrivacyUse;
|
||||
@@ -123,6 +136,7 @@ private:
|
||||
bool InstallNetplugin;
|
||||
bool network_plugin_ready {false};
|
||||
|
||||
json m_ProfileJson;
|
||||
json m_OrcaFilaList;
|
||||
std::string m_OrcaFilaLibPath;
|
||||
|
||||
|
||||
@@ -1044,46 +1044,42 @@ void PresetUpdater::priv::check_installed_vendor_profiles() const
|
||||
std::set<std::string> bundles;
|
||||
// Orca: always install filament library
|
||||
bundles.insert(PresetBundle::ORCA_FILAMENT_LIBRARY);
|
||||
for (auto &dir_entry : boost::filesystem::directory_iterator(rsrc_path)) {
|
||||
const auto &path = dir_entry.path();
|
||||
std::string file_path = path.string();
|
||||
if (is_json_file(file_path)) {
|
||||
const auto path_in_vendor = vendor_path / path.filename();
|
||||
std::string vendor_name = path.filename().string();
|
||||
// Remove the .json suffix.
|
||||
vendor_name.erase(vendor_name.size() - 5);
|
||||
if (bundles.find(vendor_name) != bundles.end())continue;
|
||||
// A vendor is named by its profile or, where the build ships preset caches
|
||||
// instead of the raw profile JSONs, by its cache alone.
|
||||
for (const std::string &vendor_name : vendor_names_in(rsrc_path)) {
|
||||
if (bundles.find(vendor_name) != bundles.end())continue;
|
||||
|
||||
const auto is_vendor_enabled = (vendor_name == PresetBundle::ORCA_DEFAULT_BUNDLE) // always update configs from resource to vendor for ORCA_DEFAULT_BUNDLE
|
||||
|| (enabled_vendors.find(vendor_name) != enabled_vendors.end());
|
||||
if (enabled_config_update) {
|
||||
if ( fs::exists(path_in_vendor)) {
|
||||
if (is_vendor_enabled) {
|
||||
Semver resource_ver = get_version_from_json(file_path);
|
||||
Semver vendor_ver = get_version_from_json(path_in_vendor.string());
|
||||
const auto is_vendor_enabled = (vendor_name == PresetBundle::ORCA_DEFAULT_BUNDLE) // always update configs from resource to vendor for ORCA_DEFAULT_BUNDLE
|
||||
|| (enabled_vendors.find(vendor_name) != enabled_vendors.end());
|
||||
if (enabled_config_update) {
|
||||
if (is_vendor_installed(vendor_name)) {
|
||||
if (is_vendor_enabled) {
|
||||
// Orca: whichever form of the vendor resources ships at the newer
|
||||
// version is the one installing lays down, and the one to judge
|
||||
// what is installed against.
|
||||
Semver resource_ver = resource_vendor_version(vendor_name);
|
||||
// Orca: a vendor installed as a preset cache has no profile
|
||||
// beside it; the version it was installed at is in the cache.
|
||||
Semver vendor_ver = installed_vendor_version(vendor_name);
|
||||
|
||||
if (vendor_ver < resource_ver) {
|
||||
BOOST_LOG_TRIVIAL(info) << "[Orca Updater]:found vendor " << vendor_name << " newer version "
|
||||
<< resource_ver.to_string() << " from resource, old version " << vendor_ver.to_string();
|
||||
bundles.insert(vendor_name);
|
||||
}
|
||||
}
|
||||
else {
|
||||
//need to be removed because not installed
|
||||
fs::remove(path_in_vendor);
|
||||
const auto path_of_vendor = vendor_path / vendor_name;
|
||||
if (fs::exists(path_of_vendor))
|
||||
fs::remove_all(path_of_vendor);
|
||||
if (vendor_ver < resource_ver) {
|
||||
BOOST_LOG_TRIVIAL(info) << "[Orca Updater]:found vendor " << vendor_name << " newer version "
|
||||
<< resource_ver.to_string() << " from resource, old version " << vendor_ver.to_string();
|
||||
bundles.insert(vendor_name);
|
||||
}
|
||||
}
|
||||
else if (is_vendor_enabled) {
|
||||
bundles.insert(vendor_name);
|
||||
else {
|
||||
//need to be removed because not installed
|
||||
remove_installed_vendor(vendor_name);
|
||||
}
|
||||
}
|
||||
else if (is_vendor_enabled) {
|
||||
bundles.insert(vendor_name);
|
||||
}
|
||||
}
|
||||
else if (is_vendor_enabled) {
|
||||
bundles.insert(vendor_name);
|
||||
}
|
||||
}
|
||||
|
||||
if (bundles.size() > 0) {
|
||||
@@ -1163,11 +1159,12 @@ Updates PresetUpdater::priv::get_config_updates(const Semver &old_slic3r_version
|
||||
auto filament_in_cache = (cache_profile_path / vendor_name / PRESET_FILAMENT_NAME);
|
||||
auto machine_in_cache = (cache_profile_path / vendor_name / PRESET_PRINTER_NAME);
|
||||
|
||||
if (( fs::exists(path_in_vendor))
|
||||
if (is_vendor_installed(vendor_name)
|
||||
|| fs::exists(print_in_cache)
|
||||
|| fs::exists(filament_in_cache)
|
||||
|| fs::exists(machine_in_cache)) {
|
||||
Semver vendor_ver = get_version_from_json(path_in_vendor.string());
|
||||
// Orca: a vendor installed as a preset cache carries its version there.
|
||||
Semver vendor_ver = installed_vendor_version(vendor_name);
|
||||
|
||||
std::map<std::string, std::string> key_values;
|
||||
std::vector<std::string> keys(3);
|
||||
|
||||
@@ -17,6 +17,7 @@ add_executable(${_TEST_NAME}_tests
|
||||
test_preset_bundle_loading.cpp
|
||||
test_preset_setting_id.cpp
|
||||
test_preset_diff.cpp
|
||||
test_vendor_cache.cpp
|
||||
test_elephant_foot_compensation.cpp
|
||||
test_fill_corner_smoothing.cpp
|
||||
test_fill_plane_path.cpp
|
||||
|
||||
@@ -574,11 +574,6 @@ TEST_CASE("Convex polygon intersection on two squares touching one vertex", "[Ge
|
||||
Polygon B = A;
|
||||
B.translate(10 / SCALING_FACTOR, 10 / SCALING_FACTOR);
|
||||
|
||||
SVG svg{std::string("one_vertex_touch") + ".svg"};
|
||||
svg.draw(A, "blue");
|
||||
svg.draw(B, "green");
|
||||
svg.Close();
|
||||
|
||||
bool is_inters = Geometry::convex_polygons_intersect(A, B);
|
||||
|
||||
REQUIRE(is_inters == false);
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
#include <catch2/catch_all.hpp>
|
||||
|
||||
#include <boost/filesystem.hpp>
|
||||
#include <fstream>
|
||||
|
||||
#include "libslic3r/PresetBundle.hpp"
|
||||
#include "libslic3r/AppConfig.hpp"
|
||||
@@ -132,7 +133,7 @@ TEST_CASE("Current vendor type tolerates missing printer model", "[Preset][Bundl
|
||||
{
|
||||
PresetBundle bundle;
|
||||
|
||||
VendorProfile orca_vendor("ORCA");
|
||||
VendorProfile orca_vendor; orca_vendor.id = "ORCA";
|
||||
VendorProfile::PrinterModel model;
|
||||
model.name = "Orca Test";
|
||||
orca_vendor.models.emplace_back(model);
|
||||
@@ -143,6 +144,31 @@ TEST_CASE("Current vendor type tolerates missing printer model", "[Preset][Bundl
|
||||
CHECK(bundle.get_current_vendor_type() == VendorType::Unknown);
|
||||
}
|
||||
|
||||
TEST_CASE("A malformed entry in a vendor's preset list is counted, not thrown", "[Preset][Bundle]")
|
||||
{
|
||||
ScopedTemporaryDir dir;
|
||||
|
||||
// A bare number where the list wants an object. An array element has no key,
|
||||
// so reporting one as if it did throws nlohmann's invalid_iterator - which is
|
||||
// not a parse_error, and escapes the catch around the vendor profile parse.
|
||||
std::ofstream((dir.path() / "Acme.json").string())
|
||||
<< R"({"version":"1.0.0","name":"Acme","process_list":[123,)"
|
||||
<< R"({"name":"0.20mm Standard @Acme","sub_path":"process/standard.json"}]})";
|
||||
fs::create_directories(dir.path() / "Acme" / "process");
|
||||
std::ofstream((dir.path() / "Acme" / "process" / "standard.json").string())
|
||||
<< R"({"type":"process","name":"0.20mm Standard @Acme","from":"system",)"
|
||||
<< R"("instantiation":"true","layer_height":"0.2"})";
|
||||
|
||||
PresetBundle bundle;
|
||||
size_t loaded = 0;
|
||||
REQUIRE_NOTHROW(loaded = bundle.load_vendor_configs_from_json(
|
||||
dir.path().string(), "Acme", PresetBundle::LoadSystem,
|
||||
ForwardCompatibilitySubstitutionRule::EnableSilent).second);
|
||||
|
||||
CHECK(bundle.error_count() > 0); // the malformed element was counted
|
||||
CHECK(loaded == 1); // the well-formed one beside it still loaded
|
||||
}
|
||||
|
||||
TEST_CASE("Printer extruder count tolerates missing nozzle diameter", "[Preset][Bundle]")
|
||||
{
|
||||
PresetBundle bundle;
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user