Compare commits

...
Author SHA1 Message Date
Hanif Koh 35dc1ca6ef Lock Config and Preset Files Across Instances and Write Them Atomically
Every running instance shares one OrcaSlicer.conf and one user preset
tree, and nothing kept their writers apart. Two instances saving at the
same moment, or the cloud preset sync thread writing while the GUI thread
saved, could interleave, and a reader in another instance could open a
preset JSON or .info file between truncate and close and get a partial
file, dropping that preset for the session with a parse error.

Add InstanceLock, a scoped guard that serialises the threads of one
process through a recursive mutex and other processes through an advisory
OS file lock: flock on POSIX, held on the guard's own descriptor so no
other close in the process can drop it, and LockFileEx on Windows. The
outermost guard opens the lock file and closes it on release, so nothing
stays open between saves and a data dir can be removed once nothing is
saving into it; the file itself is kept, since deleting it would let a
third instance lock a fresh file while the second still holds the old
one. It is best effort: when the lock file cannot be opened or locked, or
another instance still holds it after a second, the guard logs once and
lets the write proceed, then leaves the file alone for ten seconds, so a
hung instance never blocks every other one and a holder stuck in a
debugger does not cost a stall per save. The guard sits at the leaf
readers and writers: set_sync_info_and_save() calls save_info() under the
preset collection mutex, so a batch lock around save_user_presets() would
invert the order against the sync thread. The user preset scan reads its
files on worker threads without the guard, since the mutex would
serialise them, and takes it per file in the serial commit step, so a
save never waits for the whole scan. Each read keeps the bytes of the
preset and its .info as they were before parsing; commit compares them
with the disk under the guard and reads a file that changed again, so it
never deletes or writes back over another instance's newer save, nor
installs a .json and .info from two different saves; a preset another
instance removed in the meantime is not installed. Without the guard, in
a cool-down, the scan still loads the presets but leaves their files
alone: an unreadable file stays for the next scan, and a derived
compatible printer is not written back. Read-only scans, which is what
the CLI does, take no lock and create no lock file.

AppConfig holds OrcaSlicer.conf.lock in load() and save(); load is
included because the Windows path restores from the .bak copy. Every
user preset writer and reader holds user.lock: Preset::save(), which
writes no .info when the preset itself could not be written, since an
.info without its preset reads as a cloud deletion request, save_info(),
reload() and remove_files(), each preset the scan commits, the
bundle metadata reads and write, the .info removal after a cloud-confirmed
delete, the orphaned-.info scan on the sync thread, the bundle folder
removal on unsubscribe and the physical printer writers and delete
paths. A bundle import extracts under cache/ into a folder per process
and per import, where no scan reads.

Preset JSON, .info, bundle metadata, physical printer and config files,
and the caches and state files that already used a temporary by hand,
now go through write_file_atomically(), which writes <file>.<pid>.<n>.tmp
beside the target and renames it over, so a reader that never waits sees
a complete old or new file. A symlink is followed; a target that is not
a regular file is written in place; and when no temporary can be created
beside an existing target, or the rename itself is refused, by a Windows
reader holding the file open or a mount that cannot replace in one step,
the helper writes in place as before, since losing the save is worse
than a torn read. On POSIX the rename replaces the
target atomically where the old code removed it first and left a window
with no file at all; only a mount that refuses a one-step replace gets
the old remove-then-rename. A crash between temporary and rename leaves
the temporary behind, which no scan reads. Preset::save() returns
whether it wrote the preset, so the scan counts a compatible printer it
could not write back as an error. A failed config write keeps
the config dirty, and the idle handler waits ten seconds before retrying
while an explicit save always tries.
2026-09-28 16:21:31 +08:00
raistlin7447 7f30eb8a49 libslic3r: merge the loaded vendors' presets in one pass
load_vendors merged each vendor's bundle into the startup bundle in turn,
inserting every preset at its sorted place in a deque, so each insert shifted
every preset after it.

PresetBundle::merge_presets now takes every loaded bundle at once, and each
collection merges its sorted presets with theirs in one pass. A preset name this
bundle or an earlier one of the list already has is still left out and reported
under the vendor that repeats it, in the same order as before.
2026-09-27 08:42:38 -05:00
raistlin7447 b5d1866445 libslic3r: read the other vendors while the filament library loads
Installing a vendor's filaments looks up the filament library, so load_vendors
loads the library first. Reading a vendor, from its cache or its JSONs, needs
nothing from the library, yet every other vendor waited for it before starting.

load_vendor_configs_from_json splits into read_vendor and install_vendor_read.
load_vendors reads the other vendors in parallel while the library loads, and
installs each against it as soon as both its read and the library are done. A
cache the library can no longer install is still replaced by a parse of the
vendor's JSONs, now when it installs; a canceled load is not mistaken for such
a cache. Vendors start costliest first, and a cache older than the profile
beside it is ordered as a parse.

load_vendors also names the vendors that failed, the library included, which
keeps what it installed before failing and so stays among the bundle's vendors.
The setup wizard falls back to its own scan on that list rather than on which
vendors the bundle holds.
2026-09-27 08:42:37 -05:00
raistlin7447 3b1a5c0630 libslic3r: copy the configs a vendor keeps while resolving
A vendor keeps a copy of each config its other entries inherit, and the filament
library keeps every one of them for the vendors that inherit from it. The copy
was made in commit, one entry at a time on the calling thread while the workers
waited, and the library alone makes 960 of them.

Whether an entry's config is kept depends only on names known before any entry
resolves, so resolve_vendor_preset makes the copy alongside the rest of its work
and commit moves it in.
2026-09-27 08:42:36 -05:00
raistlin7447 18867094fa libslic3r: load the setup wizard's vendors the way startup does
When the wizard's profile-data cache is stale, which any vendor's version change
makes it, the printer and filament selection dialogs rebuild it by loading every
shipped vendor, 66 of them here, one after another. Startup loads its vendors in
parallel once the filament library is in.

That step moves out of load_system_presets_from_json into
PresetBundle::load_vendors, which takes each vendor with the directory it is
installed in, and the wizard calls it too. A vendor that fails to load still
sends the wizard to its hand-written JSON scan, as the exception it used to throw
did, now once the others have loaded and with the error logged. merge_presets
goes private, since the wizard no longer merges bundles itself.

Opening either dialog with a stale cache goes from 3.38-3.55 s to 2.33-2.47 s on
a 16-core desktop, and the vendor rebuild inside it from 2.34-2.48 s to
1.27-1.41 s. The catalog it writes is byte-identical to the one main writes, and
a dump of every preset startup loads is unchanged.
2026-09-27 08:42:36 -05:00
raistlin7447 5ae88ae5c7 libslic3r: install parsed and cached presets through one routine
A cache load installed the entries others inherit from or include first and
resolved the rest together, while the JSON parse installed each entry straight
after parsing it. The two orders only agree while every preset is listed after
what it depends on. An include listed after the preset naming it was missing for
one and found by the other, and every new kind of reference between presets
would have needed the same care in two places.

Both loads now go through install_vendor, which installs processes, filaments and
printers in listing order. Entries resolve together in runs and commit one at a
time, and a run ends before an entry that inherits or includes one already in
it. No entry resolves against a registration from its own run, so the result is
what installing one at a time gives, in any listing order.

The JSON parse reads every sub-file first, under one numeric locale setter, and
stops at the first that fails. The ones before it are installed before the
failure is raised, as they were one at a time, and the filament library's maps
are published only from a complete load. Each entry's parse errors are logged and
counted when it installs, so the log and a load that fails partway come out as
one-at-a-time loading leaves them.

Every preset already in the collection counts as taken for a new one's name.

resolve_vendor_preset and commit_vendor_preset take a VendorInstall with the
collection's shared state in place of a dozen parameters, const for resolve.

The first launch after an update, which parses every profile, goes from
1510-1533 ms to 806-849 ms on a 16-core desktop, and a cached launch from
342-357 ms to 328-334 ms.
2026-09-27 08:42:35 -05:00
raistlin7447 ea3a40d073 libslic3r: move the filament library's config map instead of copying it
Every preset of the filament library keeps its config for other vendors to
inherit from, and publishing that map copied all 1281 of them. The map it is
built in is cleared by the next install before anything reads it again, so the
copy was of no use to anyone.

Loading the library goes from 145 to 86 ms, and the whole preset load from 360
to 300 ms.
2026-09-27 08:42:35 -05:00
raistlin7447 806568287c libslic3r: append a vendor's presets and sort once
Installing a preset placed it at its position in the sorted collection, which
moves every preset after it, so a vendor with 2528 filaments paid an O(n)
insertion 2528 times. That is serial work at the end of a load whose per-preset
part now runs across threads, so it is what the load waits on.

The entries are appended and the collection sorted once they are all in, the way
user presets are already loaded. The sort is on a scope guard, so it also runs
for an entry that cannot be installed and throws past the end of the loop. A
repeated name was caught by find_preset against the half-filled collection,
which an unsorted one cannot answer, so the names are kept in a set, seeded with
the default presets that are in the collection before the first entry arrives.
2026-09-27 08:42:34 -05:00
raistlin7447 0342ddbe26 libslic3r: check only the keys an entry added for validity
A preset's config is a copy of the one it inherits with the entry's diff applied
over it, so the only keys it can hold that the parent does not define are the
ones in that diff. Scanning all of them instead walked 141 keys per filament
preset where 23 would do, and built a vector of every key's name to do it.

The keys the caller knows were added are checked where it has them, and the scan
of everything stays for the callers that do not. An entry that includes a
template is checked the same way, since an include diff holds only keys of the
collection default. The check also moves above Preset::normalize, which derives
keys from the ones it finds. A key derived from one the profile should not have
carried is in neither the diff nor the parent, so after normalize nothing would
name it.
2026-09-27 08:42:34 -05:00
raistlin7447 33af0651b7 libslic3r: resolve user presets together
A user preset is read from its own file and flattened against the system preset
it inherits. It cannot inherit another file of the same pass, because the presets
being read only join the collection once they have all been read, so the files
are independent of each other.

load_presets splits the way the vendor cache load already does:
resolve_user_preset reads and flattens one file and touches nothing shared, and
commit_user_preset installs it, counts its errors, and does the file work a load
can trigger (removing an unreadable preset, writing back a filament preset that
named no compatible printer). Both callers now share resolve_then_commit, which
holds the two-phase shape, works through the items in batches so what is held at
once does not grow with how many there are, and gives each piece of a batch one
CNumericLocalesSetter rather than one per file. It throws rather than commit a
batch that a canceled task group of the caller left unresolved.

A setter nested in another on the same thread does nothing while the locale is
still "C", so one setter per piece of a batch covers every file in it, and each
batch runs isolated, so a thread waiting on it picks up none of the caller's
other work. A filament preset whose derived compatible printer cannot be written
back still loads, with the failure counted.

Loading 328 user presets goes from 105 to 34 ms on a 16-core desktop.
2026-09-27 08:42:33 -05:00
raistlin7447 781a5d8516 libslic3r: compare preset json keys without the locale
Reading one preset file tries a dozen key names against every key in it with
boost::iequals, which compares through std::locale(). Constructing that takes a
lock the whole process shares in the MSVC runtime, so it is both the bulk of the
work in a preset file's parse and the reason two threads cannot do it at once.

Comparing the bytes instead takes loading 328 user presets from 129 to 105 ms
one at a time, and the same files spread across threads from 132 to 27 ms. The
first launch after an update, when the system profiles are parsed from JSON
rather than served from their cache, goes through the same function.
2026-09-27 08:42:32 -05:00
raistlin7447 38446ab77a libslic3r: resolve a vendor's cached presets together
Installing one entry is a config copy, the entry's diff applied over it,
normalization and an invalid-key scan. For the largest vendor that is around
2000 filaments at about 115 us each, and all of it reads state that does not
change while the collection loads.

load_vendor_preset splits into resolve_vendor_preset, which flattens an entry
and touches nothing shared, and commit_vendor_preset, which installs it. A
cache load installs the entries others inherit from first, since what those
register is what the rest resolve against, then resolves the rest together and
commits them one at a time. The JSON parse calls the two back to back as before.

On a 16-core desktop with 4263 system presets installed, preset loading goes
from 644-652 ms to 370-376 ms, and a bundle loaded either way comes out
identical, preset for preset and option for option.
2026-09-27 08:42:32 -05:00
raistlin7447 9080269f6f libslic3r: split the system preset load across threads reliably
The parallel_for over the vendors used an auto_partitioner, which splits its
range only while a worker is asking for work. At six vendors that often did not
happen at all. In two runs out of four on this machine every vendor in that step
ran on the calling thread, which the log shows as one thread id for all of them.

A grain of one splits the range up front. The vendors also start slowest first,
so the vendor that decides when the step ends is not the one left until last. A
vendor with no cache is parsed from its JSONs, which costs far more than any
cache load, so those go before cached ones, and within each group the bigger
file goes first, the cache or the profile index that names the vendor's presets.
2026-09-27 08:42:31 -05:00
30 changed files with 3186 additions and 859 deletions
+45 -19
View File
@@ -28,8 +28,8 @@ Per-vendor granularity is what makes the system practical:
- A vendor whose profile is bumped invalidates only its own cache. The other 60-odd
vendors keep theirs — even when the bumped vendor is the shared Orca filament
library everyone else inherits from.
- The setup wizard, which loads vendors one at a time, gets the same speedup as
startup without a second code path.
- The setup wizard loads its vendors through the same routine as startup, so it
gets the same speedup without a second code path.
- A vendor with no cache, or a broken one, costs only that vendor a parse.
A cache holds *system* presets only. User presets, project settings and modified
@@ -162,15 +162,20 @@ cache nothing can invalidate is worse than no cache.
Vendors load in a fixed order, because filament inheritance crosses exactly one
boundary: any vendor's filament may inherit from the shared Orca filament library,
and nothing else reaches across vendors — an `include` is always vendor-local. The
library therefore goes first, alone; every other vendor follows in parallel, resolving
against it; and the results are merged in a stable order:
and nothing else reaches across vendors — an `include` is always vendor-local. Only
installing a vendor's presets crosses it; reading the vendor, from its cache or its
JSONs, needs nothing from the library. So every other vendor is read while the library
loads, each is installed against it as soon as both are done, and the results are
merged in a stable order:
```mermaid
flowchart LR
lib["1 · OrcaFilamentLibrary<br/>loaded first, synchronously"] --> par["2 · every other vendor in parallel,<br/>each into its own bundle, filaments<br/>resolving against the loaded library"] --> merge["3 · bundles merged into one,<br/>sequentially, in stable vendor order"]
lib["1 · OrcaFilamentLibrary loaded;<br/>meanwhile every other vendor read<br/>from its cache or its JSONs"] --> par["2 · every other vendor installed<br/>in parallel, each into its own bundle,<br/>filaments resolving against the library"] --> merge["3 · bundles merged into one,<br/>in one pass per collection,<br/>in stable vendor order"]
```
`PresetBundle::load_vendors` runs these steps for startup and for the setup wizard,
which hand it the vendors to load and the directory each is installed in.
Whether a vendor comes from its cache or from a parse changes nothing in that
order — both produce the same bundle, so cached and parsed vendors mix freely in
one startup.
@@ -211,11 +216,12 @@ shipped cache answered first, so the profile in `<data_dir>/system/` was never p
and its cache was never written back.
Serving from a cache is not a memory-image restore. The entries are deserialized and
then installed one by one — inheritance resolved against the presets installed before
them and the currently loaded filament library, includes layered in, configs flattened
onto the collection defaults, validated and registered — by the same function the JSON
path calls straight after parsing a sub-file. An `include` layers what the included
base states, between the parent and the preset's own keys: the base's diff against the
then installed by `install_vendor`, the routine the JSON path hands the vendor's entries
to once it has parsed the sub-files: inheritance resolved against the presets installed
before them and the currently loaded filament library, includes layered in, configs
flattened onto the collection defaults, validated and registered. An `include` layers
what the included base states, between the parent and the preset's own keys: the base's
diff against the
default, taken when the base itself was installed and before the per-variant padding
`inherits` sees, so only what a template sets reaches the presets including it. The two
paths share everything below the parse, which is what makes a cache-loaded bundle
@@ -223,6 +229,16 @@ indistinguishable from a JSON-loaded one by construction rather than by test cov
Installation also rebuilds each preset's file path from the local data directory, so a
shipped cache never carries the generating machine's paths.
Installing an entry is split in two. `resolve_vendor_preset` flattens it, reading only
what is registered under the names it inherits and includes, and `commit_vendor_preset`
registers it, the only step that writes anything shared. Entries resolve across threads
in runs and commit in the order the vendor lists them. A run ends before an entry that
inherits or includes one already in it, since that one's commit registers what the
entry resolves against, so no entry in a run reads what another in it registers. An
entry's parse messages are held until it commits. The bundle, the log's parse and
install messages and the error count therefore come out as parsing and installing one
entry at a time would leave them, whatever the listing order.
App upgrades work because a cache normally survives one. Only a deliberate
`CACHE_VERSION` bump makes an installed cache unreadable, and that is handled at
install time rather than at load: a vendor whose cache this build cannot read counts
@@ -254,17 +270,20 @@ the wizard caches the *derived JSON*, not another form of the inputs:
open, the wizard computes the current stamps (one version peek per vendor) and, when
they match, serves the catalog from the file — no bundle built, no preset installed.
Caching bundle inputs instead was tried and measured: rebuilding the bundle from
per-vendor caches costs ~2 s of preset installation whatever feeds it, so only
skipping the rebuild entirely wins.
per-vendor caches costs over a second of preset installation whatever feeds it, so
only skipping the rebuild entirely wins.
Any change to the set — a vendor added, removed or updated, or its cache-only
`.opc` replaced by a newer one — changes the stamps and retires the whole file;
the wizard then rebuilds the bundle vendor by vendor (per-vendor caches serving where
they cover) and writes the catalog back. Selections, region and per-open decorations
are applied downstream of the cache either way, so a served catalog is
indistinguishable from a rebuilt one. Nothing ships this file and the updater never
touches it; it is a locally written artifact, re-derived whenever stale, written
through a temp file and rename so half a cache is never readable.
the wizard then rebuilds the bundle with `PresetBundle::load_vendors`, the load
startup uses (per-vendor caches serving where they cover), and writes the catalog
back. When a vendor fails to load, the filament library included, that open falls
back to the wizard's own scan of the vendor JSONs, as when no bundle can be built,
and writes nothing. Selections, region and per-open decorations are applied
downstream of the cache either way, so a served catalog is indistinguishable from a
rebuilt one. Nothing ships this file and the updater never touches it; it is a
locally written artifact, re-derived whenever stale, written through a temp file and
rename so half a cache is never readable.
The cache lives under `<data_dir>/cache/`, not beside the vendors: everything that
scans `<data_dir>/system/` treats any `.opc` there as a vendor, so a non-vendor
@@ -378,6 +397,13 @@ enumerates only `*.json` will find no vendors at all in a packaged build.
the `CachedPreset` field list — written and read by `visit_entry` in
`PresetCacheFormat.cpp`, one list for the save, the load and the name peek alike — or
the cache's own layout or stamps, requires bumping `CACHE_VERSION` by hand.
- **Adding a kind of reference between presets**, as `inherits` and `include` are:
parse the names into `CachedPreset` (a field change, so `CACHE_VERSION` is bumped),
have `install_vendor_entries` end a run before an entry that names one already in it
and retain what the names point at, look them up only in `resolve_vendor_preset`, and
register what they point at only in `commit_vendor_preset`. The listing-order test in
`test_vendor_cache.cpp` fails for a kind the runs do not check once its fixture uses
it.
- **The dictionary indexes with a `uint16`**, so `print_config_def` may hold at most
65535 options and one cache at most 65535 distinct enum value names.
`CacheDictionary::save` throws past that, which surfaces when CI generates the
+48 -66
View File
@@ -5,6 +5,7 @@
//BBS
#include "Preset.hpp"
#include "Exception.hpp"
#include "InstanceLock.hpp"
#include "LocalesUtils.hpp"
#include "Thread.hpp"
#include "format.hpp"
@@ -739,10 +740,13 @@ static bool verify_config_file_checksum(boost::nowide::ifstream &ifs)
#ifdef USE_JSON_CONFIG
std::string AppConfig::load()
std::string AppConfig::load(bool read_only)
{
json j;
// Keep another instance from replacing or restoring the file mid-read.
InstanceLock instance_lock(read_only ? std::string() : lock_path());
// 1) Read the complete config file into a boost::property_tree.
namespace pt = boost::property_tree;
pt::ptree tree;
@@ -992,7 +996,6 @@ void AppConfig::save()
// The config is first written to a file with a PID suffix and then moved
// to avoid race conditions with multiple instances of Slic3r
const auto path = config_path();
std::string path_pid = (boost::format("%1%.%2%") % path % get_current_pid()).str();
json j;
@@ -1122,43 +1125,18 @@ void AppConfig::save()
j["local_machines"][local_machine.first] = m_json;
}
boost::nowide::ofstream c;
c.open(path_pid, std::ios::out | std::ios::trunc);
c << j.dump(1, '\t') << std::endl;
#ifdef WIN32
// WIN32 specific: The final "rename_file()" call is not safe in case of an application crash, there is no atomic "rename file" API
// provided by Windows (sic!). Therefore we save a MD5 checksum to be able to verify file corruption. In addition,
// we save the config file into a backup first before moving it to the final destination.
c << appconfig_md5_hash_line(j.dump(1, '\t'));
#endif
c.close();
if (c.fail()) {
BOOST_LOG_TRIVIAL(error) << "Failed to write new configuration to " << path_pid << "; aborting attempt to overwrite original configuration";
return;
}
#ifdef WIN32
// Make a backup of the configuration file before copying it to the final destination.
std::string error_message;
std::string backup_path = (boost::format("%1%.bak") % path).str();
// Copy configuration file with PID suffix into the configuration file with "bak" suffix.
if (copy_file(path_pid, backup_path, error_message, false) != SUCCESS)
BOOST_LOG_TRIVIAL(error) << "Copying from " << path_pid << " to " << backup_path << " failed. Failed to create a backup configuration.";
#endif
// Rename the config atomically.
// On Windows, the rename is likely NOT atomic, thus it may fail if PrusaSlicer crashes on another thread in the meanwhile.
// To cope with that, we already made a backup of the config on Windows.
rename_file(path_pid, path);
m_dirty = false;
const std::string config_str = j.dump(1, '\t');
if (write_config_file(path, config_str + "\n", config_str))
m_dirty = false;
}
#else
std::string AppConfig::load()
std::string AppConfig::load(bool read_only)
{
// Keep another instance from replacing or restoring the file mid-read.
InstanceLock instance_lock(read_only ? std::string() : lock_path());
// 1) Read the complete config file into a boost::property_tree.
namespace pt = boost::property_tree;
pt::ptree tree;
@@ -1296,7 +1274,6 @@ void AppConfig::save()
// The config is first written to a file with a PID suffix and then moved
// to avoid race conditions with multiple instances of Slic3r
const auto path = config_path();
std::string path_pid = (boost::format("%1%.%2%") % path % get_current_pid()).str();
std::stringstream config_ss;
if (m_mode == EAppMode::Editor)
@@ -1332,39 +1309,39 @@ void AppConfig::save()
// One empty line before the MD5 sum.
config_ss << std::endl;
std::string config_str = config_ss.str();
boost::nowide::ofstream c;
c.open(path_pid, std::ios::out | std::ios::trunc);
c << config_str;
#ifdef WIN32
// WIN32 specific: The final "rename_file()" call is not safe in case of an application crash, there is no atomic "rename file" API
// provided by Windows (sic!). Therefore we save a MD5 checksum to be able to verify file corruption. In addition,
// we save the config file into a backup first before moving it to the final destination.
c << appconfig_md5_hash_line(config_str);
#endif
c.close();
if (c.fail()) {
BOOST_LOG_TRIVIAL(error) << "Failed to write new configuration to " << path_pid << "; aborting attempt to overwrite original configuration";
return;
}
#ifdef WIN32
// Make a backup of the configuration file before copying it to the final destination.
std::string error_message;
std::string backup_path = (boost::format("%1%.bak") % path).str();
// Copy configuration file with PID suffix into the configuration file with "bak" suffix.
if (copy_file(path_pid, backup_path, error_message, false) != SUCCESS)
BOOST_LOG_TRIVIAL(error) << "Copying from " << path_pid << " to " << backup_path << " failed. Failed to create a backup configuration.";
#endif
// Rename the config atomically.
// On Windows, the rename is likely NOT atomic, thus it may fail if PrusaSlicer crashes on another thread in the meanwhile.
// To cope with that, we already made a backup of the config on Windows.
rename_file(path_pid, path);
m_dirty = false;
const std::string config_str = config_ss.str();
if (write_config_file(path, config_str, config_str))
m_dirty = false;
}
#endif
bool AppConfig::write_config_file(const std::string &path, std::string body, const std::string &checksum_source)
{
// Everything before this is assembly; only the writes need the other instances kept out.
InstanceLock instance_lock(lock_path());
#ifdef WIN32
// WIN32 specific: the final replace is not safe in case of an application crash, there is no atomic "rename file" API
// provided by Windows (sic!). Therefore we save a MD5 checksum to be able to verify file corruption. In addition,
// we save the config file into a backup first before moving it to the final destination.
body += appconfig_md5_hash_line(checksum_source);
#endif
// Not flushed to the device: the idle handler saves on the GUI thread after
// any change, and the rename already gives a complete old or new file.
if (const std::error_code ec = write_file_atomically(path, body)) {
BOOST_LOG_TRIVIAL(error) << "Failed to write the configuration " << path << ": " << ec.message() << "; trying again in 10 s";
m_retry_save_at = std::chrono::steady_clock::now() + std::chrono::seconds(10);
return false;
}
m_retry_save_at = {};
#ifdef WIN32
// Written after the config, so the backup never holds a state that was not confirmed written.
const std::string backup_path = (boost::format("%1%.bak") % path).str();
if (const std::error_code ec = write_file_atomically(backup_path, body))
BOOST_LOG_TRIVIAL(error) << "Failed to write the backup configuration " << backup_path << ": " << ec.message();
#endif
return true;
}
bool AppConfig::get_variant(const std::string &vendor, const std::string &model, const std::string &variant) const
{
const auto it_v = m_vendors.find(vendor);
@@ -1853,6 +1830,11 @@ void AppConfig::reset_selections()
}
}
std::string AppConfig::lock_path()
{
return Slic3r::data_dir().empty() ? std::string() : config_path() + ".lock";
}
std::string AppConfig::config_path()
{
#ifdef USE_JSON_CONFIG
@@ -1889,7 +1871,7 @@ bool AppConfig::exists()
std::string AppConfig::load_if_exists()
{
return boost::filesystem::exists(loading_path()) ? load() : std::string();
return boost::filesystem::exists(loading_path()) ? load(/*read_only=*/true) : std::string();
}
}; // namespace Slic3r
+16 -1
View File
@@ -2,6 +2,7 @@
#define slic3r_AppConfig_hpp_
#include <set>
#include <chrono>
#include <map>
#include <string>
#include "nlohmann/json.hpp"
@@ -122,14 +123,18 @@ public:
// Load the slic3r.ini from a user profile directory (or a datadir, if configured).
// Return an error string, or an empty string on success.
std::string load();
std::string load(bool read_only = false);
// Treat a missing config as default state; otherwise load it normally.
// The CLI's load: it never saves, so it takes no lock and creates no lock file.
std::string load_if_exists();
// Store the slic3r.ini into a user profile directory (or a datadir, if configured).
void save();
// Does this config need to be saved?
bool dirty() const { return m_dirty; }
// False for ten seconds after a failed write, so the idle handler does not
// repeat a hopeless attempt on every event; an explicit save() always tries.
bool save_due() const { return std::chrono::steady_clock::now() >= m_retry_save_at; }
void set_dirty() { m_dirty = true; }
@@ -339,6 +344,8 @@ public:
// Get the default config path from Slic3r::data_dir().
std::string config_path();
// Lock file guarding config_path() against other running instances; empty without a data dir.
std::string lock_path();
// Returns true if the user's data directory comes from before Slic3r 1.40.0 (no updating)
bool legacy_datadir() const { return m_legacy_datadir; }
@@ -449,8 +456,16 @@ private:
// Preset for each machine
MachineSettingMap m_printer_settings;
// Writes the assembled config text, and on Windows its checksum and a backup copy; false when the
// config itself could not be written, in which case the caller stays dirty and retries. `checksum_source`
// is the text load() will verify, which for the JSON config ends before the trailing newline.
bool write_config_file(const std::string &path, std::string body, const std::string &checksum_source);
// Has any value been modified since the config.ini has been last saved or loaded?
bool m_dirty;
// After a failed write, save_due() is false for the next ten seconds, so the
// idle handler does not repeat a hopeless write on every event.
std::chrono::steady_clock::time_point m_retry_save_at{};
// Original version found in the ini file before it was overwritten
Semver m_orig_version;
// Whether the existing version is before system profiles & configuration updating
+3
View File
@@ -304,6 +304,8 @@ set(lisbslic3r_sources
Geometry/VoronoiUtils.cpp
Geometry/VoronoiUtils.hpp
Geometry/VoronoiVisualUtils.hpp
InstanceLock.cpp
InstanceLock.hpp
Int128.hpp
KDTreeIndirect.hpp
Layer.cpp
@@ -353,6 +355,7 @@ set(lisbslic3r_sources
Optimize/Optimizer.hpp
Orient.cpp
Orient.hpp
ParallelResolve.hpp
ParameterUtils.cpp
ParameterUtils.hpp
pchheader.cpp
+30 -19
View File
@@ -849,6 +849,19 @@ ConfigSubstitutions ConfigBase::load_from_json(const std::string &file, ForwardC
return std::move(substitutions_ctxt.substitutions);
}
// Case-insensitive compare of a JSON key against a fixed ASCII one, without
// boost::iequals, whose std::locale() takes a lock the whole process shares in the
// MSVC runtime.
static bool ascii_iequals(const std::string &key, const char *literal)
{
auto lower = [](char c) { return (c >= 'A' && c <= 'Z') ? char(c - 'A' + 'a') : c; };
size_t i = 0;
for (; i < key.size() && literal[i] != '\0'; ++ i)
if (lower(key[i]) != lower(literal[i]))
return false;
return i == key.size() && literal[i] == '\0';
}
int ConfigBase::load_from_json(const std::string &file, ConfigSubstitutionContext& substitution_context, bool load_inherits_to_config, std::map<std::string, std::string>& key_values, std::string& reason)
{
json j;
@@ -916,44 +929,44 @@ int ConfigBase::load_from_json(const std::string &file, ConfigSubstitutionContex
}
//parse the json elements
for (auto it = j.begin(); it != j.end(); it++) {
if (boost::iequals(it.key(),BBL_JSON_KEY_VERSION)) {
if (ascii_iequals(it.key(), BBL_JSON_KEY_VERSION)) {
key_values.emplace(BBL_JSON_KEY_VERSION, it.value());
}
else if (boost::iequals(it.key(), BBL_JSON_KEY_IS_CUSTOM)) {
else if (ascii_iequals(it.key(), BBL_JSON_KEY_IS_CUSTOM)) {
//skip it
}
else if (boost::iequals(it.key(), BBL_JSON_KEY_NAME)) {
else if (ascii_iequals(it.key(), BBL_JSON_KEY_NAME)) {
key_values.emplace(BBL_JSON_KEY_NAME, it.value());
if (it.value() == "project_settings")
is_project_settings = true;
}
else if (boost::iequals(it.key(), BBL_JSON_KEY_URL)) {
else if (ascii_iequals(it.key(), BBL_JSON_KEY_URL)) {
key_values.emplace(BBL_JSON_KEY_URL, it.value());
}
else if (boost::iequals(it.key(), BBL_JSON_KEY_TYPE)) {
else if (ascii_iequals(it.key(), BBL_JSON_KEY_TYPE)) {
key_values.emplace(BBL_JSON_KEY_TYPE, it.value());
}
else if (boost::iequals(it.key(), BBL_JSON_KEY_SETTING_ID)) {
else if (ascii_iequals(it.key(), BBL_JSON_KEY_SETTING_ID)) {
key_values.emplace(BBL_JSON_KEY_SETTING_ID, it.value());
}
else if (boost::iequals(it.key(), BBL_JSON_KEY_FILAMENT_ID)) {
else if (ascii_iequals(it.key(), BBL_JSON_KEY_FILAMENT_ID)) {
key_values.emplace(BBL_JSON_KEY_FILAMENT_ID, it.value());
}
else if (boost::iequals(it.key(), BBL_JSON_KEY_FROM)) {
else if (ascii_iequals(it.key(), BBL_JSON_KEY_FROM)) {
key_values.emplace(BBL_JSON_KEY_FROM, it.value());
}
else if (boost::iequals(it.key(), BBL_JSON_KEY_DESCRIPTION)) {
else if (ascii_iequals(it.key(), BBL_JSON_KEY_DESCRIPTION)) {
key_values.emplace(BBL_JSON_KEY_DESCRIPTION, it.value());
}
else if (boost::iequals(it.key(), BBL_JSON_KEY_INSTANTIATION)) {
else if (ascii_iequals(it.key(), BBL_JSON_KEY_INSTANTIATION)) {
key_values.emplace(BBL_JSON_KEY_INSTANTIATION, it.value());
}
else if (!load_inherits_to_config && boost::iequals(it.key(), BBL_JSON_KEY_INHERITS)) {
else if (!load_inherits_to_config && ascii_iequals(it.key(), BBL_JSON_KEY_INHERITS)) {
key_values.emplace(BBL_JSON_KEY_INHERITS, it.value());
}
else if (!load_inherits_to_config && boost::iequals(it.key(), BBL_JSON_KEY_INCLUDES)) {
else if (!load_inherits_to_config && ascii_iequals(it.key(), BBL_JSON_KEY_INCLUDES)) {
key_values.emplace(BBL_JSON_KEY_INCLUDES, it.value().dump());
} else if (boost::iequals(it.key(), ORCA_JSON_KEY_RENAMED_FROM)) {
} else if (ascii_iequals(it.key(), ORCA_JSON_KEY_RENAMED_FROM)) {
key_values.emplace(ORCA_JSON_KEY_RENAMED_FROM, it.value());
} else {
t_config_option_key opt_key = it.key();
@@ -1525,12 +1538,10 @@ void ConfigBase::save_to_json(const std::string &file, const std::string &name,
// Serialize first: if that throws (invalid UTF-8), the existing file stays untouched.
std::ostringstream ss;
this->save_to_json(ss, name, from, version);
boost::nowide::ofstream c;
c.open(file, std::ios::out | std::ios::trunc);
c << ss.str();
c.close();
BOOST_LOG_TRIVIAL(info) << __FUNCTION__ << ":" <<__LINE__ << boost::format(", saved config to %1%\n")%file;
if (const std::error_code ec = write_file_atomically(file, ss.str()))
BOOST_LOG_TRIVIAL(error) << __FUNCTION__ << boost::format(": failed to save config to %1%: %2%") % file % ec.message();
else
BOOST_LOG_TRIVIAL(info) << __FUNCTION__ << ":" <<__LINE__ << boost::format(", saved config to %1%\n")%file;
}
void ConfigBase::save_to_json(std::ostream &os, const std::string &name, const std::string &from, const std::string &version, bool replace_invalid_utf8) const
+165
View File
@@ -0,0 +1,165 @@
#include "InstanceLock.hpp"
#include <map>
#include <memory>
#include <system_error>
#include <thread>
#include <boost/filesystem.hpp>
#include <boost/log/trivial.hpp>
#include <boost/nowide/fstream.hpp>
#ifdef _WIN32
#include <boost/interprocess/sync/file_lock.hpp>
#include <boost/nowide/convert.hpp>
#else
#include <cerrno>
#include <fcntl.h>
#include <sys/file.h>
#include <unistd.h>
#endif
namespace Slic3r {
#ifdef _WIN32
// LockFileEx, held by this handle alone.
using NativeFileLock = boost::interprocess::file_lock;
#else
// flock(2) rather than an fcntl lock: it belongs to this open file description,
// so any other code in the process that opens and closes the lock file, as a
// backup or an export walking the data dir might, cannot drop it. An fcntl
// lock would go with the first such close.
class NativeFileLock
{
public:
explicit NativeFileLock(const char *path) : m_fd(::open(path, O_RDWR | O_CREAT | O_CLOEXEC, 0644))
{
if (m_fd < 0)
throw std::system_error(errno, std::generic_category(), path);
}
~NativeFileLock() { ::close(m_fd); }
bool try_lock()
{
if (::flock(m_fd, LOCK_EX | LOCK_NB) == 0)
return true;
// A signal (a child exiting, for one) interrupts the call like any other; the caller polls again.
if (errno == EWOULDBLOCK || errno == EINTR)
return false;
throw std::system_error(errno, std::generic_category(), "flock");
}
void unlock() { ::flock(m_fd, LOCK_UN); }
private:
int m_fd;
};
#endif
// One slot per lock file, shared by every guard in the process: one lock
// object per path behind a mutex is what makes the guard re-entrant and safe
// to use from the preset sync thread and the GUI thread at once. The lock
// file is opened by the outermost guard and closed when it goes, so the file
// is never held open between guards: whatever is at the path is what gets
// locked, and a data dir can be removed once nothing is saving into it. The
// file is kept rather than deleted on release because the lock state lives in
// the kernel on the open file, and deleting it would let a third instance
// lock a fresh file while the second still holds the old one.
struct InstanceLock::Slot
{
std::recursive_mutex mutex;
// Non-null exactly while this process holds the file lock.
std::unique_ptr<NativeFileLock> file_lock;
int depth{0};
// Until this point, after a guard could not open, lock or wait out the
// file, guards do not touch it.
std::chrono::steady_clock::time_point cooldown_until{};
};
InstanceLock::Slot &InstanceLock::slot_for(const std::string &lock_file_path)
{
// Never freed: a save during static destruction still needs its slot.
static auto *registry_mutex = new std::mutex();
static auto *registry = new std::map<std::string, std::unique_ptr<Slot>>();
std::lock_guard<std::mutex> guard(*registry_mutex);
std::unique_ptr<Slot> &slot = (*registry)[lock_file_path];
if (! slot)
slot = std::make_unique<Slot>();
return *slot;
}
// Starts the cool-down. Called with the slot mutex held.
void InstanceLock::defer(Slot &slot, const std::string &reason)
{
slot.cooldown_until = std::chrono::steady_clock::now() + cooldown;
BOOST_LOG_TRIVIAL(warning) << reason << "; proceeding without the lock for the next " << cooldown.count() << " ms";
}
// Creates the lock file if needed and opens it, or starts the cool-down.
// Called with the slot mutex held.
bool InstanceLock::open_lock_file(Slot &slot, const std::string &lock_file_path)
{
try {
#ifdef _WIN32
// The lock opens an existing file; created once, on the first miss.
const std::wstring wide_path = boost::nowide::widen(lock_file_path);
try {
slot.file_lock = std::make_unique<NativeFileLock>(wide_path.c_str());
} catch (const std::exception &) {
boost::nowide::ofstream(lock_file_path, std::ios::app).close();
slot.file_lock = std::make_unique<NativeFileLock>(wide_path.c_str());
}
#else
slot.file_lock = std::make_unique<NativeFileLock>(lock_file_path.c_str());
#endif
return true;
} catch (const std::exception &e) {
defer(slot, "Cannot open lock file " + lock_file_path + ": " + e.what() + " (check its owner and permissions)");
return false;
}
}
InstanceLock::InstanceLock(const std::string &lock_file_path, std::chrono::milliseconds timeout)
{
if (lock_file_path.empty())
return;
m_slot = &slot_for(lock_file_path);
m_slot_guard = std::unique_lock<std::recursive_mutex>(m_slot->mutex);
const auto now = std::chrono::steady_clock::now();
if (m_slot->depth == 0 && now >= m_slot->cooldown_until && open_lock_file(*m_slot, lock_file_path)) {
const auto deadline = now + timeout;
bool taken = false;
for (;;) {
try {
if ((taken = m_slot->file_lock->try_lock()))
break;
} catch (const std::exception &e) {
defer(*m_slot, "Cannot lock " + lock_file_path + ": " + e.what());
break;
}
if (std::chrono::steady_clock::now() >= deadline) {
defer(*m_slot, "Another instance has held " + lock_file_path + " for over " + std::to_string(timeout.count()) + " ms");
break;
}
std::this_thread::sleep_for(std::chrono::milliseconds(5));
}
if (! taken)
m_slot->file_lock.reset();
}
// Counted last, so a throw above leaves the slot exactly as it was found.
++ m_slot->depth;
m_locked = m_slot->file_lock != nullptr;
}
InstanceLock::~InstanceLock()
{
if (m_slot == nullptr)
return;
if (-- m_slot->depth == 0 && m_slot->file_lock) {
try {
m_slot->file_lock->unlock();
} catch (const std::exception &e) {
BOOST_LOG_TRIVIAL(warning) << "Cannot unlock instance lock: " << e.what();
}
m_slot->file_lock.reset();
}
}
} // namespace Slic3r
+60
View File
@@ -0,0 +1,60 @@
#pragma once
#include <chrono>
#include <mutex>
#include <string>
namespace Slic3r {
// Scoped write lock on a file shared by every running instance of the
// application, such as the app config or the user preset directory: threads
// of this process are serialised through a recursive mutex, other processes
// through an advisory OS file lock on `lock_file_path`. The lock file is
// created on first use and kept; the OS releases the lock when its holder
// exits, so a crashed instance never leaves a stale lock behind.
//
// Best effort: when the lock file cannot be opened or locked, or another
// instance still holds it after `timeout`, the guard keeps only the in-process
// mutex, locked() reports false and the write proceeds, since a hung instance
// must never block another one from saving. For `cooldown` afterwards guards
// leave the file alone. The wait for the in-process mutex is bounded only by
// the longest critical section, so a guard covers a few file operations and
// nothing slower.
//
// Lock order: the preset collection mutex may be held when a guard is taken
// (set_sync_info_and_save() calls save_info() under it), never the reverse;
// that is why the guards sit at the leaf readers and writers and why a guard
// must not be added around save_user_presets(), which takes the collection
// mutex through delete_preset().
class InstanceLock
{
public:
// Long against a critical section of milliseconds, short against the GUI
// thread, which is where most guards are taken.
static constexpr std::chrono::milliseconds default_timeout{1000};
// Long enough that a holder stuck in a debugger does not cost a stall per
// save; mutable so tests can shorten it.
static inline std::chrono::milliseconds cooldown{10000};
// An empty path makes the guard a no-op.
explicit InstanceLock(const std::string &lock_file_path, std::chrono::milliseconds timeout = default_timeout);
~InstanceLock();
InstanceLock(const InstanceLock &) = delete;
InstanceLock &operator=(const InstanceLock &) = delete;
// True while this process holds the cross-process file lock.
bool locked() const { return m_locked; }
private:
struct Slot;
static Slot &slot_for(const std::string &lock_file_path);
static bool open_lock_file(Slot &slot, const std::string &lock_file_path);
static void defer(Slot &slot, const std::string &reason);
Slot *m_slot{nullptr};
std::unique_lock<std::recursive_mutex> m_slot_guard;
bool m_locked{false};
};
} // namespace Slic3r
+16
View File
@@ -13,8 +13,19 @@
namespace Slic3r {
// How many setters this thread holds, so the ones nested in another can skip
// setlocale, which takes a lock the whole process shares on Windows.
static thread_local int s_numeric_locale_depth = 0;
CNumericLocalesSetter::CNumericLocalesSetter()
{
// Nested in another setter on this thread, whose "C" the separator check
// confirms is still set.
if (s_numeric_locale_depth > 0 && is_decimal_separator_point()) {
m_nested = true;
++ s_numeric_locale_depth;
return;
}
#ifdef _WIN32
_configthreadlocale(_ENABLE_PER_THREAD_LOCALE);
m_orig_numeric_locale = std::setlocale(LC_NUMERIC, nullptr);
@@ -29,12 +40,17 @@ CNumericLocalesSetter::CNumericLocalesSetter()
m_new_locale = newlocale(LC_NUMERIC_MASK, "C", m_new_locale);
uselocale(m_new_locale);
#endif
// Counted last, since the destructor does not run for a constructor that throws.
++ s_numeric_locale_depth;
}
CNumericLocalesSetter::~CNumericLocalesSetter()
{
-- s_numeric_locale_depth;
if (m_nested)
return;
#ifdef _WIN32
std::setlocale(LC_NUMERIC, m_orig_numeric_locale.data());
#else
+5
View File
@@ -19,8 +19,13 @@ class CNumericLocalesSetter {
public:
CNumericLocalesSetter();
~CNumericLocalesSetter();
// A copy would restore the locale twice, and count down once more than up.
CNumericLocalesSetter(const CNumericLocalesSetter&) = delete;
CNumericLocalesSetter& operator=(const CNumericLocalesSetter&) = delete;
private:
// Inside another setter on this thread, which does the setting and restoring.
bool m_nested { false };
#ifdef _WIN32
std::string m_orig_numeric_locale;
#else
+65
View File
@@ -0,0 +1,65 @@
#pragma once
#include <algorithm>
#include <cstddef>
#include <type_traits>
#include <utility>
#include <variant>
#include <vector>
#include <tbb/blocked_range.h>
#include <tbb/parallel_for.h>
#include <tbb/task_arena.h>
#include <tbb/task_group.h>
#include "Exception.hpp"
namespace Slic3r {
// A batch of resolved presets, each a whole config, is what resolve_then_commit adds
// to peak memory, so it stays far below a vendor's preset count and above any core count.
inline constexpr size_t resolve_batch_size = 64;
// Resolve `count` items that do not depend on each other and install them one at
// a time.
//
// `resolve(i)` runs on any thread and must read only, since the items are
// resolved side by side. `commit(i, resolved)` is called for every item in index
// order, on the calling thread, and is where shared state is written.
//
// The items are worked through in batches, so what is resolved and held at once
// does not grow with `count`. An exception from either callable propagates after
// the batches before it have been committed. A cancellation of the caller's task
// group, which stops a batch partway without an exception, throws RuntimeError
// before that batch commits.
//
// `ChunkSetup`, when given, is constructed once for each piece of a batch TBB hands
// out, for per-thread state a resolve would otherwise set up per item, such as the
// C numeric locale, whose setting takes a lock the whole process shares.
template<class ChunkSetup = std::monostate, class Resolve, class Commit>
void resolve_then_commit(size_t count, Resolve resolve, Commit commit)
{
using Resolved = std::invoke_result_t<Resolve&, size_t>;
std::vector<Resolved> resolved(std::min(count, resolve_batch_size));
for (size_t first = 0; first < count; first += resolve_batch_size) {
const size_t last = std::min(first + resolve_batch_size, count);
// Isolated, so a thread waiting on the batch runs none of the caller's other
// tasks before finishing it.
tbb::this_task_arena::isolate([&] {
tbb::parallel_for(tbb::blocked_range<size_t>(first, last),
[&](const tbb::blocked_range<size_t>& range) {
ChunkSetup setup;
(void) setup;
for (size_t i = range.begin(); i < range.end(); ++ i)
resolved[i - first] = resolve(i);
});
});
if (tbb::is_current_task_group_canceling())
throw RuntimeError("resolve_then_commit: canceled before the batch was resolved");
for (size_t i = first; i < last; ++ i)
commit(i, std::move(resolved[i - first]));
}
}
} // namespace Slic3r
+352 -169
View File
@@ -5,6 +5,8 @@
#include "Preset.hpp"
#include "PresetBundle.hpp"
#include "AppConfig.hpp"
#include "LocalesUtils.hpp"
#include "ParallelResolve.hpp"
#ifdef _MSC_VER
#define WIN32_LEAN_AND_MEAN
@@ -49,6 +51,9 @@
#include "libslic3r.h"
#include "LifecycleEvents.hpp"
#include "Utils.hpp"
#include "InstanceLock.hpp"
#include <sstream>
#include "Time.hpp"
#include "PlaceholderParser.hpp"
#include "libslic3r/GCode/Thumbnails.hpp"
@@ -105,6 +110,32 @@ std::string get_preset_canonical_name(const std::string &preset_bare_name, const
}
}
std::string user_presets_lock_path(bool read_only)
{
return read_only || data_dir().empty() ? std::string() : (fs::path(data_dir()) / (PRESET_USER_DIR ".lock")).string();
}
// Removes a preset file the scan could not load, and its .info, under the lock.
// The scan passes read_only when it could not take the lock, since the file may
// then be another instance's fresh write that it merely raced.
static void remove_preset_files(const std::string &preset_file, bool read_only)
{
if (read_only)
return;
InstanceLock instance_lock(user_presets_lock_path());
// A symlink whose target is missing stays, since the target may come back; one
// that cannot even be followed is removed like any other unreadable file.
auto remove = [](const fs::path &file) {
boost::system::error_code ec;
if (fs::status(file, ec).type() != fs::file_not_found)
fs::remove(file, ec);
};
fs::path file_path(preset_file);
remove(file_path);
file_path.replace_extension(".info");
remove(file_path);
}
std::string get_preset_bare_name(const std::string &canonical_name)
{
const auto pos = canonical_name.find_last_of('/');
@@ -506,10 +537,11 @@ void Preset::normalize(DynamicPrintConfig &config)
handle_legacy_sla(config);
}
std::string Preset::remove_invalid_keys(DynamicPrintConfig &config, const DynamicPrintConfig &default_config)
std::string Preset::remove_invalid_keys(DynamicPrintConfig &config, const DynamicPrintConfig &default_config,
const DynamicPrintConfig *added)
{
std::string incorrect_keys;
for (const std::string &key : config.keys())
for (const std::string &key : (added != nullptr ? *added : config).keys())
if (! default_config.has(key)) {
if (incorrect_keys.empty())
incorrect_keys = key;
@@ -621,7 +653,6 @@ void Preset::load_info(const std::string& file)
}
else if (v.first.compare("base_id") == 0) {
this->base_id = v.second.get_value<std::string>();
BOOST_LOG_TRIVIAL(info) << __FUNCTION__ << " load info from: " << file << " and base_id: " << this->base_id;
if (this->base_id.compare("null") == 0)
this->base_id.clear();
}
@@ -647,18 +678,20 @@ void Preset::save_info(std::string file)
file = idx_file.string();
}
boost::nowide::ofstream c;
c.open(file, std::ios::out | std::ios::trunc);
std::string sync_info_to_save;
//BBS: hold is used for stop requesting to server this time
if (this->sync_info.compare("hold") != 0)
sync_info_to_save = this->sync_info;
std::ostringstream c;
c << "sync_info" << " = " << sync_info_to_save << std::endl;
c << "user_id" << " = " << this->user_id << std::endl;
c << "setting_id" << " = " << this->setting_id << std::endl;
c << "base_id" << " = " << this->base_id << std::endl;
c << "updated_time" << " = " << std::to_string(this->updated_time) << std::endl;
c.close();
InstanceLock instance_lock(user_presets_lock_path());
if (const std::error_code ec = write_file_atomically(file, c.str()))
BOOST_LOG_TRIVIAL(error) << __FUNCTION__ << ": failed to save " << file << ": " << ec.message();
}
void Preset::remove_files(bool cloud_already_deleted)
@@ -667,6 +700,7 @@ void Preset::remove_files(bool cloud_already_deleted)
if (this->is_project_embedded) {
return;
}
InstanceLock instance_lock(user_presets_lock_path());
// Erase the preset file.
boost::nowide::remove(this->file.c_str());
fs::path idx_path(this->file);
@@ -684,11 +718,11 @@ void Preset::remove_files(bool cloud_already_deleted)
}
//BBS: add logic for only difference save
void Preset::save(DynamicPrintConfig* parent_config)
bool Preset::save(DynamicPrintConfig* parent_config)
{
//BBS: add project embedded preset logic
if (this->is_project_embedded)
return;
return true;
//BBS: change to json format
//this->config.save(this->file);
std::string from_str;
@@ -703,12 +737,16 @@ void Preset::save(DynamicPrintConfig* parent_config)
else
from_str = std::string("Default");
boost::filesystem::create_directories(fs::path(this->file).parent_path());
const std::string bare_name = get_preset_bare_name(this->name);
// What gets written: the diff against the parent, the config plus its
// filament id, or the config as is. Built before the lock is taken so the
// exclusive window covers only the file writes.
DynamicPrintConfig temp_config;
const DynamicPrintConfig *to_save = &this->config;
//BBS: only save difference if it has parent
if (parent_config) {
DynamicPrintConfig temp_config;
std::vector<std::string> dirty_options = config.diff(*parent_config);
std::string extruder_id_name, extruder_variant_name;
@@ -744,13 +782,22 @@ void Preset::save(DynamicPrintConfig* parent_config)
opt_dst->set(opt_src);
}
}
temp_config.save_to_json(this->file, bare_name, from_str, this->version.to_string());
to_save = &temp_config;
} else if (!filament_id.empty() && inherits().empty()) {
DynamicPrintConfig temp_config = config;
temp_config = config;
temp_config.set_key_value(BBL_JSON_KEY_FILAMENT_ID, new ConfigOptionString(filament_id));
temp_config.save_to_json(this->file, bare_name, from_str, this->version.to_string());
} else {
this->config.save_to_json(this->file, bare_name, from_str, this->version.to_string());
to_save = &temp_config;
}
std::ostringstream json;
to_save->save_to_json(json, bare_name, from_str, this->version.to_string());
InstanceLock instance_lock(user_presets_lock_path());
boost::filesystem::create_directories(fs::path(this->file).parent_path());
if (const std::error_code ec = write_file_atomically(this->file, json.str())) {
// No .info either: one without its preset reads as a cloud deletion request.
BOOST_LOG_TRIVIAL(error) << __FUNCTION__ << ": failed to save " << this->file << ": " << ec.message();
return false;
}
BOOST_LOG_TRIVIAL(info) << __FUNCTION__ << " save config for: " << this->name << " and filament_id: " << filament_id << " and base_id: " << this->base_id;
@@ -760,6 +807,7 @@ void Preset::save(DynamicPrintConfig* parent_config)
idx_file.replace_extension(".info");
this->save_info(idx_file.string());
}
return true;
}
void Preset::reload(Preset const &parent)
@@ -771,6 +819,7 @@ void Preset::reload(Preset const &parent)
std::string reason;
ForwardCompatibilitySubstitutionRule substitution_rule = ForwardCompatibilitySubstitutionRule::Disable;
try {
InstanceLock instance_lock(user_presets_lock_path());
ConfigSubstitutions config_substitutions = config.load_from_json(file, substitution_rule, key_values, reason);
this->config = parent.config;
this->config.apply(std::move(config));
@@ -1669,6 +1718,186 @@ std::string PresetCollection::canonical_preset_name(const std::string &name, con
return get_preset_canonical_name(parsed.bare, origin);
}
PresetCollection::PresetFilesOnDisk PresetCollection::PresetFilesOnDisk::read(const boost::filesystem::path &file)
{
auto read_if_present = [](const fs::path &path) -> std::optional<std::string> {
std::string bytes;
try {
load_string_file(path, bytes);
} catch (const std::exception &) {
return std::nullopt;
}
return bytes;
};
fs::path info_path(file);
info_path.replace_extension(".info");
return { read_if_present(file), read_if_present(info_path) };
}
PresetCollection::UserPresetLoad PresetCollection::resolve_user_preset(
const boost::filesystem::path &file, const std::string &canonical_name,
const PresetOrigin &load_origin, ForwardCompatibilitySubstitutionRule substitution_rule,
const std::string &extruder_id_name, const std::string &extruder_variant_name,
std::set<std::string> *key_set1, std::set<std::string> *key_set2) const
{
UserPresetLoad out;
out.preset = Preset(m_type, canonical_name, false);
Preset &preset = out.preset;
preset.bundle_id = load_origin.bundle_id;
preset.file = file.string();
// Before either file is parsed, so a save that lands during the parse still shows
// up as a difference when commit compares.
out.on_disk = PresetFilesOnDisk::read(file);
// Load the preset file, apply preset values on top of defaults.
try {
fs::path idx_path(preset.file);
idx_path.replace_extension(".info");
if (fs::exists(idx_path)) {
out.info_file = idx_path.string();
preset.load_info(out.info_file);
}
DynamicPrintConfig config;
//BBS: change to json format
//ConfigSubstitutions config_substitutions = config.load_from_ini(preset.file, substitution_rule);
std::map<std::string, std::string> key_values;
std::string reason;
ConfigSubstitutions config_substitutions = config.load_from_json(preset.file, substitution_rule, key_values, reason);
if (! config_substitutions.empty())
out.substitutions.push_back({ preset.name, m_type, PresetConfigSubstitutions::Source::UserFile, preset.file, std::move(config_substitutions) });
if (!reason.empty()) {
out.discard_file = true;
out.errors.push_back((boost::format("parse config %1% failed") % preset.file).str());
return out;
}
std::string version_str = key_values[BBL_JSON_KEY_VERSION];
boost::optional<Semver> version = Semver::parse(version_str);
if (!version) return out;
preset.version = *version;
if (key_values.find(BBL_JSON_KEY_FILAMENT_ID) != key_values.end())
preset.filament_id = key_values[BBL_JSON_KEY_FILAMENT_ID];
if (key_values.find(BBL_JSON_KEY_DESCRIPTION) != key_values.end())
preset.description = key_values[BBL_JSON_KEY_DESCRIPTION];
if (key_values.find(BBL_JSON_KEY_INSTANTIATION) != key_values.end())
preset.is_visible = key_values[BBL_JSON_KEY_INSTANTIATION] != "false";
//Orca: find and use the inherit config as the base
const Preset* inherit_preset = nullptr;
ConfigOption* inherits_config = config.option(BBL_JSON_KEY_INHERITS);
// check inherits_config
if (inherits_config) {
ConfigOptionString * option_str = dynamic_cast<ConfigOptionString *> (inherits_config);
std::string inherits_value = option_str->value;
// Orca: try to find if the parent preset has been renamed
inherit_preset = this->find_preset2(inherits_value);
Preset::normalize_inherits(config, inherit_preset);
}
const Preset& default_preset = this->default_preset_for(config);
if (inherit_preset) {
preset.config = inherit_preset->config;
preset.filament_id = inherit_preset->filament_id;
extend_default_config_length(config, false, {});
preset.config.update_diff_values_to_child_config(config, extruder_id_name, extruder_variant_name, *key_set1, *key_set2);
}
else {
auto inherits_config2 = dynamic_cast<ConfigOptionString *>(inherits_config);
if ((inherits_config2 && !inherits_config2->value.empty())) {
out.errors.push_back((boost::format("can not find parent %1% for config %2%!") % inherits_config2->value % preset.file).str());
return out;
}
// We support custom root preset now
// Find a default preset for the config. The PrintPresetCollection provides different default preset based on the "printer_technology" field.
preset.config = default_preset.config;
preset.config.apply(std::move(config));
extend_default_config_length(preset.config, true, default_preset.config);
}
Preset::normalize(preset.config);
// Report configuration fields, which are misplaced into a wrong group.
std::string incorrect_keys = Preset::remove_invalid_keys(preset.config, default_preset.config);
if (!incorrect_keys.empty())
out.errors.push_back("Error in a preset file: The preset \"" + preset.file +
"\" contains the following incorrect keys: " + incorrect_keys + ", which were removed");
if (preset.type == Preset::TYPE_FILAMENT && preset.is_user() && preset.inherits().empty()) {
auto compatible_printers = dynamic_cast<ConfigOptionStrings *>(preset.config.option("compatible_printers", true));
if (compatible_printers && compatible_printers->values.empty()) {
size_t at_pos = canonical_name.find('@');
if (at_pos != std::string::npos && at_pos + 1 < canonical_name.length()) {
compatible_printers->values.push_back(canonical_name.substr(at_pos + 1));
out.save_compatible_printers = true;
}
}
}
preset.loaded = true;
out.complete = true;
} catch (const std::ifstream::failure &err) {
out.discard_file = true;
out.errors.push_back((boost::format("The user-config cannot be loaded: %1%. Reason: %2%") % preset.file % err.what()).str());
//throw Slic3r::RuntimeError(std::string("The selected preset cannot be loaded: ") + preset.file + "\n\tReason: " + err.what());
} catch (const std::runtime_error &err) {
out.discard_file = true;
out.errors.push_back((boost::format("Failed loading the user-config file: %1%. Reason: %2%") % preset.file % err.what()).str());
//throw Slic3r::RuntimeError(std::string("Failed loading the preset file: ") + preset.file + "\n\tReason: " + err.what());
}
out.install = true;
return out;
}
void PresetCollection::commit_user_preset(UserPresetLoad &&loaded, std::deque<Preset> &presets_loaded,
PresetsConfigSubstitutions &substitutions,
const std::function<void(Preset&)> &preset_loaded_fn,
bool read_only)
{
Preset &preset = loaded.preset;
if (! loaded.info_file.empty())
BOOST_LOG_TRIVIAL(info) << __FUNCTION__ << " load info from: " << loaded.info_file << " and base_id: " << preset.base_id;
append(substitutions, std::move(loaded.substitutions));
for (const std::string &error : loaded.errors) {
++m_errors;
BOOST_LOG_TRIVIAL(error) << error;
}
if (loaded.discard_file)
remove_preset_files(loaded.preset.file, read_only);
if (!loaded.install)
return;
if (loaded.complete) {
if (loaded.save_compatible_printers) {
// The preset loads with the derived printer whether or not its file
// can be rewritten.
try {
if (read_only)
BOOST_LOG_TRIVIAL(info) << __FUNCTION__ << " added compatible_printers for preset: " << preset.name << " (not written back)";
else if (preset.save(nullptr))
BOOST_LOG_TRIVIAL(info) << __FUNCTION__ << " added compatible_printers for preset: " << preset.name;
else
++m_errors; // save() logged why
} catch (const std::runtime_error &err) {
++m_errors;
BOOST_LOG_TRIVIAL(error) << __FUNCTION__ << " can not write compatible_printers back to " << preset.file << ": " << err.what();
}
}
//BBS: add some workaround for previous incorrect settings
if ((!preset.setting_id.empty())&&(preset.setting_id == preset.base_id))
preset.setting_id.clear();
BOOST_LOG_TRIVIAL(info) << __FUNCTION__ << " load preset: " << preset.name << " and filament_id: " << preset.filament_id << " and base_id: " << preset.base_id;
//BBS: add config related logs
BOOST_LOG_TRIVIAL(debug) << __FUNCTION__ << boost::format(", preset type %1%, name %2%, path %3%, is_system %4%, is_default %5% is_visible %6%")%Preset::get_type_string(m_type) %preset.name %preset.file %preset.is_system %preset.is_default %preset.is_visible;
// add alias for custom filament preset
set_custom_preset_alias(preset);
}
if (preset_loaded_fn != nullptr)
preset_loaded_fn(preset);
BOOST_LOG_TRIVIAL(info) << __FUNCTION__ << __LINE__ << " load config successful and preset name is:" << preset.name;
presets_loaded.emplace_back(std::move(preset));
}
// Load all presets found in dir_path.
// Throws an exception on error.
void PresetCollection::load_presets(
@@ -1706,6 +1935,8 @@ void PresetCollection::load_presets(
std::set<std::string> *key_set1 = nullptr, *key_set2 = nullptr;
Preset::get_extruder_names_and_keysets(m_type, extruder_id_name, extruder_variant_name, &key_set1, &key_set2);
struct UserPresetFile { fs::path path; std::string canonical_name; };
std::vector<UserPresetFile> files;
//BBS: change to json format
for (auto &dir_entry : boost::filesystem::directory_iterator(dir))
{
@@ -1721,149 +1952,47 @@ void PresetCollection::load_presets(
BOOST_LOG_TRIVIAL(warning) << "Preset already present, not loading: " << canonical_name;
continue;
}
try {
Preset preset(m_type, canonical_name, false);
preset.bundle_id = resolved_origin.bundle_id;
preset.file = dir_entry.path().string();
// Load the preset file, apply preset values on top of defaults.
try {
fs::path idx_path(preset.file);
idx_path.replace_extension(".info");
if (fs::exists(idx_path)) {
preset.load_info(idx_path.string());
}
DynamicPrintConfig config;
//BBS: change to json format
//ConfigSubstitutions config_substitutions = config.load_from_ini(preset.file, substitution_rule);
std::map<std::string, std::string> key_values;
std::string reason;
ConfigSubstitutions config_substitutions = config.load_from_json(preset.file, substitution_rule, key_values, reason);
if (! config_substitutions.empty())
substitutions.push_back({ preset.name, m_type, PresetConfigSubstitutions::Source::UserFile, preset.file, std::move(config_substitutions) });
if (!reason.empty()) {
fs::path file_path(preset.file);
if (!read_only && fs::exists(file_path))
fs::remove(file_path);
file_path.replace_extension(".info");
if (!read_only && fs::exists(file_path))
fs::remove(file_path);
BOOST_LOG_TRIVIAL(error) << boost::format("parse config %1% failed")%preset.file;
++m_errors;
continue;
}
files.push_back({ dir_entry.path(), std::move(canonical_name) });
}
}
std::string version_str = key_values[BBL_JSON_KEY_VERSION];
boost::optional<Semver> version = Semver::parse(version_str);
if (!version) continue;
preset.version = *version;
if (key_values.find(BBL_JSON_KEY_FILAMENT_ID) != key_values.end())
preset.filament_id = key_values[BBL_JSON_KEY_FILAMENT_ID];
if (key_values.find(BBL_JSON_KEY_DESCRIPTION) != key_values.end())
preset.description = key_values[BBL_JSON_KEY_DESCRIPTION];
if (key_values.find(BBL_JSON_KEY_INSTANTIATION) != key_values.end())
preset.is_visible = key_values[BBL_JSON_KEY_INSTANTIATION] != "false";
//Orca: find and use the inherit config as the base
Preset* inherit_preset = nullptr;
ConfigOption* inherits_config = config.option(BBL_JSON_KEY_INHERITS);
// check inherits_config
if (inherits_config) {
ConfigOptionString * option_str = dynamic_cast<ConfigOptionString *> (inherits_config);
std::string inherits_value = option_str->value;
// Orca: try to find if the parent preset has been renamed
inherit_preset = this->find_preset2(inherits_value);
Preset::normalize_inherits(config, inherit_preset);
} else {
;
}
const Preset& default_preset = this->default_preset_for(config);
if (inherit_preset) {
preset.config = inherit_preset->config;
preset.filament_id = inherit_preset->filament_id;
extend_default_config_length(config, false, {});
preset.config.update_diff_values_to_child_config(config, extruder_id_name, extruder_variant_name, *key_set1, *key_set2);
}
else {
auto inherits_config2 = dynamic_cast<ConfigOptionString *>(inherits_config);
if ((inherits_config2 && !inherits_config2->value.empty())) {
BOOST_LOG_TRIVIAL(error) << boost::format("can not find parent %1% for config %2%!")%inherits_config2->value %preset.file;
++m_errors;
continue;
}
// We support custom root preset now
// Find a default preset for the config. The PrintPresetCollection provides different default preset based on the "printer_technology" field.
preset.config = default_preset.config;
preset.config.apply(std::move(config));
extend_default_config_length(preset.config, true, default_preset.config);
}
BOOST_LOG_TRIVIAL(info) << __FUNCTION__ << " load preset: " << name << " and filament_id: " << preset.filament_id << " and base_id: " << preset.base_id;
Preset::normalize(preset.config);
// Report configuration fields, which are misplaced into a wrong group.
std::string incorrect_keys = Preset::remove_invalid_keys(preset.config, default_preset.config);
if (!incorrect_keys.empty()) {
++m_errors;
BOOST_LOG_TRIVIAL(error)
<< "Error in a preset file: The preset \"" << preset.file
<< "\" contains the following incorrect keys: " << incorrect_keys << ", which were removed";
}
if (preset.type == Preset::TYPE_FILAMENT && preset.is_user() && preset.inherits().empty()) {
auto compatible_printers = dynamic_cast<ConfigOptionStrings *>(preset.config.option("compatible_printers", true));
if (compatible_printers && compatible_printers->values.empty()) {
size_t at_pos = name.find('@');
if (at_pos != std::string::npos && at_pos + 1 < name.length()) {
compatible_printers->values.push_back(name.substr(at_pos + 1));
if (!read_only)
preset.save(nullptr);
BOOST_LOG_TRIVIAL(info) << __FUNCTION__ << " added compatible_printers for preset: " << name;
}
}
}
preset.loaded = true;
//BBS: add some workaround for previous incorrect settings
if ((!preset.setting_id.empty())&&(preset.setting_id == preset.base_id))
preset.setting_id.clear();
//BBS: add config related logs
BOOST_LOG_TRIVIAL(debug) << __FUNCTION__ << boost::format(", preset type %1%, name %2%, path %3%, is_system %4%, is_default %5% is_visible %6%")%Preset::get_type_string(m_type) %preset.name %preset.file %preset.is_system %preset.is_default %preset.is_visible;
// add alias for custom filament preset
set_custom_preset_alias(preset);
} catch (const std::ifstream::failure &err) {
++m_errors;
BOOST_LOG_TRIVIAL(error) << boost::format("The user-config cannot be loaded: %1%. Reason: %2%")%preset.file %err.what();
fs::path file_path(preset.file);
if (!read_only && fs::exists(file_path))
fs::remove(file_path);
file_path.replace_extension(".info");
if (!read_only && fs::exists(file_path))
fs::remove(file_path);
//throw Slic3r::RuntimeError(std::string("The selected preset cannot be loaded: ") + preset.file + "\n\tReason: " + err.what());
} catch (const std::runtime_error &err) {
++m_errors;
BOOST_LOG_TRIVIAL(error) << boost::format("Failed loading the user-config file: %1%. Reason: %2%")%preset.file %err.what();
//throw Slic3r::RuntimeError(std::string("Failed loading the preset file: ") + preset.file + "\n\tReason: " + err.what());
fs::path file_path(preset.file);
if (!read_only && fs::exists(file_path))
fs::remove(file_path);
file_path.replace_extension(".info");
if (!read_only && fs::exists(file_path))
fs::remove(file_path);
resolve_then_commit<CNumericLocalesSetter>(files.size(),
[&](size_t i) {
return this->resolve_user_preset(files[i].path, files[i].canonical_name, resolved_origin, substitution_rule,
extruder_id_name, extruder_variant_name, key_set1, key_set2);
},
[&](size_t i, UserPresetLoad &&loaded) {
// Resolve read the files without the lock, so another instance may have saved
// over them since. A file that changed is resolved again under the lock before
// commit removes or rewrites it, which also keeps its .json and .info from two
// different saves apart. Without the lock nothing is checked, so commit leaves
// the files alone.
const std::string lock_path = user_presets_lock_path(read_only);
InstanceLock instance_lock(lock_path);
if (instance_lock.locked()) {
const PresetFilesOnDisk on_disk = PresetFilesOnDisk::read(files[i].path);
boost::system::error_code ec;
// The link itself, and a stat error counts as present, so a file that is
// there but cannot be read is still counted and removed by commit.
if (! on_disk.json && fs::symlink_status(files[i].path, ec).type() == fs::file_not_found)
return; // removed by another instance since it was read
if (! (on_disk == loaded.on_disk)) {
CNumericLocalesSetter locales_setter;
loaded = this->resolve_user_preset(files[i].path, files[i].canonical_name, resolved_origin, substitution_rule,
extruder_id_name, extruder_variant_name, key_set1, key_set2);
}
if (preset_loaded_fn != nullptr)
preset_loaded_fn(preset);
presets_loaded.emplace_back(preset);
BOOST_LOG_TRIVIAL(info) << __FUNCTION__ << __LINE__ << " load config successful and preset name is:" << preset.name;
}
const bool leave_files = read_only || (! lock_path.empty() && ! instance_lock.locked());
// Committing can remove an unreadable preset's file, and a filesystem error
// there is reported without stopping the rest of the directory.
try {
this->commit_user_preset(std::move(loaded), presets_loaded, substitutions, preset_loaded_fn, leave_files);
} catch (const std::runtime_error &err) {
errors_cummulative += err.what();
errors_cummulative += "\n";
}
}
}
});
if (presets_loaded.size() > 0)
m_presets.insert(m_presets.end(), std::make_move_iterator(presets_loaded.begin()), std::make_move_iterator(presets_loaded.end()));
sort_presets();
@@ -2167,7 +2296,10 @@ void PresetCollection::set_sync_info_and_save(std::string name, std::string sett
preset->setting_id = setting_id;
if (update_time > 0)
preset->updated_time = update_time;
preset->sync_info == "update" ? preset->save(nullptr) : preset->save_info();
if (preset->sync_info == "update")
preset->save(nullptr);
else
preset->save_info();
break;
}
}
@@ -2827,6 +2959,18 @@ std::pair<Preset*, bool> PresetCollection::load_external_preset(
return std::make_pair(&preset, false);
}
Preset& PresetCollection::append_preset(std::string &&path, const std::string &name, DynamicPrintConfig &&config)
{
lock();
Preset &preset = m_presets.emplace_back(m_type, name, false);
preset.file = std::move(path);
preset.config = std::move(config);
preset.loaded = true;
preset.is_dirty = false;
unlock();
return preset;
}
Preset& PresetCollection::load_preset(const std::string &path, const std::string &name, DynamicPrintConfig &&config, bool select, Semver file_version)
{
lock();
@@ -3846,28 +3990,53 @@ bool PresetCollection::select_preset_by_name_strict(const std::string &name)
return false;
}
// Merge one vendor's presets with the other vendor's presets, report duplicates.
std::vector<std::string> PresetCollection::merge_presets(PresetCollection &&other, const VendorMap &new_vendors)
std::vector<std::vector<std::string>> PresetCollection::merge_presets(const std::vector<PresetCollection*> &others, const VendorMap &new_vendors)
{
std::vector<std::string> duplicates;
for (Preset &preset : other.m_presets) {
if (preset.is_default || preset.is_external)
continue;
Preset key(m_type, preset.name);
auto it = (m_type == Preset::TYPE_FILAMENT)
? std::lower_bound(m_presets.begin() + m_num_default_presets, m_presets.end(), key, filament_preset_less)
: std::lower_bound(m_presets.begin() + m_num_default_presets, m_presets.end(), key);
if (it == m_presets.end() || it->name != preset.name) {
auto less = [this](const Preset &a, const Preset &b) {
return m_type == Preset::TYPE_FILAMENT ? filament_preset_less(a, b) : a < b;
};
struct Incoming { Preset *preset; size_t source; };
auto incoming_less = [&less](const Incoming &a, const Incoming &b) { return less(*a.preset, *b.preset); };
// Each of `others` is sorted, so its presets form one sorted run.
std::vector<Incoming> incoming;
std::vector<size_t> run_ends { 0 };
for (size_t source = 0; source < others.size(); ++ source) {
for (Preset &preset : others[source]->m_presets)
if (! preset.is_default && ! preset.is_external)
incoming.push_back({ &preset, source });
assert(std::is_sorted(incoming.begin() + run_ends.back(), incoming.end(), incoming_less));
run_ends.push_back(incoming.size());
}
// Merged pairwise and stably, so equal names stay in the order of `others`.
const size_t runs = others.size();
for (size_t width = 1; width < runs; width *= 2)
for (size_t i = 0; i + width < runs; i += 2 * width)
std::inplace_merge(incoming.begin() + run_ends[i], incoming.begin() + run_ends[i + width],
incoming.begin() + run_ends[std::min(i + 2 * width, runs)], incoming_less);
std::vector<std::vector<std::string>> duplicates(others.size());
std::deque<Preset> merged;
auto own = m_presets.begin() + m_num_default_presets;
std::move(m_presets.begin(), own, std::back_inserter(merged));
// On equal names this collection's preset is kept, else the earliest of `others`,
// and each repeat is listed under the collection it came from.
for (auto next = incoming.begin(); own != m_presets.end() || next != incoming.end();) {
if (next == incoming.end() || (own != m_presets.end() && ! less(*next->preset, *own)))
merged.emplace_back(std::move(*own ++));
else {
Preset &preset = *(next ++)->preset;
if (preset.vendor != nullptr) {
// Re-assign a pointer to the vendor structure in the new PresetBundle.
auto it = new_vendors.find(preset.vendor->id);
assert(it != new_vendors.end());
preset.vendor = &it->second;
}
m_presets.emplace(it, std::move(preset));
} else
duplicates.emplace_back(std::move(preset.name));
merged.emplace_back(std::move(preset));
}
for (; next != incoming.end() && next->preset->name == merged.back().name; ++ next)
duplicates[next->source].emplace_back(next->preset->name);
}
m_presets = std::move(merged);
return duplicates;
}
@@ -4212,8 +4381,15 @@ void PhysicalPrinter::update_preset_names_in_config()
}
}
void PhysicalPrinter::save(DynamicPrintConfig* /* parent_config */)
{
InstanceLock instance_lock(user_presets_lock_path());
this->config.save_to_json(this->file, std::string("Physical_Printer"), std::string("User"), std::string(SLIC3R_VERSION));
}
void PhysicalPrinter::save(const std::string& file_name_from, const std::string& file_name_to)
{
InstanceLock instance_lock(user_presets_lock_path());
// rename the file
boost::nowide::rename(file_name_from.data(), file_name_to.data());
this->file = file_name_to;
@@ -4345,6 +4521,7 @@ void PhysicalPrinterCollection::load_printers(
continue;
}
try {
InstanceLock instance_lock(user_presets_lock_path());
PhysicalPrinter printer(name, this->default_config());
printer.file = dir_entry.path().string();
// Load the preset file, apply preset values on top of defaults.
@@ -4537,7 +4714,10 @@ bool PhysicalPrinterCollection::delete_printer(const std::string& name)
const PhysicalPrinter& printer = *it;
// Erase the preset file.
boost::nowide::remove(printer.file.c_str());
{
InstanceLock instance_lock(user_presets_lock_path());
boost::nowide::remove(printer.file.c_str());
}
m_printers.erase(it);
return true;
}
@@ -4549,7 +4729,10 @@ bool PhysicalPrinterCollection::delete_selected_printer()
const PhysicalPrinter& printer = this->get_selected_printer();
// Erase the preset file.
boost::nowide::remove(printer.file.c_str());
{
InstanceLock instance_lock(user_presets_lock_path());
boost::nowide::remove(printer.file.c_str());
}
// Remove the preset from the list.
m_printers.erase(m_printers.begin() + m_idx_selected);
// unselect all printers
+74 -5
View File
@@ -8,6 +8,7 @@
#include <unordered_set>
#include <functional>
#include <mutex>
#include <optional>
#include <boost/algorithm/string/predicate.hpp>
#include <boost/filesystem/path.hpp>
#include <boost/property_tree/ptree_fwd.hpp>
@@ -330,7 +331,8 @@ public:
//BBS: add logic for only difference save
//if parent_config is null, save all keys, otherwise, only save difference
void save(DynamicPrintConfig* parent_config);
// Returns false when the preset file could not be written.
bool save(DynamicPrintConfig* parent_config);
void reload(Preset const & parent);
// Return a label of this preset, consisting of a name and a "(modified)" suffix, if this preset is dirty.
@@ -445,7 +447,10 @@ public:
static std::string remove_suffix_modified(const std::string& name);
static void normalize(DynamicPrintConfig &config);
// Report configuration fields, which are misplaced into a wrong group, remove them from the config.
static std::string remove_invalid_keys(DynamicPrintConfig &config, const DynamicPrintConfig &default_config);
// `added`, when given, is the diff applied over a copy of default_config, and only
// its keys are checked, since no other key can be missing from default_config.
static std::string remove_invalid_keys(DynamicPrintConfig &config, const DynamicPrintConfig &default_config,
const DynamicPrintConfig *added = nullptr);
// BBS: move constructor to public
Preset(Type type, const std::string &name, bool is_default = false) : type(type), is_default(is_default), name(name) {}
@@ -485,6 +490,12 @@ std::string get_preset_canonical_name(const std::string &preset_bare_name, const
// Tail segment of a canonical name — what's written to the bundle's .json filename and JSON "name" field.
std::string get_preset_bare_name(const std::string &canonical_name);
// Lock file guarding every user preset file under data_dir() against other
// running instances and the preset sync thread. Empty without a data dir, and
// for a read-only load (the CLI), which never rewrites or deletes and may run
// many jobs on one data dir.
std::string user_presets_lock_path(bool read_only = false);
// Resolve an origin from a directory path when the caller passes Kind::Auto.
PresetOrigin detect_origin_from_path(const boost::filesystem::path &path, const PresetOrigin &explicit_origin = PresetOrigin());
@@ -801,6 +812,8 @@ public:
// Return number of presets including the "- default -" preset.
size_t size() const { return m_presets.size(); }
bool has_defaults_only() const { return m_presets.size() <= m_num_default_presets; }
// How many presets this collection refused or repaired while loading.
int error_count() const { return m_errors; }
// For Print / Filament presets, disable those, which are not compatible with the printer.
template<typename PreferedCondition>
@@ -874,8 +887,10 @@ protected:
// This is a temporary state, which shall be fixed immediately by the following step.
bool select_preset_by_name_strict(const std::string &name);
// Merge one vendor's presets with the other vendor's presets, report duplicates.
std::vector<std::string> merge_presets(PresetCollection &&other, const VendorMap &new_vendors);
// Move the presets of `others` into this collection in one pass. A name this
// collection or an earlier one of `others` already has is left out, and reported
// in the list of the collection that repeats it.
std::vector<std::vector<std::string>> merge_presets(const std::vector<PresetCollection*> &others, const VendorMap &new_vendors);
// Update m_map_alias_to_profile_name from loaded system profiles.
void update_map_alias_to_profile_name();
@@ -890,6 +905,56 @@ protected:
void set_custom_preset_alias(Preset &preset);
private:
// A preset file and its .info as read from disk, std::nullopt for one that is missing.
struct PresetFilesOnDisk
{
std::optional<std::string> json;
std::optional<std::string> info;
static PresetFilesOnDisk read(const boost::filesystem::path &file);
bool operator==(const PresetFilesOnDisk &rhs) const { return json == rhs.json && info == rhs.info; }
};
// One preset file read and flattened against the presets already in this
// collection, before anything the collection shares has been touched.
struct UserPresetLoad
{
Preset preset;
// Joins the collection. A file that threw partway still joins it, without
// the steps that did not run.
bool install { false };
// The whole of the load ran, so the preset is ready to be aliased.
bool complete { false };
// A filament preset that named no compatible printer and was given one from
// its name, which commit writes back to its file.
bool save_compatible_printers { false };
// Unreadable, so commit removes it and its .info file.
bool discard_file { false };
// The .info file read beside the preset, which commit logs.
std::string info_file;
// Both files as they were before the preset was read from them. Resolve runs
// without the instance lock, so commit compares this with the disk under it.
PresetFilesOnDisk on_disk;
// Counted and logged by commit, in the order the directory listed the files.
std::vector<std::string> errors;
PresetsConfigSubstitutions substitutions;
};
// Read and flatten one preset file. It reads only, and resolves against the presets
// loaded before this pass, never another file of the same pass, so the files of a
// pass are independent of each other.
UserPresetLoad resolve_user_preset(const boost::filesystem::path &file, const std::string &canonical_name,
const PresetOrigin &load_origin, ForwardCompatibilitySubstitutionRule substitution_rule,
const std::string &extruder_id_name, const std::string &extruder_variant_name,
std::set<std::string> *key_set1, std::set<std::string> *key_set2) const;
// Install one resolved preset. The collection, its alias maps, the error count
// and the preset files on disk are touched here and only here.
void commit_user_preset(UserPresetLoad &&loaded, std::deque<Preset> &presets_loaded,
PresetsConfigSubstitutions &substitutions,
const std::function<void(Preset&)> &preset_loaded_fn,
bool read_only);
std::string canonical_preset_name(const std::string &name, const PresetOrigin &load_origin = PresetOrigin()) const;
// Comparator that sorts "Generic " prefixed presets before others, then alphabetically within each group.
@@ -901,6 +966,10 @@ private:
return a.name < b.name;
}
// Append a preset without keeping the collection sorted, for a caller installing
// many at once; find_preset() is unusable until sort_presets() runs.
Preset& append_preset(std::string &&path, const std::string &name, DynamicPrintConfig &&config);
// Sort presets: filament presets use generic-first ordering, others sort alphabetically.
void sort_presets() {
if (m_type == Preset::TYPE_FILAMENT)
@@ -1054,7 +1123,7 @@ public:
//BBS: change to json format
//void save() { this->config.save(this->file); }
void save(DynamicPrintConfig* parent_config) { this->config.save_to_json(this->file, std::string("Physical_Printer"), std::string("User"), std::string(SLIC3R_VERSION)); }
void save(DynamicPrintConfig* parent_config);
void save(const std::string& file_name_from, const std::string& file_name_to);
void update_from_preset(const Preset& preset);
File diff suppressed because it is too large Load Diff
+172 -27
View File
@@ -15,6 +15,7 @@
#include <unordered_map>
#include <optional>
#include <array>
#include <atomic>
#include <boost/filesystem/path.hpp>
#include <unordered_set>
@@ -622,36 +623,180 @@ public:
// default_filament_profile must resolve to a system filament.
bool check_printer_default_materials() const;
// Merge one vendor's presets with the other vendor's presets, report duplicates.
// Public so per-vendor-cache consumers (e.g. the setup wizard) can assemble a
// bundle out of several per-vendor caches loaded into separate PresetBundle instances.
std::vector<std::string> merge_presets(PresetBundle &&other);
// One vendor to load, and the directory it is installed in.
struct VendorSource
{
std::string name;
boost::filesystem::path dir;
};
// Load `vendors` into this bundle, the Orca filament library directly and every
// other vendor in parallel into a bundle of its own that inherits from it, merged
// in the order given. A vendor that cannot be loaded has its error added to the
// returned text, or thrown in validation mode, and its name to `failed`; it is
// left out, except for the library, which keeps what it installed before the
// failure. Once `cancel` is set, no further vendor starts loading.
std::pair<PresetsConfigSubstitutions, std::string> load_vendors(const std::vector<VendorSource>& vendors,
ForwardCompatibilitySubstitutionRule compatibility_rule, bool allow_cache,
const std::atomic<bool>* cancel = nullptr, std::vector<std::string>* failed = nullptr);
private:
// Load one vendor from the preset cache installed in `dir`, judged against
// the vendor profile there. False, with this bundle left clean, when there
// is no usable cache and the vendor has to be parsed. This is how
// load_vendor_configs_from_json reads a cache.
bool load_vendor_cache(const boost::filesystem::path& dir, const std::string& vendor_name, const PresetBundle* base_bundle);
// Move the presets and vendor profiles of `others` into this bundle, in one pass
// over each collection. A preset whose name this bundle or an earlier one of
// `others` already has is left out and listed under the bundle that repeats it.
std::vector<std::vector<std::string>> merge_presets(const std::vector<PresetBundle*> &others);
// Load one source-form preset entry into this bundle: resolve `inherits`
// and `include`, flatten, validate and register the preset. Returns the
// reason loading failed, empty on success. See the definition for the
// sharing contract between the JSON parse and the cache load.
// retain_configs / retain_includes, when non-null, name the only presets
// registered into config_maps / include_maps (a config copy each). The
// cache load passes the names its entries inherit / include — the only
// ones ever looked up again; the JSON parse retains all, not knowing what
// later subfiles name.
std::string load_vendor_preset(const CachedPreset& entry,
const std::string& path, const std::string& vendor_name,
const PresetBundle* base_bundle,
LoadConfigBundleAttributes flags,
ConfigSubstitutionContext& substitution_context, PresetsConfigSubstitutions& substitutions,
std::map<std::string, DynamicPrintConfig>& config_maps, std::map<std::string, DynamicPrintConfig>& include_maps,
std::map<std::string, std::string>& filament_id_maps,
PresetCollection* presets_collection, size_t& count, bool is_from_lib,
const std::set<std::string>* retain_configs = nullptr, const std::set<std::string>* retain_includes = nullptr);
// What parsing one entry's JSON sub-file reported. Its errors and warnings are
// logged when the entry installs, so they come out in listing order with the
// entry's install errors, as parsing and installing one entry at a time leaves them.
struct EntryParse
{
ConfigSubstitutions substitutions;
// Counted in the bundle's error count.
std::vector<std::string> errors;
std::vector<std::string> warnings;
};
// An EntryParse for each entry of the VendorCacheData list of the same name.
struct VendorParse
{
std::vector<EntryParse> process_entries;
std::vector<EntryParse> filament_entries;
std::vector<EntryParse> machine_entries;
};
// One vendor read from its cache or its JSONs by read_vendor, for
// install_vendor_read to install.
struct VendorRead
{
std::string dir;
std::string vendor_name;
LoadConfigBundleAttributes flags;
ForwardCompatibilitySubstitutionRule compatibility_rule;
// The errors this bundle had counted before the read, which the cache stamp leaves out.
int errors_at_entry { 0 };
// A whole-vendor load, which can read a cache and write one.
bool cacheable { false };
// Read from the cache at cache_path, which install can still reject.
bool from_cache { false };
std::string cache_path;
// Only the vendor profile was asked for.
bool vendor_only { false };
VendorCacheData data;
// What each entry's JSON parse reported, and whether and with which version
// the cache is written once the entries install.
VendorParse parsed;
bool will_cache { false };
std::string version;
// The sub-file the parse stopped at, its kind, why, and the errors it reported.
std::string reason;
std::string failed_subfile;
const char* failed_kind { nullptr };
std::vector<std::string> failed_errors;
};
// Read a vendor into this bundle's vendor profiles and `data`, from its cache
// when one covers it, else from its JSONs; nothing is installed. Throws
// ConfigurationError when the vendor's own JSON cannot be parsed.
VendorRead read_vendor(const std::string& dir, const std::string& vendor_name, LoadConfigBundleAttributes flags,
ForwardCompatibilitySubstitutionRule compatibility_rule, bool allow_cache);
// Parse the vendor's JSONs into `read`, up to the first sub-file that fails.
void parse_vendor_json(VendorRead& read);
// Install what read_vendor read, against base_bundle's filament library. A cache
// that cannot be installed is replaced by a parse of the JSONs. Throws
// ConfigurationError at the first entry that cannot be installed, or after
// installing the entries before a sub-file that could not be parsed.
std::pair<PresetsConfigSubstitutions, size_t> install_vendor_read(VendorRead&& read, const PresetBundle* base_bundle);
// Install a cache's entries. False, with this bundle left clean, when one of them
// cannot be installed.
bool install_vendor_cache(const std::string& cache_path, const std::string& vendor_name, VendorCacheData&& data,
const PresetBundle* base_bundle);
// Log and count errors reported by a resolve or a parse.
void log_errors(const std::vector<std::string>& errors);
// The state of installing one collection of one vendor, which
// resolve_vendor_preset reads through a const reference and only
// commit_vendor_preset writes.
struct VendorInstall
{
// The directory holding <vendor_name>/, whose sub-paths the entries name.
std::string path;
std::string vendor_name;
const VendorProfile* vendor_profile;
const PresetBundle* base_bundle;
LoadConfigBundleAttributes flags;
PresetCollection* presets;
// The Orca filament library, which keeps every config for other vendors to
// resolve against.
bool is_from_lib;
PresetsConfigSubstitutions* substitutions;
// The names some entry inherits / includes, the only ones whose configs /
// include diffs are looked up again.
std::set<std::string> inherited;
std::set<std::string> included;
std::map<std::string, DynamicPrintConfig> config_maps;
std::map<std::string, DynamicPrintConfig> include_maps;
std::map<std::string, std::string> filament_id_maps;
std::unordered_set<std::string> installed_names;
size_t count { 0 };
};
// Install a vendor's source-form entries, parsed from its JSON or read from its
// cache: processes, then filaments, then printers. Both loads go through here,
// so a cache-loaded bundle cannot come out different from a JSON-loaded one.
// `parsed` is given for entries parsed just now. `complete` says the entries
// are the vendor's whole lists; only then are the filament library's configs
// and filament ids left in m_config_maps and m_filament_id_maps. Returns the
// number of presets installed, and throws ConfigurationError at the first
// entry that cannot be installed.
size_t install_vendor(const std::string& path, const std::string& vendor_name, const PresetBundle* base_bundle,
LoadConfigBundleAttributes flags, const VendorCacheData& entries,
VendorParse* parsed, bool complete, PresetsConfigSubstitutions& substitutions);
// Install one collection's entries in the order they are listed.
void install_vendor_entries(VendorInstall& install, const std::vector<CachedPreset>& entries,
std::vector<EntryParse>* parsed);
// One entry flattened against the preset it inherits, before anything this
// bundle shares has been touched.
struct PresetInstall
{
DynamicPrintConfig config;
std::string file_path;
// Empty when the preset is its own alias.
std::string alias;
std::string filament_id;
std::vector<std::string> renamed_from;
// Reported by commit, so resolving entries together leaves the log and
// the error count as one entry at a time produces them.
std::vector<std::string> errors;
// What a base states for the presets that include it, when it is retained.
std::optional<DynamicPrintConfig> included;
// The config kept for the entries that inherit this one, or for other
// vendors when this is the filament library.
std::optional<DynamicPrintConfig> retained;
// Not instantiated, so it contributes a config and no preset.
bool config_only { false };
// Non-empty when the entry is rejected, and says why.
std::string reason;
};
// Flatten one entry against the config it inherits, from this collection's
// config_maps or base_bundle's filament library, with the include diffs it
// names layered in. It looks up nothing but the names the entry inherits and
// includes, and writes nothing.
PresetInstall resolve_vendor_preset(const CachedPreset& entry, const VendorInstall& install) const;
// Install a resolved entry. The collections, the maps in `install` and the
// error count are touched here and only here, one entry at a time.
// Presets are appended, so a repeated name is caught with `installed_names`,
// and the collection is sorted once every entry is in.
std::string commit_vendor_preset(const CachedPreset& entry, PresetInstall&& resolved,
ConfigSubstitutions&& substitutions, VendorInstall& install);
// Clear every collection's m_printer_hold_alias, which reset() leaves alone.
void clear_printer_hold_aliases();
+11 -33
View File
@@ -400,46 +400,24 @@ bool write_cache_blob(const std::string& path, const std::string& blob)
{
boost::crc_32_type crc;
crc.process_bytes(blob.data(), blob.size());
// Written beside the target and moved into place, as AppConfig::save does:
// a cache is truncated and rewritten in full, so a write that dies partway
// would otherwise leave a header claiming more body than the file holds.
// The PID suffix also keeps two instances writing the same vendor from
// interleaving.
const std::string tmp_path = path + "." + std::to_string(get_current_pid()) + ".tmp";
// Written beside the target and moved into place: a cache is truncated and
// rewritten in full, so a write that dies partway would otherwise leave a
// header claiming more body than the file holds.
try {
boost::filesystem::create_directories(boost::filesystem::path(path).parent_path());
{
boost::nowide::ofstream ofs(tmp_path, std::ios::binary | std::ios::trunc);
if (!ofs.is_open()) {
BOOST_LOG_TRIVIAL(warning) << "VendorCacheFile: cannot open for writing: " << tmp_path;
return false;
}
CacheFileHeader fhdr;
fhdr.magic = CACHE_MAGIC;
fhdr.version = CACHE_VERSION;
fhdr.data_size = static_cast<uint64_t>(blob.size());
fhdr.crc32 = crc.checksum();
ofs.write(reinterpret_cast<const char*>(&fhdr), sizeof(fhdr));
ofs.write(blob.data(), static_cast<std::streamsize>(blob.size()));
ofs.close(); // flush; close() raises failbit on error
if (! ofs.good()) {
BOOST_LOG_TRIVIAL(warning) << "VendorCacheFile: write failed (" << tmp_path << ")";
boost::system::error_code ec;
boost::filesystem::remove(tmp_path, ec);
return false;
}
}
if (const std::error_code ec = rename_file(tmp_path, path)) {
BOOST_LOG_TRIVIAL(warning) << "VendorCacheFile: could not move " << tmp_path << " into place: " << ec.message();
boost::system::error_code rm;
boost::filesystem::remove(tmp_path, rm);
CacheFileHeader fhdr;
fhdr.magic = CACHE_MAGIC;
fhdr.version = CACHE_VERSION;
fhdr.data_size = static_cast<uint64_t>(blob.size());
fhdr.crc32 = crc.checksum();
const std::string_view header(reinterpret_cast<const char*>(&fhdr), sizeof(fhdr));
if (const std::error_code ec = write_file_atomically(path, { header, std::string_view(blob) }, /*binary=*/true)) {
BOOST_LOG_TRIVIAL(warning) << "VendorCacheFile: write failed (" << path << "): " << ec.message();
return false;
}
return true;
} catch (const std::exception& e) {
BOOST_LOG_TRIVIAL(warning) << "VendorCacheFile: write failed (" << path << "): " << e.what();
boost::system::error_code ec;
boost::filesystem::remove(tmp_path, ec);
return false;
}
}
+1 -1
View File
@@ -109,7 +109,7 @@ void skip_config(cereal::BinaryInputArchive& ar, const CacheDictionary& dict);
// One preset as its JSON subfile states it: the config diff, the names of the
// preset it inherits and the presets it includes, and the parse metadata —
// everything the parse phase of load_vendor_configs_from_json extracts and
// everything PresetBundle::parse_vendor_json extracts and
// nothing it derives. Inheritance and includes are resolved when the entry is
// installed, against whatever filament library is loaded then, so a cache
// carries no other vendor's values and no other vendor's update can make it
+17
View File
@@ -8,6 +8,8 @@
#include <functional>
#include <type_traits>
#include <system_error>
#include <initializer_list>
#include <string_view>
#include <regex>
#include <boost/system/error_code.hpp>
@@ -224,6 +226,21 @@ extern std::vector<std::string> split_string(const std::string &str, char delimi
// On Windows, the file explorer (or anti-virus or whatever else) often locks the file
// for a short while, so the file may not be movable. Retry while we see recoverable errors.
extern std::error_code rename_file(const std::string &from, const std::string &to);
// Write `chunks`, in order, to `path` through a temporary file beside it that is
// then renamed over the target, so a concurrent reader sees the old or the new
// file, never a partial one. The temporary is removed on failure and an existing
// target keeps its permissions. Text mode unless `binary`, so Windows writes CRLF
// as the streams this replaces did. A target that is not a regular file (a
// device or pipe) is written in place, since replacing it would change what it
// is, and so is an existing target beside which no temporary can be created or
// whose replace the filesystem refuses; a symlink is followed and the file it
// names is replaced. On Windows a reader holding the
// target open without sharing its deletion, which the C runtime does not, makes
// the replace fall back to the in-place write too, so an unlocked reader there
// can still see a partial file.
extern std::error_code write_file_atomically(const std::string &path, std::initializer_list<std::string_view> chunks, bool binary = false);
inline std::error_code write_file_atomically(const std::string &path, const std::string &content, bool binary = false)
{ return write_file_atomically(path, { std::string_view(content) }, binary); }
enum CopyFileResult {
SUCCESS = 0,
+90 -2
View File
@@ -9,6 +9,8 @@
#include <stdio.h>
#include <filesystem>
#include <sstream>
#include <cerrno>
#include <mutex>
#include <iomanip>
#include <algorithm>
#include <cmath>
@@ -702,13 +704,99 @@ namespace WindowsSupport
std::error_code rename_file(const std::string &from, const std::string &to)
{
#ifdef _WIN32
// Retries and moves an open destination aside itself.
return WindowsSupport::rename(from, to);
#else
boost::nowide::remove(to.c_str());
return std::make_error_code(static_cast<std::errc>(boost::nowide::rename(from.c_str(), to.c_str())));
// rename(2) replaces an existing target atomically; removing it first would
// leave a window in which the file does not exist at all.
if (boost::nowide::rename(from.c_str(), to.c_str()) == 0)
return {};
const int err = errno;
// Some mounts (sshfs, gvfs, MTP and a few SMB setups) refuse to replace an
// existing target in one step, each with the error it sees fit; every error
// is worth the remove-then-rename this always did, except the ones no retry
// can help: nothing at the source, a different device, or a directory where
// a file was expected and the reverse.
const bool worth_retrying = err != ENOENT && err != EXDEV && err != ENOTDIR && err != EISDIR;
if (worth_retrying && boost::nowide::remove(to.c_str()) == 0 && boost::nowide::rename(from.c_str(), to.c_str()) == 0)
return {};
return std::make_error_code(static_cast<std::errc>(err));
#endif
}
static std::error_code write_whole_file(const std::string &path, std::initializer_list<std::string_view> chunks, bool binary)
{
errno = 0;
FILE *file = boost::nowide::fopen(path.c_str(), binary ? "wb" : "w");
if (file == nullptr)
return std::make_error_code(errno != 0 ? static_cast<std::errc>(errno) : std::errc::io_error);
bool ok = true;
for (const std::string_view chunk : chunks)
ok = ok && std::fwrite(chunk.data(), 1, chunk.size(), file) == chunk.size();
ok = ok && std::fflush(file) == 0;
const int err = ok ? 0 : errno;
ok = std::fclose(file) == 0 && ok;
if (ok)
return {};
return std::make_error_code(err != 0 ? static_cast<std::errc>(err) : std::errc::io_error);
}
// The in-place fallback truncates the target, so two threads of this process
// on the same file must not both be in it. One mutex for all such writes: they
// are the rare case. Never freed, like the InstanceLock registry, so a save
// during static destruction still finds it.
static std::error_code write_in_place(const std::string &path, std::initializer_list<std::string_view> chunks, bool binary)
{
static auto *mutex = new std::mutex();
std::lock_guard<std::mutex> guard(*mutex);
return write_whole_file(path, chunks, binary);
}
std::error_code write_file_atomically(const std::string &path, std::initializer_list<std::string_view> chunks, bool binary)
{
boost::system::error_code bec;
const boost::filesystem::file_status target = boost::filesystem::symlink_status(path, bec);
const bool target_exists = ! bec && boost::filesystem::exists(target);
if (target_exists && boost::filesystem::is_symlink(target)) {
// A config or preset kept in a dotfiles repository: the link stays,
// the file it points to is replaced like any other.
const boost::filesystem::path resolved = boost::filesystem::canonical(path, bec);
if (! bec && boost::filesystem::is_regular_file(resolved, bec))
return write_file_atomically(resolved.string(), chunks, binary);
}
if (target_exists && ! boost::filesystem::is_regular_file(target))
return write_in_place(path, chunks, binary);
// Unique per process and per call, so two threads writing one target
// without a lock never share a temporary.
static std::atomic<unsigned> counter{0};
const std::string tmp_path = path + "." + std::to_string(get_current_pid()) + "." + std::to_string(counter++) + ".tmp";
if (const std::error_code ec = write_whole_file(tmp_path, chunks, binary)) {
boost::nowide::remove(tmp_path.c_str());
if (! target_exists)
return ec;
// A directory that lets this process write its files but not create
// one: losing the save is worse than a reader seeing a partial file.
BOOST_LOG_TRIVIAL(warning) << "Cannot create a temporary beside " << path << " (" << ec.message() << "); writing in place";
return write_in_place(path, chunks, binary);
}
#ifndef _WIN32
// Not on Windows, where a read-only bit on the temporary would stop the rename itself.
if (target_exists)
boost::filesystem::permissions(tmp_path, target.permissions(), bec);
#endif
if (const std::error_code ec = rename_file(tmp_path, path)) {
boost::nowide::remove(tmp_path.c_str());
// A reader on Windows holding the target open without FILE_SHARE_DELETE,
// or a mount that cannot replace a file at all. Losing the save is worse
// than a reader seeing a partial file, so write in place the way this
// used to work before the atomic path existed.
BOOST_LOG_TRIVIAL(warning) << "Cannot replace " << path << " (" << ec.message() << "); writing in place";
return write_in_place(path, chunks, binary);
}
return {};
}
#ifdef __linux__
// Copied from boost::filesystem.
// Called by copy_file_linux() in case linux sendfile() API is not supported.
+21 -12
View File
@@ -85,6 +85,7 @@
#include "libslic3r/Model.hpp"
#include "libslic3r/I18N.hpp"
#include "libslic3r/PresetBundle.hpp"
#include "libslic3r/InstanceLock.hpp"
#include "libslic3r/Thread.hpp"
#include "libslic3r/miniz_extension.hpp"
#include "libslic3r/Utils.hpp"
@@ -3549,7 +3550,7 @@ bool GUI_App::on_init_inner()
update_publish_status();
}
if (m_post_initialized && app_config->dirty())
if (m_post_initialized && app_config->dirty() && app_config->save_due())
app_config->save();
});
@@ -7643,8 +7644,11 @@ void GUI_App::start_sync_user_preset(bool with_progress_dlg)
// Delete the bundle folder and bundle
fs::path bundle_folder = fs::path(bundle.path.c_str()).parent_path();
boost::system::error_code ec;
boost::filesystem::remove_all(bundle_folder, ec);
{
boost::system::error_code ec;
InstanceLock instance_lock(user_presets_lock_path());
boost::filesystem::remove_all(bundle_folder, ec);
}
preset_bundle->bundles.WriteLock();
preset_bundle->bundles.m_bundles.erase(bundle.id);
@@ -8931,6 +8935,7 @@ void GUI_App::preset_deleted_from_cloud(std::string setting_id)
// Delete the .info file after cloud deletion is confirmed
if (!preset_file_path.empty() && fs::exists(fs::path(preset_file_path))) {
InstanceLock instance_lock(user_presets_lock_path());
boost::nowide::remove(preset_file_path.c_str());
BOOST_LOG_TRIVIAL(info) << "Deleted .info file after cloud confirmation: " << preset_file_path;
}
@@ -8993,15 +8998,19 @@ void GUI_App::scan_orphaned_info_files()
fs::path preset_file = info_file;
preset_file.replace_extension(".json");
// If .json doesn't exist, .info is orphaned
if (!fs::exists(preset_file)) {
// Extract setting_id from .info file
std::string setting_id = extract_setting_id_from_info(info_file.string());
if (!setting_id.empty()) {
// Add to need_delete_presets
delete_preset_from_cloud(setting_id, info_file.string());
BOOST_LOG_TRIVIAL(info) << "Found orphaned .info file on startup: " << info_file.string();
}
// If .json doesn't exist, .info is orphaned. Read under the lock, so a
// remove_files() in another instance is seen whole or not at all; the
// delete queue's own mutex is taken after the lock is released.
std::string setting_id;
{
InstanceLock instance_lock(user_presets_lock_path());
if (!fs::exists(preset_file))
setting_id = extract_setting_id_from_info(info_file.string());
}
if (!setting_id.empty()) {
// Add to need_delete_presets
delete_preset_from_cloud(setting_id, info_file.string());
BOOST_LOG_TRIVIAL(info) << "Found orphaned .info file on startup: " << info_file.string();
}
}
if (ec)
+24 -43
View File
@@ -1405,23 +1405,26 @@ bool GuideFrame::BuildProfileDataFromVendors()
// is served from the shipped profiles. Each is stamped by name and
// version alone: a profile change requires a version bump, so those two
// determine content wherever the vendor's copy sits.
struct VendorSource { std::string name; boost::filesystem::path dir; std::string version; };
std::vector<VendorSource> ordered;
auto add_vendor = [&ordered](const std::string& name, const boost::filesystem::path& dir) {
std::vector<PresetBundle::VendorSource> ordered;
json stamps = json::array();
auto add_vendor = [&ordered, &stamps](const std::string& name, const boost::filesystem::path& dir) {
// The version a load from `dir` would serve: the profile's where one
// exists (a cache is only served while it covers the profile beside
// it), the cache's own stamp where the cache is the whole vendor.
// A profile without a version (blacklist.json) carries no presets
// and is passed over.
const boost::filesystem::path profile = dir / (name + ".json");
std::string version;
if (boost::filesystem::exists(profile)) {
const Semver v = get_version_from_json(profile.string());
if (v.valid())
ordered.push_back({name, dir, v.to_string()});
if (! v.valid())
return;
version = v.to_string();
} else {
ordered.push_back({name, dir,
VendorCacheFile::peek_version((dir / (name + ".opc")).string(), name)});
version = VendorCacheFile::peek_version((dir / (name + ".opc")).string(), name);
}
ordered.push_back({name, dir});
stamps.push_back({name, version});
};
const std::string filament_library(PresetBundle::ORCA_FILAMENT_LIBRARY);
if (auto it = vendor_sources.find(filament_library); it != vendor_sources.end())
@@ -1431,9 +1434,6 @@ bool GuideFrame::BuildProfileDataFromVendors()
add_vendor(name, dir);
if (ordered.empty())
return false;
json stamps = json::array();
for (const VendorSource& v : ordered)
stamps.push_back({v.name, v.version});
// What this function derives is a pure function of that stamped set, so
// the derived JSON is cached whole: a fresh cache makes an open one
@@ -1461,34 +1461,24 @@ bool GuideFrame::BuildProfileDataFromVendors()
}
// Each vendor comes from its preset cache where one covers it, which is
// what makes this worth doing instead of the scan below; loading into a
// bundle per vendor keeps the install order the startup path has.
PresetBundle bundle;
auto load_vendor = [](PresetBundle& into, const std::string& vendor,
const boost::filesystem::path& dir, const PresetBundle* base) {
into.load_vendor_configs_from_json(dir.string(), vendor, PresetBundle::LoadSystem,
ForwardCompatibilitySubstitutionRule::EnableSilent, base);
};
for (const VendorSource& v : ordered) {
if (*m_cancel_token)
return false; // as in the scan below: a vendor without a cache is parsed, and that takes time
if (v.name == filament_library) {
load_vendor(bundle, v.name, v.dir, nullptr);
} else {
PresetBundle tmp;
load_vendor(tmp, v.name, v.dir, &bundle);
bundle.merge_presets(std::move(tmp));
}
}
if (bundle.vendors.empty())
// what makes this worth doing instead of the scan below.
PresetBundle bundle;
std::vector<std::string> failed;
const std::string errors = bundle.load_vendors(ordered, ForwardCompatibilitySubstitutionRule::EnableSilent,
/*allow_cache=*/true, m_cancel_token.get(), &failed).second;
if (*m_cancel_token || bundle.vendors.empty())
return false;
if (! errors.empty())
BOOST_LOG_TRIVIAL(warning) << "GuideFrame: loading the vendors reported: " << errors;
// A vendor that failed to load sends this open to the scan below, which lists
// what it can read of every vendor.
if (! failed.empty())
return false;
if (! BuildProfileJson(bundle, /*require_all_resource_vendors=*/false))
return false;
// Written through a temp file and moved into place, as the preset caches
// are: half a cache must never be readable, and the PID suffix keeps two
// instances from interleaving on one temp file.
const std::string tmp_path = cache_file.string() + "." + std::to_string(get_current_pid()) + ".tmp";
// are: half a cache must never be readable.
try {
json out;
out["format"] = 1;
@@ -1497,18 +1487,9 @@ bool GuideFrame::BuildProfileDataFromVendors()
for (const char* key : { "model", "machine", "filament", "process" })
profile[key] = m_ProfileJson[key];
boost::filesystem::create_directories(cache_file.parent_path());
{
boost::nowide::ofstream ofs(tmp_path, std::ios::binary | std::ios::trunc);
ofs << out.dump(-1, ' ', false, json::error_handler_t::ignore);
ofs.close();
if (! ofs.good())
throw std::runtime_error("write failed");
}
if (const std::error_code ec = rename_file(tmp_path, cache_file.string()))
if (const std::error_code ec = write_file_atomically(cache_file.string(), out.dump(-1, ' ', false, json::error_handler_t::ignore), /*binary=*/true))
throw std::runtime_error(ec.message());
} catch (const std::exception& e) {
boost::system::error_code rm;
boost::filesystem::remove(tmp_path, rm);
BOOST_LOG_TRIVIAL(warning) << "GuideFrame: could not write the profile data cache: " << e.what();
}
return true;
+5 -3
View File
@@ -2,6 +2,7 @@
#include <algorithm>
#include <sstream>
#include <system_error>
#include <exception>
#include <boost/format.hpp>
#include <boost/log/trivial.hpp>
@@ -580,9 +581,10 @@ bool C3DPrinterOS::save_api_session(const std::string &session, const std::strin
j.put("session", session);
j.put("email", email);
try {
auto temp_path = m_api_session_file_path + ".tmp";
pt::write_json(temp_path, j);
boost::filesystem::rename(temp_path, m_api_session_file_path);
std::ostringstream json;
pt::write_json(json, j);
if (const std::error_code ec = write_file_atomically(m_api_session_file_path, json.str()))
throw std::system_error(ec);
} catch (const std::exception &err) {
BOOST_LOG_TRIVIAL(error) << __FUNCTION__ << ": failed to write json to file. Path = "
<< m_api_session_file_path
+6 -25
View File
@@ -1475,15 +1475,8 @@ void OrcaCloudServiceAgent::save_sync_state()
if (sync_state_path.empty())
return;
try {
std::string tmp_path = sync_state_path + ".tmp";
std::ofstream ofs(tmp_path, std::ios::out | std::ios::trunc);
if (ofs.good()) {
ofs << std::to_string(sync_state.last_sync_timestamp);
ofs.close();
boost::filesystem::rename(tmp_path, sync_state_path);
}
} catch (...) {}
if (const std::error_code ec = write_file_atomically(sync_state_path, std::to_string(sync_state.last_sync_timestamp)))
BOOST_LOG_TRIVIAL(warning) << "OrcaCloudServiceAgent: failed to save the sync state: " << ec.message();
}
void OrcaCloudServiceAgent::clear_sync_state()
@@ -1572,22 +1565,10 @@ void OrcaCloudServiceAgent::persist_user_secret(const std::string& secret)
wxFileName::Mkdir(path.GetPath(), wxS_DIR_DEFAULT, wxPATH_MKDIR_FULL);
}
const std::string tmp_path = secret_fallback_path + ".tmp";
std::ofstream ofs(tmp_path, std::ios::out | std::ios::trunc | std::ios::binary);
if (ofs.good()) {
ofs << signed_payload;
ofs.flush();
ofs.close();
if (wxRenameFile(wxString::FromUTF8(tmp_path.c_str()), wxString::FromUTF8(secret_fallback_path.c_str()), true)) {
stored = true;
} else {
wxRemoveFile(wxString::FromUTF8(tmp_path.c_str()));
BOOST_LOG_TRIVIAL(warning) << "OrcaCloudServiceAgent: failed to atomically replace user secret file";
}
} else {
BOOST_LOG_TRIVIAL(warning) << "OrcaCloudServiceAgent: cannot open user secret file for write - " << secret_fallback_path;
}
if (const std::error_code ec = write_file_atomically(secret_fallback_path, signed_payload, /*binary=*/true))
BOOST_LOG_TRIVIAL(warning) << "OrcaCloudServiceAgent: cannot write user secret file " << secret_fallback_path << ": " << ec.message();
else
stored = true;
} else {
// Use wxSecretStore only
wxSecretStore store = wxSecretStore::GetDefault();
+4 -15
View File
@@ -249,21 +249,10 @@ bool PluginConfig::save()
return false;
}
// Write to a PID-suffixed file and rename it into place, so a crash mid-write cannot truncate an
// existing config. Same approach as AppConfig::save().
const std::string path_pid = (boost::format("%1%.%2%") % path % get_current_pid()).str();
boost::nowide::ofstream file;
file.open(path_pid, std::ios::out | std::ios::trunc);
file << root.dump(1, '\t') << std::endl;
file.close();
if (file.fail()) {
BOOST_LOG_TRIVIAL(error) << "PluginConfig: failed to write " << path_pid << "; keeping the existing config";
return false;
}
if (const std::error_code rename_ec = rename_file(path_pid, path)) {
BOOST_LOG_TRIVIAL(error) << "PluginConfig: failed to move " << path_pid << " onto " << path << ": " << rename_ec.message();
// Written beside the target and moved into place, so a crash mid-write cannot truncate an
// existing config.
if (const std::error_code ec = write_file_atomically(path, root.dump(1, '\t') + "\n")) {
BOOST_LOG_TRIVIAL(error) << "PluginConfig: failed to write " << path << ": " << ec.message() << "; keeping the existing config";
return false;
}
+3
View File
@@ -14,7 +14,9 @@ add_executable(${_TEST_NAME}_tests
test_clipper_utils.cpp
test_config.cpp
test_config_variant_expansion.cpp
test_locales_utils.cpp
test_toolordering_nozzle_group.cpp
test_parallel_resolve.cpp
test_preset_bundle_loading.cpp
test_preset_setting_id.cpp
test_preset_diff.cpp
@@ -49,6 +51,7 @@ add_executable(${_TEST_NAME}_tests
test_ordering_strategies.cpp
# test_png_io.cpp
test_indexed_triangle_set.cpp
test_instance_lock.cpp
../libnest2d/printer_parts.cpp
)
+204
View File
@@ -0,0 +1,204 @@
#include <catch2/catch_all.hpp>
#include <atomic>
#include <chrono>
#include <thread>
#include <boost/filesystem.hpp>
#include "libslic3r/InstanceLock.hpp"
#include "test_utils.hpp"
#ifndef _WIN32
#include <fcntl.h>
#include <sys/file.h>
#include <sys/wait.h>
#include <unistd.h>
#endif
using namespace Slic3r;
using namespace std::chrono_literals;
// Sets a process-wide knob for one test and restores it however the test ends.
template<typename T> struct ScopedStaticValue
{
T &ref;
T saved;
ScopedStaticValue(T &ref, T value) : ref(ref), saved(ref) { ref = value; }
~ScopedStaticValue() { ref = saved; }
};
TEST_CASE("InstanceLock creates its lock file and holds it for the guard's scope", "[InstanceLock]")
{
ScopedTemporaryFile lock_file(".lock");
const std::string path = lock_file.string();
{
InstanceLock lock(path);
REQUIRE(lock.locked());
REQUIRE(boost::filesystem::exists(path));
}
// Released: a fresh guard gets the lock at once instead of waiting out a timeout.
const auto started = std::chrono::steady_clock::now();
InstanceLock again(path, 5000ms);
REQUIRE(again.locked());
// Well inside the timeout it would otherwise have waited out; loose enough for a loaded runner.
REQUIRE(std::chrono::steady_clock::now() - started < 4000ms);
}
TEST_CASE("InstanceLock nests within one thread", "[InstanceLock]")
{
ScopedTemporaryFile lock_file(".lock");
const std::string path = lock_file.string();
InstanceLock outer(path);
{
InstanceLock inner(path, 100ms);
REQUIRE(inner.locked());
}
// The inner guard leaving does not release the outer one.
REQUIRE(outer.locked());
}
TEST_CASE("InstanceLock is a no-op for an empty path and survives an unwritable one", "[InstanceLock]")
{
ScopedTemporaryDir dir;
InstanceLock none("");
REQUIRE_FALSE(none.locked());
// The directory does not exist, so the lock file cannot be created; the
// guard still constructs and the write it guards can go ahead.
InstanceLock unwritable((dir.path() / "missing" / "shared.lock").string(), 100ms);
REQUIRE_FALSE(unwritable.locked());
}
TEST_CASE("InstanceLock retries a lock file it could not open once the cool-down passes", "[InstanceLock]")
{
ScopedTemporaryDir dir;
const std::string path = (dir.path() / "later" / "shared.lock").string();
ScopedStaticValue cooldown(InstanceLock::cooldown, 300ms);
bool before_dir, during_cooldown, after_cooldown;
const auto started = std::chrono::steady_clock::now();
{
InstanceLock lock(path, 100ms);
before_dir = lock.locked();
}
boost::filesystem::create_directories(dir.path() / "later");
{
InstanceLock lock(path, 100ms);
during_cooldown = lock.locked();
}
const bool second_guard_inside_cooldown = std::chrono::steady_clock::now() - started < InstanceLock::cooldown;
std::this_thread::sleep_for(400ms);
{
InstanceLock lock(path, 100ms);
after_cooldown = lock.locked();
}
REQUIRE_FALSE(before_dir);
// A loaded runner may take longer than the cool-down to get here; then the
// second guard legitimately retried, so only assert when the timing held.
if (second_guard_inside_cooldown)
REQUIRE_FALSE(during_cooldown);
REQUIRE(after_cooldown);
}
TEST_CASE("InstanceLock reopens a lock file that was replaced on disk", "[InstanceLock]")
{
ScopedTemporaryFile lock_file(".lock");
const std::string path = lock_file.string();
{
InstanceLock lock(path);
REQUIRE(lock.locked());
}
boost::filesystem::remove(path);
InstanceLock lock(path);
REQUIRE(lock.locked());
// Each outermost guard opens the file afresh, so the deleted path is back.
REQUIRE(boost::filesystem::exists(path));
}
TEST_CASE("InstanceLock serialises the threads of one process", "[InstanceLock]")
{
ScopedTemporaryFile lock_file(".lock");
const std::string path = lock_file.string();
std::atomic<bool> holder_ready{false};
std::atomic<bool> holder_released{false};
std::thread holder([&] {
InstanceLock lock(path);
holder_ready = true;
std::this_thread::sleep_for(150ms);
holder_released = true;
});
while (! holder_ready)
std::this_thread::yield();
bool released_before_acquire = false;
{
InstanceLock lock(path);
released_before_acquire = holder_released;
}
holder.join();
REQUIRE(released_before_acquire);
}
#ifndef _WIN32
// The cross-process side of the lock is a POSIX flock, which a child process
// takes here directly; LockFileEx backs the guard on Windows, but spawning a
// child there is not worth a test.
TEST_CASE("InstanceLock yields to another process and reports it", "[InstanceLock]")
{
ScopedTemporaryFile lock_file(".lock");
const std::string path = lock_file.string();
ScopedStaticValue cooldown(InstanceLock::cooldown, 300ms);
int child_holds[2], child_may_exit[2];
REQUIRE(::pipe(child_holds) == 0);
REQUIRE(::pipe(child_may_exit) == 0);
const pid_t child = ::fork();
REQUIRE(child >= 0);
if (child == 0) {
int fd = ::open(path.c_str(), O_RDWR | O_CREAT, 0644);
char byte = ::flock(fd, LOCK_EX | LOCK_NB) == 0 ? '1' : '0';
if (::write(child_holds[1], &byte, 1) != 1 || ::read(child_may_exit[0], &byte, 1) != 1)
::_exit(1);
::_exit(0);
}
char byte = '0';
REQUIRE(::read(child_holds[0], &byte, 1) == 1);
REQUIRE(byte == '1');
bool locked_while_child_holds;
{
InstanceLock lock(path, 100ms);
locked_while_child_holds = lock.locked();
}
// The timed-out wait starts a cool-down: the next guard does not touch the file.
const auto started = std::chrono::steady_clock::now();
bool locked_during_cooldown;
{
InstanceLock lock(path, 5000ms);
locked_during_cooldown = lock.locked();
}
const auto cooldown_wait = std::chrono::steady_clock::now() - started;
REQUIRE(::write(child_may_exit[1], "x", 1) == 1);
int status = 0;
REQUIRE(::waitpid(child, &status, 0) == child);
for (int fd : {child_holds[0], child_holds[1], child_may_exit[0], child_may_exit[1]})
::close(fd);
REQUIRE_FALSE(locked_while_child_holds);
REQUIRE_FALSE(locked_during_cooldown);
REQUIRE(cooldown_wait < 4000ms);
// Once the cool-down passes, the lock the child released is taken again.
std::this_thread::sleep_for(400ms);
InstanceLock lock(path);
REQUIRE(lock.locked());
}
#endif
+69
View File
@@ -0,0 +1,69 @@
#include <catch2/catch_all.hpp>
#include <clocale>
#include "libslic3r/LocalesUtils.hpp"
using namespace Slic3r;
namespace {
// Switches this thread's numeric locale to one whose decimal separator is a comma,
// when the system has one installed.
struct CommaNumericLocale
{
#ifdef _WIN32
bool apply()
{
for (const char* name : { "de-DE", "German_Germany.1252" })
if (std::setlocale(LC_NUMERIC, name) != nullptr)
return true;
return false;
}
#else
locale_t locale { (locale_t) 0 };
bool apply()
{
for (const char* name : { "de_DE.UTF-8", "de_DE.utf8", "de_DE" })
if ((locale = newlocale(LC_NUMERIC_MASK, name, (locale_t) 0)) != (locale_t) 0) {
uselocale(locale);
return true;
}
return false;
}
// Freed once the setters around apply() have put the thread's own locale back.
~CommaNumericLocale()
{
if (locale != (locale_t) 0)
freelocale(locale);
}
#endif
};
} // namespace
TEST_CASE("a setter nested in another leaves the C locale in place for the outer one", "[LocalesUtils]")
{
CNumericLocalesSetter outer;
{
CNumericLocalesSetter inner;
CHECK(is_decimal_separator_point());
}
CHECK(is_decimal_separator_point());
}
TEST_CASE("a setter nested in another sets C again when the locale changed between them", "[LocalesUtils]")
{
CommaNumericLocale comma;
{
CNumericLocalesSetter outer;
if (! comma.apply())
SKIP("no locale with a comma decimal separator is installed");
REQUIRE_FALSE(is_decimal_separator_point());
{
CNumericLocalesSetter inner;
CHECK(is_decimal_separator_point());
}
CHECK_FALSE(is_decimal_separator_point());
}
}
+122
View File
@@ -0,0 +1,122 @@
#include <catch2/catch_all.hpp>
#include <numeric>
#include <stdexcept>
#include <thread>
#include <vector>
#include <tbb/blocked_range.h>
#include <tbb/parallel_for.h>
#include <tbb/task_group.h>
#include "libslic3r/ParallelResolve.hpp"
using namespace Slic3r;
namespace {
thread_local int t_live_setups = 0;
// Counts how many are alive on the constructing thread.
struct CountingSetup
{
CountingSetup() { ++ t_live_setups; }
~CountingSetup() { -- t_live_setups; }
};
std::vector<size_t> first_indices(size_t count)
{
std::vector<size_t> indices(count);
std::iota(indices.begin(), indices.end(), size_t(0));
return indices;
}
} // namespace
TEST_CASE("every item resolves once and commits in index order on the calling thread", "[ParallelResolve]")
{
const size_t count = 200;
std::vector<int> resolves(count, 0);
std::vector<size_t> committed, values;
std::vector<bool> on_caller;
const std::thread::id caller = std::this_thread::get_id();
resolve_then_commit(count,
[&](size_t i) { ++ resolves[i]; return i * 3; },
[&](size_t i, size_t resolved) {
committed.push_back(i);
values.push_back(resolved);
on_caller.push_back(std::this_thread::get_id() == caller);
});
CHECK(committed == first_indices(count));
for (size_t i = 0; i < count; ++ i) {
CHECK(resolves[i] == 1);
CHECK(values[i] == i * 3);
CHECK(on_caller[i]);
}
}
TEST_CASE("every item resolves inside one chunk setup", "[ParallelResolve]")
{
const size_t count = 200;
std::vector<int> live(count, 0);
resolve_then_commit<CountingSetup>(count,
[&](size_t i) { live[i] = t_live_setups; return 0; },
[](size_t, int) {});
for (size_t i = 0; i < count; ++ i)
CHECK(live[i] == 1);
}
TEST_CASE("an exception from resolve leaves the batches before it committed", "[ParallelResolve]")
{
const size_t count = 200, fails_at = 150;
REQUIRE(fails_at >= resolve_batch_size);
std::vector<size_t> committed;
CHECK_THROWS_AS(resolve_then_commit(count,
[&](size_t i) {
if (i == fails_at)
throw std::runtime_error("resolve failed");
return i;
},
[&](size_t i, size_t) { committed.push_back(i); }),
std::runtime_error);
CHECK(committed == first_indices(fails_at / resolve_batch_size * resolve_batch_size));
}
TEST_CASE("an exception from commit stops at the item that threw", "[ParallelResolve]")
{
const size_t count = 200, fails_at = 90;
std::vector<size_t> committed;
CHECK_THROWS_AS(resolve_then_commit(count,
[](size_t i) { return i; },
[&](size_t i, size_t) {
if (i == fails_at)
throw std::runtime_error("commit failed");
committed.push_back(i);
}),
std::runtime_error);
CHECK(committed == first_indices(fails_at));
}
TEST_CASE("a canceled task group stops before committing an item it did not resolve", "[ParallelResolve]")
{
std::vector<size_t> committed;
bool threw = false;
tbb::task_group_context context;
tbb::parallel_for(tbb::blocked_range<size_t>(0, 1), [&](const tbb::blocked_range<size_t>&) {
context.cancel_group_execution();
try {
resolve_then_commit(200,
[](size_t i) { return i + 1; },
[&](size_t, size_t resolved) { committed.push_back(resolved); });
} catch (const std::runtime_error&) {
threw = true;
}
}, context);
CHECK(threw);
CHECK(committed.empty());
}
@@ -5,6 +5,7 @@
#include <fstream>
#include "libslic3r/PresetBundle.hpp"
#include "libslic3r/ParallelResolve.hpp"
#include "libslic3r/AppConfig.hpp"
#include "libslic3r/Model.hpp"
#include "libslic3r/TriangleMesh.hpp"
@@ -17,6 +18,10 @@
#include <iostream>
#include <initializer_list>
#ifndef _WIN32
#include <unistd.h> // geteuid
#endif
using namespace Slic3r;
namespace {
@@ -59,6 +64,16 @@ void write_preset_with_inherits(const DynamicPrintConfig &default_config, const
config.save_to_json(file.string(), name, "User", "1.0.0");
}
// A user preset file stating nothing but the preset it inherits, so every value it
// ends up with came from resolving that parent.
void write_minimal_child(const fs::path &file, const std::string &name, const std::string &inherits)
{
fs::create_directories(file.parent_path());
std::ofstream(file.string())
<< R"({"type":"process","name":")" << name << R"(","from":"User","version":"1.0.0","inherits":")"
<< inherits << R"("})";
}
// Add an in-memory preset (no file) with the given inherits value (empty => root preset).
Preset &add_inmemory_preset(PresetCollection &coll, const std::string &name, const std::string &inherits = {})
{
@@ -261,6 +276,162 @@ TEST_CASE("Selected printer uses its default or saved bed type", "[Preset][Bundl
CHECK(app_config.get_printer_setting("Test Printer", "curr_bed_type") == std::to_string(static_cast<int>(expected_bed_type)));
}
TEST_CASE("A directory of user presets loads with each one resolved against its parent", "[Preset][Bundle]")
{
ScopedTemporaryDir temp_dir;
RenameTestCollection coll;
Preset &parent = add_inmemory_preset(coll, "Parent Process");
parent.config.option<ConfigOptionFloat>("layer_height", true)->value = 0.24;
parent.is_system = true;
constexpr int children = 400;
for (int i = 0; i < children; ++ i)
write_minimal_child(temp_dir.path() / PRESET_PRINT_NAME / ("Child " + std::to_string(i) + ".json"),
"Child " + std::to_string(i), "Parent Process");
PresetsConfigSubstitutions substitutions;
coll.load_presets(temp_dir.path().string(), PRESET_PRINT_NAME, substitutions,
ForwardCompatibilitySubstitutionRule::Disable);
CHECK(coll.size() == size_t(children) + 2); // the children, the default preset and the parent
CHECK(coll.error_count() == 0);
for (int i = 0; i < children; ++ i) {
const Preset *child = coll.find_preset("Child " + std::to_string(i));
REQUIRE(child != nullptr);
CHECK(child->inherits() == "Parent Process");
CHECK(child->alias == "Child " + std::to_string(i));
CHECK(child->loaded);
REQUIRE(child->config.option<ConfigOptionFloat>("layer_height") != nullptr);
CHECK_THAT(child->config.opt_float("layer_height"), Catch::Matchers::WithinAbs(0.24, 1e-9));
}
}
TEST_CASE("Repeated loads of a user preset directory produce the same presets", "[Preset][Bundle]")
{
ScopedTemporaryDir temp_dir;
auto seed_directory = [&]() {
for (int i = 0; i < 200; ++ i)
write_minimal_child(temp_dir.path() / PRESET_PRINT_NAME / ("Child " + std::to_string(i) + ".json"),
"Child " + std::to_string(i), "Parent Process");
};
std::vector<std::vector<std::string>> names_per_run;
for (int run = 0; run < 3; ++ run) {
RenameTestCollection coll;
Preset &parent = add_inmemory_preset(coll, "Parent Process");
parent.is_system = true;
if (run == 0)
seed_directory();
PresetsConfigSubstitutions substitutions;
coll.load_presets(temp_dir.path().string(), PRESET_PRINT_NAME, substitutions,
ForwardCompatibilitySubstitutionRule::Disable);
std::vector<std::string> names;
for (auto it = coll.begin(); it != coll.end(); ++ it)
names.push_back(it->name + "|" + it->alias + "|" + it->inherits());
names_per_run.push_back(std::move(names));
}
REQUIRE(names_per_run[0].size() > 200);
CHECK(names_per_run[1] == names_per_run[0]);
CHECK(names_per_run[2] == names_per_run[0]);
}
TEST_CASE("An unreadable user preset is counted and removed while the rest still load", "[Preset][Bundle]")
{
ScopedTemporaryDir temp_dir;
RenameTestCollection coll;
const fs::path dir = temp_dir.path() / PRESET_PRINT_NAME;
for (int i = 0; i < 20; ++ i)
write_preset_with_inherits(coll.default_preset().config, dir / ("Good " + std::to_string(i) + ".json"),
"Good " + std::to_string(i), std::string());
fs::create_directories(dir);
std::ofstream((dir / "Broken.json").string()) << "{not-json";
PresetsConfigSubstitutions substitutions;
coll.load_presets(temp_dir.path().string(), PRESET_PRINT_NAME, substitutions,
ForwardCompatibilitySubstitutionRule::EnableSilent);
CHECK(coll.error_count() == 1);
CHECK(coll.find_preset("Broken") == nullptr);
CHECK_FALSE(fs::exists(dir / "Broken.json"));
for (int i = 0; i < 20; ++ i)
CHECK(coll.find_preset("Good " + std::to_string(i)) != nullptr);
}
TEST_CASE("A user filament naming no compatible printer gets the one after its @, in memory and on disk", "[Preset][Bundle]")
{
ScopedTemporaryDir temp_dir;
PresetBundle bundle;
const fs::path file = temp_dir.path() / PRESET_FILAMENT_NAME / "My PLA @Test Printer.json";
REQUIRE(bundle.filaments.default_preset().config.option<ConfigOptionStrings>("compatible_printers")->values.empty());
write_preset_with_inherits(bundle.filaments.default_preset().config, file, "My PLA @Test Printer", std::string());
PresetsConfigSubstitutions substitutions;
bundle.filaments.load_presets(temp_dir.path().string(), PRESET_FILAMENT_NAME, substitutions,
ForwardCompatibilitySubstitutionRule::EnableSilent);
const std::vector<std::string> expected { "Test Printer" };
const Preset *preset = bundle.filaments.find_preset("My PLA @Test Printer");
REQUIRE(preset != nullptr);
CHECK(preset->config.option<ConfigOptionStrings>("compatible_printers")->values == expected);
DynamicPrintConfig saved;
std::map<std::string, std::string> key_values;
std::string reason;
saved.load_from_json(file.string(), ForwardCompatibilitySubstitutionRule::EnableSilent, key_values, reason);
REQUIRE(reason.empty());
REQUIRE(saved.option<ConfigOptionStrings>("compatible_printers") != nullptr);
CHECK(saved.option<ConfigOptionStrings>("compatible_printers")->values == expected);
}
TEST_CASE("A user preset's setting id equal to its base id is dropped in memory, not in the .info written back", "[Preset][Bundle]")
{
ScopedTemporaryDir temp_dir;
PresetBundle bundle;
const fs::path file = temp_dir.path() / PRESET_FILAMENT_NAME / "My PLA @Test Printer.json";
write_preset_with_inherits(bundle.filaments.default_preset().config, file, "My PLA @Test Printer", std::string());
fs::path info = file;
info.replace_extension(".info");
std::ofstream(info.string()) << "sync_info = \nuser_id = \nsetting_id = PFUS1\nbase_id = PFUS1\nupdated_time = 0\n";
PresetsConfigSubstitutions substitutions;
bundle.filaments.load_presets(temp_dir.path().string(), PRESET_FILAMENT_NAME, substitutions,
ForwardCompatibilitySubstitutionRule::EnableSilent);
const Preset *preset = bundle.filaments.find_preset("My PLA @Test Printer");
REQUIRE(preset != nullptr);
CHECK(preset->setting_id.empty());
CHECK(preset->base_id == "PFUS1");
Preset reloaded(Preset::TYPE_FILAMENT, "My PLA @Test Printer");
reloaded.load_info(info.string());
CHECK(reloaded.setting_id == "PFUS1");
}
TEST_CASE("A user preset that is not loaded still reports its substituted values", "[Preset][Bundle]")
{
ScopedTemporaryDir temp_dir;
RenameTestCollection coll;
const fs::path dir = temp_dir.path() / PRESET_PRINT_NAME;
fs::create_directories(dir);
std::ofstream((dir / "Orphan.json").string())
<< R"({"type":"process","name":"Orphan","from":"User","version":"1.0.0","inherits":"No Such Parent",)"
<< R"("wall_generator":"no_such_generator"})";
PresetsConfigSubstitutions substitutions;
coll.load_presets(temp_dir.path().string(), PRESET_PRINT_NAME, substitutions,
ForwardCompatibilitySubstitutionRule::Enable);
CHECK(coll.find_preset("Orphan") == nullptr);
CHECK(coll.error_count() == 1);
REQUIRE(substitutions.size() == 1);
CHECK(substitutions.front().preset_name == "Orphan");
}
TEST_CASE("find_preset resolves a system preset's renamed_from", "[Preset][Rename]")
{
RenameTestCollection coll;
@@ -5482,3 +5653,214 @@ TEST_CASE("Config import confines zip entries, preset names and bundle ids to th
CHECK_FALSE(any_filename_contains(temp_dir.path(), "bundle-escape"));
}
}
TEST_CASE("A user preset saved over by another instance while its directory loads is read again under the lock", "[Preset][Bundle][InstanceLock]")
{
ScopedTemporaryDir temp_dir;
ScopedDataDir data_dir_scope(temp_dir.path());
RenameTestCollection coll;
Preset &parent = add_inmemory_preset(coll, "Parent Process");
parent.config.option<ConfigOptionFloat>("layer_height", true)->value = 0.24;
parent.is_system = true;
// Fewer than one batch, so every file is read before the first one is committed.
constexpr int children = 8;
static_assert(children <= int(resolve_batch_size));
const fs::path dir = temp_dir.path() / PRESET_PRINT_NAME;
for (int i = 0; i < children; ++ i)
write_minimal_child(dir / ("Child " + std::to_string(i) + ".json"), "Child " + std::to_string(i), "Parent Process");
// The first commit stands in for another instance saving every other preset after
// they were read and before they are committed.
std::string first;
auto save_the_others = [&](Preset &preset) {
if (! first.empty())
return;
first = preset.name;
for (int i = 0; i < children; ++ i) {
const std::string name = "Child " + std::to_string(i);
if (name != first)
std::ofstream((dir / (name + ".json")).string())
<< R"({"type":"process","name":")" << name
<< R"(","from":"User","version":"1.0.0","inherits":"Parent Process","layer_height":"0.3"})";
}
};
PresetsConfigSubstitutions substitutions;
coll.load_presets(temp_dir.path().string(), PRESET_PRINT_NAME, substitutions,
ForwardCompatibilitySubstitutionRule::Disable, save_the_others);
REQUIRE_FALSE(first.empty());
CHECK(coll.error_count() == 0);
for (int i = 0; i < children; ++ i) {
const std::string name = "Child " + std::to_string(i);
const Preset *child = coll.find_preset(name);
REQUIRE(child != nullptr);
CHECK_THAT(child->config.opt_float("layer_height"), Catch::Matchers::WithinAbs(name == first ? 0.24 : 0.3, 1e-9));
}
}
TEST_CASE("A user preset removed by another instance while its directory loads is not installed", "[Preset][Bundle][InstanceLock]")
{
ScopedTemporaryDir temp_dir;
ScopedDataDir data_dir_scope(temp_dir.path());
RenameTestCollection coll;
Preset &parent = add_inmemory_preset(coll, "Parent Process");
parent.is_system = true;
// Fewer than one batch, so every file is read before the first one is committed.
constexpr int children = 8;
static_assert(children <= int(resolve_batch_size));
const fs::path dir = temp_dir.path() / PRESET_PRINT_NAME;
for (int i = 0; i < children; ++ i)
write_minimal_child(dir / ("Child " + std::to_string(i) + ".json"), "Child " + std::to_string(i), "Parent Process");
std::string first;
auto remove_the_others = [&](Preset &preset) {
if (! first.empty())
return;
first = preset.name;
for (int i = 0; i < children; ++ i)
if (const std::string name = "Child " + std::to_string(i); name != first)
fs::remove(dir / (name + ".json"));
};
PresetsConfigSubstitutions substitutions;
coll.load_presets(temp_dir.path().string(), PRESET_PRINT_NAME, substitutions,
ForwardCompatibilitySubstitutionRule::Disable, remove_the_others);
REQUIRE_FALSE(first.empty());
CHECK(coll.error_count() == 0);
CHECK(coll.size() == 3); // the default preset, the parent and the first child
CHECK(coll.find_preset(first) != nullptr);
}
TEST_CASE("Without the instance lock a user preset loads but its file is not written back", "[Preset][Bundle][InstanceLock]")
{
ScopedTemporaryDir temp_dir;
ScopedDataDir data_dir_scope(temp_dir.path());
// A directory where the lock file belongs, so the lock cannot be taken.
fs::create_directories(fs::path(user_presets_lock_path()));
PresetBundle bundle;
const fs::path file = temp_dir.path() / PRESET_FILAMENT_NAME / "My PLA @Test Printer.json";
write_preset_with_inherits(bundle.filaments.default_preset().config, file, "My PLA @Test Printer", std::string());
const std::string before = read_file(file);
PresetsConfigSubstitutions substitutions;
bundle.filaments.load_presets(temp_dir.path().string(), PRESET_FILAMENT_NAME, substitutions,
ForwardCompatibilitySubstitutionRule::EnableSilent);
const Preset *preset = bundle.filaments.find_preset("My PLA @Test Printer");
REQUIRE(preset != nullptr);
CHECK(preset->config.option<ConfigOptionStrings>("compatible_printers")->values == std::vector<std::string>{ "Test Printer" });
CHECK(read_file(file) == before);
}
#ifndef _WIN32
// File permissions stop reading only on POSIX.
TEST_CASE("A user preset that cannot be read under the instance lock is counted and removed", "[Preset][Bundle][InstanceLock]")
{
if (::geteuid() == 0)
SKIP("file permissions do not stop root");
ScopedTemporaryDir temp_dir;
ScopedDataDir data_dir_scope(temp_dir.path());
RenameTestCollection coll;
Preset &parent = add_inmemory_preset(coll, "Parent Process");
parent.is_system = true;
const fs::path file = temp_dir.path() / PRESET_PRINT_NAME / "Unreadable.json";
write_minimal_child(file, "Unreadable", "Parent Process");
fs::permissions(file, fs::no_perms);
PresetsConfigSubstitutions substitutions;
coll.load_presets(temp_dir.path().string(), PRESET_PRINT_NAME, substitutions,
ForwardCompatibilitySubstitutionRule::Disable);
CHECK(coll.error_count() == 1);
CHECK(coll.find_preset("Unreadable") == nullptr);
CHECK_FALSE(fs::exists(file));
}
#endif
#ifndef _WIN32
// Creating a symlink needs no privilege only on POSIX.
TEST_CASE("A user preset that is a symlink to itself is counted and removed while the rest still load", "[Preset][Bundle][InstanceLock]")
{
ScopedTemporaryDir temp_dir;
ScopedDataDir data_dir_scope(temp_dir.path());
RenameTestCollection coll;
Preset &parent = add_inmemory_preset(coll, "Parent Process");
parent.is_system = true;
const fs::path dir = temp_dir.path() / PRESET_PRINT_NAME;
write_minimal_child(dir / "Good A.json", "Good A", "Parent Process");
write_minimal_child(dir / "Good B.json", "Good B", "Parent Process");
const fs::path loop = dir / "Loop.json";
fs::create_symlink(loop.filename(), loop);
PresetsConfigSubstitutions substitutions;
REQUIRE_NOTHROW(coll.load_presets(temp_dir.path().string(), PRESET_PRINT_NAME, substitutions,
ForwardCompatibilitySubstitutionRule::Disable));
CHECK(coll.error_count() == 1);
CHECK(coll.find_preset("Good A") != nullptr);
CHECK(coll.find_preset("Good B") != nullptr);
CHECK(coll.find_preset("Loop") == nullptr);
boost::system::error_code ec;
CHECK(fs::symlink_status(loop, ec).type() == fs::file_not_found);
}
#endif
#ifndef _WIN32
// Creating a symlink needs no privilege only on POSIX.
TEST_CASE("A user preset symlinked to a missing target is counted but the link is kept", "[Preset][Bundle][InstanceLock]")
{
ScopedTemporaryDir temp_dir;
ScopedDataDir data_dir_scope(temp_dir.path());
RenameTestCollection coll;
const fs::path dir = temp_dir.path() / PRESET_PRINT_NAME;
const fs::path link = dir / "Linked.json";
fs::create_directories(dir);
fs::create_symlink(temp_dir.path() / "unmounted" / "Linked.json", link);
PresetsConfigSubstitutions substitutions;
coll.load_presets(temp_dir.path().string(), PRESET_PRINT_NAME, substitutions,
ForwardCompatibilitySubstitutionRule::Disable);
CHECK(coll.error_count() == 1);
CHECK(coll.find_preset("Linked") == nullptr);
boost::system::error_code ec;
CHECK(fs::symlink_status(link, ec).type() == fs::symlink_file);
}
#endif
#ifndef _WIN32
// The read-only bit on a directory stops file creation only on POSIX.
TEST_CASE("A user filament whose derived compatible printer cannot be written back still loads with the failure counted", "[Preset][Bundle]")
{
if (::geteuid() == 0)
SKIP("a read-only directory does not stop root");
ScopedTemporaryDir temp_dir;
PresetBundle bundle;
const fs::path dir = temp_dir.path() / PRESET_FILAMENT_NAME;
const fs::path file = dir / "My PLA @Test Printer.json";
write_preset_with_inherits(bundle.filaments.default_preset().config, file, "My PLA @Test Printer", std::string());
const std::string before = read_file(file);
fs::permissions(file, fs::owner_read);
fs::permissions(dir, fs::owner_read | fs::owner_exe);
PresetsConfigSubstitutions substitutions;
bundle.filaments.load_presets(temp_dir.path().string(), PRESET_FILAMENT_NAME, substitutions,
ForwardCompatibilitySubstitutionRule::EnableSilent);
// Restored before any assertion, so a failure never leaves an unremovable directory behind.
fs::permissions(dir, fs::owner_all);
fs::permissions(file, fs::owner_read | fs::owner_write);
const Preset *preset = bundle.filaments.find_preset("My PLA @Test Printer");
REQUIRE(preset != nullptr);
CHECK(preset->config.option<ConfigOptionStrings>("compatible_printers")->values == std::vector<std::string>{ "Test Printer" });
CHECK(bundle.filaments.error_count() == 1);
CHECK(read_file(file) == before);
}
#endif
+109
View File
@@ -10,6 +10,8 @@
#include <cctype>
#include <fstream>
#include <string>
#include <thread>
#include <system_error>
#ifndef _WIN32
#include <unistd.h> // getuid
@@ -62,6 +64,113 @@ TEST_CASE("per-user temp root is unchanged on Windows, isolated elsewhere", "[ut
#endif
}
TEST_CASE("write_file_atomically replaces the target and leaves no temporary file", "[utils]") {
ScopedTemporaryDir dir;
const boost::filesystem::path target = dir.path() / "preset.json";
REQUIRE_FALSE(write_file_atomically(target.string(), "first"));
REQUIRE_FALSE(write_file_atomically(target.string(), "second"));
std::string content;
load_string_file(target, content);
REQUIRE(content == "second");
size_t entries = 0;
for (auto &entry : boost::filesystem::directory_iterator(dir.path())) {
(void) entry;
++entries;
}
REQUIRE(entries == 1);
}
TEST_CASE("write_file_atomically reports a missing directory and writes nothing", "[utils]") {
ScopedTemporaryDir dir;
const boost::filesystem::path target = dir.path() / "missing" / "preset.json";
const std::error_code ec = write_file_atomically(target.string(), "x");
REQUIRE(ec == std::errc::no_such_file_or_directory);
REQUIRE_FALSE(boost::filesystem::exists(target));
}
#ifndef _WIN32
// The read-only bit on a directory stops file creation only on POSIX.
TEST_CASE("write_file_atomically writes in place when no temporary can be created beside an existing target", "[utils]") {
if (::geteuid() == 0)
SKIP("a read-only directory does not stop root");
ScopedTemporaryDir dir;
const boost::filesystem::path target = dir.path() / "preset.json";
REQUIRE_FALSE(write_file_atomically(target.string(), "first"));
boost::filesystem::permissions(dir.path(), boost::filesystem::owner_read | boost::filesystem::owner_exe);
const std::error_code replaced = write_file_atomically(target.string(), "second");
const std::error_code created = write_file_atomically((dir.path() / "new.json").string(), "x");
// Restored before any assertion, so a failure never leaves an unremovable directory behind.
boost::filesystem::permissions(dir.path(), boost::filesystem::owner_all);
REQUIRE_FALSE(replaced);
REQUIRE(created == std::errc::permission_denied);
std::string content;
load_string_file(target, content);
REQUIRE(content == "second");
}
#endif
TEST_CASE("write_file_atomically keeps bytes intact in binary mode", "[utils]") {
ScopedTemporaryDir dir;
const boost::filesystem::path target = dir.path() / "blob.bin";
const std::string bytes("a\r\nb\0c", 6);
REQUIRE_FALSE(write_file_atomically(target.string(), bytes, /*binary=*/true));
REQUIRE(boost::filesystem::file_size(target) == bytes.size());
}
#ifndef _WIN32
TEST_CASE("write_file_atomically writes through a symlink and keeps the target's permissions", "[utils]") {
ScopedTemporaryDir dir;
const boost::filesystem::path real = dir.path() / "real.json";
const boost::filesystem::path link = dir.path() / "link.json";
REQUIRE_FALSE(write_file_atomically(real.string(), "first"));
boost::filesystem::permissions(real, boost::filesystem::owner_read | boost::filesystem::owner_write);
boost::filesystem::create_symlink(real, link);
REQUIRE_FALSE(write_file_atomically(link.string(), "second"));
REQUIRE(boost::filesystem::is_symlink(boost::filesystem::symlink_status(link)));
std::string content;
load_string_file(real, content);
REQUIRE(content == "second");
REQUIRE_FALSE(write_file_atomically(real.string(), "third"));
const auto perms = boost::filesystem::status(real).permissions() & boost::filesystem::all_all;
REQUIRE(perms == (boost::filesystem::owner_read | boost::filesystem::owner_write));
}
#endif
TEST_CASE("write_file_atomically survives two threads writing one target", "[utils]") {
ScopedTemporaryDir dir;
const boost::filesystem::path target = dir.path() / "shared.json";
const std::string a(20000, 'a'), b(20000, 'b');
std::thread other([&] {
for (int i = 0; i < 50; ++i)
write_file_atomically(target.string(), a);
});
for (int i = 0; i < 50; ++i)
write_file_atomically(target.string(), b);
other.join();
std::string content;
load_string_file(target, content);
const bool whole = content == a || content == b;
REQUIRE(whole);
// No temporary may be left; a scanner on Windows may briefly hold the old
// file under another name, so only the temporaries are counted.
size_t temporaries = 0;
for (auto &entry : boost::filesystem::directory_iterator(dir.path()))
if (entry.path().extension() == ".tmp")
++temporaries;
REQUIRE(temporaries == 0);
}
TEST_CASE("copy_file reports the OS error when the destination cannot be written", "[utils]") {
ScopedTemporaryFile source(".txt");
{
+480 -18
View File
@@ -3,12 +3,21 @@
#include <boost/filesystem.hpp>
#include <boost/crc.hpp>
#include <cereal/archives/binary.hpp>
#include <algorithm>
#include <atomic>
#include <cstring>
#include <fstream>
#include <functional>
#include <memory>
#include <numeric>
#include <random>
#include <set>
#include <sstream>
#include <tbb/blocked_range.h>
#include <tbb/parallel_for.h>
#include <tbb/task_group.h>
#include "libslic3r/PresetBundle.hpp"
#include "libslic3r/PresetCacheFormat.hpp"
#include "libslic3r/Preset.hpp"
@@ -52,6 +61,27 @@ void write_vendor_tree(const fs::path& dir, const std::string& vendor, const std
<< R"(","from":"system","instantiation":"true","layer_height":"0.2"})";
}
// A vendor whose process list is `processes`, each a preset name and the text of
// its sub-file, listed in that order.
void write_process_vendor(const fs::path& dir, const std::string& vendor,
const std::vector<std::pair<std::string, std::string>>& processes)
{
fs::create_directories(dir / vendor / "process");
std::ofstream index((dir / (vendor + ".json")).string());
index << R"({"version":"1.0.0","name":")" << vendor << R"(","process_list":[)";
for (size_t i = 0; i < processes.size(); ++ i) {
const std::string sub_path = "process/p" + std::to_string(i) + ".json";
index << (i ? "," : "") << R"({"name":")" << processes[i].first << R"(","sub_path":")" << sub_path << R"("})";
std::ofstream((dir / vendor / sub_path).string()) << processes[i].second;
}
index << "]}";
}
std::string process_json(const std::string& name, const std::string& extra = std::string())
{
return R"({"type":"process","name":")" + name + R"(","from":"system",)" + extra + R"("layer_height":"0.2"})";
}
// A small but complete vendor: one machine model, one process, a non-instantiated
// base filament with an instantiated child that inherits it and includes a
// dual-extruder template, a second standalone filament carrying explicit
@@ -128,6 +158,80 @@ void write_lib_tree(const fs::path& dir, const std::string& version, const std::
<< R"("filament_id":"GFL99","filament_cost":")" << cost << R"("})";
}
struct FixtureEntry {
std::string name, sub_path;
std::vector<size_t> deps; // indices of the entries this one inherits or includes
};
// Filaments with two levels of inheritance, a template to include, and `leaves`
// presets that inherit either level, some including the template. Only the
// subfiles are written; write_filament_list lists them.
std::vector<FixtureEntry> write_layered_filaments(const fs::path& dir, const std::string& vendor, int leaves)
{
fs::create_directories(dir / vendor / "filament");
std::vector<FixtureEntry> entries {
{ vendor + " Base PLA", "filament/base.json", {} },
{ vendor + " Mid PLA", "filament/mid.json", {0} },
{ vendor + " dual template", "filament/template.json", {} },
};
std::ofstream((dir / vendor / "filament" / "base.json").string())
<< R"({"type":"filament","name":")" << entries[0].name << R"(","from":"system","instantiation":"false",)"
<< R"("filament_id":"GFA_base","filament_cost":"42","filament_max_volumetric_speed":["12"]})";
std::ofstream((dir / vendor / "filament" / "mid.json").string())
<< R"({"type":"filament","name":")" << entries[1].name << R"(","from":"system","instantiation":"false",)"
<< R"("inherits":")" << entries[0].name << R"(","filament_flow_ratio":"0.95"})";
std::ofstream((dir / vendor / "filament" / "template.json").string())
<< R"({"type":"filament","name":")" << entries[2].name << R"(","from":"system","instantiation":"false",)"
<< R"("filament_extruder_variant":["Direct Drive Standard","Direct Drive High Flow"],)"
<< R"("filament_max_volumetric_speed":["20","22"]})";
for (int i = 0; i < leaves; ++ i) {
const size_t parent = i % 2;
const bool include = i % 3 == 0;
entries.push_back({ vendor + " PLA " + std::to_string(i) + " @0.4", "filament/leaf" + std::to_string(i) + ".json",
include ? std::vector<size_t>{parent, 2} : std::vector<size_t>{parent} });
std::ofstream((dir / vendor / entries.back().sub_path).string())
<< R"({"type":"filament","name":")" << entries.back().name << R"(","from":"system","instantiation":"true",)"
<< R"("inherits":")" << entries[parent].name << R"(",)"
<< (include ? R"("include":[")" + entries[2].name + R"("],)" : std::string())
<< R"("nozzle_temperature":[")" << (200 + i % 40) << R"("]})";
}
return entries;
}
// The vendor profile for write_layered_filaments, listing its entries in `order`,
// or as they were written when `order` is empty.
void write_filament_list(const fs::path& dir, const std::string& vendor, const std::vector<FixtureEntry>& entries,
std::vector<size_t> order = {})
{
if (order.empty()) {
order.resize(entries.size());
std::iota(order.begin(), order.end(), 0);
}
std::ofstream f((dir / (vendor + ".json")).string());
f << R"({"version":"1.0.0","name":")" << vendor << R"(","filament_list":[)";
for (size_t k = 0; k < order.size(); ++ k)
f << (k ? "," : "") << R"({"name":")" << entries[order[k]].name << R"(","sub_path":")"
<< entries[order[k]].sub_path << R"("})";
f << "]}";
}
// A random order of `entries` that lists each one after everything it depends on.
std::vector<size_t> shuffled_after_dependencies(const std::vector<FixtureEntry>& entries, std::mt19937& rng)
{
std::vector<size_t> order;
std::vector<bool> listed(entries.size(), false);
while (order.size() < entries.size()) {
std::vector<size_t> ready;
for (size_t i = 0; i < entries.size(); ++ i)
if (! listed[i] && std::all_of(entries[i].deps.begin(), entries[i].deps.end(), [&](size_t d) { return listed[d]; }))
ready.push_back(i);
const size_t next = ready[std::uniform_int_distribution<size_t>(0, ready.size() - 1)(rng)];
listed[next] = true;
order.push_back(next);
}
return order;
}
// A vendor whose one filament inherits the library's base and states nothing of
// its own — everything it shows comes from the library it is resolved against.
void write_vendor_with_lib_filament(const fs::path& dir, const std::string& vendor, const std::string& version)
@@ -624,6 +728,298 @@ TEST_CASE("a cache-loaded vendor is indistinguishable from a JSON-loaded one", "
CHECK(pr->config.opt_string("machine_start_gcode") == "G28 ; template"); // through the include
}
TEST_CASE("a wide vendor loads from its cache exactly as it loads from JSON", "[VendorCache]")
{
InstallDirs dirs;
constexpr int leaves = 400;
write_filament_list(dirs.system, "Acme", write_layered_filaments(dirs.system, "Acme", leaves));
PresetBundle from_json;
from_json.set_generate_vendor_caches(true);
from_json.load_vendor_configs_from_json(dirs.system.string(), "Acme", PresetBundle::LoadSystem,
ForwardCompatibilitySubstitutionRule::EnableSilent);
REQUIRE(fs::exists(dirs.system / "Acme.opc"));
fs::remove_all(dirs.system / "Acme");
PresetBundle from_cache;
from_cache.load_vendor_configs_from_json(dirs.system.string(), "Acme", PresetBundle::LoadSystem,
ForwardCompatibilitySubstitutionRule::EnableSilent);
auto a = presets_for(from_json.filaments, "Acme");
auto b = presets_for(from_cache.filaments, "Acme");
REQUIRE(a.size() == size_t(leaves));
REQUIRE(b.size() == a.size());
for (size_t i = 0; i < a.size(); ++ i) {
CHECK(a[i]->name == b[i]->name);
CHECK(preset_deep_equal(*a[i], *b[i]));
}
CHECK(from_cache.error_count() == from_json.error_count());
// The two-level chain resolved, so the leaf has the middle profile's flow ratio
// and the base profile's filament id, neither of which it states.
const Preset* leaf = from_cache.filaments.find_preset("Acme PLA 7 @0.4", false);
REQUIRE(leaf != nullptr);
CHECK(leaf->filament_id == "GFA_base");
const auto* flow = leaf->config.option<ConfigOptionFloats>("filament_flow_ratio");
REQUIRE(flow != nullptr);
CHECK_THAT(flow->values.front(), WithinAbs(0.95, 1e-9));
}
TEST_CASE("presets install the same in any order that lists each after what it depends on", "[VendorCache]")
{
InstallDirs dirs;
const std::vector<FixtureEntry> entries = write_layered_filaments(dirs.system, "Acme", 150);
auto load = [&](const std::vector<size_t>& order) {
write_filament_list(dirs.system, "Acme", entries, order);
auto bundle = std::make_unique<PresetBundle>();
bundle->load_vendor_configs_from_json(dirs.system.string(), "Acme", PresetBundle::LoadSystem,
ForwardCompatibilitySubstitutionRule::EnableSilent);
return bundle;
};
const auto reference = load({});
CHECK(reference->error_count() == 0);
const auto expected = presets_for(reference->filaments, "Acme");
REQUIRE(expected.size() == 150);
// Every seed checks against one reference load, which GENERATE would repeat per seed.
for (unsigned seed = 0; seed < 8; ++ seed) {
CAPTURE(seed);
std::mt19937 rng(seed);
const auto shuffled = load(shuffled_after_dependencies(entries, rng));
CHECK(shuffled->error_count() == 0);
const auto actual = presets_for(shuffled->filaments, "Acme");
REQUIRE(actual.size() == expected.size());
for (size_t i = 0; i < actual.size(); ++ i)
CHECK(preset_deep_equal(*actual[i], *expected[i]));
}
}
TEST_CASE("a key misplaced into a vendor preset is reported and removed", "[VendorCache]")
{
InstallDirs dirs;
write_process_vendor(dirs.system, "Acme", {
{ "Acme template", R"({"type":"process","name":"Acme template","from":"system","instantiation":"false","wall_loops":"5"})" },
{ "Acme base", R"({"type":"process","name":"Acme base","from":"system","instantiation":"false","filament_cost":"5"})" },
{ "0.20mm Standard @Acme", process_json("0.20mm Standard @Acme",
R"("instantiation":"true","inherits":"Acme base","include":["Acme template"],"nozzle_temperature":["210"],)") } });
PresetBundle bundle;
bundle.load_vendor_configs_from_json(dirs.system.string(), "Acme", PresetBundle::LoadSystem,
ForwardCompatibilitySubstitutionRule::EnableSilent);
CHECK(bundle.error_count() == 2);
const Preset* preset = bundle.prints.find_preset("0.20mm Standard @Acme", false);
REQUIRE(preset != nullptr);
CHECK_FALSE(preset->config.has("filament_cost"));
CHECK_FALSE(preset->config.has("nozzle_temperature"));
CHECK(preset->config.opt_int("wall_loops") == 5);
}
TEST_CASE("each vendor loads from the directory it is listed with", "[VendorCache]")
{
InstallDirs dirs;
write_vendor_tree(dirs.system, "Acme", "1.0.0");
write_vendor_tree(dirs.profiles, "Zeta", "2.0.0");
PresetBundle bundle;
bundle.load_vendors({ { "Acme", dirs.system }, { "Zeta", dirs.profiles } },
ForwardCompatibilitySubstitutionRule::EnableSilent, /*allow_cache=*/false);
CHECK(bundle.vendors.count("Acme") == 1);
CHECK(bundle.vendors.count("Zeta") == 1);
CHECK(bundle.prints.find_preset("0.20mm Standard @Acme", false) != nullptr);
CHECK(bundle.prints.find_preset("0.20mm Standard @Zeta", false) != nullptr);
}
TEST_CASE("a vendor that fails to load is left out, reported, and the others still load", "[VendorCache]")
{
InstallDirs dirs;
write_vendor_tree(dirs.system, "Acme", "1.0.0");
write_process_vendor(dirs.system, "Broken", { { "Broken A", "{not-json" } });
PresetBundle bundle;
std::vector<std::string> failed;
const std::string errors = bundle.load_vendors({ { "Acme", dirs.system }, { "Broken", dirs.system } },
ForwardCompatibilitySubstitutionRule::EnableSilent,
/*allow_cache=*/false, nullptr, &failed).second;
CHECK(bundle.vendors.count("Acme") == 1);
CHECK(bundle.vendors.count("Broken") == 0);
CHECK(bundle.prints.find_preset("0.20mm Standard @Acme", false) != nullptr);
CHECK(errors.find("Broken") != std::string::npos);
CHECK(failed == std::vector<std::string>{ "Broken" });
}
TEST_CASE("a filament library that fails partway is reported, and vendors inheriting from it are left out", "[VendorCache]")
{
InstallDirs dirs;
const std::string lib(PresetBundle::ORCA_FILAMENT_LIBRARY);
write_lib_tree(dirs.system, "1.0.0", "20");
std::ofstream((dirs.system / (lib + ".json")).string())
<< R"({"version":"1.0.0","name":")" << lib << R"(","filament_list":[)"
<< R"({"name":"Generic PLA","sub_path":"filament/generic_pla.json"},)"
<< R"({"name":"Generic PETG","sub_path":"filament/generic_petg.json"}]})";
std::ofstream((dirs.system / lib / "filament" / "generic_petg.json").string()) << "{not-json";
write_vendor_with_lib_filament(dirs.system, "Acme", "1.0.0");
write_vendor_tree(dirs.system, "Zeta", "1.0.0");
PresetBundle bundle;
std::vector<std::string> failed;
const std::string errors = bundle.load_vendors({ { "Acme", dirs.system }, { lib, dirs.system }, { "Zeta", dirs.system } },
ForwardCompatibilitySubstitutionRule::EnableSilent,
/*allow_cache=*/false, nullptr, &failed).second;
CHECK(failed == std::vector<std::string>{ lib, "Acme" });
CHECK(errors.find("generic_petg.json") != std::string::npos);
CHECK(bundle.vendors.count(lib) == 1);
CHECK(bundle.vendors.count("Acme") == 0);
CHECK(bundle.filaments.find_preset("Acme PLA @0.4", false) == nullptr);
CHECK(bundle.vendors.count("Zeta") == 1);
CHECK(bundle.prints.find_preset("0.20mm Standard @Zeta", false) != nullptr);
}
TEST_CASE("a vendor whose profile cannot be read is left out, reported, and the others still load", "[VendorCache]")
{
InstallDirs dirs;
write_vendor_tree(dirs.system, "Acme", "1.0.0");
std::ofstream((dirs.system / "Broken.json").string()) << "{not-json";
PresetBundle bundle;
const std::string errors = bundle.load_vendors({ { "Acme", dirs.system }, { "Broken", dirs.system } },
ForwardCompatibilitySubstitutionRule::EnableSilent,
/*allow_cache=*/false).second;
CHECK(bundle.vendors.count("Acme") == 1);
CHECK(bundle.vendors.count("Broken") == 0);
CHECK(bundle.prints.find_preset("0.20mm Standard @Acme", false) != nullptr);
CHECK(errors.find("Broken.json") != std::string::npos);
}
TEST_CASE("a canceled vendor load starts no vendor", "[VendorCache]")
{
InstallDirs dirs;
write_vendor_tree(dirs.system, "Acme", "1.0.0");
write_vendor_tree(dirs.system, "Zeta", "1.0.0");
const std::atomic<bool> cancel { true };
PresetBundle bundle;
bundle.load_vendors({ { "Acme", dirs.system }, { "Zeta", dirs.system } },
ForwardCompatibilitySubstitutionRule::EnableSilent, /*allow_cache=*/false, &cancel);
CHECK(bundle.vendors.empty());
CHECK(bundle.prints.find_preset("0.20mm Standard @Acme", false) == nullptr);
}
TEST_CASE("a preset two vendors both define is kept from the first listed and reported under the others", "[VendorCache]")
{
InstallDirs dirs;
for (const std::string vendor : { "Acme", "Mira", "Zeta" })
write_process_vendor(dirs.system, vendor, {
{ "Shared", process_json("Shared", R"("instantiation":"true",)") },
{ "Own @" + vendor, process_json("Own @" + vendor, R"("instantiation":"true",)") } });
PresetBundle bundle;
const std::string errors = bundle.load_vendors({ { "Mira", dirs.system }, { "Acme", dirs.system }, { "Zeta", dirs.system } },
ForwardCompatibilitySubstitutionRule::EnableSilent,
/*allow_cache=*/false).second;
const Preset* shared = bundle.prints.find_preset("Shared", false);
REQUIRE(shared != nullptr);
REQUIRE(shared->vendor != nullptr);
CHECK(shared->vendor->id == "Mira");
CHECK(errors.find("vendor Mira") == std::string::npos);
CHECK(errors.find("Found duplicated settings in vendor Acme's json file lists: Shared") != std::string::npos);
CHECK(errors.find("Found duplicated settings in vendor Zeta's json file lists: Shared") != std::string::npos);
CHECK(bundle.error_count() == 2);
for (const std::string vendor : { "Acme", "Mira", "Zeta" }) {
const Preset* own = bundle.prints.find_preset("Own @" + vendor, false);
REQUIRE(own != nullptr);
CHECK(own->vendor == &bundle.vendors.at(vendor));
}
}
TEST_CASE("filaments merged from several vendors come out generic first, then by name", "[VendorCache]")
{
InstallDirs dirs;
auto write_filaments = [&](const std::string& vendor, const std::vector<std::string>& names) {
fs::create_directories(dirs.system / vendor / "filament");
std::ofstream index((dirs.system / (vendor + ".json")).string());
index << R"({"version":"1.0.0","name":")" << vendor << R"(","filament_list":[)";
for (size_t i = 0; i < names.size(); ++ i) {
const std::string sub_path = "filament/f" + std::to_string(i) + ".json";
index << (i ? "," : "") << R"({"name":")" << names[i] << R"(","sub_path":")" << sub_path << R"("})";
std::ofstream((dirs.system / vendor / sub_path).string())
<< R"({"type":"filament","name":")" << names[i] << R"(","from":"system","instantiation":"true",)"
<< R"("filament_id":"GF)" << vendor << i << R"("})";
}
index << "]}";
};
write_filaments("Zeta", { "Zeta PLA @0.4", "Generic PETG @Zeta" });
write_filaments("Acme", { "Acme PLA @0.4", "Generic PLA @Acme" });
PresetBundle bundle;
bundle.load_vendors({ { "Zeta", dirs.system }, { "Acme", dirs.system } },
ForwardCompatibilitySubstitutionRule::EnableSilent, /*allow_cache=*/false);
std::vector<std::string> names;
for (const Preset& preset : bundle.filaments.get_presets())
if (! preset.is_default)
names.push_back(preset.name);
CHECK(names == std::vector<std::string>{ "Generic PETG @Zeta", "Generic PLA @Acme", "Acme PLA @0.4", "Zeta PLA @0.4" });
for (const std::string& name : names)
CHECK(bundle.filaments.find_preset(name, false) != nullptr);
}
TEST_CASE("a vendor read while the filament library loads resolves against it, from JSON and from its cache", "[VendorCache]")
{
InstallDirs dirs;
const std::string lib(PresetBundle::ORCA_FILAMENT_LIBRARY);
write_lib_tree(dirs.system, "1.0.0", "20");
write_vendor_with_lib_filament(dirs.system, "Acme", "1.0.0");
auto load = [&] {
auto bundle = std::make_unique<PresetBundle>();
bundle->set_generate_vendor_caches(true);
bundle->load_vendors({ { "Acme", dirs.system }, { lib, dirs.system } },
ForwardCompatibilitySubstitutionRule::EnableSilent, /*allow_cache=*/true);
return bundle;
};
const auto from_json = load();
REQUIRE(fs::exists(dirs.system / "Acme.opc"));
fs::remove_all(dirs.system / "Acme");
const auto from_cache = load();
for (const PresetBundle* bundle : { from_json.get(), from_cache.get() }) {
const Preset* pla = bundle->filaments.find_preset("Acme PLA @0.4", false);
REQUIRE(pla != nullptr);
CHECK(pla->filament_id == "GFL99");
CHECK_THAT(pla->config.option<ConfigOptionFloats>("filament_cost")->values.front(), WithinAbs(20., 1e-9));
CHECK(bundle->error_count() == 0);
}
}
TEST_CASE("repeated cache loads of one vendor produce the same presets", "[VendorCache]")
{
InstallDirs dirs;
write_filament_list(dirs.system, "Acme", write_layered_filaments(dirs.system, "Acme", 400));
PresetBundle seed;
seed.set_generate_vendor_caches(true);
seed.load_vendor_configs_from_json(dirs.system.string(), "Acme", PresetBundle::LoadSystem,
ForwardCompatibilitySubstitutionRule::EnableSilent);
REQUIRE(fs::exists(dirs.system / "Acme.opc"));
fs::remove_all(dirs.system / "Acme");
std::vector<PresetBundle> loads(3);
for (PresetBundle& bundle : loads)
bundle.load_vendor_configs_from_json(dirs.system.string(), "Acme", PresetBundle::LoadSystem,
ForwardCompatibilitySubstitutionRule::EnableSilent);
auto first = presets_for(loads[0].filaments, "Acme");
REQUIRE(!first.empty());
for (size_t run = 1; run < loads.size(); ++ run) {
auto again = presets_for(loads[run].filaments, "Acme");
REQUIRE(again.size() == first.size());
for (size_t i = 0; i < first.size(); ++ i) {
CHECK(first[i]->name == again[i]->name);
CHECK(preset_deep_equal(*first[i], *again[i]));
}
CHECK(loads[run].error_count() == loads[0].error_count());
}
}
TEST_CASE("a cache-served vendor reports the errors its parse counted", "[VendorCache]")
{
TempDir tmp;
@@ -866,6 +1262,27 @@ TEST_CASE("a vendor installed as its cache alone still loads after a library upd
CHECK_THAT(cost->values.front(), WithinAbs(30., 1e-9));
}
TEST_CASE("a cache install stopped by a canceled task group throws instead of rejecting the cache", "[VendorCache]")
{
TempDir tmp;
const std::string cache = (tmp.path / "Acme.opc").string();
REQUIRE(save_one_vendor(cache, one_vendor("Acme"), "Acme", "1.0.0", {filament_entry("Acme PLA @0.4")}));
PresetBundle bundle;
bool threw = false;
tbb::task_group_context context;
tbb::parallel_for(tbb::blocked_range<size_t>(0, 1), [&](const tbb::blocked_range<size_t>&) {
context.cancel_group_execution();
try {
bundle.load_vendor_cache(cache, "Acme", Semver(1, 0, 0));
} catch (const std::runtime_error&) {
threw = true;
}
}, context);
CHECK(threw);
}
TEST_CASE("a cache entry whose parent is missing falls back to the vendor's JSONs", "[VendorCache]")
{
TempDir tmp;
@@ -893,6 +1310,12 @@ TEST_CASE("a cache entry whose parent is missing falls back to the vendor's JSON
user.string(), "Acme", PresetBundle::LoadSystem, ForwardCompatibilitySubstitutionRule::EnableSilent);
CHECK(presets_loaded == 1);
CHECK(out.vendors.at("Acme").name == "Acme"); // the profile's name, not the cache's
// Through load_vendors, which reads the cache before installing it.
PresetBundle several;
several.load_vendors({ { "Acme", user } }, ForwardCompatibilitySubstitutionRule::EnableSilent, /*allow_cache=*/true);
CHECK(several.vendors.at("Acme").name == "Acme");
CHECK(several.prints.find_preset("0.20mm Standard @Acme", false) != nullptr);
}
TEST_CASE("a profile with no usable version is never served from cache", "[VendorCache]")
@@ -1380,24 +1803,6 @@ TEST_CASE("a header claiming more body than the file holds is rejected", "[Vendo
REQUIRE_FALSE(bundle.load_vendor_cache(cache, "Bounded", Semver(1, 0, 0)));
}
TEST_CASE("a failed write leaves the previous cache in place", "[VendorCache]")
{
TempDir tmp;
const std::string cache = (tmp.path / "Durable.opc").string();
REQUIRE(save_one_vendor(cache, one_vendor("Durable"), "Durable", "1.0.0"));
const std::string before = slurp(cache);
// A directory where the temp file wants to go: the write cannot complete,
// and must not have destroyed what was already there to find that out.
const fs::path blocker = fs::path(cache + "." + std::to_string(get_current_pid()) + ".tmp");
fs::create_directories(blocker);
REQUIRE_FALSE(save_one_vendor(cache, one_vendor("Durable"), "Durable", "2.0.0"));
CHECK(slurp(cache) == before);
fs::remove_all(blocker);
}
TEST_CASE("a cache written by another build's option ordering still loads", "[VendorCache]")
{
// The regression the fingerprint used to prevent by refusing the file
@@ -1691,6 +2096,31 @@ TEST_CASE("an include listed after the preset that names it is an error, and the
CHECK(silk->config.option<ConfigOptionFloats>("filament_max_volumetric_speed")->values == std::vector<double>{12.});
}
TEST_CASE("an include listed after the preset that names it is missing from the cache load too", "[VendorCache]")
{
InstallDirs dirs;
write_full_vendor_tree(dirs.system, "Acme", "1.0.0", /*templates_last=*/true);
PresetBundle from_json;
from_json.set_generate_vendor_caches(true);
from_json.load_vendor_configs_from_json(dirs.system.string(), "Acme", PresetBundle::LoadSystem,
ForwardCompatibilitySubstitutionRule::EnableSilent);
REQUIRE(fs::exists(dirs.system / "Acme.opc"));
fs::remove_all(dirs.system / "Acme");
PresetBundle from_cache;
from_cache.load_vendor_configs_from_json(dirs.system.string(), "Acme", PresetBundle::LoadSystem,
ForwardCompatibilitySubstitutionRule::EnableSilent);
CHECK(from_cache.error_count() == 2);
const Preset* pr = from_cache.printers.find_preset("Acme 0.4 nozzle", false);
REQUIRE(pr != nullptr);
CHECK(pr->config.opt_string("machine_start_gcode") != "G28 ; template");
const Preset* silk = from_cache.filaments.find_preset("Acme Silk PLA @0.4", false);
REQUIRE(silk != nullptr);
const auto& speed = silk->config.option<ConfigOptionFloats>("filament_max_volumetric_speed")->values;
REQUIRE(speed.size() == 1);
CHECK_THAT(speed.front(), WithinAbs(12., 1e-9));
}
TEST_CASE("a G-code template that states no instantiation is included, not loaded as a preset", "[VendorCache]")
{
InstallDirs dirs;
@@ -1712,3 +2142,35 @@ TEST_CASE("a G-code template that states no instantiation is included, not loade
CHECK(pr->config.opt_string("machine_start_gcode") == "G28 ; template");
CHECK(presets_for(bundle.printers, "Acme").size() == 1);
}
TEST_CASE("a sub-file that fails to parse leaves the ones listed before it installed", "[VendorCache]")
{
InstallDirs dirs;
write_process_vendor(dirs.system, "Acme", {
{ "Acme A", process_json("Acme A", R"("instantiation":"true",)") },
{ "Acme B", "{not-json" },
{ "Acme C", process_json("Acme C", R"("instantiation":"true",)") } });
PresetBundle bundle;
CHECK_THROWS_AS(bundle.load_vendor_configs_from_json(dirs.system.string(), "Acme", PresetBundle::LoadSystem,
ForwardCompatibilitySubstitutionRule::EnableSilent),
ConfigurationError);
CHECK(bundle.prints.find_preset("Acme A", false) != nullptr);
CHECK(bundle.prints.find_preset("Acme C", false) == nullptr);
}
TEST_CASE("a load that fails to install partway counts no errors of the sub-files after it", "[VendorCache]")
{
InstallDirs dirs;
// B names a parent nobody defines. C states no instantiation, an error of its own.
write_process_vendor(dirs.system, "Acme", {
{ "Acme A", process_json("Acme A", R"("instantiation":"true",)") },
{ "Acme B", process_json("Acme B", R"("instantiation":"true","inherits":"Nobody",)") },
{ "Acme C", process_json("Acme C") } });
PresetBundle bundle;
CHECK_THROWS_AS(bundle.load_vendor_configs_from_json(dirs.system.string(), "Acme", PresetBundle::LoadSystem,
ForwardCompatibilitySubstitutionRule::EnableSilent),
ConfigurationError);
// B's missing parent and B's failed install.
CHECK(bundle.error_count() == 2);
CHECK(bundle.prints.find_preset("Acme A", false) != nullptr);
}