juli/OrcaSlicer
1
1
Fork 0
mirror of https://github.com/OrcaSlicer/OrcaSlicer.git synced 2026-09-29 11:51:24 +00:00
Code Issues Packages Projects Releases Wiki Activity

libslic3r: install parsed and cached presets through one routine

Browse Source 
A cache load installed the entries others inherit from or include first and
resolved the rest together, while the JSON parse installed each entry straight
after parsing it. The two orders only agree while every preset is listed after
what it depends on. An include listed after the preset naming it was missing for
one and found by the other, and every new kind of reference between presets
would have needed the same care in two places.

Both loads now go through install_vendor, which installs processes, filaments and
printers in listing order. Entries resolve together in runs and commit one at a
time, and a run ends before an entry that inherits or includes one already in
it. No entry resolves against a registration from its own run, so the result is
what installing one at a time gives, in any listing order.

The JSON parse reads every sub-file first, under one numeric locale setter, and
stops at the first that fails. The ones before it are installed before the
failure is raised, as they were one at a time, and the filament library's maps
are published only from a complete load. Each entry's parse errors are logged and
counted when it installs, so the log and a load that fails partway come out as
one-at-a-time loading leaves them.

Every preset already in the collection counts as taken for a new one's name.

resolve_vendor_preset and commit_vendor_preset take a VendorInstall with the
collection's shared state in place of a dozen parameters, const for resolve.

The first launch after an update, which parses every profile, goes from
1510-1533 ms to 806-849 ms on a 16-core desktop, and a cached launch from
342-357 ms to 328-334 ms.
This commit is contained in:
raistlin7447
2026-09-27 08:42:35 -05:00
parent ea3a40d073
commit 5ae88ae5c7
4 changed files with 484 additions and 346 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/HLSD/preset-cache.md
+23 -5
View File
@@ -211,11 +211,12 @@ shipped cache answered first, so the profile in `<data_dir>/system/` was never p
and its cache was never written back.
Serving from a cache is not a memory-image restore. The entries are deserialized and
then installed one by one — inheritance resolved against the presets installed before
them and the currently loaded filament library, includes layered in, configs flattened
onto the collection defaults, validated and registered — by the same function the JSON
path calls straight after parsing a sub-file. An `include` layers what the included
base states, between the parent and the preset's own keys: the base's diff against the
then installed by `install_vendor`, the routine the JSON path hands the vendor's entries
to once it has parsed the sub-files: inheritance resolved against the presets installed
before them and the currently loaded filament library, includes layered in, configs
flattened onto the collection defaults, validated and registered. An `include` layers
what the included base states, between the parent and the preset's own keys: the base's
diff against the
default, taken when the base itself was installed and before the per-variant padding
`inherits` sees, so only what a template sets reaches the presets including it. The two
paths share everything below the parse, which is what makes a cache-loaded bundle
@@ -223,6 +224,16 @@ indistinguishable from a JSON-loaded one by construction rather than by test cov
Installation also rebuilds each preset's file path from the local data directory, so a
shipped cache never carries the generating machine's paths.
Installing an entry is split in two. `resolve_vendor_preset` flattens it, reading only
what is registered under the names it inherits and includes, and `commit_vendor_preset`
registers it, the only step that writes anything shared. Entries resolve across threads
in runs and commit in the order the vendor lists them. A run ends before an entry that
inherits or includes one already in it, since that one's commit registers what the
entry resolves against, so no entry in a run reads what another in it registers. An
entry's parse messages are held until it commits. The bundle, the log's parse and
install messages and the error count therefore come out as parsing and installing one
entry at a time would leave them, whatever the listing order.
App upgrades work because a cache normally survives one. Only a deliberate
`CACHE_VERSION` bump makes an installed cache unreadable, and that is handled at
install time rather than at load: a vendor whose cache this build cannot read counts
@@ -378,6 +389,13 @@ enumerates only `*.json` will find no vendors at all in a packaged build.
the `CachedPreset` field list — written and read by `visit_entry` in
`PresetCacheFormat.cpp`, one list for the save, the load and the name peek alike — or
the cache's own layout or stamps, requires bumping `CACHE_VERSION` by hand.
- **Adding a kind of reference between presets**, as `inherits` and `include` are:
parse the names into `CachedPreset` (a field change, so `CACHE_VERSION` is bumped),
have `install_vendor_entries` end a run before an entry that names one already in it
and retain what the names point at, look them up only in `resolve_vendor_preset`, and
register what they point at only in `commit_vendor_preset`. The listing-order test in
`test_vendor_cache.cpp` fails for a kind the runs do not check once its fixture uses
it.
- **The dictionary indexes with a `uint16`**, so `print_config_def` may hold at most
65535 options and one cache at most 65535 distinct enum value names.
`CacheDictionary::save` throws past that, which surfaces when CI generates the
src/libslic3r/PresetBundle.cpp
+199 -243
View File
@@ -6620,18 +6620,26 @@ void PresetBundle::load_config_file_config(const std::string &name_or_path, bool
BOOST_LOG_TRIVIAL(debug) << __FUNCTION__ << boost::format(": finished");
}
PresetBundle::PresetInstall PresetBundle::resolve_vendor_preset(
const CachedPreset& entry,
const std::string& path, const std::string& vendor_name,
const PresetBundle* base_bundle,
const std::map<std::string, DynamicPrintConfig>& config_maps,
const std::map<std::string, DynamicPrintConfig>& include_maps,
const std::map<std::string, std::string>& filament_id_maps,
const PresetCollection& presets_collection,
const std::set<std::string>* retain_includes) const
static ConfigurationError failed_loading_error(const std::string& file, const std::string& dir)
{
const VendorProfile& current_vendor_profile = this->vendors.at(vendor_name);
const std::string subfile = path + "/" + vendor_name + "/" + entry.sub_path;
return ConfigurationError((boost::format("Failed loading configuration file %1%\nSuggest cleaning the directory %2% firstly") % file % dir).str());
}
void PresetBundle::log_errors(const std::vector<std::string>& errors)
{
for (const std::string& error : errors) {
++m_errors;
BOOST_LOG_TRIVIAL(error) << error;
}
}
PresetBundle::PresetInstall PresetBundle::resolve_vendor_preset(const CachedPreset& entry, const VendorInstall& install) const
{
const std::string& path = install.path;
const std::string& vendor_name = install.vendor_name;
const PresetBundle* base_bundle = install.base_bundle;
const PresetCollection& presets_collection = *install.presets;
const VendorProfile& current_vendor_profile = *install.vendor_profile;
const std::string& preset_name = entry.name;
const DynamicPrintConfig* default_config = nullptr;
@@ -6641,8 +6649,8 @@ PresetBundle::PresetInstall PresetBundle::resolve_vendor_preset(
//check whether it inherits other preset or not
if (! entry.inherits.empty()) {
auto it2 = config_maps.find(entry.inherits);
if (it2 != config_maps.end())
auto it2 = install.config_maps.find(entry.inherits);
if (it2 != install.config_maps.end())
default_config = &(it2->second);
if (default_config == nullptr && base_bundle != nullptr) {
auto base_it2 = base_bundle->m_config_maps.find(entry.inherits);
@@ -6651,8 +6659,8 @@ PresetBundle::PresetInstall PresetBundle::resolve_vendor_preset(
}
if (default_config != nullptr) {
if (out.filament_id.empty() && (presets_collection.type() == Preset::TYPE_FILAMENT)) {
auto filament_id_map_iter = filament_id_maps.find(entry.inherits);
if (filament_id_map_iter != filament_id_maps.end()) {
auto filament_id_map_iter = install.filament_id_maps.find(entry.inherits);
if (filament_id_map_iter != install.filament_id_maps.end()) {
out.filament_id = filament_id_map_iter->second;
}
if (out.filament_id.empty() && base_bundle != nullptr) {
@@ -6675,8 +6683,8 @@ PresetBundle::PresetInstall PresetBundle::resolve_vendor_preset(
// Layer each included preset's own keys over the parent, in the order listed;
// this preset's own keys go on top.
for (const std::string& name : entry.includes) {
auto it = include_maps.find(name);
if (it == include_maps.end()) {
auto it = install.include_maps.find(name);
if (it == install.include_maps.end()) {
out.errors.push_back(std::string(__FUNCTION__) + ": can not find include " + name + " for " + preset_name);
continue;
}
@@ -6687,7 +6695,7 @@ PresetBundle::PresetInstall PresetBundle::resolve_vendor_preset(
// that include it. It is taken before extend_default_config_length pads every
// per-variant key to the base's variant count: the padded defaults would
// otherwise override the values each includer inherits.
if (entry.instantiation == "false" && (retain_includes == nullptr || retain_includes->count(preset_name) != 0)) {
if (entry.instantiation == "false" && install.included.count(preset_name) != 0) {
out.included.emplace();
out.included->apply_only(out.config, out.config.diff(presets_collection.default_preset_for(out.config).config));
}
@@ -6697,8 +6705,8 @@ PresetBundle::PresetInstall PresetBundle::resolve_vendor_preset(
// collection default, so only the entry's own keys can be missing from default_config.
std::string incorrect_keys = Preset::remove_invalid_keys(out.config, *default_config, &entry.config_src);
if (!incorrect_keys.empty())
out.errors.push_back(std::string(__FUNCTION__) + ": The config " + subfile + " contains incorrect keys: " +
incorrect_keys + ", which were removed");
out.errors.push_back(std::string(__FUNCTION__) + ": The config " + path + "/" + vendor_name + "/" + entry.sub_path +
" contains incorrect keys: " + incorrect_keys + ", which were removed");
if (entry.instantiation == "false" && "Template" != vendor_name) {
out.config_only = true;
return out;
@@ -6795,39 +6803,32 @@ PresetBundle::PresetInstall PresetBundle::resolve_vendor_preset(
return out;
}
std::string PresetBundle::commit_vendor_preset(
const CachedPreset& entry, PresetInstall&& resolved,
const std::string& path, const std::string& vendor_name,
LoadConfigBundleAttributes flags,
ConfigSubstitutionContext& substitution_context, PresetsConfigSubstitutions& substitutions,
std::map<std::string, DynamicPrintConfig>& config_maps, std::map<std::string, DynamicPrintConfig>& include_maps,
std::map<std::string, std::string>& filament_id_maps,
PresetCollection* presets_collection, size_t& count, bool is_from_lib,
std::unordered_set<std::string>& installed_names,
const std::set<std::string>* retain_configs)
std::string PresetBundle::commit_vendor_preset(const CachedPreset& entry, PresetInstall&& resolved,
ConfigSubstitutions&& substitutions, VendorInstall& install)
{
const VendorProfile* current_vendor_profile = &this->vendors.at(vendor_name);
const std::string& path = install.path;
const std::string& vendor_name = install.vendor_name;
PresetCollection* presets_collection = install.presets;
const VendorProfile* current_vendor_profile = install.vendor_profile;
const std::string& preset_name = entry.name;
const std::string subfile = path + "/" + vendor_name + "/" + entry.sub_path;
for (const std::string& error : resolved.errors) {
++m_errors;
BOOST_LOG_TRIVIAL(error) << error;
}
log_errors(resolved.errors);
if (resolved.included)
include_maps.emplace(preset_name, std::move(*resolved.included));
install.include_maps.emplace(preset_name, std::move(*resolved.included));
if (! resolved.reason.empty())
return resolved.reason;
const bool retain_config = install.is_from_lib || install.inherited.count(preset_name) != 0;
if (resolved.config_only) {
if (retain_configs == nullptr || retain_configs->count(preset_name) != 0)
config_maps.emplace(preset_name, std::move(resolved.config));
if (retain_config)
install.config_maps.emplace(preset_name, std::move(resolved.config));
if ((presets_collection->type() == Preset::TYPE_FILAMENT) && (!resolved.filament_id.empty()))
filament_id_maps.emplace(preset_name, resolved.filament_id);
install.filament_id_maps.emplace(preset_name, resolved.filament_id);
return std::string();
}
if (! installed_names.insert(preset_name).second) {
if (! install.installed_names.insert(preset_name).second) {
++m_errors;
BOOST_LOG_TRIVIAL(error) << "Error in a Vendor Config Bundle \"" << path << "\": The printer preset \"" <<
preset_name << "\" has already been loaded from another Config Bundle.";
@@ -6836,7 +6837,7 @@ std::string PresetBundle::commit_vendor_preset(
// Load the preset into the list of presets, save it to disk.
Preset &loaded = presets_collection->append_preset(std::move(resolved.file_path), preset_name, std::move(resolved.config));
if (flags.has(LoadConfigBundleAttribute::LoadSystem)) {
if (install.flags.has(LoadConfigBundleAttribute::LoadSystem)) {
loaded.is_system = true;
loaded.vendor = current_vendor_profile;
loaded.version = current_vendor_profile->config_version;
@@ -6850,8 +6851,7 @@ std::string PresetBundle::commit_vendor_preset(
loaded.setting_id = generate_preset_setting_id(
vendor_name, Preset::get_type_string(presets_collection->type()), preset_name);
loaded.filament_id = resolved.filament_id;
loaded.m_from_orca_filament_lib = is_from_lib;
BOOST_LOG_TRIVIAL(debug) << __FUNCTION__ << " " << __LINE__ << ", " << loaded.name << " load filament_id: " << resolved.filament_id;
loaded.m_from_orca_filament_lib = install.is_from_lib;
if (presets_collection->type() == Preset::TYPE_FILAMENT) {
if (resolved.filament_id.empty() && "Template" != vendor_name) {
++m_errors;
@@ -6860,7 +6860,7 @@ std::string PresetBundle::commit_vendor_preset(
return "Can not find filament_id for " + preset_name;
}
else {
filament_id_maps.emplace(preset_name, resolved.filament_id);
install.filament_id_maps.emplace(preset_name, resolved.filament_id);
}
}
}
@@ -6872,48 +6872,104 @@ std::string PresetBundle::commit_vendor_preset(
filaments.set_printer_hold_alias(loaded.alias, loaded);
}
loaded.renamed_from = std::move(resolved.renamed_from);
if (! substitution_context.empty())
substitutions.push_back({
if (! substitutions.empty())
install.substitutions->push_back({
preset_name, presets_collection->type(), PresetConfigSubstitutions::Source::ConfigBundle,
std::string(), std::move(substitution_context.substitutions) });
if (retain_configs == nullptr || retain_configs->count(preset_name) != 0)
config_maps.emplace(preset_name, loaded.config);
++count;
std::string(), std::move(substitutions) });
if (retain_config)
install.config_maps.emplace(preset_name, loaded.config);
++install.count;
//BBS: add config related logs
BOOST_LOG_TRIVIAL(debug) << __FUNCTION__ << boost::format(", got preset %1%, from %2%")%loaded.name %subfile;
BOOST_LOG_TRIVIAL(debug) << __FUNCTION__ << ", got preset " << loaded.name << ", filament_id " << loaded.filament_id
<< ", from " << path << "/" << vendor_name << "/" << entry.sub_path;
return std::string();
}
// The names a vendor may not reuse. A preset is installed by appending it, so the
// collection cannot be searched for a repeated name while one is in progress, and
// the default presets are already in it before the first entry arrives.
void PresetBundle::seed_installed_names(std::unordered_set<std::string>& installed_names,
const PresetCollection& presets) const
void PresetBundle::install_vendor_entries(VendorInstall& install, const std::vector<CachedPreset>& entries,
std::vector<EntryParse>* parsed)
{
for (size_t i = 0; i < presets.m_num_default_presets; ++ i)
installed_names.insert(presets.m_presets[i].name);
assert(parsed == nullptr || parsed->size() == entries.size());
for (const CachedPreset& entry : entries) {
if (! entry.inherits.empty())
install.inherited.insert(entry.inherits);
install.included.insert(entry.includes.begin(), entry.includes.end());
}
// The names an appended preset may not repeat, since the collection cannot be
// searched until it is sorted again.
for (const Preset& preset : install.presets->m_presets)
install.installed_names.insert(preset.name);
// Held until the collection is sorted again, also when an entry throws, so a reader
// holding the lock never finds it unsorted.
install.presets->lock();
ScopeGuard sort_on_exit([&install] {
install.presets->sort_presets();
install.presets->unlock();
});
auto resolve = [&](size_t i) { return resolve_vendor_preset(entries[i], install); };
auto commit = [&](size_t i, PresetInstall&& resolved) {
ConfigSubstitutions substitutions;
if (parsed != nullptr) {
log_errors((*parsed)[i].errors);
for (const std::string& warning : (*parsed)[i].warnings)
BOOST_LOG_TRIVIAL(error) << warning;
substitutions = std::move((*parsed)[i].substitutions);
}
const std::string reason = commit_vendor_preset(entries[i], std::move(resolved), std::move(substitutions), install);
if (! reason.empty()) {
++m_errors;
const std::string subfile_path = install.path + "/" + install.vendor_name + "/" + entries[i].sub_path;
BOOST_LOG_TRIVIAL(error) << __FUNCTION__ << ": can not install " << entries[i].name << " from " << subfile_path << ": " << reason;
throw failed_loading_error(subfile_path, install.path);
}
};
// A run of entries resolves together and commits in listing order, and ends
// before an entry that inherits or includes one already in it. So no entry
// resolves against a registration from its own run, and the result matches
// installing one at a time.
std::unordered_set<std::string> run_names;
size_t run_begin = 0;
auto install_run = [&](size_t run_end) {
resolve_then_commit(run_end - run_begin,
[&](size_t k) { return resolve(run_begin + k); },
[&](size_t k, PresetInstall&& resolved) { commit(run_begin + k, std::move(resolved)); });
run_names.clear();
run_begin = run_end;
};
auto in_run = [&](const std::string& name) { return run_names.count(name) != 0; };
for (size_t i = 0; i < entries.size(); ++ i) {
const CachedPreset& entry = entries[i];
if (in_run(entry.inherits) || std::any_of(entry.includes.begin(), entry.includes.end(), in_run))
install_run(i);
if (install.inherited.count(entry.name) != 0 || install.included.count(entry.name) != 0)
run_names.insert(entry.name);
}
install_run(entries.size());
}
// Orca: load one source-form preset entry — parsed from its JSON subfile just
// now, or deserialized from the vendor's cache; the code is shared so a
// cache-loaded bundle cannot come out different from a JSON-loaded one.
std::string PresetBundle::load_vendor_preset(
const CachedPreset& entry,
const std::string& path, const std::string& vendor_name,
const PresetBundle* base_bundle,
LoadConfigBundleAttributes flags,
ConfigSubstitutionContext& substitution_context, PresetsConfigSubstitutions& substitutions,
std::map<std::string, DynamicPrintConfig>& config_maps, std::map<std::string, DynamicPrintConfig>& include_maps,
std::map<std::string, std::string>& filament_id_maps,
PresetCollection* presets_collection, size_t& count, bool is_from_lib,
std::unordered_set<std::string>& installed_names,
const std::set<std::string>* retain_configs, const std::set<std::string>* retain_includes)
size_t PresetBundle::install_vendor(const std::string& path, const std::string& vendor_name, const PresetBundle* base_bundle,
LoadConfigBundleAttributes flags, const VendorCacheData& entries,
VendorParse* parsed, bool complete, PresetsConfigSubstitutions& substitutions)
{
PresetInstall resolved = this->resolve_vendor_preset(entry, path, vendor_name, base_bundle,
config_maps, include_maps, filament_id_maps, *presets_collection, retain_includes);
return this->commit_vendor_preset(entry, std::move(resolved), path, vendor_name, flags,
substitution_context, substitutions, config_maps, include_maps, filament_id_maps, presets_collection,
count, is_from_lib, installed_names, retain_configs);
const bool is_orca_lib = vendor_name == ORCA_FILAMENT_LIBRARY;
const VendorProfile* vendor_profile = &this->vendors.at(vendor_name);
size_t count = 0;
auto install_collection = [&](const std::vector<CachedPreset>& list, std::vector<EntryParse>* list_parsed,
PresetCollection* presets, bool is_from_lib) {
VendorInstall install { path, vendor_name, vendor_profile, base_bundle, flags, presets, is_from_lib, &substitutions };
install_vendor_entries(install, list, list_parsed);
count += install.count;
return install;
};
install_collection(entries.process_entries, parsed ? &parsed->process_entries : nullptr, &this->prints, false);
VendorInstall filaments = install_collection(entries.filament_entries, parsed ? &parsed->filament_entries : nullptr,
&this->filaments, is_orca_lib);
if (is_orca_lib && complete) {
m_config_maps = std::move(filaments.config_maps);
m_filament_id_maps = std::move(filaments.filament_id_maps);
}
install_collection(entries.machine_entries, parsed ? &parsed->machine_entries : nullptr, &this->printers, false);
return count;
}
//BBS: Load a config bundle file from json
@@ -7170,17 +7226,16 @@ std::pair<PresetsConfigSubstitutions, size_t> PresetBundle::load_vendor_configs_
return std::make_pair(PresetsConfigSubstitutions{}, 0);
// 3) paste the process/filament/print configs
PresetCollection *presets = nullptr;
size_t presets_loaded = 0;
// Parse one subfile into a source-form entry — everything the JSON states,
// nothing resolved. Loading the entry (load_vendor_preset) is the
// same code whether the entry was parsed just now or deserialized from the
// nothing resolved. Installing the entries (install_vendor) is the
// same code whether they were parsed just now or deserialized from the
// vendor's cache.
auto parse_subfile = [this, dir, vendor_name](
auto parse_subfile = [&dir, &vendor_name](
ConfigSubstitutionContext& substitution_context,
const std::pair<std::string, std::string>& subfile_iter,
CachedPreset& entry) -> std::string {
CachedPreset& entry, std::vector<std::string>& errors, std::vector<std::string>& warnings) -> std::string {
std::string subfile = dir + "/" + vendor_name + "/" + subfile_iter.second;
std::string reason;
@@ -7192,8 +7247,7 @@ std::pair<PresetsConfigSubstitutions, size_t> PresetBundle::load_vendor_configs_
entry.sub_path = subfile_iter.second;
entry.config_src.load_from_json(subfile, substitution_context, false, key_values, reason);
if (!reason.empty()) {
++m_errors;
BOOST_LOG_TRIVIAL(error) << __FUNCTION__<< ": load config file "<<subfile<<" Failed!";
errors.push_back(std::string(__FUNCTION__) + ": load config file " + subfile + " Failed!");
return reason;
}
entry.name = key_values[BBL_JSON_KEY_NAME];
@@ -7208,15 +7262,10 @@ std::pair<PresetsConfigSubstitutions, size_t> PresetBundle::load_vendor_configs_
entry.name = subfile_iter.first;
}
if(key_values.find(BBL_JSON_KEY_INSTANTIATION) == key_values.end())
{
BOOST_LOG_TRIVIAL(error) << __FUNCTION__ << ": Missing instantiation attribute for " << entry.name;
++m_errors;
}
errors.push_back(std::string(__FUNCTION__) + ": Missing instantiation attribute for " + entry.name);
entry.instantiation = key_values[BBL_JSON_KEY_INSTANTIATION];
if(entry.instantiation != "false" && entry.instantiation != "true"){
BOOST_LOG_TRIVIAL(error) << __FUNCTION__ << ": Missing instantiation attribute for " << entry.name;
++m_errors;
}
if(entry.instantiation != "false" && entry.instantiation != "true")
errors.push_back(std::string(__FUNCTION__) + ": Missing instantiation attribute for " + entry.name);
auto setting_it = key_values.find(BBL_JSON_KEY_SETTING_ID);
if (setting_it != key_values.end())
entry.setting_id = setting_it->second;
@@ -7229,8 +7278,7 @@ std::pair<PresetsConfigSubstitutions, size_t> PresetBundle::load_vendor_configs_
// An `inherits` key naming nothing can never resolve; fail it
// here so install can key off the empty string as "no inherits".
if (entry.inherits.empty()) {
++m_errors;
BOOST_LOG_TRIVIAL(error) << __FUNCTION__ << ": can not find inherits " << entry.inherits << " for " << entry.name;
errors.push_back(std::string(__FUNCTION__) + ": can not find inherits " + entry.inherits + " for " + entry.name);
reason = "Can not find inherits: " + entry.inherits;
return reason;
}
@@ -7243,30 +7291,24 @@ std::pair<PresetsConfigSubstitutions, size_t> PresetBundle::load_vendor_configs_
for (const auto& name : includes) {
if (name.is_string())
entry.includes.push_back(name.get<std::string>());
else {
++m_errors;
BOOST_LOG_TRIVIAL(error) << __FUNCTION__ << ": invalid include " << name.dump() << " for " << entry.name;
}
else
errors.push_back(std::string(__FUNCTION__) + ": invalid include " + name.dump() + " for " + entry.name);
}
}
if (key_values.find(ORCA_JSON_KEY_RENAMED_FROM) != key_values.end()) {
if (!unescape_strings_cstyle(key_values[ORCA_JSON_KEY_RENAMED_FROM], entry.renamed_from)) {
BOOST_LOG_TRIVIAL(error) << "Error in a Config \"" << dir << "\": The preset \"" << entry.name
<< "\" contains invalid \"renamed_from\" key, which is being ignored.";
}
if (!unescape_strings_cstyle(key_values[ORCA_JSON_KEY_RENAMED_FROM], entry.renamed_from))
warnings.push_back("Error in a Config \"" + dir + "\": The preset \"" + entry.name +
"\" contains invalid \"renamed_from\" key, which is being ignored.");
}
}
catch(nlohmann::detail::parse_error &err) {
++m_errors;
BOOST_LOG_TRIVIAL(error) << __FUNCTION__<< ": parse "<< subfile <<" got a nlohmann::detail::parse_error, reason = " << err.what();
errors.push_back(std::string(__FUNCTION__) + ": parse " + subfile + " got a nlohmann::detail::parse_error, reason = " + err.what());
reason = std::string("json parse error") + err.what();
return reason;
}
return reason;
};
std::map<std::string, DynamicPrintConfig> configs, include_maps;
std::map<std::string, std::string> filament_id_maps;
// Orca: whether to (re)write the vendor's cache after this parse, leaving it
// in step with the profile so the next run reads it instead. It is written
// where the vendor was looked for, even when the profile came from resources,
@@ -7275,69 +7317,58 @@ std::pair<PresetsConfigSubstitutions, size_t> PresetBundle::load_vendor_configs_
// worse than none.
const bool will_cache = cacheable && m_generate_vendor_caches && vendor_profile.config_version.valid();
VendorCacheData cache_data;
// Parsed up to the first sub-file that fails, and the ones before it are
// installed before that failure is raised, since the filament library and a
// filament-only scan keep what a throwing load installed.
VendorParse parsed;
std::string reason;
std::string failed_subfile;
const char* failed_kind = nullptr;
std::vector<std::string> failed_errors;
auto parse_subfiles = [&](const std::vector<std::pair<std::string, std::string>>& subfiles,
std::vector<CachedPreset>& entries, std::vector<EntryParse>& entries_parsed, const char* kind) {
for (const auto& subfile : subfiles) {
CachedPreset entry;
std::vector<std::string> errors, warnings;
reason = parse_subfile(substitution_context, subfile, entry, errors, warnings);
if (! reason.empty()) {
failed_subfile = subfile.second;
failed_kind = kind;
failed_errors = std::move(errors);
return false;
}
entries.emplace_back(std::move(entry));
entries_parsed.push_back({ std::move(substitution_context.substitutions), std::move(errors), std::move(warnings) });
}
return true;
};
{
// One setter for every sub-file, so the ones load_from_json makes have
// nothing to set.
CNumericLocalesSetter locales_setter;
if (parse_subfiles(process_subfiles, cache_data.process_entries, parsed.process_entries, "process") &&
parse_subfiles(filament_subfiles, cache_data.filament_entries, parsed.filament_entries, "filament"))
parse_subfiles(machine_subfiles, cache_data.machine_entries, parsed.machine_entries, "printer");
}
int parse_errors = 0;
for (const std::vector<EntryParse>* list : { &parsed.process_entries, &parsed.filament_entries, &parsed.machine_entries })
for (const EntryParse& entry_parsed : *list)
parse_errors += int(entry_parsed.errors.size());
const int errors_before_install = m_errors;
presets_loaded = install_vendor(dir, vendor_name, base_bundle, flags, cache_data, &parsed, reason.empty(), substitutions);
// Errors added by install are counted apart: a cache load runs install again,
// so the parse_errors stamped into the cache must hold only what a cache load
// will not recount.
int install_errors = 0;
auto load_subfiles = [&](std::vector<std::pair<std::string, std::string>>& subfiles,
std::vector<CachedPreset>& entries, const char* kind, bool is_from_lib = false) {
configs.clear();
include_maps.clear();
filament_id_maps.clear();
std::unordered_set<std::string> installed_names;
seed_installed_names(installed_names, *presets);
// The presets below are appended, so the collection is out of the order
// find_preset() searches until it is sorted. A subfile that cannot be
// loaded throws past the end of the loop, and a caller that carries on
// from that would be left searching an unsorted collection.
struct SortOnExit {
PresetCollection* presets;
~SortOnExit() { presets->sort_presets(); }
} sort_on_exit { presets };
for (auto& subfile : subfiles) {
CachedPreset entry;
std::string reason = parse_subfile(substitution_context, subfile, entry);
if (reason.empty()) {
const int errors_before_install = m_errors;
reason = load_vendor_preset(entry, dir, vendor_name, base_bundle, flags,
substitution_context, substitutions, configs, include_maps, filament_id_maps,
presets, presets_loaded, is_from_lib, installed_names);
install_errors += m_errors - errors_before_install;
}
if (!reason.empty()) {
++m_errors;
//parse error
std::string subfile_path = dir + "/" + vendor_name + "/" + subfile.second;
BOOST_LOG_TRIVIAL(error) << __FUNCTION__ << boost::format(", got error when parse %1% setting from %2%") % kind % subfile_path;
throw ConfigurationError((boost::format("Failed loading configuration file %1%\nSuggest cleaning the directory %2% firstly") % subfile_path % dir).str());
}
if (will_cache)
entries.emplace_back(std::move(entry));
}
};
// The section order below — process, filaments (with the ORCA-lib map copy),
// printers — is mirrored by load_vendor_cache's install loops; keep the two
// in lockstep.
//3.1) paste the process
presets = &this->prints;
load_subfiles(process_subfiles, cache_data.process_entries, "process");
//3.2) paste the filaments
presets = &this->filaments;
const auto is_orca_lib = vendor_name == ORCA_FILAMENT_LIBRARY;
load_subfiles(filament_subfiles, cache_data.filament_entries, "filament", is_orca_lib);
if (is_orca_lib) {
// The next install clears `configs` before using it again, and for the
// library it holds a full config for every one of its presets.
m_config_maps = std::move(configs);
m_filament_id_maps = std::move(filament_id_maps);
const int install_errors = m_errors - errors_before_install - parse_errors;
if (! reason.empty()) {
log_errors(failed_errors);
++m_errors;
//parse error
std::string subfile_path = dir + "/" + vendor_name + "/" + failed_subfile;
BOOST_LOG_TRIVIAL(error) << __FUNCTION__ << boost::format(", got error when parse %1% setting from %2%") % failed_kind % subfile_path;
throw failed_loading_error(subfile_path, dir);
}
//3.3) paste the printers
presets = &this->printers;
load_subfiles(machine_subfiles, cache_data.machine_entries, "printer");
if (will_cache) {
// Clamped: the count is a difference of three tallies, and a stamp that
// wrapped would be added to every future load of this vendor.
@@ -8092,85 +8123,10 @@ bool PresetBundle::load_vendor_cache(const std::string& cache_path, const std::s
// own below, so m_errors comes out as a JSON parse would leave it.
m_errors += int(data.parse_errors);
// Install the entries through the same code load_vendor_configs_from_json
// installs them with straight after parsing. The entries others inherit
// from or include go in first here rather than in listing order, which the
// collection being sorted afterwards makes unobservable. The substitution context
// stays empty (the entries were substituted when they were parsed), so no
// substitutions are reported, as before.
ConfigSubstitutionContext substitution_context { ForwardCompatibilitySubstitutionRule::EnableSilent };
// Stays empty, since the entries were substituted when they were parsed.
PresetsConfigSubstitutions substitutions;
std::map<std::string, DynamicPrintConfig> configs, include_maps;
std::map<std::string, std::string> filament_id_maps;
const std::string path = boost::filesystem::path(cache_path).parent_path().string();
size_t count = 0;
auto install_entries = [&](const std::vector<CachedPreset>& entries, PresetCollection* presets, bool is_from_lib) {
configs.clear();
include_maps.clear();
filament_id_maps.clear();
// Only configs of presets that other entries inherit or include are
// ever looked up again; registering just those skips one full config
// copy for every leaf preset. The library's filaments are all retained
// — they become the m_config_maps other vendors resolve against.
std::set<std::string> inherited, included;
for (const CachedPreset& entry : entries) {
if (! entry.inherits.empty())
inherited.insert(entry.inherits);
included.insert(entry.includes.begin(), entry.includes.end());
}
const std::set<std::string>* retain_configs = is_from_lib ? nullptr : &inherited;
std::unordered_set<std::string> installed_names;
seed_installed_names(installed_names, *presets);
// The entries below are appended, so the collection is out of the
// order find_preset() searches until it is sorted, and an entry that
// cannot be installed throws past the end of this lambda.
struct SortOnExit {
PresetCollection* presets;
~SortOnExit() { presets->sort_presets(); }
} sort_on_exit { presets };
auto commit = [&](const CachedPreset& entry, PresetInstall&& resolved) {
const std::string reason = commit_vendor_preset(entry, std::move(resolved), path, vendor_name,
LoadConfigBundleAttribute::LoadSystem, substitution_context, substitutions,
configs, include_maps, filament_id_maps, presets, count, is_from_lib, installed_names, retain_configs);
if (! reason.empty())
throw std::runtime_error("entry " + entry.name + " failed to install: " + reason);
};
auto resolve = [&](const CachedPreset& entry) {
return resolve_vendor_preset(entry, path, vendor_name, base_bundle, configs, include_maps, filament_id_maps,
*presets, &included);
};
// An entry others inherit from or include is installed before them, in
// the order the vendor lists them, because what it registers in `configs`
// or `include_maps` is what they resolve against. So is an entry that
// includes one listed after it, which it must find missing. The rest
// depend on nothing else.
std::vector<size_t> leaves;
leaves.reserve(entries.size());
for (size_t i = 0; i < entries.size(); ++ i) {
const CachedPreset& entry = entries[i];
const bool includes_installed = std::all_of(entry.includes.begin(), entry.includes.end(),
[&](const std::string& name) { return include_maps.count(name) != 0; });
if (inherited.count(entry.name) == 0 && included.count(entry.name) == 0 && includes_installed)
leaves.push_back(i);
else
commit(entry, resolve(entry));
}
resolve_then_commit(leaves.size(),
[&](size_t k) { return resolve(entries[leaves[k]]); },
[&](size_t k, PresetInstall&& resolved) { commit(entries[leaves[k]], std::move(resolved)); });
};
install_entries(data.process_entries, &this->prints, false);
const bool is_orca_lib = vendor_name == ORCA_FILAMENT_LIBRARY;
install_entries(data.filament_entries, &this->filaments, is_orca_lib);
if (is_orca_lib) {
// The next install clears `configs` before using it again, and for the
// library it holds a full config for every one of its presets.
m_config_maps = std::move(configs);
m_filament_id_maps = std::move(filament_id_maps);
}
install_entries(data.machine_entries, &this->printers, false);
install_vendor(boost::filesystem::path(cache_path).parent_path().string(), vendor_name, base_bundle,
LoadConfigBundleAttribute::LoadSystem, data, nullptr, true, substitutions);
return true;
} catch (const std::exception& e) {
BOOST_LOG_TRIVIAL(warning) << "PresetBundle: rejecting vendor cache " << cache_path << ": " << e.what();
src/libslic3r/PresetBundle.hpp
+73 -48
View File
@@ -634,25 +634,70 @@ private:
// load_vendor_configs_from_json reads a cache.
bool load_vendor_cache(const boost::filesystem::path& dir, const std::string& vendor_name, const PresetBundle* base_bundle);
// Load one source-form preset entry into this bundle: resolve `inherits`
// and `include`, flatten, validate and register the preset. Returns the
// reason loading failed, empty on success. See the definition for the
// sharing contract between the JSON parse and the cache load.
// retain_configs / retain_includes, when non-null, name the only presets
// registered into config_maps / include_maps (a config copy each). The
// cache load passes the names its entries inherit / include — the only
// ones ever looked up again; the JSON parse retains all, not knowing what
// later subfiles name.
std::string load_vendor_preset(const CachedPreset& entry,
const std::string& path, const std::string& vendor_name,
const PresetBundle* base_bundle,
LoadConfigBundleAttributes flags,
ConfigSubstitutionContext& substitution_context, PresetsConfigSubstitutions& substitutions,
std::map<std::string, DynamicPrintConfig>& config_maps, std::map<std::string, DynamicPrintConfig>& include_maps,
std::map<std::string, std::string>& filament_id_maps,
PresetCollection* presets_collection, size_t& count, bool is_from_lib,
std::unordered_set<std::string>& installed_names,
const std::set<std::string>* retain_configs = nullptr, const std::set<std::string>* retain_includes = nullptr);
// What parsing one entry's JSON sub-file reported. Its errors and warnings are
// logged when the entry installs, so they come out in listing order with the
// entry's install errors, as parsing and installing one entry at a time leaves them.
struct EntryParse
{
ConfigSubstitutions substitutions;
// Counted in the bundle's error count.
std::vector<std::string> errors;
std::vector<std::string> warnings;
};
// An EntryParse for each entry of the VendorCacheData list of the same name.
struct VendorParse
{
std::vector<EntryParse> process_entries;
std::vector<EntryParse> filament_entries;
std::vector<EntryParse> machine_entries;
};
// Log and count errors reported by a resolve or a parse.
void log_errors(const std::vector<std::string>& errors);
// The state of installing one collection of one vendor, which
// resolve_vendor_preset reads through a const reference and only
// commit_vendor_preset writes.
struct VendorInstall
{
// The directory holding <vendor_name>/, whose sub-paths the entries name.
std::string path;
std::string vendor_name;
const VendorProfile* vendor_profile;
const PresetBundle* base_bundle;
LoadConfigBundleAttributes flags;
PresetCollection* presets;
// The Orca filament library, which keeps every config for other vendors to
// resolve against.
bool is_from_lib;
PresetsConfigSubstitutions* substitutions;
// The names some entry inherits / includes, the only ones whose configs /
// include diffs are looked up again.
std::set<std::string> inherited;
std::set<std::string> included;
std::map<std::string, DynamicPrintConfig> config_maps;
std::map<std::string, DynamicPrintConfig> include_maps;
std::map<std::string, std::string> filament_id_maps;
std::unordered_set<std::string> installed_names;
size_t count { 0 };
};
// Install a vendor's source-form entries, parsed from its JSON or read from its
// cache: processes, then filaments, then printers. Both loads go through here,
// so a cache-loaded bundle cannot come out different from a JSON-loaded one.
// `parsed` is given for entries parsed just now. `complete` says the entries
// are the vendor's whole lists; only then are the filament library's configs
// and filament ids left in m_config_maps and m_filament_id_maps. Returns the
// number of presets installed, and throws ConfigurationError at the first
// entry that cannot be installed.
size_t install_vendor(const std::string& path, const std::string& vendor_name, const PresetBundle* base_bundle,
LoadConfigBundleAttributes flags, const VendorCacheData& entries,
VendorParse* parsed, bool complete, PresetsConfigSubstitutions& substitutions);
// Install one collection's entries in the order they are listed.
void install_vendor_entries(VendorInstall& install, const std::vector<CachedPreset>& entries,
std::vector<EntryParse>* parsed);
// One entry flattened against the preset it inherits, before anything this
// bundle shares has been touched.
@@ -669,44 +714,24 @@ private:
std::vector<std::string> errors;
// What a base states for the presets that include it, when it is retained.
std::optional<DynamicPrintConfig> included;
// Inherited from and nothing else, so it contributes a config and no preset.
// Not instantiated, so it contributes a config and no preset.
bool config_only { false };
// Non-empty when the entry is rejected, and says why.
std::string reason;
};
// Flatten one entry against config_maps (this bundle's presets) or
// base_bundle's filament library, with the bases it includes from
// include_maps. It reads only, so entries whose parents are already
// installed resolve independently of each other.
PresetInstall resolve_vendor_preset(const CachedPreset& entry,
const std::string& path, const std::string& vendor_name,
const PresetBundle* base_bundle,
const std::map<std::string, DynamicPrintConfig>& config_maps,
const std::map<std::string, DynamicPrintConfig>& include_maps,
const std::map<std::string, std::string>& filament_id_maps,
const PresetCollection& presets_collection,
const std::set<std::string>* retain_includes = nullptr) const;
// Flatten one entry against the config it inherits, from this collection's
// config_maps or base_bundle's filament library, with the include diffs it
// names layered in. It looks up nothing but the names the entry inherits and
// includes, and writes nothing.
PresetInstall resolve_vendor_preset(const CachedPreset& entry, const VendorInstall& install) const;
// Install a resolved entry. The collections, the inheritance maps and the
// Install a resolved entry. The collections, the maps in `install` and the
// error count are touched here and only here, one entry at a time.
// Presets are appended, so a repeated name is caught with `installed_names`,
// and the caller sorts the collection once every entry is in.
// and the collection is sorted once every entry is in.
std::string commit_vendor_preset(const CachedPreset& entry, PresetInstall&& resolved,
const std::string& path, const std::string& vendor_name,
LoadConfigBundleAttributes flags,
ConfigSubstitutionContext& substitution_context, PresetsConfigSubstitutions& substitutions,
std::map<std::string, DynamicPrintConfig>& config_maps, std::map<std::string, DynamicPrintConfig>& include_maps,
std::map<std::string, std::string>& filament_id_maps,
PresetCollection* presets_collection, size_t& count, bool is_from_lib,
std::unordered_set<std::string>& installed_names,
const std::set<std::string>* retain_configs = nullptr);
// The names an install may not give a preset. A preset is installed by
// appending it, so the collection is too unsorted to be searched for a
// repeated name, and the default presets are in it before the first entry.
void seed_installed_names(std::unordered_set<std::string>& installed_names,
const PresetCollection& presets) const;
ConfigSubstitutions&& substitutions, VendorInstall& install);
// Clear every collection's m_printer_hold_alias, which reset() leaves alone.
void clear_printer_hold_aliases();
tests/libslic3r/test_vendor_cache.cpp
+189 -50
View File
@@ -3,9 +3,13 @@
#include <boost/filesystem.hpp>
#include <boost/crc.hpp>
#include <cereal/archives/binary.hpp>
#include <algorithm>
#include <cstring>
#include <fstream>
#include <functional>
#include <memory>
#include <numeric>
#include <random>
#include <set>
#include <sstream>
@@ -52,6 +56,27 @@ void write_vendor_tree(const fs::path& dir, const std::string& vendor, const std
<< R"(","from":"system","instantiation":"true","layer_height":"0.2"})";
}
// A vendor whose process list is `processes`, each a preset name and the text of
// its sub-file, listed in that order.
void write_process_vendor(const fs::path& dir, const std::string& vendor,
const std::vector<std::pair<std::string, std::string>>& processes)
{
fs::create_directories(dir / vendor / "process");
std::ofstream index((dir / (vendor + ".json")).string());
index << R"({"version":"1.0.0","name":")" << vendor << R"(","process_list":[)";
for (size_t i = 0; i < processes.size(); ++ i) {
const std::string sub_path = "process/p" + std::to_string(i) + ".json";
index << (i ? "," : "") << R"({"name":")" << processes[i].first << R"(","sub_path":")" << sub_path << R"("})";
std::ofstream((dir / vendor / sub_path).string()) << processes[i].second;
}
index << "]}";
}
std::string process_json(const std::string& name, const std::string& extra = std::string())
{
return R"({"type":"process","name":")" + name + R"(","from":"system",)" + extra + R"("layer_height":"0.2"})";
}
// A small but complete vendor: one machine model, one process, a non-instantiated
// base filament with an instantiated child that inherits it and includes a
// dual-extruder template, a second standalone filament carrying explicit
@@ -128,33 +153,78 @@ void write_lib_tree(const fs::path& dir, const std::string& version, const std::
<< R"("filament_id":"GFL99","filament_cost":")" << cost << R"("})";
}
// A vendor wide enough for a cache load to resolve its entries together, with
// two levels of inheritance the load has to work through in order and `leaves`
// presets that depend on nothing but those.
void write_wide_vendor_tree(const fs::path& dir, const std::string& vendor, const std::string& version, int leaves)
struct FixtureEntry {
std::string name, sub_path;
std::vector<size_t> deps; // indices of the entries this one inherits or includes
};
// Filaments with two levels of inheritance, a template to include, and `leaves`
// presets that inherit either level, some including the template. Only the
// subfiles are written; write_filament_list lists them.
std::vector<FixtureEntry> write_layered_filaments(const fs::path& dir, const std::string& vendor, int leaves)
{
fs::create_directories(dir / vendor / "filament");
std::ostringstream filament_list;
filament_list << R"({"name":")" << vendor << R"( Base PLA","sub_path":"filament/base.json"},)"
<< R"({"name":")" << vendor << R"( Mid PLA","sub_path":"filament/mid.json"})";
for (int i = 0; i < leaves; ++ i)
filament_list << R"(,{"name":")" << vendor << " PLA " << i << R"( @0.4","sub_path":"filament/leaf)"
<< i << R"(.json"})";
std::ofstream((dir / (vendor + ".json")).string())
<< R"({"version":")" << version << R"(","name":")" << vendor << R"(","filament_list":[)"
<< filament_list.str() << "]}";
std::vector<FixtureEntry> entries {
{ vendor + " Base PLA", "filament/base.json", {} },
{ vendor + " Mid PLA", "filament/mid.json", {0} },
{ vendor + " dual template", "filament/template.json", {} },
};
std::ofstream((dir / vendor / "filament" / "base.json").string())
<< R"({"type":"filament","name":")" << vendor
<< R"( Base PLA","from":"system","instantiation":"false","filament_id":"GFA_base","filament_cost":"42"})";
<< R"({"type":"filament","name":")" << entries[0].name << R"(","from":"system","instantiation":"false",)"
<< R"("filament_id":"GFA_base","filament_cost":"42","filament_max_volumetric_speed":["12"]})";
std::ofstream((dir / vendor / "filament" / "mid.json").string())
<< R"({"type":"filament","name":")" << vendor
<< R"( Mid PLA","from":"system","instantiation":"false","inherits":")" << vendor
<< R"( Base PLA","filament_flow_ratio":"0.95"})";
for (int i = 0; i < leaves; ++ i)
std::ofstream((dir / vendor / "filament" / ("leaf" + std::to_string(i) + ".json")).string())
<< R"({"type":"filament","name":")" << vendor << " PLA " << i
<< R"( @0.4","from":"system","instantiation":"true","inherits":")" << vendor
<< R"( Mid PLA","nozzle_temperature":")" << (200 + i % 40) << R"("})";
<< R"({"type":"filament","name":")" << entries[1].name << R"(","from":"system","instantiation":"false",)"
<< R"("inherits":")" << entries[0].name << R"(","filament_flow_ratio":"0.95"})";
std::ofstream((dir / vendor / "filament" / "template.json").string())
<< R"({"type":"filament","name":")" << entries[2].name << R"(","from":"system","instantiation":"false",)"
<< R"("filament_extruder_variant":["Direct Drive Standard","Direct Drive High Flow"],)"
<< R"("filament_max_volumetric_speed":["20","22"]})";
for (int i = 0; i < leaves; ++ i) {
const size_t parent = i % 2;
const bool include = i % 3 == 0;
entries.push_back({ vendor + " PLA " + std::to_string(i) + " @0.4", "filament/leaf" + std::to_string(i) + ".json",
include ? std::vector<size_t>{parent, 2} : std::vector<size_t>{parent} });
std::ofstream((dir / vendor / entries.back().sub_path).string())
<< R"({"type":"filament","name":")" << entries.back().name << R"(","from":"system","instantiation":"true",)"
<< R"("inherits":")" << entries[parent].name << R"(",)"
<< (include ? R"("include":[")" + entries[2].name + R"("],)" : std::string())
<< R"("nozzle_temperature":[")" << (200 + i % 40) << R"("]})";
}
return entries;
}
// The vendor profile for write_layered_filaments, listing its entries in `order`,
// or as they were written when `order` is empty.
void write_filament_list(const fs::path& dir, const std::string& vendor, const std::vector<FixtureEntry>& entries,
std::vector<size_t> order = {})
{
if (order.empty()) {
order.resize(entries.size());
std::iota(order.begin(), order.end(), 0);
}
std::ofstream f((dir / (vendor + ".json")).string());
f << R"({"version":"1.0.0","name":")" << vendor << R"(","filament_list":[)";
for (size_t k = 0; k < order.size(); ++ k)
f << (k ? "," : "") << R"({"name":")" << entries[order[k]].name << R"(","sub_path":")"
<< entries[order[k]].sub_path << R"("})";
f << "]}";
}
// A random order of `entries` that lists each one after everything it depends on.
std::vector<size_t> shuffled_after_dependencies(const std::vector<FixtureEntry>& entries, std::mt19937& rng)
{
std::vector<size_t> order;
std::vector<bool> listed(entries.size(), false);
while (order.size() < entries.size()) {
std::vector<size_t> ready;
for (size_t i = 0; i < entries.size(); ++ i)
if (! listed[i] && std::all_of(entries[i].deps.begin(), entries[i].deps.end(), [&](size_t d) { return listed[d]; }))
ready.push_back(i);
const size_t next = ready[std::uniform_int_distribution<size_t>(0, ready.size() - 1)(rng)];
listed[next] = true;
order.push_back(next);
}
return order;
}
// A vendor whose one filament inherits the library's base and states nothing of
@@ -655,27 +725,19 @@ TEST_CASE("a cache-loaded vendor is indistinguishable from a JSON-loaded one", "
TEST_CASE("a wide vendor loads from its cache exactly as it loads from JSON", "[VendorCache]")
{
TempDir tmp;
const fs::path rsrc = tmp.path / "resources" / "profiles";
const fs::path user = tmp.path / "data" / PRESET_SYSTEM_DIR;
fs::create_directories(rsrc);
fs::create_directories(user);
// Enough presets that the cache load resolves them across threads while the
// JSON parse below keeps installing them one at a time.
InstallDirs dirs;
constexpr int leaves = 400;
write_wide_vendor_tree(user, "Acme", "1.0.0", leaves);
ScopedDirs dirs(tmp.path / "data", tmp.path / "resources");
write_filament_list(dirs.system, "Acme", write_layered_filaments(dirs.system, "Acme", leaves));
PresetBundle from_json;
from_json.set_generate_vendor_caches(true);
from_json.load_vendor_configs_from_json(user.string(), "Acme", PresetBundle::LoadSystem,
from_json.load_vendor_configs_from_json(dirs.system.string(), "Acme", PresetBundle::LoadSystem,
ForwardCompatibilitySubstitutionRule::EnableSilent);
REQUIRE(fs::exists(user / "Acme.opc"));
REQUIRE(fs::exists(dirs.system / "Acme.opc"));
fs::remove_all(user / "Acme");
fs::remove_all(dirs.system / "Acme");
PresetBundle from_cache;
from_cache.load_vendor_configs_from_json(user.string(), "Acme", PresetBundle::LoadSystem,
from_cache.load_vendor_configs_from_json(dirs.system.string(), "Acme", PresetBundle::LoadSystem,
ForwardCompatibilitySubstitutionRule::EnableSilent);
auto a = presets_for(from_json.filaments, "Acme");
@@ -697,27 +759,70 @@ TEST_CASE("a wide vendor loads from its cache exactly as it loads from JSON", "[
CHECK_THAT(flow->values.front(), WithinAbs(0.95, 1e-9));
}
TEST_CASE("presets install the same in any order that lists each after what it depends on", "[VendorCache]")
{
InstallDirs dirs;
const std::vector<FixtureEntry> entries = write_layered_filaments(dirs.system, "Acme", 150);
auto load = [&](const std::vector<size_t>& order) {
write_filament_list(dirs.system, "Acme", entries, order);
auto bundle = std::make_unique<PresetBundle>();
bundle->load_vendor_configs_from_json(dirs.system.string(), "Acme", PresetBundle::LoadSystem,
ForwardCompatibilitySubstitutionRule::EnableSilent);
return bundle;
};
const auto reference = load({});
CHECK(reference->error_count() == 0);
const auto expected = presets_for(reference->filaments, "Acme");
REQUIRE(expected.size() == 150);
// Every seed checks against one reference load, which GENERATE would repeat per seed.
for (unsigned seed = 0; seed < 8; ++ seed) {
CAPTURE(seed);
std::mt19937 rng(seed);
const auto shuffled = load(shuffled_after_dependencies(entries, rng));
CHECK(shuffled->error_count() == 0);
const auto actual = presets_for(shuffled->filaments, "Acme");
REQUIRE(actual.size() == expected.size());
for (size_t i = 0; i < actual.size(); ++ i)
CHECK(preset_deep_equal(*actual[i], *expected[i]));
}
}
TEST_CASE("a key misplaced into a vendor preset is reported and removed", "[VendorCache]")
{
InstallDirs dirs;
write_process_vendor(dirs.system, "Acme", {
{ "Acme template", R"({"type":"process","name":"Acme template","from":"system","instantiation":"false","wall_loops":"5"})" },
{ "Acme base", R"({"type":"process","name":"Acme base","from":"system","instantiation":"false","filament_cost":"5"})" },
{ "0.20mm Standard @Acme", process_json("0.20mm Standard @Acme",
R"("instantiation":"true","inherits":"Acme base","include":["Acme template"],"nozzle_temperature":["210"],)") } });
PresetBundle bundle;
bundle.load_vendor_configs_from_json(dirs.system.string(), "Acme", PresetBundle::LoadSystem,
ForwardCompatibilitySubstitutionRule::EnableSilent);
CHECK(bundle.error_count() == 2);
const Preset* preset = bundle.prints.find_preset("0.20mm Standard @Acme", false);
REQUIRE(preset != nullptr);
CHECK_FALSE(preset->config.has("filament_cost"));
CHECK_FALSE(preset->config.has("nozzle_temperature"));
CHECK(preset->config.opt_int("wall_loops") == 5);
}
TEST_CASE("repeated cache loads of one vendor produce the same presets", "[VendorCache]")
{
TempDir tmp;
const fs::path rsrc = tmp.path / "resources" / "profiles";
const fs::path user = tmp.path / "data" / PRESET_SYSTEM_DIR;
fs::create_directories(rsrc);
fs::create_directories(user);
write_wide_vendor_tree(user, "Acme", "1.0.0", 400);
ScopedDirs dirs(tmp.path / "data", tmp.path / "resources");
InstallDirs dirs;
write_filament_list(dirs.system, "Acme", write_layered_filaments(dirs.system, "Acme", 400));
PresetBundle seed;
seed.set_generate_vendor_caches(true);
seed.load_vendor_configs_from_json(user.string(), "Acme", PresetBundle::LoadSystem,
seed.load_vendor_configs_from_json(dirs.system.string(), "Acme", PresetBundle::LoadSystem,
ForwardCompatibilitySubstitutionRule::EnableSilent);
REQUIRE(fs::exists(user / "Acme.opc"));
fs::remove_all(user / "Acme");
REQUIRE(fs::exists(dirs.system / "Acme.opc"));
fs::remove_all(dirs.system / "Acme");
std::vector<PresetBundle> loads(3);
for (PresetBundle& bundle : loads)
bundle.load_vendor_configs_from_json(user.string(), "Acme", PresetBundle::LoadSystem,
bundle.load_vendor_configs_from_json(dirs.system.string(), "Acme", PresetBundle::LoadSystem,
ForwardCompatibilitySubstitutionRule::EnableSilent);
auto first = presets_for(loads[0].filaments, "Acme");
@@ -1820,7 +1925,9 @@ TEST_CASE("an include listed after the preset that names it is missing from the
CHECK(pr->config.opt_string("machine_start_gcode") != "G28 ; template");
const Preset* silk = from_cache.filaments.find_preset("Acme Silk PLA @0.4", false);
REQUIRE(silk != nullptr);
CHECK(silk->config.option<ConfigOptionFloats>("filament_max_volumetric_speed")->values == std::vector<double>{12.});
const auto& speed = silk->config.option<ConfigOptionFloats>("filament_max_volumetric_speed")->values;
REQUIRE(speed.size() == 1);
CHECK_THAT(speed.front(), WithinAbs(12., 1e-9));
}
TEST_CASE("a G-code template that states no instantiation is included, not loaded as a preset", "[VendorCache]")
@@ -1844,3 +1951,35 @@ TEST_CASE("a G-code template that states no instantiation is included, not loade
CHECK(pr->config.opt_string("machine_start_gcode") == "G28 ; template");
CHECK(presets_for(bundle.printers, "Acme").size() == 1);
}
TEST_CASE("a sub-file that fails to parse leaves the ones listed before it installed", "[VendorCache]")
{
InstallDirs dirs;
write_process_vendor(dirs.system, "Acme", {
{ "Acme A", process_json("Acme A", R"("instantiation":"true",)") },
{ "Acme B", "{not-json" },
{ "Acme C", process_json("Acme C", R"("instantiation":"true",)") } });
PresetBundle bundle;
CHECK_THROWS_AS(bundle.load_vendor_configs_from_json(dirs.system.string(), "Acme", PresetBundle::LoadSystem,
ForwardCompatibilitySubstitutionRule::EnableSilent),
ConfigurationError);
CHECK(bundle.prints.find_preset("Acme A", false) != nullptr);
CHECK(bundle.prints.find_preset("Acme C", false) == nullptr);
}
TEST_CASE("a load that fails to install partway counts no errors of the sub-files after it", "[VendorCache]")
{
InstallDirs dirs;
// B names a parent nobody defines. C states no instantiation, an error of its own.
write_process_vendor(dirs.system, "Acme", {
{ "Acme A", process_json("Acme A", R"("instantiation":"true",)") },
{ "Acme B", process_json("Acme B", R"("instantiation":"true","inherits":"Nobody",)") },
{ "Acme C", process_json("Acme C") } });
PresetBundle bundle;
CHECK_THROWS_AS(bundle.load_vendor_configs_from_json(dirs.system.string(), "Acme", PresetBundle::LoadSystem,
ForwardCompatibilitySubstitutionRule::EnableSilent),
ConfigurationError);
// B's missing parent and B's failed install.
CHECK(bundle.error_count() == 2);
CHECK(bundle.prints.find_preset("Acme A", false) != nullptr);
}