juli/OrcaSlicer
1
1
Fork 0
mirror of https://github.com/OrcaSlicer/OrcaSlicer.git synced 2026-07-15 15:03:49 +00:00
Code Issues Packages Projects Releases Wiki Activity

test: fix flaky k-medoids goldens added with H2C/A2L support (#14773)

Browse Source 
The filament-group golden harness landed with H2C/A2L support (#14685). Its
"FilamentGroup golden regression" / stress_66 case fails intermittently on
Windows x64, on main and on unrelated PRs alike. The test depends on how fast the
runner is.

The k-medoids clustering these goldens exercise is an anytime search bounded by a
3 second wall clock. Every restart is seeded from its own index, so nothing about
it is random. What varies is how many restarts fit in the budget, and the best
cost is a minimum over completed restarts, so a slower runner is never better.
Grading a score produced that way measures the machine as much as the code.

Add a ClusteringBudget struct and let the tests set it. The defaults are the
current 3 seconds and 30 restarts, so slicing behavior is unchanged. A
non-positive timeout removes the wall clock and bounds the search by restart
count alone.

The goldens are then graded under a fixed budget of four restarts, where every
one of them reaches the BambuStudio reference within 3%, so the score becomes a
property of the code. This retires the machine-specific 125103 lock on stress_66.

The default wall-clock path keeps its own test, asserting the grouping is valid
and the search does not run away. It makes no score assertion, because under a
wall clock that number is not a property of the code.

The golden test also checks the run fits in ten times the default wall clock.
Slicing quality depends on how many restarts fit in the budget, so a search an
order of magnitude slower would degrade real groupings while a fixed-budget score
gate stayed green.

The 3% tolerance stays as the parity allowance against the goldens. It also
covers a small spread across standard libraries: the k-medoids search seeds each
restart with std::shuffle, whose algorithm the C++ standard leaves unspecified,
so libstdc++, libc++ and the MSVC STL permute the same seed differently, start
from different medoids, and settle on slightly different groupings, about 3e-4
apart and only on the goldens heavy enough to reach the k-medoids search.
This commit is contained in:
Kris Austin
2026-07-15 03:59:55 -05:00
committed by GitHub
parent f626382209
commit d500ba4e9e
4 changed files with 75 additions and 33 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
src/libslic3r/FilamentGroup.cpp
View File

@@ -352,8 +352,7 @@ namespace Slic3r
int k,
const std::vector<unsigned int>& used_filaments,
const std::unordered_map<int, std::vector<int>>& unplaceable_limits,
int* cost,
int timeout_ms)
int* cost)
{
auto distance_evaluator = std::make_shared<FlushDistanceEvaluator>(ctx.model_info.flush_matrix, used_filaments, ctx.model_info.layer_filaments);
KMediods PAM(k, (int)used_filaments.size(), distance_evaluator, ctx.machine_info.master_extruder_id);
@@ -369,7 +368,7 @@ namespace Slic3r
}
PAM.set_cluster_group_size(cluster_size_limit);
PAM.do_clustering(ctx, timeout_ms, 30);
PAM.do_clustering(ctx, m_clustering_budget);
m_memoryed_heap = PAM.get_memoryed_groups();
@@ -793,7 +792,7 @@ namespace Slic3r
2.1 In each cluster, make the point that minimizes the sum of distances within the cluster the medoid
2.2 Reassign each point to the cluster defined by the closest medoid determined in the previous step
*/
void KMediods::do_clustering(const FilamentGroupContext &context, int timeout_ms, int retry)
void KMediods::do_clustering(const FilamentGroupContext& context, const ClusteringBudget& budget)
{
FlushTimeMachine T;
T.time_machine_start();
@@ -817,7 +816,11 @@ namespace Slic3r
double best_cluster_cost = std::numeric_limits<double>::max();
int retry_count = 0;
while (retry_count < retry && T.time_machine_end() < timeout_ms) {
// Run at least one restart; otherwise every filament would stay in the default group.
const int retry = std::max(1, budget.max_restarts);
auto within_budget = [&]() { return budget.timeout_ms <= 0 || T.time_machine_end() < budget.timeout_ms; };
while (retry_count < retry && within_budget()) {
std::vector<int> curr_cluster_centers = init_cluster_center(m_placeable_limits, m_unplaceable_limits, m_max_cluster_size, m_cluster_group_size, retry_count);
std::vector<int> curr_cluster_labels = assign_cluster_label(curr_cluster_centers, m_placeable_limits, m_unplaceable_limits, m_max_cluster_size, m_cluster_group_size);
double curr_cluster_cost = evaluate_labels(curr_cluster_labels);
@@ -826,7 +829,7 @@ namespace Slic3r
update_memoryed_groups(g, memory_threshold, memoryed_groups);
bool mediods_changed = true;
while (mediods_changed && T.time_machine_end() < timeout_ms) {
while (mediods_changed && within_budget()) {
mediods_changed = false;
double best_swap_cost = curr_cluster_cost;
int best_swap_cluster = -1;
@@ -889,7 +892,7 @@ namespace Slic3r
if (estimated < ENUM_THRESHOLD)
result = calc_group_by_enum(k, used_filaments, unplaceable_limits, cost);
else
result = calc_group_by_kmedoids(k, used_filaments, unplaceable_limits, cost, 3000);
result = calc_group_by_kmedoids(k, used_filaments, unplaceable_limits, cost);
change_memoryed_heaps_to_arrays(m_memoryed_heap, ctx.group_info.total_filament_num, used_filaments, m_memoryed_groups);

17
src/libslic3r/FilamentGroup.hpp
View File

@@ -142,6 +142,16 @@ namespace Slic3r
FilamentGroupContext::SpeedInfo m_speed_info;
};
// Search budget for the k-medoids clustering, an anytime search. Each restart is seeded from its
// own index, so what it returns depends on how many restarts complete before the clock expires,
// and therefore on the speed of the machine. A timeout_ms <= 0 removes the clock and bounds the
// search by max_restarts alone.
struct ClusteringBudget
{
int timeout_ms = 3000;
int max_restarts = 30;
};
class FilamentGroup
{
using MemoryedGroup = FilamentGroupUtils::MemoryedGroup;
@@ -149,6 +159,8 @@ namespace Slic3r
public:
explicit FilamentGroup(const FilamentGroupContext& ctx_) :ctx(ctx_) {}
public:
void set_clustering_budget(const ClusteringBudget& budget) { m_clustering_budget = budget; }
std::vector<int> calc_filament_group(int * cost = nullptr);
std::vector<std::vector<int>> get_memoryed_groups()const { return m_memoryed_groups; }
@@ -162,7 +174,7 @@ namespace Slic3r
std::vector<int> calc_group_by_enum(int k, const std::vector<unsigned int>& used_filaments,
const std::unordered_map<int, std::vector<int>>& unplaceable_limits, int* cost = nullptr);
std::vector<int> calc_group_by_kmedoids(int k, const std::vector<unsigned int>& used_filaments,
const std::unordered_map<int, std::vector<int>>& unplaceable_limits, int* cost = nullptr, int timeout_ms = 500);
const std::unordered_map<int, std::vector<int>>& unplaceable_limits, int* cost = nullptr);
std::map<int, int> rebuild_unprintables(const std::vector<unsigned int>& used_filaments, const std::map<int,int>& extruder_unprintables);
std::unordered_map<int, std::vector<int>> rebuild_nozzle_unprintables(const std::vector<unsigned int>& used_filaments, const std::unordered_map<int, std::vector<int>>& extruder_unprintables, const std::vector<int>& filament_volume_map);
@@ -175,6 +187,7 @@ namespace Slic3r
FilamentGroupContext ctx;
MemoryedGroupHeap m_memoryed_heap;
std::vector<std::vector<int>> m_memoryed_groups;
ClusteringBudget m_clustering_budget;
public:
std::optional<std::function<bool(int, std::vector<int>&)>> get_custom_seq;
};
@@ -220,7 +233,7 @@ namespace Slic3r
void set_memory_threshold(double threshold) { memory_threshold = threshold; }
MemoryedGroupHeap get_memoryed_groups()const { return memoryed_groups; }
void do_clustering(const FilamentGroupContext& context, int timeout_ms = 100, int retry = 10);
void do_clustering(const FilamentGroupContext& context, const ClusteringBudget& budget);
std::vector<int> get_cluster_labels()const { return m_cluster_labels; }
protected:

4
tests/filament_group/fg_test_evaluator.hpp
View File

@@ -210,13 +210,15 @@ inline FullEvalResult full_evaluate_map(const FilamentGroupContext& ctx,
return result;
}
inline TestResult run_and_evaluate(const FilamentGroupContext& ctx) {
inline TestResult run_and_evaluate(const FilamentGroupContext& ctx,
const ClusteringBudget& budget = {}) {
TestResult result;
auto start = std::chrono::high_resolution_clock::now();
int algo_cost = 0;
FilamentGroup fg(ctx);
fg.set_clustering_budget(budget);
result.filament_map = fg.calc_filament_group(&algo_cost);
auto end = std::chrono::high_resolution_clock::now();

70
tests/filament_group/filament_group_regression_main.cpp
View File

@@ -211,19 +211,23 @@ static std::vector<PropertySpec>& get_property_specs() {
return specs;
}
// Orca: a small number of config_c "stress" goldens run the nozzle-centric kmedoids clustering path,
// which is bounded by a 3000 ms wall-clock budget (FilamentGroup.cpp calc_group_by_kmedoids). On
// slower hardware the clustering explores fewer restarts and lands on a deterministically-worse-but-
// valid grouping than the stored golden. We regression-lock those against Orca's own deterministic
// score (bit-stable across runs on this machine — verified twice) so the gate stays green while the
// divergence is documented; every other golden is a true parity gate at 3% tolerance.
static std::optional<double> orca_locked_base_score(const std::string& stem) {
if (stem == "stress_66") return 125103.0; // config_c 15-filament kmedoids case; golden 117843
return std::nullopt;
}
// Under the default wall clock the result depends on how fast the machine is (see ClusteringBudget),
// so the goldens are graded under a fixed budget instead. Two restarts is the fewest that reaches
// parity with the reference on every golden, stress_79 being the last to get there. Four leaves
// margin, since the search follows a different path on each standard library (see below).
static constexpr ClusteringBudget FIXED_SEARCH_BUDGET{
/*timeout_ms*/ 0, // no wall clock
/*max_restarts*/ 4};
// ============ Layer 1: Golden Regression (all configs) ============
// Graded against the BambuStudio golden the harness was ported from, one-directional at 3%.
//
// The tolerance is a parity allowance, and it also covers a small spread across standard libraries.
// The k-medoids search seeds each restart with std::shuffle, whose algorithm the C++ standard leaves
// unspecified, so libstdc++, libc++ and the MSVC STL permute the same seed differently, start from
// different medoids, and settle on slightly different groupings, about 3e-4 apart on either side of
// the reference, and only on the goldens heavy enough to reach the k-medoids search.
TEST_CASE("FilamentGroup golden regression", "[filament_group][golden]") {
auto files = get_golden_files();
if (files.empty()) {
@@ -238,29 +242,49 @@ TEST_CASE("FilamentGroup golden regression", "[filament_group][golden]") {
auto tc = load_test_case(file_path);
REQUIRE(tc.base_result.has_value());
auto result = run_and_evaluate(tc.context);
auto result = run_and_evaluate(tc.context, FIXED_SEARCH_BUDGET);
auto eval = full_evaluate_map(tc.context, result.filament_map);
auto& base = *tc.base_result;
// Reference score: the stored golden by default; Orca's deterministic score for the
// documented heuristic-divergent config_c stress golden (see orca_locked_base_score).
std::string stem = fs::path(file_path).stem().string();
double base_score = base.full_score;
if (auto locked = orca_locked_base_score(stem))
base_score = *locked;
INFO("Case: " << tc.metadata.id);
INFO("Reference score: " << base_score << " (BBS golden " << base.full_score << ")");
INFO("Golden score: " << base.full_score);
INFO("Actual score: " << eval.full_score);
INFO("Flush cost: " << eval.flush_cost << " (BBS golden " << base.flush_cost << ")");
INFO("Flush cost: " << eval.flush_cost << " (golden " << base.flush_cost << ")");
INFO("Elapsed: " << result.elapsed_ms << " ms");
int tolerance = std::max(50, (int)(base_score * 0.03));
int tolerance = std::max(50, (int)(base.full_score * 0.03));
REQUIRE(result.constraints_ok);
REQUIRE(eval.full_score <= base_score + tolerance);
// RelWithDebInfo runaway guard; the Release-calibrated 20 s limit is raised for the slower build.
REQUIRE(eval.full_score <= base.full_score + tolerance);
// A slower search still scores the same above, since it searches just as far, but in slicing
// it would mean fewer restarts fit in the wall clock and so worse groupings. Loose on
// purpose, so it never becomes a proxy for how loaded the runner is.
const double throughput_ceiling_ms = 10.0 * ClusteringBudget{}.timeout_ms;
REQUIRE(result.elapsed_ms < throughput_ceiling_ms);
}
}
// Covers the path real slicing takes, under the default wall clock. The score there depends on the
// runner rather than on the code (see FIXED_SEARCH_BUDGET), so the only things worth asserting are
// that the grouping comes back valid and that the search terminates.
TEST_CASE("FilamentGroup returns a valid grouping under the default budget", "[filament_group][budget]") {
auto files = get_golden_files();
REQUIRE(!files.empty());
auto file_path = GENERATE_REF(from_range(files));
DYNAMIC_SECTION("Golden: " << fs::path(file_path).stem().string()) {
auto tc = load_test_case(file_path);
auto result = run_and_evaluate(tc.context); // the default budget, as real slicing runs it
INFO("Case: " << tc.metadata.id);
INFO("Elapsed: " << result.elapsed_ms << " ms");
REQUIRE(result.constraints_ok);
// A hang guard. The clock is only checked between swaps, so a sweep can overshoot.
REQUIRE(result.elapsed_ms < 40000.0);
}
}