Tag the two known-broken tests [NotWorking] so CI stops being red on every commit

This fork's Unit Tests job failed on every single commit, because CI runs the
whole ctest suite including the two cases tagged [known-broken] that
scripts/kernel-test.sh has always excluded locally. A job that is red
unconditionally is worse than no job: it trains everyone to ignore it, so the
next genuine regression arrives invisible.

No workflow change was needed. scripts/run_unit_tests.sh already passes
-LE NotWorking, and tests/CMakeLists.txt registers Catch2 tags as ctest labels
via catch_discover_tests(ADD_TAGS_AS_LABELS) — so the exclusion upstream
already ships works as soon as the cases carry the tag. Verified against the
built test tree: 337 tests unfiltered, 335 with -LE NotWorking, i.e. exactly
these two dropped and nothing else.

The second cause recorded in the issue, the test-reporter step failing with
"Resource not accessible by integration: 403" on a fork, is already fixed
upstream: the Publish Test Results step now carries continue-on-error: true.

The comment these cases carried claimed CI kept the bugs visible by reporting
them forever. That is now false and was never a good mechanism anyway, so
visibility moves to the tracker: snaporca-tkz for the solver SIGABRT, and
snaporca-kzy, filed now, for the thread groove volume. Neither is fixed;
neither is forgotten.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Tommaso Bianchi
2026-07-26 08:13:08 +02:00
co-authored by Claude Opus 5
parent b25335e1b3
commit f13f2876f6
2 changed files with 16 additions and 7 deletions
+3 -1
View File
@@ -34,7 +34,9 @@ VOL="${BUILD_VOL:-orcacad_kerneltest}"
# Two pre-existing failures are excluded by default (see [known-broken] in
# test_caddocument.cpp): one of them SIGABRTs inside the vendored solver and takes the
# whole process down, so without this exclusion a green run is simply unreachable and the
# suite stops after ~12 of 32 cases. CI runs everything and still reports both.
# suite stops after ~12 of 32 cases. Both are also tagged [NotWorking], the ctest label
# the mainline fork's CI excludes, so they no longer redden its Unit Tests job on every
# commit; they stay tracked as snaporca-tkz and snaporca-kzy rather than as CI noise.
TAGS="${TAGS:-[CadDocument]~[known-broken]}"
HOST=""
+13 -6
View File
@@ -611,9 +611,15 @@ TEST_CASE("entity constraints: point-on-line positions a centre onto an axis", "
// [known-broken]: the "tangent line to circle" SECTION below aborts inside the vendored
// solver (slvs/dsc.h FindById, "Cannot find handle"). SIGABRT is fatal to the whole Catch2
// process, so this one case takes the entire suite down with it and no later test runs.
// Tagged so the delegated dev loop (scripts/kernel-test.sh) can exclude it and still reach
// a green baseline; CI runs every test and keeps reporting it, so the bug stays visible.
TEST_CASE("entity constraints: tangent/midpoint/symmetric/angle", "[CadDocument][known-broken]")
// [known-broken] lets the delegated dev loop (scripts/kernel-test.sh) exclude it and still
// reach a green baseline. [NotWorking] is the ctest label the mainline-based fork's CI already
// excludes (run_unit_tests.sh passes -LE NotWorking, and catch_discover_tests registers Catch2
// tags as ctest labels via ADD_TAGS_AS_LABELS) — so its Unit Tests job stops being red on every
// single commit, which was training everyone to ignore it and would have hidden the next real
// regression. The tag is inert on the Snapmaker fork, which has neither that runner nor
// catch_discover_tests; the two files are kept identical because they are ported wholesale
// between the forks. The bug stays visible in the tracker instead — see snaporca-tkz.
TEST_CASE("entity constraints: tangent/midpoint/symmetric/angle", "[CadDocument][known-broken][NotWorking]")
{
using R = SketchPointRole;
using T = SketchConstraintType;
@@ -962,9 +968,10 @@ TEST_CASE("extrude taper + up-to-face distance", "[CadDocument]")
}
// [known-broken]: pre-existing failure, the cut groove volume does not meet the asserted
// threshold. Excluded from the delegated dev loop so a green run means "I broke nothing";
// CI still runs and reports it.
TEST_CASE("internal thread cuts a visible groove into the bore wall", "[CadDocument][known-broken]")
// threshold. Excluded from the delegated dev loop so a green run means "I broke nothing",
// and [NotWorking] excludes it from CI's ctest gate for the same reason as the case above.
// Tracked in the issue tracker instead — see snaporca-kzy.
TEST_CASE("internal thread cuts a visible groove into the bore wall", "[CadDocument][known-broken][NotWorking]")
{
using namespace Slic3r;
SketchPlane xy = SketchPlane::XY();