diff --git a/AGENTS.md b/AGENTS.md index 3b5620181b..f0a6fea64f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -17,7 +17,7 @@ cmake --build . --config %build_type% --target ALL_BUILD -- -m ## Testing -Catch2 framework. Tests in `tests/` directory. +Catch2 framework. Tests in `tests/`; see [tests/AGENTS.md](tests/AGENTS.md) for where a new test belongs and the conventions to follow. ```bash cd build && ctest --output-on-failure # all tests diff --git a/tests/AGENTS.md b/tests/AGENTS.md new file mode 100644 index 0000000000..c62a1d12ec --- /dev/null +++ b/tests/AGENTS.md @@ -0,0 +1,57 @@ +# Test suite rules + +Rules for writing tests under `tests/`. [CATCH2.md](CATCH2.md) is the Catch2 reference. Building and running the suites is covered on the wiki, at . + +## The suites + +- `libslic3r`: the core library. Geometry, meshes, file formats, config and presets, Clipper, algorithms, data structures. +- `fff_print`: the FFF slicing pipeline, from a `Model` plus config through `Print` and `PrintObject` to emitted G-code. +- `sla_print`: SLA support-tree and pad geometry, support-point generation, raycast. +- `libnest2d`: 2D nesting and packing. +- `slic3rutils`: the Python plugin system and its slicing-pipeline bindings. +- `filament_group`: filament-to-extruder grouping, checked against golden files. + +## Building and running + +Tests are off by default, so the build has to be told to include them. + +- Windows: `build_release_vs.bat tests`, then `ctest --test-dir build/tests -C Release` +- macOS: `./build_release_macos.sh -s -a arm64 -T`, which builds and runs them +- Linux: `./build_linux.sh -t`, then `ctest --test-dir build/tests` + +Rebuild a single suite with `cmake --build build --config Release --target _tests`. Visual Studio and Xcode are multi-configuration generators, so `ctest` needs `-C` there; on Linux it does not. + +## Where a test goes + +- Pick the suite by the production code the test exercises, not by how the test is written. +- A property of a class that holds with no `Print` involved belongs in `libslic3r`. Behavior that depends on print settings, or produces or consumes G-code or slicing state, belongs in `fff_print`. +- One file per subsystem, named `test_.cpp`. It owns every test for that subsystem, whether the test reads in-memory state or generated output. +- When you add a file, list it in that suite's `CMakeLists.txt` in the same change. + +## Use the existing helpers + +Check these before writing your own setup or output-parsing code. + +- `tests/test_utils.hpp` is shared by every suite. `load_model()` loads a mesh from `tests/data/`, and `ScopedTemporaryFile` gives a temp path that removes itself. +- `fff_print/test_helpers.hpp` builds and slices a `Print` and parses the emitted G-code. Read it before writing an fff_print test rather than assembling a `Print` by hand. +- The other suites have their own: `sla_print/sla_test_utils.hpp`, `libnest2d/libnest2d_test_utils.hpp`, `slic3rutils/plugin_test_utils.hpp`, `filament_group/fg_test_utils.hpp`. `libslic3r` has none and uses the shared header. +- Test data lives in `tests/data/` and is reached through the `TEST_DATA_DIR` define. Wrap it in `std::string(...)` before joining a path onto it. + +## Writing the test + +- Name the test case as a plain behavioral sentence in the present tense. No `Subsystem:` prefix. +- Tag it with the subsystem it covers, matching the file, in PascalCase. That tag is what people filter on, so every test needs one. +- Add further tags where they help: a narrower one to slice a large file (`[Rotcalip]`, `[Placer]`), a shared one for something spanning files (`[Python]`, `[H2C]`, `[Regression]`), or `[NotWorking]` / `[.]` to disable or hide a test. Say why in a comment if you disable or hide. +- Prefer a flat `TEST_CASE` per behavior, with `GENERATE` for parameterized cases. Reserve `SCENARIO` / `GIVEN` / `WHEN` / `THEN` for genuine shared setup that branches into a few close variations. +- Set the config keys your test depends on, and derive the expected values from what you set. A 20mm cube sliced at `layer_height` 2 is 10 layers, and the test should state both parts. If a number in your assertion comes from a key you never set, the test is also testing that default. +- Assert the defining property, not an incidental value. "Skirt present" or "at least 2 brim loops" survives a refactor; exact coordinates and byte counts do not. +- Name a regression test for the behavior it protects, never for an issue or PR number. +- When asserting on G-code, match the meaningful token such as `; skirt` rather than whole lines, whitespace or comment wording. Depend on ordering only when ordering is the contract. + +## Catch2 rules that cause real breakage + +- Never reuse a `SECTION` name inside a loop. Use `DYNAMIC_SECTION` so each iteration is unique. +- Never assert from a spawned thread. Catch2 assertions are not thread-safe. Collect results in the thread and assert on the main thread. +- Never combine conditions with `&&` or `||` inside one assertion. Split them so Catch2 can print both operands on failure. +- Compare floats with `WithinAbs` or `WithinRel`, never `==`. Prefer these over `Approx` in new tests. +- Keep tests self-contained: no shared state, green under `--order rand`. diff --git a/tests/CATCH2.md b/tests/CATCH2.md new file mode 100644 index 0000000000..1b528b3c01 --- /dev/null +++ b/tests/CATCH2.md @@ -0,0 +1,369 @@ +# Catch2 reference + +How to write and structure test code with Catch2 in OrcaSlicer. For where a test belongs, how to name and tag it, and how to build and run the suites, see [AGENTS.md](AGENTS.md). + +OrcaSlicer uses **Catch2 v3.11.0**, vendored in `tests/catch2/`. Include it with the single-header convenience include: + +```cpp +#include +``` + +## Critical rules + +These three mistakes produce undefined behavior, crashes, or useless failure output rather than a normal test failure. Avoid them everywhere. + +### 1. Never reuse a section name inside a loop + +A repeated `SECTION` name in a loop makes Catch2's section tracking behave unpredictably. Use `DYNAMIC_SECTION` so each iteration is unique. + +```cpp +// WRONG: same name every iteration +for (int i = 0; i < 3; ++i) + SECTION("Same name") { REQUIRE(i >= 0); } + +// CORRECT +for (int i = 0; i < 3; ++i) + DYNAMIC_SECTION("Section " << i) { REQUIRE(i >= 0); } +``` + +### 2. Assertions are not thread-safe + +Catch2 assertions are not thread-safe by default. A `REQUIRE`/`CHECK` from a spawned thread corrupts internal state or terminates the process. Collect results in the thread, assert on the main thread. + +```cpp +// WRONG +std::thread t([&]{ REQUIRE(work() == expected); }); + +// CORRECT +std::atomic passed{0}; +std::thread t([&]{ if (work() == expected) passed++; }); +t.join(); +REQUIRE(passed == 1); +``` + +> Catch2 v3.9.0+ has opt-in thread-safe assertions via `CATCH_CONFIG_EXPERIMENTAL_THREAD_SAFE_ASSERTIONS`. OrcaSlicer does not enable that flag, so assertions remain non-thread-safe. See [Thread safety](#thread-safety) below for the full rule list. + +### 3. Do not combine conditions with binary operators + +Catch2 decomposes a single comparison to show both operands on failure. A `&&`/`||` inside one assertion collapses to `false` with no values. Split it. + +```cpp +REQUIRE(a > 0 && b < 10); // WRONG: prints "false" +REQUIRE(a > 0); // CORRECT: each prints its operands +REQUIRE(b < 10); +``` + +## Test structure + +```cpp +#include +#include "libslic3r/Point.hpp" + +using namespace Slic3r; + +TEST_CASE("Behavioral description", "[SubsystemTag]") { + // ... +} +``` + +## Assertions + +```cpp +// Stop the test on failure +REQUIRE(expression); +REQUIRE_FALSE(expression); + +// Continue the test after failure (report all failures in the case) +CHECK(expression); +CHECK_FALSE(expression); + +// Record the result without failing (for assumptions that may be violated) +CHECK_NOFAIL(expression); +``` + +### Exceptions + +```cpp +REQUIRE_NOTHROW(function_call()); +REQUIRE_THROWS(risky_function()); +REQUIRE_THROWS_AS(function_call(), SpecificException); +REQUIRE_THROWS_WITH(function_call(), "Expected error message"); +REQUIRE_THROWS_MATCHES(function_call(), SpecificException, + Catch::Matchers::Message("contains this")); +``` + +Prefer these over a hand-rolled `try`/`catch` with a bool flag. + +## Matchers + +```cpp +#include + +// String matchers +using Catch::Matchers::StartsWith; +using Catch::Matchers::EndsWith; +using Catch::Matchers::ContainsSubstring; // v2's "Contains" no longer exists +using Catch::Matchers::Equals; +using Catch::Matchers::Matches; // regex + +REQUIRE_THAT(result, StartsWith("Expected prefix")); +REQUIRE_THAT(result, ContainsSubstring("middle part")); +REQUIRE_THAT(result, Matches(".*pattern.*")); + +// Float matchers - always prefer these over Approx +using Catch::Matchers::WithinAbs; +using Catch::Matchers::WithinRel; +using Catch::Matchers::WithinULP; + +REQUIRE_THAT(v, WithinAbs(expected, 0.001)); +REQUIRE_THAT(v, WithinRel(expected, 0.01)); +REQUIRE_THAT(v, WithinULP(expected, 4)); + +// Combine: relative OR absolute (useful when the value can be near zero) +REQUIRE_THAT(v, WithinRel(expected, 0.001) || WithinAbs(0.0, 0.000001)); +``` + +## Sections + +Each `SECTION` re-runs the enclosing `TEST_CASE` body from the top, so setup declared before the sections is fresh for each one. + +```cpp +TEST_CASE("Complex feature", "[Feature]") { + SomeObject obj; // rebuilt for every section + + SECTION("First scenario") { REQUIRE(obj.method1() == expected_value); } + SECTION("Second scenario") { REQUIRE(obj.method2() == other_expected); } +} +``` + +## BDD-style tests + +`SCENARIO` / `GIVEN` / `WHEN` / `THEN` are aliases for `TEST_CASE` and `SECTION` with prefixed names. New tests should prefer a flat `TEST_CASE`; reserve BDD for genuine shared setup that branches into closely related variations (see the test-design guidance in [AGENTS.md](AGENTS.md)). + +```cpp +SCENARIO("User performs an operation", "[UserStory]") { + GIVEN("A setup condition") { + GCodeWriter writer; + WHEN("The user acts") { + auto result = writer.some_operation(); + THEN("The outcome holds") { + REQUIRE(result.size() > 0); + } + } + } +} +``` + +## Generators + +```cpp +// Value list +auto v = GENERATE(1, 3, 5, 7, 11, 13); + +// Range +auto i = GENERATE(range(1, 10)); // 1..9 + +// From a variable (use GENERATE_REF / GENERATE_COPY for captured references) +std::vector values = {1, 2, 3, 4, 5}; +auto x = GENERATE_REF(from_range(values)); + +// Random +auto r = GENERATE(take(100, random(-1000, 1000))); +``` + +## Fixtures + +```cpp +class GeometryFixture { +public: + Point origin{0, 0}; + Point unit_x{1, 0}; +}; + +TEST_CASE_METHOD(GeometryFixture, "Point operations", "[Geometry]") { + REQUIRE(origin.distance_to(unit_x) == 1.0); +} +``` + +Persistent (`TEST_CASE_PERSISTENT_FIXTURE`, one instance for the whole case) and type-parameterized (`TEMPLATE_TEST_CASE_METHOD`) variants also exist; neither is used in the suite today. + +## Advanced features + +### Logging and control + +```cpp +INFO("Persists until end of scope"); +UNSCOPED_INFO("Survives beyond its scope"); // v2.7.0+ +CAPTURE(some_variable, another_var); // logs names and values + +WARN("Warns without failing"); +SKIP("Reason"); // marks the test skipped (v3.3.0+) +FAIL("Stops the test"); +SUCCEED("Explicit success marker"); +``` + +### Other macros + +Available but currently unused in the suite; see the upstream docs for details. + +- **Compile-time asserts**: `STATIC_REQUIRE` / `STATIC_CHECK` (v3.0.1+) check type traits at compile time. +- **Conditional blocks**: `CHECKED_IF` / `CHECKED_ELSE` record a branch condition without counting it as a failure. +- **Benchmarking** (v2.9.0+): `BENCHMARK("name") { return work(); };`, or `BENCHMARK_ADVANCED` when setup must be excluded from the measurement. + +## Usage patterns in OrcaSlicer + +Concrete shapes for exercising the codebase's own types. Test data is reached through the `TEST_DATA_DIR` define; always wrap it in `std::string(...)` before concatenating a path. + +```cpp +// Geometry, with epsilon tolerance +TEST_CASE("Line operations", "[Geometry]") { + Line line{{100000, 0}, {0, 0}}; + Line rotated(line); + rotated.rotate(0.9 * EPSILON, {0, 0}); + REQUIRE(line.parallel_to(rotated)); +} + +// Config from an ini +TEST_CASE("Config loading", "[Config]") { + DynamicPrintConfig config; + REQUIRE_NOTHROW(config.load_from_ini(std::string(TEST_DATA_DIR) + "/test_config/sample.ini", + ForwardCompatibilitySubstitutionRule::Disable)); + REQUIRE(config.has("layer_height")); +} + +// File I/O +TEST_CASE("STL file parsing", "[FileFormat]") { + TriangleMesh mesh; + REQUIRE_NOTHROW(mesh.ReadSTLFile((std::string(TEST_DATA_DIR) + "/test_stl/20mmbox.stl").c_str())); + REQUIRE_FALSE(mesh.empty()); + REQUIRE(mesh.volume() > 0); +} + +// G-code emission, matched by token (see test_gcodewriter.cpp) +TEST_CASE("z_hop lifts the nozzle", "[GCodeWriter]") { + GCodeWriter writer; + writer.set_extruders({0}); + writer.set_extruder(0); + writer.travel_to_z(10.0); + writer.config.z_hop.values = {1.0}; + REQUIRE_THAT(writer.eager_lift(LiftType::NormalLift), Catch::Matchers::ContainsSubstring("Z11")); +} +``` + +### Custom string conversions + +Give Catch2 a way to print a custom type on failure. The usual case is an `operator<<` overload: + +```cpp +std::ostream& operator<<(std::ostream& os, const Point& p) { + return os << "Point(" << p.x << ", " << p.y << ")"; +} +``` + +When you cannot add `operator<<`, specialize `Catch::StringMaker`. Enums can be registered with `CATCH_REGISTER_ENUM` (at global scope) and exceptions translated with `CATCH_TRANSLATE_EXCEPTION`; see the upstream docs for those. + +## Command line + +[AGENTS.md](AGENTS.md) covers the everyday commands (CTest, per-suite runs, tag filtering as CTest labels). The flags below are Catch2's own, available when you run a suite executable directly. + +```bash +# Filtering +suite_tests "[Geometry]" # by tag +suite_tests "*geometry*" # by name pattern +suite_tests "~[Performance]" # exclude a tag +suite_tests "[Geometry][Config],[Algorithm]" # (Geometry AND Config) OR Algorithm + +# Discovery +suite_tests --list-tests +suite_tests --list-tags +suite_tests --list-reporters + +# Debugging a failure +suite_tests --break # break into the debugger on failure +suite_tests --success # show passing assertions too +suite_tests --durations yes # per-test timing +suite_tests --abort # stop at the first failure +``` + +### Ordering and sharding + +Run in random order so tests stay independent. For parallel shards, all shards must share one seed. + +```bash +suite_tests --order rand --warn NoAssertions + +suite_tests --order rand --shard-index 0 --shard-count 4 --rng-seed 0xBEEF +suite_tests --order rand --shard-index 1 --shard-count 4 --rng-seed 0xBEEF +# ...one invocation per shard index +``` + +### Reporters + +```bash +suite_tests --reporter console # default, human-readable +suite_tests --reporter compact +suite_tests --reporter xml # Catch2 XML +suite_tests --reporter junit # JUnit XML (CI) +suite_tests --reporter tap +suite_tests --reporter console --reporter junit::out=results.xml # multiple at once +``` + +## Common pitfalls + +### Floating-point comparison + +Compare floats with the float matchers, never with `==`. New tests should prefer the `Within*` matchers over `Approx`. Many existing tests still use `Approx`, which works but is: + +- **Asymmetric**: `Approx(10).epsilon(0.1) != 11.1` yet `Approx(11.1).epsilon(0.1) == 10`. +- **Double-only**: all math is done in `double`, which misbehaves for `float` inputs. +- **Relative by default**: `Approx(0) == X` holds only for `X == 0`. + +Use `WithinAbs` near zero, `WithinRel` across magnitudes, `WithinULP` for the tightest check, or combine them. `Catch::StringMaker::precision = 15;` widens printed precision. + +### Exception testing + +Use `REQUIRE_THROWS` / `REQUIRE_THROWS_AS` rather than a `try`/`catch` with a bool flag. + +### Thread safety + +Assertions are not thread-safe (see [Critical rule 2](#2-assertions-are-not-thread-safe)). The full list of macros that must stay on the main thread: + +- **`REQUIRE` family**: throws in a spawned thread with no handler, terminating the process. +- **`CHECK` family**: can corrupt internal state. +- **`SKIP`, `FAIL`, `SUCCEED`**: unsafe even with v3's opt-in thread-safe assertions. +- **Message macros** (`INFO`, `CAPTURE`, `WARN`): unsafe. +- **`STATIC_REQUIRE` / `STATIC_CHECK`**: unsafe (rely on runtime registration). + +### Path handling + +Wrap `TEST_DATA_DIR` in `std::string(...)` before concatenating, or use `boost::filesystem`: + +```cpp +std::string path = std::string(TEST_DATA_DIR) + "/model.obj"; +``` + +### Memory + +Prefer RAII and smart pointers so a failing assertion cleans up automatically. + +## Compilation and performance flags + +```cpp +#define CATCH_CONFIG_FAST_COMPILE // ~20% faster compile, disables some features +#define CATCH_CONFIG_DISABLE_STRINGIFICATION // works around the VS2017 raw-string bug +#define CATCH_CONFIG_WINDOWS_CRTDBG // memory-leak detection (whole build) +``` + +The test build already defines `CATCH_CONFIG_FAST_COMPILE` (via `test_common` in `tests/CMakeLists.txt`). + +## Platform-specific workarounds + +- **MinGW/Cygwin** slow linking: build with `-fuse-ld=lld`. +- **Visual Studio 2017** raw-string-literal bug: define `CATCH_CONFIG_DISABLE_STRINGIFICATION` (disables expression stringification). +- **Visual Studio 2022** spaceship operator: `REQUIRE((a <=> b) == 0)` may not compile; use clang-cl or avoid `<=>` in assertions. + +## Catch2 v3 notes + +Available on v3.11.0: `SKIP()` (v3.3.0+), opt-in thread-safe assertions (v3.9.0+, not enabled here), built-in `BENCHMARK`, multiple simultaneous reporters (v3.0.1+), `STATIC_CHECK` (v3.0.1+), built-in sharding (`--shard-*`). + +Two behavior notes: the string matcher is `ContainsSubstring` (v2's `Contains` is gone), and a section is re-run when a later sibling section fails (unchanged from v2). diff --git a/tests/CLAUDE.md b/tests/CLAUDE.md index dcbeab7cbd..43c994c2d3 100644 --- a/tests/CLAUDE.md +++ b/tests/CLAUDE.md @@ -1,964 +1 @@ -# CLAUDE.md - Testing Guide for OrcaSlicer - -This guide provides comprehensive instructions for Claude Code when writing, maintaining, and understanding tests in the OrcaSlicer codebase. - -> **Adding or organizing `fff_print` tests?** See -> [fff_print/README.md](fff_print/README.md) for where a test belongs and how to -> name it. This guide covers Catch2 mechanics; that README is the suite's -> organizing contract. - -## ⚠️ CRITICAL RULES - MUST FOLLOW - -### 1. **SECTIONS IN LOOPS - NEVER REUSE NAMES** -❌ **WRONG**: Will cause unpredictable behavior -```cpp -TEST_CASE("Bad loop sections") { - for (int i = 0; i < 3; ++i) { - SECTION("Same name") { // WRONG! Same name used multiple times - REQUIRE(i >= 0); - } - } -} -``` - -✅ **CORRECT**: Use DYNAMIC_SECTION or incorporate counter -```cpp -TEST_CASE("Good loop sections") { - for (int i = 0; i < 3; ++i) { - DYNAMIC_SECTION("Section " << i) { // Unique name per iteration - REQUIRE(i >= 0); - } - } -} -``` - -### 2. **THREAD SAFETY - ASSERTIONS ARE NOT THREAD-SAFE** -❌ **WRONG**: Will cause undefined behavior or crashes -```cpp -TEST_CASE("Multi-threaded test") { - std::vector threads; - for (int i = 0; i < 4; ++i) { - threads.emplace_back([]() { - REQUIRE(some_calculation() == expected); // NOT THREAD-SAFE! - }); - } -} -``` - -✅ **CORRECT**: Synchronize results, test on main thread -```cpp -TEST_CASE("Multi-threaded test") { - std::vector threads; - std::atomic passed{0}; - - for (int i = 0; i < 4; ++i) { - threads.emplace_back([&passed]() { - if (some_calculation() == expected) { - passed++; - } - }); - } - - for (auto& t : threads) t.join(); - REQUIRE(passed == 4); // Test results on main thread -} -``` - -### 3. **EXPRESSION DECOMPOSITION - AVOID BINARY OPERATORS** -❌ **WRONG**: Cannot decompose properly -```cpp -REQUIRE(a > 0 && b < 10); // Shows "false" on failure, not individual values -``` - -✅ **CORRECT**: Split into separate assertions -```cpp -REQUIRE(a > 0); -REQUIRE(b < 10); // Each shows individual values on failure -``` - -### 4. **FLOATING POINT - NEVER USE APPROX** -❌ **WRONG**: Approx is deprecated and asymmetric -```cpp -REQUIRE(calculated_value == Catch::Approx(expected)); // Deprecated! -``` - -✅ **CORRECT**: Use floating point matchers -```cpp -REQUIRE_THAT(calculated_value, WithinAbs(expected, 0.001)); -REQUIRE_THAT(calculated_value, WithinRel(expected, 0.01)); // 1% tolerance -REQUIRE_THAT(calculated_value, WithinULP(expected, 4)); // 4 ULPs apart -``` - -### 5. **TEST ORDERING - ALWAYS USE RANDOM ORDER** -✅ **REQUIRED**: For CI/CD and development -```bash -# Essential flags for running tests -./tests --order rand --warn NoAssertions - -# For test sharding (parallel execution), share random seed -./tests --order rand --shard-index 0 --shard-count 3 --rng-seed 0xBEEF -./tests --order rand --shard-index 1 --shard-count 3 --rng-seed 0xBEEF -./tests --order rand --shard-index 2 --shard-count 3 --rng-seed 0xBEEF -``` - -## Overview of OrcaSlicer's Testing Framework - -OrcaSlicer uses **Catch2 v3** (currently v3.11.0, vendored in `tests/catch2/`) as its primary testing framework. The test suite is organized into several modules that mirror the project's architectural components: - -> **Note**: Test files include the framework via `#include ` (the v3 single-header convenience include). All v3 features described in this guide are available. - -### Test Structure -``` -tests/ -├── CMakeLists.txt # Main test configuration -├── catch_main.hpp # Custom test reporter -├── libslic3r/ # Core library tests (21 test files) -├── fff_print/ # FFF printing tests (12 test files) -├── sla_print/ # SLA printing tests (4 test files) -├── libnest2d/ # 2D nesting tests -├── slic3rutils/ # Utility tests -├── data/ # Test data files and meshes -└── catch2/ # Catch2 framework files -``` - -### Build Integration -- Tests are built using CMake with `catch_discover_tests()` integration -- Each test module creates a separate executable (e.g., `libslic3r_tests`, `fff_print_tests`) -- Test data directory is available via `TEST_DATA_DIR` preprocessor definition -- Custom verbose console reporter provides detailed test output - -## Test Suite Organization - -### libslic3r Tests -Core slicing engine tests covering: -- **Geometry operations**: Points, polygons, lines, Voronoi diagrams -- **File formats**: STL, 3MF, AMF parsing and validation -- **Algorithms**: Clipper operations, mesh boolean operations, optimization -- **Configuration**: Print settings validation and parsing -- **Utilities**: String processing, time utilities, data structures - -### fff_print Tests -Fused Filament Fabrication specific tests: -- **G-code generation**: Writer functionality, cooling, lift/unlift -- **Slicing algorithms**: Layer generation, infill patterns -- **Print mechanics**: Flow calculations, extrusion, support material -- **Model processing**: Print objects, skirt/brim generation - -### sla_print Tests -Stereolithography specific tests: -- **SLA print processing**: Layer curing, support generation -- **Raycast operations**: Light path calculations -- **Test utilities**: SLA-specific helper functions - -## Writing New Tests - Best Practices - -### File Organization -1. **Naming Convention**: `test_.cpp` (e.g., `test_geometry.cpp`) -2. **Header Structure**: Include `` first, then relevant headers -3. **Namespace Usage**: Use `using namespace Slic3r;` for convenience -4. **File Placement**: Add to appropriate test directory and update CMakeLists.txt - -### Test Naming and Structure -```cpp -#include -#include "libslic3r/Point.hpp" - -using namespace Slic3r; - -TEST_CASE("Feature description", "[category_tag]") { - // Test implementation -} -``` - -### Tagging System -Use descriptive tags for test categorization: -- `[Geometry]` - Geometric operations and calculations -- `[GCodeWriter]` - G-code generation functionality -- `[Config]` - Configuration and settings tests -- `[FileFormat]` - File I/O operations (STL, 3MF, etc.) -- `[Algorithm]` - Core algorithms and processing -- `[Performance]` - Performance benchmarks (if applicable) - -## Catch2 Features Guide - -### Basic Assertions -```cpp -// Primary assertions - stop test on failure -REQUIRE(expression); -REQUIRE_FALSE(expression); - -// Continuing assertions - continue test after failure -CHECK(expression); -CHECK_FALSE(expression); - -// Non-failing checks - record result but don't fail test -CHECK_NOFAIL(expression); // Useful for assumptions that might be violated -``` - -### Exception Testing -```cpp -// Verify no exception is thrown -REQUIRE_NOTHROW(function_call()); - -// Verify any exception is thrown -REQUIRE_THROWS(risky_function()); - -// Verify specific exception type -REQUIRE_THROWS_AS(function_call(), SpecificException); - -// Verify exception message -REQUIRE_THROWS_WITH(function_call(), "Expected error message"); - -// Verify exception with matchers (for partial matching) -REQUIRE_THROWS_MATCHES(function_call(), SpecificException, - Catch::Matchers::Message("contains this")); -``` - -### Complex Assertions with Matchers -```cpp -#include - -// String matchers -using Catch::Matchers::StartsWith; -using Catch::Matchers::EndsWith; -using Catch::Matchers::ContainsSubstring; -using Catch::Matchers::Equals; -using Catch::Matchers::Matches; // Regex matching - -REQUIRE_THAT(result_string, StartsWith("Expected prefix")); -REQUIRE_THAT(result_string, ContainsSubstring("middle part")); -REQUIRE_THAT(result_string, Matches(".*pattern.*")); - -// Floating point matchers - ALWAYS use these instead of Approx! -using Catch::Matchers::WithinAbs; -using Catch::Matchers::WithinRel; -using Catch::Matchers::WithinULP; - -REQUIRE_THAT(float_value, WithinAbs(expected, 0.001)); // Absolute tolerance -REQUIRE_THAT(float_value, WithinRel(expected, 0.01)); // Relative tolerance (1%) -REQUIRE_THAT(float_value, WithinULP(expected, 4)); // ULP difference (requires IEEE-754) - -// Combining matchers -REQUIRE_THAT(value, WithinRel(expected, 0.001) || WithinAbs(0.0, 0.000001)); -``` - -### Sections for Test Organization -```cpp -TEST_CASE("Complex feature testing", "[Feature]") { - // Common setup code - SomeObject obj; - - SECTION("First scenario") { - // Specific test case - REQUIRE(obj.method1() == expected_value); - } - - SECTION("Second scenario") { - // Another test case with same setup - REQUIRE(obj.method2() == other_expected); - } -} -``` - -### BDD-Style Tests -Use for complex scenarios and user story testing: - -> **Note**: BDD macros are aliases for TEST_CASE and SECTION with prefixed names -```cpp -SCENARIO("User performs complex operation", "[UserStory]") { - GIVEN("A specific setup condition") { - GCodeWriter writer; - // Setup code - - WHEN("User performs action") { - auto result = writer.some_operation(); - - THEN("Expected outcome occurs") { - REQUIRE(result.size() > 0); - - AND_WHEN("Follow-up action occurs") { - auto next_result = writer.next_operation(); - - THEN("Final outcome is correct") { - REQUIRE(next_result == expected); - } - } - } - } - } -} -``` - -### Data Generators for Parameterized Tests -```cpp -TEST_CASE("Function works with various inputs", "[Algorithm]") { - auto test_value = GENERATE(1, 3, 5, 7, 11, 13); - - REQUIRE(is_odd(test_value)); - REQUIRE(test_value > 0); -} - -// Range-based generators -TEST_CASE("Range testing", "[Algorithm]") { - auto i = GENERATE(range(1, 10)); // 1 to 9 - REQUIRE(process_value(i) > i); -} - -// Using variables in generators (requires GENERATE_COPY or GENERATE_REF) -TEST_CASE("Generator with variables", "[Algorithm]") { - std::vector values = {1, 2, 3, 4, 5}; - auto test_value = GENERATE_REF(from_range(values)); // Use GENERATE_REF for references - - REQUIRE(test_value > 0); -} - -// Custom generators -TEST_CASE("Random values", "[Algorithm]") { - auto random_int = GENERATE(take(100, random(-1000, 1000))); // 100 random values - REQUIRE(process_random_value(random_int)); -} -``` - -### Test Fixtures -```cpp -class GeometryFixture { -public: - Point origin{0, 0}; - Point unit_x{1, 0}; - Point unit_y{0, 1}; - - mutable double tolerance = EPSILON; // Use mutable for data that might change -}; - -// Standard fixture - new instance per test run -TEST_CASE_METHOD(GeometryFixture, "Point operations", "[Geometry]") { - REQUIRE(origin.distance_to(unit_x) == 1.0); -} - -// Persistent fixture - single instance for entire test case (v3.2.0+) -TEST_CASE_PERSISTENT_FIXTURE(GeometryFixture, "Persistent operations", "[Geometry]") { - static int call_count = 0; - ++call_count; - INFO("This fixture persists across sections, call: " << call_count); - - SECTION("First section") { - REQUIRE(origin.distance_to(unit_x) == 1.0); - } - - SECTION("Second section") { - REQUIRE(origin.distance_to(unit_y) == 1.0); - // call_count will be 2 here with persistent fixture - } -} - -// Template fixtures for type-parameterized tests -template -class NumericFixture { -public: - T zero = T{0}; - T one = T{1}; -}; - -TEMPLATE_TEST_CASE_METHOD(NumericFixture, "Numeric operations", "[Template]", int, float, double) { - REQUIRE(TestType{} == this->zero); - REQUIRE(TestType{1} == this->one); -} -``` - -### Advanced Testing Features - -#### Logging and Information Macros -```cpp -TEST_CASE("Advanced logging", "[Logging]") { - INFO("This info persists until end of scope"); - - SECTION("Section A") { - INFO("Section A specific info"); - CAPTURE(some_variable, another_var); // Captures variable names and values - CHECK(some_condition); - } - - SECTION("Section B") { - UNSCOPED_INFO("This survives beyond its scope"); // v2.7.0+ - CHECK(other_condition); - } -} - -// Warning and explicit control -TEST_CASE("Explicit test control", "[Control]") { - WARN("This warns but doesn't fail the test"); - - if (precondition_not_met) { - SKIP("Reason"); // Marks the test as skipped (v3.3.0+, available) - return; - } - - if (critical_failure) { - FAIL("Critical condition failed"); // Fails and stops test - } - - SUCCEED("Reached successful completion"); // Explicit success marker -} -``` - -#### Static Assertions (Compile-time Testing) -```cpp -TEST_CASE("Compile-time checks", "[Static]") { - STATIC_REQUIRE(sizeof(int) >= 4); // Checked at compile time - STATIC_REQUIRE_FALSE(std::is_void_v); - - // For traits and template metaprogramming - STATIC_CHECK(std::is_trivially_copyable_v); // v3.0.1+ -} -``` - -#### Conditional Testing -```cpp -TEST_CASE("Conditional blocks", "[Conditional]") { - int value = get_test_value(); - - // These record the expression but don't count as test failures (v3.0.1+) - CHECKED_IF(value > 0) { - // This block runs if value > 0 - REQUIRE(value <= 100); - } CHECKED_ELSE(value > 0) { - // This block runs if value <= 0 - REQUIRE(value >= -100); - } -} -``` - -#### Benchmarking (v2.9.0+) -```cpp -TEST_CASE("Performance testing", "[Benchmark]") { - // Simple benchmarking - BENCHMARK("Algorithm performance") { - return expensive_algorithm(); - }; - - // Advanced benchmarking with setup - BENCHMARK_ADVANCED("Advanced benchmark")(Catch::Benchmark::Chronometer meter) { - std::vector data = setup_test_data(); // Setup not measured - - meter.measure([&] { - return process_data(data); // Only this is measured - }); - }; -} -``` - -## OrcaSlicer-Specific Testing Patterns - -### Geometry Testing -```cpp -TEST_CASE("Line operations", "[Geometry]") { - Line line{{100000, 0}, {0, 0}}; - Line parallel{{200000, 0}, {0, 0}}; - - REQUIRE(line.parallel_to(line)); - REQUIRE(line.parallel_to(parallel)); - - // Test with epsilon tolerance - Line rotated(parallel); - rotated.rotate(0.9 * EPSILON, {0, 0}); - REQUIRE(line.parallel_to(rotated)); -} -``` - -### Configuration Testing -```cpp -TEST_CASE("Config loading", "[Config]") { - DynamicPrintConfig config; - std::string config_path = std::string(TEST_DATA_DIR) + "/test_config/sample.ini"; - - REQUIRE_NOTHROW(config.load_from_ini(config_path)); - REQUIRE(config.has("layer_height")); -} -``` - -### File I/O Testing -```cpp -TEST_CASE("STL file parsing", "[FileFormat]") { - std::string stl_path = std::string(TEST_DATA_DIR) + "/test_stl/20mmbox.stl"; - - TriangleMesh mesh; - REQUIRE_NOTHROW(mesh.ReadSTLFile(stl_path.c_str())); - REQUIRE(!mesh.empty()); - REQUIRE(mesh.volume() > 0); -} -``` - -### G-code Generation Testing -```cpp -TEST_CASE("G-code writer functionality", "[GCodeWriter]") { - GCodeWriter writer; - - // Load test configuration - std::string config_path = std::string(TEST_DATA_DIR) + "/fff_print_tests/test_config.ini"; - writer.config.load(config_path, ForwardCompatibilitySubstitutionRule::Disable); - - // Test specific G-code generation - std::string result = writer.lift(); - REQUIRE(!result.empty()); - REQUIRE_THAT(result, Catch::Matchers::ContainsSubstring("G1")); -} -``` - -### Performance Testing Patterns -```cpp -TEST_CASE("Algorithm performance", "[Performance][Algorithm]") { - // Large test data - std::vector points = generate_large_point_set(10000); - - // Time the operation (manual timing example; the BENCHMARK macro is also available) - auto start = std::chrono::high_resolution_clock::now(); - auto result = convex_hull(points); - auto end = std::chrono::high_resolution_clock::now(); - - auto duration = std::chrono::duration_cast(end - start); - - REQUIRE(result.size() > 0); - REQUIRE(duration.count() < 1000); // Should complete in < 1 second -} -``` - -### Custom String Conversions - -#### For Custom Types -```cpp -// Method 1: operator<< overload (preferred) -std::ostream& operator<<(std::ostream& os, const Point& point) { - os << "Point(" << point.x << ", " << point.y << ")"; - return os; -} - -// Method 2: StringMaker specialization -namespace Catch { - template<> - struct StringMaker { - static std::string convert(const MyCustomType& value) { - return "MyCustomType{" + std::to_string(value.data) + "}"; - } - }; -} - -// Method 3: Enum registration (v2.8.0+) -enum class Status { Ready, Processing, Complete, Error }; - -// Must be at global scope! -CATCH_REGISTER_ENUM(Status, Status::Ready, Status::Processing, Status::Complete, Status::Error); - -// Method 4: Exception translation -CATCH_TRANSLATE_EXCEPTION(MyCustomException const& ex) { - return "MyCustomException: " + std::string(ex.what()); -} - -// Method 5: Disable range iteration for problematic types -namespace Catch { - template<> - struct is_range { - static const bool value = false; - }; -} -``` - -## Running and Debugging Tests - -### Building Tests -```bash -# Build all tests -cd build && make - -# Build specific test suite -cd build && make libslic3r_tests - -# Build and run tests -cd build && make && ctest -``` - -### Running Tests - -#### Essential Test Execution Patterns -```bash -# REQUIRED: Random order with assertion warnings (best practice) -cd build && ./tests/libslic3r/libslic3r_tests --order rand --warn NoAssertions - -# Run all tests with verbose output via CTest -cd build && ctest --output-on-failure - -# Run specific test suite with best practices -cd build && ./tests/libslic3r/libslic3r_tests --order rand --warn NoAssertions - -# Filter tests with specific tags -cd build && ./tests/libslic3r/libslic3r_tests "[Geometry]" --order rand - -# Filter by test name patterns -cd build && ./tests/libslic3r/libslic3r_tests "*geometry*" --order rand - -# Exclude tests (negation) -cd build && ./tests/libslic3r/libslic3r_tests "~[Performance]" --order rand - -# Combine filters: (Geometry AND Config) OR Algorithm -cd build && ./tests/libslic3r/libslic3r_tests "[Geometry][Config],[Algorithm]" --order rand - -# List available tests, tags, and reporters -cd build && ./tests/libslic3r/libslic3r_tests --list-tests -cd build && ./tests/libslic3r/libslic3r_tests --list-tags -cd build && ./tests/libslic3r/libslic3r_tests --list-reporters - -# Debug failing tests -cd build && ./tests/libslic3r/libslic3r_tests --break # Break into debugger on failure -cd build && ./tests/libslic3r/libslic3r_tests --success # Show passing tests too -cd build && ./tests/libslic3r/libslic3r_tests --durations yes # Show timing info - -# Abort on first failure -cd build && ./tests/libslic3r/libslic3r_tests --abort - -# Test sharding for parallel execution (MUST share random seed) -cd build && ./tests/libslic3r/libslic3r_tests --order rand --shard-index 0 --shard-count 4 --rng-seed 0xBEEF & -cd build && ./tests/libslic3r/libslic3r_tests --order rand --shard-index 1 --shard-count 4 --rng-seed 0xBEEF & -cd build && ./tests/libslic3r/libslic3r_tests --order rand --shard-index 2 --shard-count 4 --rng-seed 0xBEEF & -cd build && ./tests/libslic3r/libslic3r_tests --order rand --shard-index 3 --shard-count 4 --rng-seed 0xBEEF & -wait # Wait for all to complete -``` - -#### Reporter Options for CI Integration -```bash -# Different output formats for CI systems -cd build && ./tests/libslic3r/libslic3r_tests --reporter console # Default human-readable -cd build && ./tests/libslic3r/libslic3r_tests --reporter compact # Minimal output -cd build && ./tests/libslic3r/libslic3r_tests --reporter xml # Catch2 XML format -cd build && ./tests/libslic3r/libslic3r_tests --reporter junit # JUnit XML (widely supported) -cd build && ./tests/libslic3r/libslic3r_tests --reporter tap # Test Anything Protocol -cd build && ./tests/libslic3r/libslic3r_tests --reporter teamcity # TeamCity integration -cd build && ./tests/libslic3r/libslic3r_tests --reporter sonarqube # SonarQube integration -cd build && ./tests/libslic3r/libslic3r_tests --reporter automake # Automake integration - -# Multiple reporters simultaneously (if supported) -cd build && ./tests/libslic3r/libslic3r_tests --reporter console --reporter junit::out=results.xml -``` - -### Test Output Control -The custom `VerboseConsoleReporter` provides enhanced output: -- Test case start/end notifications with timing -- Section execution tracking -- Color-coded success/failure indicators -- Duration reporting for performance analysis - -## Test Data Management - -### Using TEST_DATA_DIR -All test data is accessible via the `TEST_DATA_DIR` preprocessor definition: - -```cpp -std::string mesh_path = std::string(TEST_DATA_DIR) + "/20mm_cube.obj"; -std::string config_path = std::string(TEST_DATA_DIR) + "/test_config/printer.ini"; -``` - -### Available Test Assets - -#### 3D Models -- **Basic shapes**: `20mm_cube.obj`, `pyramid.obj`, `sphere.obj` -- **Complex geometry**: `extruder_idler.obj`, `ipadstand.obj`, `bridge.obj` -- **Edge cases**: `cube_with_hole.obj`, `sloping_hole.obj`, `small_dorito.obj` - -#### File Format Tests -- **STL variants**: ASCII/binary, different line endings, Unicode names -- **3MF files**: Multi-material, complex assemblies -- **Configuration files**: Various printer/material profiles - -#### Test Utilities -The `Test` namespace provides helper functions: -```cpp -using namespace Slic3r::Test; - -// Load standard test meshes -TriangleMesh mesh = mesh(TestMesh::cube_20x20x20); - -// Standard test configurations -DynamicPrintConfig config = config(TestConfig::PLA_default); -``` - -## Common Pitfalls and Solutions - -### Floating-Point Comparisons - -> **CRITICAL**: Never use Approx - it's deprecated due to asymmetry and other issues - -❌ **Incorrect**: -```cpp -REQUIRE(calculated_volume == expected_volume); // Exact equality -REQUIRE(calculated_volume == Catch::Approx(expected)); // Deprecated! Asymmetric! -``` - -✅ **Correct**: Always use floating point matchers -```cpp -// Absolute tolerance - good when values are near zero -REQUIRE_THAT(calculated_volume, WithinAbs(expected_volume, 0.001)); - -// Relative tolerance - good for values with different magnitudes -REQUIRE_THAT(calculated_volume, WithinRel(expected_volume, 0.01)); // 1% tolerance - -// ULP (Units in Last Place) - most precise, requires IEEE-754 -REQUIRE_THAT(calculated_volume, WithinULP(expected_volume, 4)); - -// Combined approach - relative OR absolute -REQUIRE_THAT(calculated_volume, - WithinRel(expected_volume, 0.001) || WithinAbs(0.0, 0.000001)); - -// Precision control for output -Catch::StringMaker::precision = 15; // Show more decimal places -``` - -### Why Approx is Problematic: -- **Asymmetric**: `Approx(10).epsilon(0.1) != 11.1` but `Approx(11.1).epsilon(0.1) == 10` -- **Double-only**: All computation done in `double`, causes issues with `float` inputs -- **Default behavior**: Only uses relative comparison, so `Approx(0) == X` only works for `X == 0` - -### Path Handling -❌ **Incorrect**: -```cpp -std::string path = TEST_DATA_DIR + "/model.obj"; // May have path separator issues -``` - -✅ **Correct**: -```cpp -std::string path = std::string(TEST_DATA_DIR) + "/model.obj"; -// or use boost::filesystem for complex path operations -``` - -### Exception Testing -❌ **Incorrect**: -```cpp -bool threw_exception = false; -try { - risky_function(); -} catch (...) { - threw_exception = true; -} -REQUIRE(threw_exception); -``` - -✅ **Correct**: -```cpp -REQUIRE_THROWS(risky_function()); -// or for specific exceptions -REQUIRE_THROWS_AS(risky_function(), SpecificException); -``` - -### Thread Safety - -⚠️ **CRITICAL**: Catch2 assertions are **NOT thread-safe** by default! - -> **Note**: Catch2 v3.9.0+ has opt-in thread-safe assertions via `CATCH_CONFIG_EXPERIMENTAL_THREAD_SAFE_ASSERTIONS`. OrcaSlicer is on v3.11.0 but does not enable this flag, so assertions remain non-thread-safe by default. - -❌ **Incorrect**: Will cause undefined behavior or crashes -```cpp -std::thread t([&]() { - REQUIRE(threaded_operation() == expected); // NOT THREAD-SAFE! - CHECK(other_operation()); // NOT THREAD-SAFE! -}); -``` - -✅ **Correct**: Collect results, assert on main thread -```cpp -std::atomic success{false}; -std::atomic error_count{0}; - -std::thread t([&]() { - // Do work in thread, collect results - bool result1 = (threaded_operation() == expected); - bool result2 = other_operation(); - - if (result1 && result2) { - success = true; - } else { - error_count++; - } -}); - -t.join(); - -// Assert results on main thread -REQUIRE(success); -REQUIRE(error_count == 0); -``` - -#### Thread Safety Rules: -- **REQUIRE family**: Would terminate process in spawned threads (throws exception with no try-catch) -- **CHECK family**: Not thread-safe, can corrupt internal state -- **SKIP, FAIL, SUCCEED**: Not thread-safe even with v3 thread-safe assertions -- **Message macros**: INFO, CAPTURE, WARN - not thread-safe -- **STATIC_REQUIRE/CHECK**: Not thread-safe (relies on runtime registration) - -### Memory Management -Use RAII and smart pointers in tests: -```cpp -TEST_CASE("Resource management", "[Memory]") { - auto model = std::make_unique(); - // Automatic cleanup on test completion/failure - - REQUIRE(model->objects.empty()); -} -``` - -## Performance Considerations - -### Compilation Optimizations -```cpp -// In CMakeLists.txt or as preprocessor definition -#define CATCH_CONFIG_FAST_COMPILE // 20% faster compilation, disables some features - -// For faster test iteration during development -#define CATCH_CONFIG_DISABLE_STRINGIFICATION // Workaround for VS2017 raw string bug -``` - -### Runtime Performance -```cpp -TEST_CASE("Performance-sensitive test", "[Performance]") { - // Manual timing example (Catch2's built-in BENCHMARK macro is also available) - auto start = std::chrono::high_resolution_clock::now(); - - auto result = expensive_operation(); - - auto end = std::chrono::high_resolution_clock::now(); - auto duration = std::chrono::duration_cast(end - start); - - REQUIRE(result.is_valid()); - REQUIRE(duration.count() < 1000); // Should complete in < 1 second - - INFO("Operation took " << duration.count() << "ms"); -} -``` - -### Memory Leak Detection -```cpp -// For Windows builds - detects memory leaks -#define CATCH_CONFIG_WINDOWS_CRTDBG // Must be defined for whole build -``` - -## Integration with CMake - -### Adding New Test Files -1. Create test file: `test_new_feature.cpp` -2. Add to appropriate `CMakeLists.txt`: -```cmake -add_executable(${_TEST_NAME}_tests - ${_TEST_NAME}_tests.cpp - test_existing_feature.cpp - test_new_feature.cpp # Add here -) -``` - -### Advanced Test Discovery -```cmake -# Basic test discovery -catch_discover_tests(${_TEST_NAME}_tests TEST_PREFIX "${_TEST_NAME}: ") - -# Advanced test discovery with customization -catch_discover_tests(${_TEST_NAME}_tests - TEST_PREFIX "${_TEST_NAME}: " - TEST_SUFFIX " (auto)" - WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} - EXTRA_ARGS --order rand --warn NoAssertions - PROPERTIES - TIMEOUT 300 - LABELS "unit;core" - DISCOVERY_MODE PRE_TEST # or POST_BUILD - REPORTER junit - OUTPUT_DIR ${CMAKE_BINARY_DIR}/test-results - OUTPUT_PREFIX "results_" - OUTPUT_SUFFIX ".xml" -) - -# Test sharding for parallel execution -include(CatchShardTests) # If available -catch_shard_tests(${_TEST_NAME}_tests - SHARD_COUNT 4 - TEST_PREFIX "${_TEST_NAME}_shard: " -) -``` - -### Conditional Test Compilation -```cmake -# Feature-dependent tests -if (TARGET OpenVDB::openvdb) - target_sources(${_TEST_NAME}_tests PRIVATE test_hollowing.cpp) -endif() - -# Platform-specific tests -if(WIN32) - target_sources(${_TEST_NAME}_tests PRIVATE test_windows_specific.cpp) -elseif(UNIX) - target_sources(${_TEST_NAME}_tests PRIVATE test_unix_specific.cpp) -endif() - -# Compiler-specific workarounds -if(MSVC) - target_compile_definitions(${_TEST_NAME}_tests PRIVATE CATCH_CONFIG_DISABLE_STRINGIFICATION) -endif() - -# Fast compile mode for development -if(CMAKE_BUILD_TYPE STREQUAL "Debug") - target_compile_definitions(${_TEST_NAME}_tests PRIVATE CATCH_CONFIG_FAST_COMPILE) -endif() -``` - -## Known Issues and Workarounds - -### Platform-Specific Issues -```cpp -// MinGW/CygWin slow linking workaround -// Use: -fuse-ld=lld flag to speed up linking significantly - -// Visual Studio 2017 raw string literal bug -#define CATCH_CONFIG_DISABLE_STRINGIFICATION -// This disables expression stringification but works around the compiler bug - -// Visual Studio 2022 spaceship operator issue -// REQUIRE((a <=> b) == 0); // May not compile with MSVC -// Workaround: use clang-cl or avoid spaceship in assertions - -// QNX/VxWorks C stdlib issues -#include // Use C++ headers -std::foo_function(); // Always call qualified -// NOT: #include and foo_function(); -``` - -### Catch2 v3 Features Available -```cpp -// OrcaSlicer is on Catch2 v3.11.0 - all of these ARE available: -// SKIP() macro - v3.3.0+ -// Opt-in thread-safe assertions - v3.9.0+ (NOT enabled here; see Thread Safety) -// Built-in BENCHMARK / BENCHMARK_ADVANCED - v3.x -// testCasePartial events - v3.0.1+ -// Multiple reporters simultaneously - v3.0.1+ -// STATIC_CHECK macro - v3.0.1+ -// Built-in test sharding (--shard-*) - v3.x - -// v3 notes to remember: -// - String matcher is "ContainsSubstring" (v2's "Contains" no longer exists) -// - Sections can still be re-run if a later section fails (unchanged from v2) -``` - -### Test Organization Best Practices - -#### Project Structure Rules -1. **1:1 correspondence**: One test binary per library/module -2. **Hidden tests**: Use `[.]` or `[!benchmark]` tags for tests that shouldn't run by default -3. **Tag hierarchy**: Use consistent tagging scheme across the project -4. **File naming**: Follow `test_.cpp` pattern - -#### CI/CD Integration -```bash -# Essential CI test command -./tests --order rand --warn NoAssertions --reporter junit::out=results.xml - -# For coverage analysis -./tests --order rand --warn NoAssertions --reporter console --success - -# For performance tracking -./tests --order rand --warn NoAssertions --durations yes -``` - -This comprehensive guide ensures robust, maintainable, and efficient testing practices for OrcaSlicer development with Claude Code, incorporating all critical knowledge from the official Catch2 documentation. \ No newline at end of file +@AGENTS.md diff --git a/tests/README.md b/tests/README.md new file mode 100644 index 0000000000..bd490ae38f --- /dev/null +++ b/tests/README.md @@ -0,0 +1,8 @@ +# OrcaSlicer tests + +Building, running and writing tests is documented on the wiki, under [How to Test](https://www.orcaslicer.com/wiki/developer_reference/how_to_test.html). + +Two files here rather than there, because coding agents only read what is in the repository: + +- [AGENTS.md](AGENTS.md) is the same guidance in short form, and is what an agent working under `tests/` picks up. +- [CATCH2.md](CATCH2.md) is the Catch2 reference, including the mistakes that break a test at runtime. diff --git a/tests/fff_print/README.md b/tests/fff_print/README.md deleted file mode 100644 index c8ed85e367..0000000000 --- a/tests/fff_print/README.md +++ /dev/null @@ -1,93 +0,0 @@ -# fff_print test suite - -Component- and pipeline-level tests for FFF slicing: the path from a `Model` plus config, through `Print` / `PrintObject`, to emitted G-code. - -For Catch2 mechanics (assertions, generators, matchers, random ordering, thread-safety), see [../CLAUDE.md](../CLAUDE.md). This document is the organizing contract for the suite: where a test goes, and how it is named. - -## Organizing principle - -**One file per subsystem. A subsystem is usually a single production class (`Flow`, `PrintObject`), but may be a cohesive feature that spans several (skirt/brim lives in `Brim.cpp`, `Print.cpp`, and `GCode.cpp`). That file owns every test for the subsystem: in-memory-state assertions and emitted-G-code assertions alike.** - -A test's home is decided by *what production code it exercises*, never by *how it observes the result*. A skirt test that inspects `print.skirt()` and one that greps the G-code for `; skirt` live in the same file. - -If you touched code in a subsystem, its test file is where your test goes. If a subsystem has no file yet, add `test_.cpp` and list it in `CMakeLists.txt`. - -## File ownership - -### Building blocks (one class, exercised through its API) - -| File | Source (`src/libslic3r/`) | Covers | -|---|---|---| -| `test_trianglemesh` | `TriangleMesh.{c,h}pp` | mesh stats, transforms, slicing, split/merge/cut | -| `test_flow` | `Flow.{c,h}pp` | extrusion width / area math | -| `test_extrusion_entity` | `ExtrusionEntity.{c,h}pp` | extrusion-collection geometry | -| `test_gcodewriter` | `GCodeWriter.{c,h}pp`, `GCode.cpp` | low-level G-code emit primitives, origin | -| `test_model` | `Model.{c,h}pp` | object / volume / instance construction | - -### Slicing pipeline (build a `Print`, then assert state or G-code) - -| File | Source (`src/libslic3r/`) | Covers | -|---|---|---| -| `test_printobject` | `PrintObject.cpp` | layer heights, perimeter generation | -| `test_fill` | `Fill/` | infill patterns and infill G-code | -| `test_skirt_brim` | `Brim.cpp`, `Print.cpp` | skirt/brim loop counts, grouping, brim ears, emission order | -| `test_support_material` | `Support/` | support & raft layers, contact distance | -| `test_cooling` | `GCode/CoolingBuffer.cpp` | fan control, speed-marker consumption | -| `test_multifilament` | `GCode/ToolOrdering.cpp` | per-feature and per-object filament routing | -| `test_print` | `Print.{c,h}pp` | `validate()`, solid-shell behavior, sequential printing, custom G-code & config comments, default-slice smoke | - -Paths are under `src/libslic3r/`. A trailing `/` is a directory of related files; otherwise it is a single class. `{c,h}pp` means the `.cpp`/`.hpp` pair. - -## Naming and tags - -- **File:** `test_.cpp`. -- **Test name:** a plain behavioral sentence, present tense, stating the contract the test pins down. No `Subsystem:` prefix (the tag carries that). - - Good: `TEST_CASE("Skirt is emitted once per layer it spans", "[SkirtBrim]")` - - Avoid: `TEST_CASE("Print: Skirt generation", "[Print]")` -- **Tags:** - - Exactly one **subsystem** tag, PascalCase, matching the file (`[SkirtBrim]`, `[PrintObject]`, `[Fill]`). This is the grouping / filter key. - - Optional **cross-cutting** tags for a concern that genuinely spans files (`[validate]`, `[Regression]`). - - **Status** tags: `[NotWorking]` marks a test disabled for a known, documented reason; CI excludes it via `~[NotWorking]` (it does not hide itself). Use `[.]` to hide a test from default runs entirely. Either way, say why in a one-line comment. - -## Test style - -Prefer a flat `TEST_CASE` per behavior, with `GENERATE` for parameterized cases and shared setup factored into helpers. The test name carries the behavior, so the BDD scaffolding is usually redundant. Reserve `SCENARIO` / `GIVEN` / `WHEN` / `THEN` for a test with genuine shared setup that branches into a few closely related variations, and never let a `SCENARIO` accumulate unrelated `WHEN`s: that grab-bag is what this contract exists to prevent (and it hides failures behind a single coarse test case). - -## Robust tests - -A test should fail only when the behavior it names breaks, not from unrelated changes (the "change-detector" anti-pattern). Test behavior, not incidentals, and aim for one reason to fail. Concretely: - -- Don't depend on or assert defaults: set the config keys the behavior needs, and derive expected values from those inputs (a 20mm cube at 0.2mm = 100 layers), not from a default that may change. -- Assert the defining property, not an incidental value: prefer "skirt present", "at least 2 brim loops", or "ears vs none" over exact coordinates, extrusion amounts, line counts, or byte sizes. -- Compare floats with a tolerance (`WithinAbs` / `WithinRel`), never `==`. -- Match the meaningful G-code token (`; skirt`), not whole lines, whitespace, or comment wording. -- Rely on ordering only when it is the contract (as `role_sequence` does). -- Keep tests self-contained: no shared state, green under `--order rand`. - -## Helpers - -Reuse these instead of building a `Print` or parsing G-code by hand. - -- **Global** (`tests/test_utils.hpp`, available to every suite): - - `load_model("file.obj")`: load a `TriangleMesh` from `tests/data/`. - - `ScopedTemporaryFile`: an RAII temp-file path, removed on scope exit. -- **Suite harness** (`fff_print/test_helpers.{hpp,cpp}`): - - Build and run: `init_print(...)`, `init_and_process_print(...)`, `slice(...)` (returns the G-code string), and `gcode(print)`. - - Two-cube placement: `slice_two_cubes_arranged(...)` (arranger-positioned), and `place_two_cubes_apart(...)` / `slice_two_cubes_apart(...)` (a fixed gap, not arranged). - - Meshes: `cube(size)` / `make_cube(...)` for simple shapes; the `TestMesh` enum with `mesh(...)` for named fixtures. - - G-code analysis: `layers_with_role(gcode, role)`, `max_z(gcode)`, `role_passes(gcode, role)`, `role_sequence(gcode, roles)`. Subsystem-specific checks stay local (for example `brim_count` in `test_skirt_brim`). - -Promote a helper into the suite harness when it is a general test primitive (not tied to one subsystem's logic), even if only one file uses it today; keep genuinely subsystem-specific helpers local (file-static). Reuse potential, not current usage count, is the test. - -## Adding a test (checklist) - -1. Find the subsystem's file in the tables; create `test_.cpp` if missing. -2. Build the print with a harness helper; set only the config keys the behavior needs. -3. Assert the behavior, in-memory or via parsed G-code, whichever is clearest. -4. Name it as a behavioral sentence and tag it `[Subsystem]`. -5. For a bug fix, add the regression test in the owning file. Name it for the behavior it protects; the test must stand on its own without relying on an external issue or PR for meaning. - -## Running - - ctest --test-dir build/tests/fff_print - build/tests/fff_print//fff_print_tests --order rand "~[NotWorking]"