Compare commits

...

5 Commits

Author SHA1 Message Date
HanifKoh
f21f062ded Cache compiled objects between CI runs (#15611)
# Description
<!--
> Please provide a summary of the changes made in this PR. Include
details such as:
  > * What issue does this PR address or fix?
  > * What new features or enhancements does this PR introduce?
> * Are there any breaking changes or dependencies that need to be
considered?
-->

Every CI build leg compiles the whole tree from scratch: 42 to 57
minutes of each build job, on every push and every pull request, roughly
200 runs a week. This PR caches the compiled objects with ccache so that
a run only compiles what changed since the last push to main. With a
warm cache the compile steps take 1 to 4 minutes on all six legs and a
pull-request run finishes in about 30 minutes instead of 75.

Three prerequisites landed last week and made this measurable: #15537
took `GIT_COMMIT_HASH` off the compile line, #15552 made a build without
the precompiled header work on Windows, and #15501 stopped the Flatpak
job from rebuilding its dependencies.

## Changes

### Compiler cache in `build_orca.yml`

Each build leg (Linux x86_64/aarch64, Windows x64/arm64, macOS
arm64/x86_64) restores a cache entry keyed by that leg, compiles through
`ccache` via `CMAKE_<LANG>_COMPILER_LAUNCHER`, and prints its hit
statistics at the end of the job. The macOS universal combine does not
compile and is left out.

Who writes the cache is the important part. Cache entries are immutable
and a restore always takes the newest matching one, so every save is a
new entry that is never read again once a newer one exists. Therefore:

- **Pushes save.** After a successful save, the older entries for the
same leg on the same ref are deleted, so a branch holds exactly one
entry per leg. The save comes first, so a failed save leaves the
previous entry in place.
- **Pull requests restore only.** They read main's entries (GitHub lets
a PR read the base branch's caches) and keep nothing. Saving from PRs
would add about 6 GB per run that no other run can read.

The store is therefore a flat ~7 GB (one entry per leg: Linux ~1 GB,
Windows ~2 GB, macOS ~0.6 GB), not a growing one. The
`hendrikmuhs/ccache-action` only installs and configures ccache; restore
and save go through `actions/cache` with one path string, because the
cache service only matches entries saved under the identical path and
the action spells it differently on Windows. A failed ccache install
falls back to an uncached build rather than failing the job.

### Precompiled header off when the cache is on

With `SLIC3R_PCH` left on, a warm cache hit only 19 % of compiles: Clang
stamps the PCH with the build time, CMake does not pass
`-fno-pch-timestamp`, and everything that includes the PCH (libslic3r
and libslic3r_gui, ~750 files) missed every run. `build_linux.sh -p`
exists for exactly this reason. The workflow now exports
`ORCA_EXTRA_BUILD_ARGS=-DSLIC3R_PCH=OFF` whenever ccache is enabled,
which brings the warm hit rate to 98.4–98.9 %.

The cost is on cold compiles, which are 25–60 % slower than today's PCH
build (ccache preprocesses every miss before compiling it, and the miss
compiles without PCH). Main pays this once after an image update or a
wide header change; PRs pay it only for the files their change
invalidates. A change to a header included by half the tree
(`PrintConfig.hpp`, `Preset.hpp`, `Model.hpp`) lands a run at 1.2–1.9×
today's time. `ccache`'s depend mode would remove the preprocessor pass
and is the natural follow-up.

### Includes the precompiled header was supplying on macOS

A build without PCH had never been tried on macOS. Three files used what
`pchheader.hpp` happened to include: `LocalesUtils.cpp` needs
`<sstream>` and `<iomanip>`, and `AmsMappingPopup.cpp` /
`PhysicalPrinterDialog.cpp` need `<wx/tooltip.h>`. libstdc++ and the GTK
wx port pull these in transitively; libc++ and the Cocoa port do not.
This is the macOS counterpart of #15552 and is worth merging on its own.

### `ORCA_EXTRA_BUILD_ARGS` pass-through

`build_linux.sh` already forwarded this variable to the slicer
configure. `build_release_macos.sh` now reads it into an array
(shellcheck-clean), and `build_release_vs.bat` appends it on both
configure lines, so CI can add a CMake option without editing three
scripts.

## Behaviour reviewers should know about

- **Main-only cache writes need `actions: write`** on the workflow token
to delete the previous entry. The default token already has it (the
nightly deploy steps write with it), so no `permissions:` block was
added. A fork PR's read-only token never reaches the delete step.
- **A runner image update cold-starts the cache** as configured, because
ccache keys the compiler by its mtime and every image rebuild reinstalls
it. Images updated 20260819 → 20260828 during this work, about every one
to two weeks. Keying on the compiler version string (`compiler_check`)
would avoid that; left as a follow-up since it changes every hash.
- **What is now the critical path:** the two Flatpak jobs (46–66 min,
untouched here), the orca-test-repo regression suite run inline in the
Linux job (7 min), and NSIS/PDB/MSIX packaging on Windows (6 min). Those
are the next wins.
- **Open question:** CI still drives `build_release_vs.bat`. #15552 gave
`build_win.bat` a `--cache ccache --no-pch` option; moving the Windows
job onto it would replace the batch-file change here.

# Screenshots/Recordings/Graphs

<!--
> Please attach relevant screenshots to showcase the UI changes.
> Please attach images that can help explain the changes.
-->

Compile step of each build leg, minutes. Main's numbers are from run
34324625046.

| Leg | main | cold, PCH on | warm, PCH on | cold, PCH off | warm, PCH
off | ~50 % of headers changed | 5 source files changed |
|---|---|---|---|---|---|---|---|
| Linux x86_64 | 48 | ~75 | (19 % hits) | ~110 | **2.6** | 91.3 (452/928
misses) | 3.4 |
| Linux aarch64 | 41.7 | 53.4 | 52.2 (178/927 hits) | 58.8 | **2.5** |
55.8 (452/928) | 2.9 |
| Windows x64 | 57 | 85.5 | — | ~105 | **2.4** | 76.8 (449/974) | 2.5 |
| Windows arm64 | ~45 | 64.4 | — | ~78 | **4.3** | 59.6 (450/974) | 4.2
|
| macOS arm64 | 51 | ~71 | — | — | **0.8** | 79.7 (453/947) | 0.9 |
| macOS x86_64 | ~43 | 70.2 | — | — | **1.0** | 68.1 (411/742) | 1.0 |

Warm hit rates: 98.4–98.9 % on every leg; the 11–14 misses are what any
commit changes (version stamp and its includers). The "50 % of headers"
column is a real event: #15251 and #15416 merged into main between two
runs, changing 20 headers that reach 453 of 870 translation units.

Whole run, before and after (a pull-request run; wall clock to the last
non-Flatpak job):

| Job | main (run 34324625046) | warm cache (run 34444305385) | what
remains |
|---|---|---|---|
| Windows arm64 | 50.0 | 15.7 | compile 4.3, NSIS 3.5, cache save 1.6,
deps restore 1.1, cache restore 1.0 |
| Windows x64 | 67.4 | 13.2 | NSIS 3.2, PDB 2.6, compile 2.4, MSIX 0.5 |
| Linux x86_64 | 57.5 | 12.6 | orca-test-repo regression 7.6, compile
2.6 |
| macOS x86_64 | 46.9 | 6.1 | free disk space 2.3, compile 1.0 |
| Linux aarch64 | 43.9 | 4.6 | compile 2.5, apt 0.9 |
| macOS arm64 | 54.9 | 4.4 | free disk space 1.7, compile 0.8 |
| macOS universal | 7.7 | 2.2 | signing and notarisation only on main |
| Flatpak x86_64 / aarch64 | 66.6 / 46.5 | unchanged | full compile
inside flatpak-builder |
| **Wall clock** | **75 min** | **31 min** (Flatpak excluded; 66 with
it) | macOS runner queueing now exceeds job time |

Cache storage: one generation per leg is 400–680 MB compressed at PCH
on, 0.6–2 GB at PCH off; six legs ≈ 7 GB. Without the delete step, 21
main pushes a week would hold ~80 GB of entries that are never read.

## Tests

<!--
> Please describe the tests that you have conducted to verify the
changes made in this PR.
-->

- Thirteen CI runs on this PR, one change per run, with the ccache
statistics printed by every leg: cold (34336272234), warm with PCH
(34346737197), cold and warm without PCH (34352209577, 34364791732), the
macOS include fixes (34435044411, 34435968806 with `ninja -k 0` to list
every remaining file, 34439532375), all legs warm (34444305385), the
keep-only-newest cleanup (34450381312, then 34452640274 after the
Windows CRLF fix), the half-tree invalidation (34452640274), the
five-file change (34463517683, 34464720539), and this final shape
(34466377763, restore-only).
- Unit tests on all five platforms, the profile slice check, the Windows
build-script suite, Shellcheck and the universal DMG build all pass on
the cached binaries.
- The cleanup was verified against the PR's own cache scope: 44 entries
from the earlier runs reduced to exactly one per leg, on all three
platforms, after fixing the CRLF that made `gh cache delete` fail on
Windows.
- A libc++ syntax-only pass over all 1986 C++ translation units on Linux
found the `LocalesUtils.cpp` include; the two wx includes only surface
in a real macOS build and were found with a keep-going build in one
round.

<!--
> A guide for users on how to download the artifacts from this PR.
-->

[How to Download Pull Requests Artifacts for
Testing](https://www.orcaslicer.com/wiki/how_to_download_pr_artifacts)
2026-09-12 00:46:24 +08:00
Hanif Koh
6a88f0790e Enable ccache Depend Mode
A miss used to cost a preprocessor pass for the hash and then the real
compile. With the depend mode ccache hashes the include list the
compiler reports, so a miss costs only the compile. Ninja already asks
every compiler here for that list.
2026-09-11 23:02:31 +08:00
Hanif Koh
6f90ff6e93 Allow ccache with PCH
Clang records the modification time of every input in the precompiled
header, so a fresh checkout produces a different header and every file
that includes it misses the compiler cache. -fno-pch-timestamp makes the
header reproducible, and pch_defines lets ccache cache the header itself.
The precompiled header no longer has to be turned off when the cache is
on.
2026-09-11 23:02:21 +08:00
Hanif Koh
67f77e16c3 ci: cache compiled objects between runs
Every CI leg compiled the whole tree from scratch, 42 to 57 minutes of
each build job. Objects are now cached with ccache, one entry per leg
kept on the branch that built it: a push saves the cache and drops the
previous entry, a pull request restores main's and keeps nothing.

The precompiled header is turned off whenever the cache is on: Clang
stamps it with the build time, so every file including it missed. With
it off, a warm run hits 98.5 to 98.9 % of compiles and the compile steps
take 1 to 4 minutes; a cold run costs 25 to 60 % more than before, and a
change to a widely included header lands in between.
2026-09-11 23:02:21 +08:00
Hanif Koh
a6cf5cc1e3 Add the includes the precompiled header was supplying on macOS
A build without SLIC3R_PCH had never been tried on macOS. Three files
used what pchheader.hpp happened to include: LocalesUtils.cpp needs
<sstream> and <iomanip>, and the two dialogs need <wx/tooltip.h>. The
GTK port's headers and libstdc++ pull these in transitively, the Cocoa
port's headers and libc++ do not.
2026-09-11 13:02:57 +08:00
5 changed files with 91 additions and 0 deletions

View File

@@ -76,6 +76,57 @@ jobs:
if (-not (Test-Path "$cmakeBin\cmake.exe")) { throw "cmake.exe not found at $cmakeBin" }
Add-Content -Path $env:GITHUB_PATH -Value $cmakeBin
# Compiler cache. Pushes save it, so main keeps it warm; pull requests
# restore it and discard what they compiled. Objects are keyed on the
# preprocessed source, the compiler and the flags, so a leg only ever
# hits its own entries. A failed install costs the caching, not the build.
- name: Name the compiler cache leg
if: ${{ !inputs.macos-combine-only }}
shell: bash
run: |
leg="${{ runner.os }}-${{ inputs.arch || 'amd64' }}${{ runner.os == 'Windows' && format('-{0}', inputs.compiler) || '' }}"
echo "CCACHE_LEG=$leg" >> "$GITHUB_ENV"
echo "CCACHE_ENTRY=ccache-$leg-${{ github.run_id }}-${{ github.run_attempt }}" >> "$GITHUB_ENV"
# The action only installs and configures ccache. Restore and save go
# through actions/cache with one path string, since the cache service
# only matches entries saved under the identical path and the action
# spells it differently on Windows.
- name: Compiler cache
id: ccache
if: ${{ !inputs.macos-combine-only }}
continue-on-error: true
uses: hendrikmuhs/ccache-action@v1.2
with:
key: ${{ env.CCACHE_LEG }}
max-size: 3G
restore: false
save: false
- name: Restore compiler cache
if: ${{ steps.ccache.outcome == 'success' }}
uses: actions/cache/restore@v6
with:
path: ${{ github.workspace }}/.ccache
key: ${{ env.CCACHE_ENTRY }}
restore-keys: ccache-${{ env.CCACHE_LEG }}-
- name: Enable compiler cache
if: ${{ steps.ccache.outcome == 'success' }}
shell: bash
run: |
echo "CMAKE_C_COMPILER_LAUNCHER=ccache" >> "$GITHUB_ENV"
echo "CMAKE_CXX_COMPILER_LAUNCHER=ccache" >> "$GITHUB_ENV"
# Headers a fresh checkout has just written, the few files that
# use __DATE__ or __TIME__, and the precompiled header, whose
# macros ccache cannot see.
echo "CCACHE_SLOPPINESS=pch_defines,time_macros,include_file_mtime,include_file_ctime" >> "$GITHUB_ENV"
# Hash the includes the compiler reports instead of preprocessing
# every miss before compiling it.
echo "CCACHE_DEPEND=1" >> "$GITHUB_ENV"
# The restored directory carries the previous run's counters.
ccache -z
- name: Get the version and date on Ubuntu and macOS
if: runner.os != 'Windows'
run: |
@@ -670,3 +721,32 @@ jobs:
asset_name: orca_custom_preset_tests.zip
asset_content_type: application/octet-stream
max_releases: 1
- name: Compiler cache statistics
if: ${{ always() && steps.ccache.outcome == 'success' }}
shell: bash
run: ccache -s -v || ccache -s
# Entries are immutable, so the new one is saved first and the older
# ones for this leg on this ref are dropped afterwards: a failed save
# leaves the previous entry in place.
- name: Save compiler cache
id: ccache_save
if: ${{ steps.ccache.outcome == 'success' && github.event_name != 'pull_request' }}
uses: actions/cache/save@v6
with:
path: ${{ github.workspace }}/.ccache
key: ${{ env.CCACHE_ENTRY }}
- name: Drop older compiler cache entries
if: ${{ steps.ccache_save.outcome == 'success' }}
# A read-only token (fork PRs) cannot delete; that only costs storage.
continue-on-error: true
shell: bash
env:
GH_TOKEN: ${{ github.token }}
run: |
gh cache list --ref "$GITHUB_REF" --key "ccache-$CCACHE_LEG-" --limit 100 --json id,key \
| jq -r --arg keep "$CCACHE_ENTRY" '.[] | select(.key != $keep) | .id' \
| tr -d '\r' \
| while read -r id; do gh cache delete "$id"; done

View File

@@ -256,6 +256,13 @@ function(add_precompiled_header _target _input)
message(STATUS "Adding precompiled header ${_input} to target ${_target}.")
target_precompile_headers(${_target} PRIVATE ${_input})
# Clang records the modification time of every input in the precompiled
# header, which makes it differ between two checkouts of the same source
# and defeats a compiler cache. The build system already rebuilds the
# header when an input changes.
target_compile_options(${_target} PRIVATE
"$<$<CXX_COMPILER_ID:Clang,AppleClang>:SHELL:-Xclang -fno-pch-timestamp>")
get_target_property(_sources ${_target} SOURCES)
list(FILTER _sources INCLUDE REGEX ".*\\.mm?")

View File

@@ -3,6 +3,8 @@
#ifdef _WIN32
#include <charconv>
#endif
#include <iomanip>
#include <sstream>
#include <stdexcept>
#include <fast_float/fast_float.h>

View File

@@ -11,6 +11,7 @@
#include "MainFrame.hpp"
#include "format.hpp"
#include "Widgets/ProgressDialog.hpp"
#include <wx/tooltip.h>
#include "Widgets/RoundedRectangle.hpp"
#include "Widgets/StaticBox.hpp"

View File

@@ -9,6 +9,7 @@
#include <boost/regex.hpp>
#include <wx/sizer.h>
#include <wx/tooltip.h>
#include <wx/stattext.h>
#include <wx/textctrl.h>
#include <wx/button.h>