Lock Config and Preset Files Across Instances and Write Them Atomically

Every running instance shares one OrcaSlicer.conf and one user preset
tree, and nothing kept their writers apart. Two instances saving at the
same moment, or the cloud preset sync thread writing while the GUI thread
saved, could interleave, and a reader in another instance could open a
preset JSON or .info file between truncate and close and get a partial
file, dropping that preset for the session with a parse error.

Add InstanceLock, a scoped guard that serialises the threads of one
process through a recursive mutex and other processes through an advisory
OS file lock: flock on POSIX, held on the guard's own descriptor so no
other close in the process can drop it, and LockFileEx on Windows. The
outermost guard opens the lock file and closes it on release, so nothing
stays open between saves and a data dir can be removed once nothing is
saving into it; the file itself is kept, since deleting it would let a
third instance lock a fresh file while the second still holds the old
one. It is best effort: when the lock file cannot be opened or locked, or
another instance still holds it after a second, the guard logs once and
lets the write proceed, then leaves the file alone for ten seconds, so
a hung instance never blocks every other one and a holder stuck in a
debugger does not cost a stall per save. The
guard sits at the leaf readers and writers: set_sync_info_and_save()
calls save_info() under the preset collection mutex, so a batch lock
around save_user_presets() would invert the order against the sync
thread. The preset scan re-takes the guard every 32 files rather than
holding it across the scan, and a bundle or user folder is renamed into
cache/ under the lock and removed outside it, so a save never waits for
a whole tree. Read-only scans, which is what the CLI does, take no lock
and create no lock file.

AppConfig holds OrcaSlicer.conf.lock in load() and save(); load is
included because the Windows path restores from the .bak copy. Every
user preset writer and reader holds user.lock: Preset::save(), which
writes no .info when the preset itself could not be written, since an
.info without its preset reads as a cloud deletion request, save_info(),
reload() and remove_files(), each file read by the preset scan, the
bundle metadata reads and write, the .info removal after a cloud-confirmed
delete, the orphaned-.info scan on the sync thread, the bundle folder
removal on unsubscribe and the physical printer writers and delete
paths. A bundle import extracts under cache/ into a folder per process
and per import, where no scan reads.

Preset JSON, .info, bundle metadata, physical printer and config files,
and the caches and state files that already used a temporary by hand,
now go through write_file_atomically(), which writes <file>.<pid>.<n>.tmp
beside the target and renames it over, so a reader that never waits sees
a complete old or new file. A target this process may not write is
refused before anything is written, unless the caller says the file was
always replaced, as the config was; a symlink is followed;
a target that is not a regular file is written in place; and when the
rename itself is refused, by a Windows reader holding the file open or
a mount that cannot replace in one step, the helper writes in place as
before, since losing the save is worse than a torn read. On POSIX the
rename replaces the target atomically where the old code removed it
first and left a window with no file at all; only a mount that refuses a
one-step replace gets the old remove-then-rename. A crash between
temporary and rename leaves the temporary,
which the scans of the directories the application owns remove once it
is an hour old; only the exact shapes this code writes qualify, so a
user's numbered backup or an export folder is never touched. A failed
config write keeps the config dirty, and the idle handler waits ten
seconds before retrying while an explicit save always tries.
This commit is contained in:
Hanif Koh
2026-09-25 06:08:54 +08:00
parent 811b587eb0
commit af97d59870
21 changed files with 1166 additions and 231 deletions
+1
View File
@@ -49,6 +49,7 @@ add_executable(${_TEST_NAME}_tests
test_ordering_strategies.cpp
# test_png_io.cpp
test_indexed_triangle_set.cpp
test_instance_lock.cpp
../libnest2d/printer_parts.cpp
)
+202
View File
@@ -0,0 +1,202 @@
#include <catch2/catch_all.hpp>
#include <atomic>
#include <chrono>
#include <thread>
#include <boost/filesystem.hpp>
#include "libslic3r/InstanceLock.hpp"
#include "test_utils.hpp"
#ifndef _WIN32
#include <fcntl.h>
#include <sys/file.h>
#include <sys/wait.h>
#include <unistd.h>
#endif
using namespace Slic3r;
using namespace std::chrono_literals;
// Sets a process-wide knob for one test and restores it however the test ends.
template<typename T> struct ScopedStaticValue
{
T &ref;
T saved;
ScopedStaticValue(T &ref, T value) : ref(ref), saved(ref) { ref = value; }
~ScopedStaticValue() { ref = saved; }
};
TEST_CASE("InstanceLock creates its lock file and holds it for the guard's scope", "[InstanceLock]")
{
ScopedTemporaryFile lock_file(".lock");
const std::string path = lock_file.string();
{
InstanceLock lock(path);
REQUIRE(lock.locked());
REQUIRE(boost::filesystem::exists(path));
}
// Released: a fresh guard gets the lock at once instead of waiting out a timeout.
const auto started = std::chrono::steady_clock::now();
InstanceLock again(path, 5000ms);
REQUIRE(again.locked());
// Well inside the timeout it would otherwise have waited out; loose enough for a loaded runner.
REQUIRE(std::chrono::steady_clock::now() - started < 4000ms);
}
TEST_CASE("InstanceLock nests within one thread", "[InstanceLock]")
{
ScopedTemporaryFile lock_file(".lock");
const std::string path = lock_file.string();
InstanceLock outer(path);
{
InstanceLock inner(path, 100ms);
REQUIRE(inner.locked());
}
// The inner guard leaving does not release the outer one.
REQUIRE(outer.locked());
}
TEST_CASE("InstanceLock is a no-op for an empty path and survives an unwritable one", "[InstanceLock]")
{
ScopedTemporaryDir dir;
InstanceLock none("");
REQUIRE_FALSE(none.locked());
// The directory does not exist, so the lock file cannot be created; the
// guard still constructs and the write it guards can go ahead.
InstanceLock unwritable((dir.path() / "missing" / "shared.lock").string(), 100ms);
REQUIRE_FALSE(unwritable.locked());
}
TEST_CASE("InstanceLock retries a lock file it could not open once the cool-down passes", "[InstanceLock]")
{
ScopedTemporaryDir dir;
const std::string path = (dir.path() / "later" / "shared.lock").string();
ScopedStaticValue cooldown(InstanceLock::cooldown, 300ms);
bool before_dir, during_cooldown, after_cooldown;
const auto started = std::chrono::steady_clock::now();
{
InstanceLock lock(path, 100ms);
before_dir = lock.locked();
}
boost::filesystem::create_directories(dir.path() / "later");
{
InstanceLock lock(path, 100ms);
during_cooldown = lock.locked();
}
const bool second_guard_inside_cooldown = std::chrono::steady_clock::now() - started < InstanceLock::cooldown;
std::this_thread::sleep_for(400ms);
{
InstanceLock lock(path, 100ms);
after_cooldown = lock.locked();
}
REQUIRE_FALSE(before_dir);
// A loaded runner may take longer than the cool-down to get here; then the
// second guard legitimately retried, so only assert when the timing held.
if (second_guard_inside_cooldown)
REQUIRE_FALSE(during_cooldown);
REQUIRE(after_cooldown);
}
TEST_CASE("InstanceLock reopens a lock file that was replaced on disk", "[InstanceLock]")
{
ScopedTemporaryFile lock_file(".lock");
const std::string path = lock_file.string();
{
InstanceLock lock(path);
REQUIRE(lock.locked());
}
boost::filesystem::remove(path);
InstanceLock lock(path);
REQUIRE(lock.locked());
// Each outermost guard opens the file afresh, so the deleted path is back.
REQUIRE(boost::filesystem::exists(path));
}
TEST_CASE("InstanceLock serialises the threads of one process", "[InstanceLock]")
{
ScopedTemporaryFile lock_file(".lock");
const std::string path = lock_file.string();
std::atomic<bool> holder_ready{false};
std::atomic<bool> holder_released{false};
std::thread holder([&] {
InstanceLock lock(path);
holder_ready = true;
std::this_thread::sleep_for(150ms);
holder_released = true;
});
while (! holder_ready)
std::this_thread::yield();
bool released_before_acquire = false;
{
InstanceLock lock(path);
released_before_acquire = holder_released;
}
holder.join();
REQUIRE(released_before_acquire);
}
#ifndef _WIN32
// The cross-process side of the lock is a POSIX flock, which a child process
// takes here directly; LockFileEx backs the guard on Windows, but spawning a
// child there is not worth a test.
TEST_CASE("InstanceLock yields to another process and reports it", "[InstanceLock]")
{
ScopedTemporaryFile lock_file(".lock");
const std::string path = lock_file.string();
int child_holds[2], child_may_exit[2];
REQUIRE(::pipe(child_holds) == 0);
REQUIRE(::pipe(child_may_exit) == 0);
const pid_t child = ::fork();
REQUIRE(child >= 0);
if (child == 0) {
int fd = ::open(path.c_str(), O_RDWR | O_CREAT, 0644);
char byte = ::flock(fd, LOCK_EX | LOCK_NB) == 0 ? '1' : '0';
if (::write(child_holds[1], &byte, 1) != 1 || ::read(child_may_exit[0], &byte, 1) != 1)
::_exit(1);
::_exit(0);
}
char byte = '0';
REQUIRE(::read(child_holds[0], &byte, 1) == 1);
REQUIRE(byte == '1');
bool locked_while_child_holds;
{
InstanceLock lock(path, 100ms);
locked_while_child_holds = lock.locked();
}
// The timed-out wait starts a cool-down: the next guard does not wait again.
const auto started = std::chrono::steady_clock::now();
bool locked_during_cooldown;
{
InstanceLock lock(path, 5000ms);
locked_during_cooldown = lock.locked();
}
const auto cooldown_wait = std::chrono::steady_clock::now() - started;
REQUIRE(::write(child_may_exit[1], "x", 1) == 1);
int status = 0;
REQUIRE(::waitpid(child, &status, 0) == child);
for (int fd : {child_holds[0], child_holds[1], child_may_exit[0], child_may_exit[1]})
::close(fd);
REQUIRE_FALSE(locked_while_child_holds);
REQUIRE_FALSE(locked_during_cooldown);
REQUIRE(cooldown_wait < 4000ms);
// A guard inside the cool-down still takes the lock when it is free.
InstanceLock lock(path);
REQUIRE(lock.locked());
}
#endif
+189
View File
@@ -8,8 +8,11 @@
#include <algorithm>
#include <cctype>
#include <ctime>
#include <fstream>
#include <string>
#include <thread>
#include <system_error>
#ifndef _WIN32
#include <unistd.h> // getuid
@@ -62,6 +65,192 @@ TEST_CASE("per-user temp root is unchanged on Windows, isolated elsewhere", "[ut
#endif
}
TEST_CASE("write_file_atomically replaces the target and leaves no temporary file", "[utils]") {
ScopedTemporaryDir dir;
const boost::filesystem::path target = dir.path() / "preset.json";
REQUIRE_FALSE(write_file_atomically(target.string(), "first"));
REQUIRE_FALSE(write_file_atomically(target.string(), "second"));
std::string content;
load_string_file(target, content);
REQUIRE(content == "second");
size_t entries = 0;
for (auto &entry : boost::filesystem::directory_iterator(dir.path())) {
(void) entry;
++entries;
}
REQUIRE(entries == 1);
}
TEST_CASE("write_file_atomically reports a missing directory and writes nothing", "[utils]") {
ScopedTemporaryDir dir;
const boost::filesystem::path target = dir.path() / "missing" / "preset.json";
const std::error_code ec = write_file_atomically(target.string(), "x");
REQUIRE(ec == std::errc::no_such_file_or_directory);
REQUIRE_FALSE(boost::filesystem::exists(target));
}
TEST_CASE("write_file_atomically refuses a read-only target and leaves it untouched", "[utils]") {
#ifndef _WIN32
if (::geteuid() == 0)
SKIP("a read-only file does not stop root");
#endif
ScopedTemporaryDir dir;
const boost::filesystem::path target = dir.path() / "pinned.json";
REQUIRE_FALSE(write_file_atomically(target.string(), "pinned"));
boost::filesystem::permissions(target, boost::filesystem::owner_read | boost::filesystem::group_read | boost::filesystem::others_read);
const std::error_code ec = write_file_atomically(target.string(), "replaced");
boost::filesystem::permissions(target, boost::filesystem::owner_read | boost::filesystem::owner_write | boost::filesystem::group_read | boost::filesystem::others_read);
REQUIRE(ec == std::errc::permission_denied);
std::string content;
load_string_file(target, content);
REQUIRE(content == "pinned");
}
TEST_CASE("write_file_atomically replaces a read-only target when asked to", "[utils]") {
#ifndef _WIN32
if (::geteuid() == 0)
SKIP("a read-only file does not stop root");
#endif
ScopedTemporaryDir dir;
const boost::filesystem::path target = dir.path() / "pinned.json";
REQUIRE_FALSE(write_file_atomically(target.string(), "pinned"));
boost::filesystem::permissions(target, boost::filesystem::owner_read | boost::filesystem::group_read | boost::filesystem::others_read);
const std::error_code ec = write_file_atomically(target.string(), "replaced", { false, /*replace_read_only=*/true });
boost::filesystem::permissions(target, boost::filesystem::owner_read | boost::filesystem::owner_write | boost::filesystem::group_read | boost::filesystem::others_read);
REQUIRE_FALSE(ec);
std::string content;
load_string_file(target, content);
REQUIRE(content == "replaced");
}
TEST_CASE("write_file_atomically keeps bytes intact in binary mode", "[utils]") {
ScopedTemporaryDir dir;
const boost::filesystem::path target = dir.path() / "blob.bin";
const std::string bytes("a\r\nb\0c", 6);
REQUIRE_FALSE(write_file_atomically(target.string(), bytes, { /*binary=*/true }));
REQUIRE(boost::filesystem::file_size(target) == bytes.size());
}
#ifndef _WIN32
TEST_CASE("write_file_atomically writes through a symlink and keeps the target's permissions", "[utils]") {
ScopedTemporaryDir dir;
const boost::filesystem::path real = dir.path() / "real.json";
const boost::filesystem::path link = dir.path() / "link.json";
REQUIRE_FALSE(write_file_atomically(real.string(), "first"));
boost::filesystem::permissions(real, boost::filesystem::owner_read | boost::filesystem::owner_write);
boost::filesystem::create_symlink(real, link);
REQUIRE_FALSE(write_file_atomically(link.string(), "second"));
REQUIRE(boost::filesystem::is_symlink(boost::filesystem::symlink_status(link)));
std::string content;
load_string_file(real, content);
REQUIRE(content == "second");
REQUIRE_FALSE(write_file_atomically(real.string(), "third"));
const auto perms = boost::filesystem::status(real).permissions() & boost::filesystem::all_all;
REQUIRE(perms == (boost::filesystem::owner_read | boost::filesystem::owner_write));
}
#endif
TEST_CASE("write_file_atomically survives two threads writing one target", "[utils]") {
ScopedTemporaryDir dir;
const boost::filesystem::path target = dir.path() / "shared.json";
const std::string a(20000, 'a'), b(20000, 'b');
std::thread other([&] {
for (int i = 0; i < 50; ++i)
write_file_atomically(target.string(), a);
});
for (int i = 0; i < 50; ++i)
write_file_atomically(target.string(), b);
other.join();
std::string content;
load_string_file(target, content);
const bool whole = content == a || content == b;
REQUIRE(whole);
// No temporary may be left; a scanner on Windows may briefly hold the old
// file under another name, so only the temporaries are counted.
size_t temporaries = 0;
for (auto &entry : boost::filesystem::directory_iterator(dir.path()))
if (entry.path().extension() == ".tmp")
++temporaries;
REQUIRE(temporaries == 0);
}
TEST_CASE("remove_stale_temp_files removes an old tree set aside for removal", "[utils]") {
ScopedTemporaryDir dir;
const boost::filesystem::path doomed = dir.path() / "removing.4242.0";
boost::filesystem::create_directories(doomed / "sub");
REQUIRE_FALSE(write_file_atomically((doomed / "sub" / "x.json").string(), "x"));
boost::filesystem::last_write_time(doomed, std::time(nullptr) - 7200);
boost::filesystem::create_directories(dir.path() / "removing.4242.1"); // just set aside: in progress
boost::filesystem::create_directories(dir.path() / "import.4242.0"); // an import in progress
REQUIRE(remove_stale_temp_files(dir.path()) == 1);
REQUIRE_FALSE(boost::filesystem::exists(doomed));
REQUIRE(boost::filesystem::exists(dir.path() / "removing.4242.1"));
REQUIRE(boost::filesystem::exists(dir.path() / "import.4242.0"));
}
TEST_CASE("remove_if_stale_leftover judges one directory entry, as a scan does per file", "[utils]") {
ScopedTemporaryDir dir;
REQUIRE_FALSE(write_file_atomically((dir.path() / "a.json").string(), "{}"));
REQUIRE_FALSE(write_file_atomically((dir.path() / "a.json.123.7.tmp").string(), "{"));
REQUIRE_FALSE(write_file_atomically((dir.path() / "b.json.123.8.tmp").string(), "{"));
boost::filesystem::last_write_time(dir.path() / "a.json.123.7.tmp", std::time(nullptr) - 7200);
REQUIRE_FALSE(remove_if_stale_leftover(boost::filesystem::directory_entry(dir.path() / "a.json")));
REQUIRE(remove_if_stale_leftover(boost::filesystem::directory_entry(dir.path() / "a.json.123.7.tmp")));
REQUIRE_FALSE(remove_if_stale_leftover(boost::filesystem::directory_entry(dir.path() / "b.json.123.8.tmp"))); // too young
REQUIRE(boost::filesystem::exists(dir.path() / "a.json"));
REQUIRE_FALSE(boost::filesystem::exists(dir.path() / "a.json.123.7.tmp"));
REQUIRE(boost::filesystem::exists(dir.path() / "b.json.123.8.tmp"));
}
TEST_CASE("remove_stale_temp_files removes only old <name>.<pid>.<n>.tmp files", "[utils]") {
ScopedTemporaryDir dir;
for (const char *name : { "a.json", "a.json.123.7.tmp", "b.info.4.0.tmp", "c.json", "d.tmp", "e.json.x.1.tmp", "f.json..tmp", "a.json.99", "a.json.12.tmp", "a.json.123.5.old", "h.json.old" }) {
REQUIRE_FALSE(write_file_atomically((dir.path() / name).string(), "x"));
// Two hours old: well past the hour below which a temporary may still be in flight.
boost::filesystem::last_write_time(dir.path() / name, std::time(nullptr) - 7200);
}
// Just written: possibly another instance's in-flight save, so it stays.
REQUIRE_FALSE(write_file_atomically((dir.path() / "g.json.7.2.tmp").string(), "x"));
SECTION("with a name prefix only matching names go; a numbered backup, a one-segment name or another suffix is not removed") {
REQUIRE(remove_stale_temp_files(dir.path(), "a.json") == 1);
REQUIRE_FALSE(boost::filesystem::exists(dir.path() / "a.json.123.7.tmp"));
REQUIRE(boost::filesystem::exists(dir.path() / "a.json.123.5.old"));
REQUIRE(boost::filesystem::exists(dir.path() / "a.json.99"));
REQUIRE(boost::filesystem::exists(dir.path() / "a.json.12.tmp"));
REQUIRE(boost::filesystem::exists(dir.path() / "b.info.4.0.tmp"));
}
SECTION("without a prefix every stale temporary goes and nothing else") {
REQUIRE(remove_stale_temp_files(dir.path()) == 2);
size_t entries = 0;
for (auto &entry : boost::filesystem::directory_iterator(dir.path())) {
(void) entry;
++entries;
}
REQUIRE(entries == 10);
REQUIRE(boost::filesystem::exists(dir.path() / "a.json.123.5.old"));
REQUIRE(boost::filesystem::exists(dir.path() / "a.json"));
REQUIRE(boost::filesystem::exists(dir.path() / "h.json.old"));
REQUIRE(boost::filesystem::exists(dir.path() / "a.json.99"));
REQUIRE(boost::filesystem::exists(dir.path() / "g.json.7.2.tmp"));
}
}
TEST_CASE("copy_file reports the OS error when the destination cannot be written", "[utils]") {
ScopedTemporaryFile source(".txt");
{
+13 -6
View File
@@ -1,5 +1,9 @@
#include <catch2/catch_all.hpp>
#ifndef _WIN32
#include <unistd.h>
#endif
#include <boost/filesystem.hpp>
#include <boost/crc.hpp>
#include <cereal/archives/binary.hpp>
@@ -1358,20 +1362,23 @@ TEST_CASE("a header claiming more body than the file holds is rejected", "[Vendo
TEST_CASE("a failed write leaves the previous cache in place", "[VendorCache]")
{
#ifndef _WIN32
if (::geteuid() == 0)
SKIP("a read-only file does not stop root");
#endif
TempDir tmp;
const std::string cache = (tmp.path / "Durable.opc").string();
REQUIRE(save_one_vendor(cache, one_vendor("Durable"), "Durable", "1.0.0"));
const std::string before = slurp(cache);
// A directory where the temp file wants to go: the write cannot complete,
// and must not have destroyed what was already there to find that out.
const fs::path blocker = fs::path(cache + "." + std::to_string(get_current_pid()) + ".tmp");
fs::create_directories(blocker);
// A read-only cache (the read-only attribute on Windows) is refused before
// anything is written, so what was there must survive the attempt.
fs::permissions(cache, fs::owner_read | fs::group_read | fs::others_read);
REQUIRE_FALSE(save_one_vendor(cache, one_vendor("Durable"), "Durable", "2.0.0"));
CHECK(slurp(cache) == before);
fs::remove_all(blocker);
fs::permissions(cache, fs::owner_read | fs::owner_write | fs::group_read | fs::others_read);
CHECK(slurp(cache) == before);
}
TEST_CASE("a cache written by another build's option ordering still loads", "[VendorCache]")