refactor(automation): drop screenshot.viewport3d, keep only screenshot.window

The on-screen window capture is composited from the desktop framebuffer, so it
already includes the GL 3D viewport as currently shown (model in the editor,
toolpaths in Preview). The offscreen render_thumbnail path only ever drew the
model GLVolumeCollection — never the gcode toolpaths — and produced a blank image
after slicing because the app switches to the Preview panel. Rather than maintain a
second, more limited capture method, remove it entirely.

Removes the JSON-RPC method, IUiBackend/WxUiBackend implementation, dispatcher
route + capability entry, the now-dead opt_int/thumbnail_to_wximage helpers and
ThumbnailData include, the mock override + unit test, and the Python
screenshot_3d client method. Docs updated accordingly.
This commit is contained in:
SoftFever
2026-06-03 18:05:23 +08:00
parent 952696fd1f
commit 892b33bac5
10 changed files with 25 additions and 129 deletions

View File

@@ -4,7 +4,8 @@ OrcaSlicer ships an **opt-in, localhost-only JSON-RPC server** that lets externa
scripts introspect, drive, and screenshot the running OrcaSlicer GUI. It is built scripts introspect, drive, and screenshot the running OrcaSlicer GUI. It is built
for end-to-end testing and automation: a script can enumerate the live widget for end-to-end testing and automation: a script can enumerate the live widget
tree, click buttons, type text, send keyboard shortcuts, wait for UI state, query tree, click buttons, type text, send keyboard shortcuts, wait for UI state, query
high-level application state, and capture both window and 3D-viewport images. high-level application state, and capture window images (the on-screen capture
includes the 3D viewport).
This document is the protocol reference. It describes activation, the transport, This document is the protocol reference. It describes activation, the transport,
the JSON-RPC envelope, every method, the unified node shape, the target/locator the JSON-RPC envelope, every method, the unified node shape, the target/locator
@@ -109,8 +110,7 @@ Returns server identity and the list of supported methods. Takes no parameters.
"protocol": "2.0", "protocol": "2.0",
"capabilities": [ "capabilities": [
"tree.dump", "tree.find", "widget.get", "input.click", "input.type", "tree.dump", "tree.find", "widget.get", "input.click", "input.type",
"input.key", "sync.wait_for", "app.state", "screenshot.window", "input.key", "sync.wait_for", "app.state", "screenshot.window"
"screenshot.viewport3d"
] ]
} }
``` ```
@@ -295,28 +295,10 @@ method reads from the screen.)
- **HiDPI:** the reported `width`/`height` come from the window's logical client size, - **HiDPI:** the reported `width`/`height` come from the window's logical client size,
while the screen framebuffer is in physical pixels. On per-monitor-DPI displays the while the screen framebuffer is in physical pixels. On per-monitor-DPI displays the
two can differ; the capture may be cropped or scaled relative to the logical size. two can differ; the capture may be cropped or scaled relative to the logical size.
- For a clean, occlusion-independent, arbitrary-resolution render of the 3D scene - Because the capture is the live on-screen image, the 3D content reflects the
(including when the 3D tab is not the visible view), use **current view**: the model in the 3D editor, or the gcode toolpaths in Preview
[`screenshot.viewport3d`](#screenshotviewport3d) instead. after a slice. There is no separate offscreen 3D-render method — the window
capture already includes whatever the GL canvas is showing.
### `screenshot.viewport3d`
Render a 3D plate offscreen and return it as a PNG. Unlike `screenshot.window`, this
renders into an offscreen framebuffer, so it is independent of window size and
occlusion, works even when the 3D tab is hidden, and supports arbitrary output
resolution — making it the right choice for clean, deterministic captures.
**Params (all optional):**
| Param | Type | Default | Meaning |
|---|---|---|---|
| `plate` | int | active plate | Plate index to render. |
| `width` | int | `800` | Output width in pixels. |
| `height` | int | `600` | Output height in pixels. |
**Result:** `{ "png_base64": "<base64 PNG>", "width": <int>, "height": <int> }`.
**Errors:** `1005` on failure.
--- ---
@@ -494,8 +476,7 @@ Consequences and conventions:
- **`screenshot.window` reads the screen.** It captures the on-screen, DWM-composited - **`screenshot.window` reads the screen.** It captures the on-screen, DWM-composited
framebuffer, so the target window must be visible and unobscured, and the result is framebuffer, so the target window must be visible and unobscured, and the result is
in physical pixels (see HiDPI caveat under [`screenshot.window`](#screenshotwindow)). in physical pixels (see HiDPI caveat under [`screenshot.window`](#screenshotwindow)).
For occlusion-independent 3D captures use The capture includes the GL 3D viewport as currently shown (model or toolpaths).
[`screenshot.viewport3d`](#screenshotviewport3d).
- **Single-client / serialized.** v1 handles one request at a time; issue requests - **Single-client / serialized.** v1 handles one request at a time; issue requests
sequentially from a single client. sequentially from a single client.
- **GUI-thread marshaling.** Every backend call is marshaled onto the GUI thread - **GUI-thread marshaling.** Every backend call is marshaled onto the GUI thread
@@ -517,14 +498,14 @@ orca.click({"id": "btn_slice"}) # start slicing the plate
orca.wait_for({"id": "btn_export"}, # wait until slicing finishes orca.wait_for({"id": "btn_export"}, # wait until slicing finishes
state="enabled", timeout_ms=180000) state="enabled", timeout_ms=180000)
png = orca.screenshot_3d(width=1024, height=768) # render the 3D viewport png = orca.screenshot() # on-screen capture (incl. 3D view)
with open("preview_3d.png", "wb") as f: with open("window.png", "wb") as f:
f.write(png) f.write(png)
``` ```
For a full, runnable end-to-end example — launching OrcaSlicer with the automation For a full, runnable end-to-end example — launching OrcaSlicer with the automation
flags, loading a model, slicing, waiting for completion, and saving both a window flags, loading a model, slicing, waiting for completion, and saving a window PNG —
PNG and a 3D PNG — see `tools/automation/example_slice.py`. see `tools/automation/example_slice.py`.
--- ---

View File

@@ -92,11 +92,9 @@ public:
// Send key chords (e.g. ctrl+s) to the focused window. // Send key chords (e.g. ctrl+s) to the focused window.
virtual bool send_keys(const std::vector<KeyChord>& chords) = 0; virtual bool send_keys(const std::vector<KeyChord>& chords) = 0;
// Screenshots. target == nullptr => main frame. // Screenshot. target == nullptr => main frame. Captured from the on-screen
// composited framebuffer, so it includes the GL viewport and ImGui overlays.
virtual PngImage screenshot_window(const UiNode* target) = 0; virtual PngImage screenshot_window(const UiNode* target) = 0;
virtual PngImage screenshot_viewport3d(std::optional<int> plate,
std::optional<int> width,
std::optional<int> height) = 0;
}; };
}}} // namespace Slic3r::GUI::Automation }}} // namespace Slic3r::GUI::Automation

View File

@@ -155,12 +155,6 @@ std::string base64_encode(const std::vector<unsigned char>& data) {
return out; return out;
} }
std::optional<int> opt_int(const nlohmann::json& p, const char* key) {
if (p.is_object() && p.contains(key) && p.at(key).is_number_integer())
return p.at(key).get<int>();
return std::nullopt;
}
nlohmann::json image_to_json(const PngImage& img) { nlohmann::json image_to_json(const PngImage& img) {
if (img.png.empty()) if (img.png.empty())
throw AutomationError(kErrScreenshotFail, "screenshot produced no data"); throw AutomationError(kErrScreenshotFail, "screenshot produced no data");
@@ -174,8 +168,7 @@ nlohmann::json JsonRpcDispatcher::m_version(const nlohmann::json&) {
{"protocol", "2.0"}, {"protocol", "2.0"},
{"capabilities", nlohmann::json::array({ {"capabilities", nlohmann::json::array({
"tree.dump","tree.find","widget.get","input.click","input.type", "tree.dump","tree.find","widget.get","input.click","input.type",
"input.key","sync.wait_for","app.state","screenshot.window", "input.key","sync.wait_for","app.state","screenshot.window" })} };
"screenshot.viewport3d" })} };
} }
nlohmann::json JsonRpcDispatcher::dispatch(const nlohmann::json& request) { nlohmann::json JsonRpcDispatcher::dispatch(const nlohmann::json& request) {
@@ -200,7 +193,6 @@ nlohmann::json JsonRpcDispatcher::dispatch(const nlohmann::json& request) {
if (method == "sync.wait_for") return make_result(id, m_sync_wait_for(params)); if (method == "sync.wait_for") return make_result(id, m_sync_wait_for(params));
if (method == "app.state") return make_result(id, m_app_state(params)); if (method == "app.state") return make_result(id, m_app_state(params));
if (method == "screenshot.window") return make_result(id, m_screenshot_window(params)); if (method == "screenshot.window") return make_result(id, m_screenshot_window(params));
if (method == "screenshot.viewport3d") return make_result(id, m_screenshot_viewport3d(params));
return make_error(id, kMethodNotFound, "unknown method: " + method); return make_error(id, kMethodNotFound, "unknown method: " + method);
} catch (const AutomationError& e) { } catch (const AutomationError& e) {
return make_error(id, e.code, e.what()); return make_error(id, e.code, e.what());
@@ -350,9 +342,4 @@ nlohmann::json JsonRpcDispatcher::m_screenshot_window(const nlohmann::json& para
return image_to_json(m_backend.screenshot_window(target_ptr)); return image_to_json(m_backend.screenshot_window(target_ptr));
} }
nlohmann::json JsonRpcDispatcher::m_screenshot_viewport3d(const nlohmann::json& params) {
return image_to_json(m_backend.screenshot_viewport3d(
opt_int(params, "plate"), opt_int(params, "width"), opt_int(params, "height")));
}
}}} // namespace }}} // namespace

View File

@@ -47,7 +47,6 @@ private:
nlohmann::json m_sync_wait_for(const nlohmann::json& params); nlohmann::json m_sync_wait_for(const nlohmann::json& params);
nlohmann::json m_app_state(const nlohmann::json& params); nlohmann::json m_app_state(const nlohmann::json& params);
nlohmann::json m_screenshot_window(const nlohmann::json& params); nlohmann::json m_screenshot_window(const nlohmann::json& params);
nlohmann::json m_screenshot_viewport3d(const nlohmann::json& params);
// Resolve a unique, actionable (enabled+visible) node from params["target"]. // Resolve a unique, actionable (enabled+visible) node from params["target"].
// Throws kErrNotFound (missing/ambiguous) or kErrNotActionable (disabled/hidden). // Throws kErrNotFound (missing/ambiguous) or kErrNotActionable (disabled/hidden).

View File

@@ -5,9 +5,8 @@
#include "slic3r/GUI/GUI_App.hpp" #include "slic3r/GUI/GUI_App.hpp"
#include "slic3r/GUI/MainFrame.hpp" #include "slic3r/GUI/MainFrame.hpp"
#include "slic3r/GUI/Plater.hpp" #include "slic3r/GUI/Plater.hpp"
#include "slic3r/GUI/GLCanvas3D.hpp" #include "slic3r/GUI/GLCanvas3D.hpp" // get_current_canvas3D() for app.state
#include "libslic3r/Model.hpp" #include "libslic3r/Model.hpp"
#include "libslic3r/GCode/ThumbnailData.hpp"
#include <wx/window.h> #include <wx/window.h>
#include <wx/toplevel.h> #include <wx/toplevel.h>
@@ -270,21 +269,6 @@ PngImage wximage_to_png(const wxImage& image) {
return out; return out;
} }
// RGBA ThumbnailData -> wxImage (mirrors GLCanvas3D::debug_output_thumbnail —
// note the vertical flip GL rows require).
wxImage thumbnail_to_wximage(const ThumbnailData& td) {
wxImage image((int)td.width, (int)td.height);
image.InitAlpha();
for (unsigned int r = 0; r < td.height; ++r) {
unsigned int rr = (td.height - 1 - r) * td.width;
for (unsigned int c = 0; c < td.width; ++c) {
const unsigned char* px = td.pixels.data() + 4 * (rr + c);
image.SetRGB((int)c, (int)r, px[0], px[1], px[2]);
image.SetAlpha((int)c, (int)r, px[3]);
}
}
return image;
}
} // namespace } // namespace
PngImage WxUiBackend::screenshot_window(const UiNode* target) { PngImage WxUiBackend::screenshot_window(const UiNode* target) {
@@ -319,32 +303,4 @@ PngImage WxUiBackend::screenshot_window(const UiNode* target) {
}); });
} }
PngImage WxUiBackend::screenshot_viewport3d(std::optional<int> plate,
std::optional<int> width,
std::optional<int> height) {
return run_on_gui(m_gui_timeout_ms, [&]() -> PngImage {
Plater* p = wxGetApp().plater();
if (p == nullptr)
throw AutomationError(kErrScreenshotFail, "no plater");
GLCanvas3D* canvas = p->get_current_canvas3D();
if (canvas == nullptr)
throw AutomationError(kErrScreenshotFail, "no 3D canvas");
const unsigned int w = width ? (unsigned)*width : 800u;
const unsigned int h = height ? (unsigned)*height : 600u;
// Render the active plate's 3D scene into an offscreen RGBA buffer.
// render_thumbnail makes the canvas's GL context current itself. The
// pixel size is governed by w/h; `sizes` stays empty as elsewhere.
// Fields: {sizes, printable_only, parts_only, show_bed, transparent_background, plate_id}.
const int plate_id = plate ? *plate : 0; // v1: default active plate
const ThumbnailsParams params{ {}, false, false, true, false, plate_id };
ThumbnailData data;
canvas->render_thumbnail(data, w, h, params, Camera::EType::Ortho);
if (!data.is_valid())
throw AutomationError(kErrScreenshotFail, "thumbnail render failed");
return wximage_to_png(thumbnail_to_wximage(data));
});
}
}}} // namespace Slic3r::GUI::Automation }}} // namespace Slic3r::GUI::Automation

View File

@@ -19,8 +19,6 @@ public:
bool type_text(const std::string& text) override; bool type_text(const std::string& text) override;
bool send_keys(const std::vector<KeyChord>& chords) override; bool send_keys(const std::vector<KeyChord>& chords) override;
PngImage screenshot_window(const UiNode* target) override; PngImage screenshot_window(const UiNode* target) override;
PngImage screenshot_viewport3d(std::optional<int> plate, std::optional<int> width,
std::optional<int> height) override;
private: private:
int m_gui_timeout_ms; int m_gui_timeout_ms;

View File

@@ -18,7 +18,6 @@ public:
std::vector<std::string> typed_text; std::vector<std::string> typed_text;
std::vector<std::vector<KeyChord>> sent_keys; std::vector<std::vector<KeyChord>> sent_keys;
int screenshot_window_count = 0; int screenshot_window_count = 0;
int screenshot_viewport_count = 0;
// Canned outputs (set by tests). // Canned outputs (set by tests).
UiNode tree; // default tree for dump_tree UiNode tree; // default tree for dump_tree
@@ -50,10 +49,6 @@ public:
PngImage screenshot_window(const UiNode*) override { PngImage screenshot_window(const UiNode*) override {
++screenshot_window_count; return canned_png; ++screenshot_window_count; return canned_png;
} }
PngImage screenshot_viewport3d(std::optional<int>, std::optional<int>,
std::optional<int>) override {
++screenshot_viewport_count; return canned_png;
}
}; };
}}} // namespace }}} // namespace

View File

@@ -162,15 +162,6 @@ TEST_CASE("screenshot.window returns base64 + dims", "[automation][rpc]") {
CHECK_FALSE(resp.at("result").at("png_base64").get<std::string>().empty()); CHECK_FALSE(resp.at("result").at("png_base64").get<std::string>().empty());
} }
TEST_CASE("screenshot.viewport3d returns base64 + dims", "[automation][rpc]") {
MockUiBackend mock;
JsonRpcDispatcher d(mock);
const json resp = d.dispatch({{"jsonrpc","2.0"},{"id",3},{"method","screenshot.viewport3d"},
{"params",{{"width",256},{"height",256}}}});
CHECK(mock.screenshot_viewport_count == 1);
CHECK(resp.at("result").at("png_base64").is_string());
}
TEST_CASE("sync.wait_for succeeds once the condition holds", "[automation][rpc]") { TEST_CASE("sync.wait_for succeeds once the condition holds", "[automation][rpc]") {
MockUiBackend mock; MockUiBackend mock;
// First 2 polls: btn disabled. 3rd poll: enabled. // First 2 polls: btn disabled. 3rd poll: enabled.

View File

@@ -1,5 +1,5 @@
"""End-to-end smoke test: launch OrcaSlicer with the automation server, load a """End-to-end smoke test: launch OrcaSlicer with the automation server, load a
model, slice it, wait for completion, and save both a window PNG and a 3D PNG. model, slice it, wait for completion, and save a window PNG.
Run: Run:
python example_slice.py --orca /path/to/OrcaSlicer --model /path/to/cube.stl python example_slice.py --orca /path/to/OrcaSlicer --model /path/to/cube.stl
@@ -56,11 +56,12 @@ def main() -> int:
orca.wait_for({"id": "btn_export"}, state="enabled", timeout_ms=180000, orca.wait_for({"id": "btn_export"}, state="enabled", timeout_ms=180000,
poll_ms=500) poll_ms=500)
# The window screenshot is captured from the on-screen composited
# framebuffer, so it already includes the 3D viewport (model in the
# editor, or toolpaths in Preview after slicing).
with open("window.png", "wb") as f: with open("window.png", "wb") as f:
f.write(orca.screenshot()) f.write(orca.screenshot())
with open("preview_3d.png", "wb") as f: print("wrote window.png")
f.write(orca.screenshot_3d(width=1920, height=1080))
print("wrote window.png and preview_3d.png")
return 0 return 0
finally: finally:
proc.terminate() proc.terminate()

View File

@@ -6,8 +6,8 @@ Usage:
print(orca.version()) print(orca.version())
orca.click({"id": "btn_slice"}) orca.click({"id": "btn_slice"})
orca.wait_for({"id": "btn_export"}, state="enabled", timeout_ms=120000) orca.wait_for({"id": "btn_export"}, state="enabled", timeout_ms=120000)
png = orca.screenshot_3d(width=1024, height=768) png = orca.screenshot()
open("preview.png", "wb").write(png) open("window.png", "wb").write(png)
""" """
from __future__ import annotations from __future__ import annotations
import base64 import base64
@@ -94,18 +94,8 @@ class OrcaClient:
return self._call("app.state") return self._call("app.state")
def screenshot(self, target: Optional[dict] = None) -> bytes: def screenshot(self, target: Optional[dict] = None) -> bytes:
"""Capture a window as a PNG, exactly as composited on screen (includes the
GL 3D viewport and ImGui overlays). Defaults to the main frame."""
params = {"target": target} if target is not None else None params = {"target": target} if target is not None else None
result = self._call("screenshot.window", params) result = self._call("screenshot.window", params)
return base64.b64decode(result["png_base64"]) return base64.b64decode(result["png_base64"])
def screenshot_3d(self, plate: Optional[int] = None,
width: Optional[int] = None, height: Optional[int] = None) -> bytes:
params: dict = {}
if plate is not None:
params["plate"] = plate
if width is not None:
params["width"] = width
if height is not None:
params["height"] = height
result = self._call("screenshot.viewport3d", params or None)
return base64.b64decode(result["png_base64"])