Desktop Diorama

What began as an experiment in making a desktop feel three-dimensional has become a working spatial interface capable of streaming, resizing and interacting with real applications.

At a glance

Overview

Desktop Diorama asks a deceptively simple question:

What if the desktop felt like a place rather than a sheet of glass?

Conventional desktop environments arrange our work as overlapping rectangles on a flat plane. They are extraordinarily effective, but the basic metaphor has changed remarkably little: windows appear, disappear, stack and fight for the same two-dimensional space.

I wanted to explore something more spatial and tactile without abandoning the applications people already use.

Desktop Diorama places real Windows applications onto movable panels inside a small 3D environment built in Godot. Panels can exist at different depths, be repositioned and resized, and eventually tilted or arranged around a workspace. Webcam-based head tracking changes the camera perspective as the user moves, creating the illusion that the monitor is a window into a miniature physical space.

The important part is that these panels are not screenshots or recreations of applications.

They are representations of the real applications running underneath.

The current prototype can capture live application content, stream it into Godot, resize the underlying source window and translate interaction on the 3D surface back into coordinates understood by the original application.

Video can now play through the spatial representation at effectively the same frame rate as its source without visible delay in my testing.

That changed the project from an aesthetic experiment into a technically viable interface concept.

The starting point: making a computer feel less flat

The project began from an emotional response to conventional desktop computing rather than from a specific missing feature.

Computers have become more capable while, in many ways, becoming less tangible.

Our applications increasingly live inside similar rectangles, similar browser tabs and similar design systems. The hardware disappears behind the software, while the software itself often tries to disappear behind neutral interfaces.

There is obvious value in consistency, but there is also something appealing about older interfaces, handheld devices, physical controls and software that feels as though it belongs somewhere.

I became interested in the idea of treating the monitor less like a canvas and more like a window into a small world.

A normal desktop communicates depth abstractly. One application covering another means that it is conceptually “in front”, but both remain on exactly the same physical plane.

A spatial desktop could make that relationship literal.

A reference window might sit slightly behind an editor. A music player might live off to one side. A communication window could occupy its own part of the environment rather than being another entry in an invisible stack.

Head movement could then reveal those relationships naturally, in the same way that leaning slightly to one side lets somebody look around an object on a desk.

The original idea was therefore not:

“Build another desktop environment.”

It was much smaller:

Can a normal monitor convincingly create the feeling that application windows exist in physical depth?

That became the first hypothesis.

Product hypothesis

The early product hypothesis was:

If desktop windows could be treated as persistent spatial objects rather than overlapping rectangles, then ordinary computer use could become more legible, tactile and personal without requiring users to abandon their existing applications.

There were two halves to that statement.

The first was experiential:

• could spatial organisation feel useful rather than gimmicky?
• could head-tracked perspective make depth convincing?
• could interaction remain understandable?
• could the desktop feel more like a workspace somebody inhabits?

The second was technical:

• could real applications be captured reliably?
• could captured applications remain live when hidden behind the 3D shell?
• could video remain smooth?
• could the source application be resized?
• could mouse input travel back through a transformed 3D surface?
• could all of this happen without the latency destroying the illusion?

The second group contained the assumptions capable of killing the project completely.

I therefore made an early product decision that shaped the rest of development:

prove the dangerous assumptions before polishing the attractive ones.

Building the idea in stages

Phase 1 — Creating spatial windows before using real ones

The first version contained no real Windows applications at all.

I built a reusable panel system inside Godot so that I could first understand what it meant for something resembling a desktop window to behave as a 3D object.

Panels were composed from elements including:

• a frame;
• a content surface;
• collision geometry;
• resize handles;
• and the logic required to position and manipulate them independently.

The important word was independently.

An early implementation exposed a useful Godot-specific problem: some mesh and collision resources were shared between instances. Resizing one panel could therefore modify another because both were referencing the same underlying resource.

Correcting this required creating instance-specific resources rather than treating each visual panel as though its data were automatically isolated.

It was a small engineering problem, but a useful architectural lesson. The interface might look like a collection of rectangles, but the product needed each one to behave as a genuine persistent object.

The panel interaction model gradually developed into:

• selection;
• right-click dragging;
• corner-based resizing;
• forward and backward movement in depth;
• dynamic resize handles;
• spatial ordering;
• optional focus behaviour;
• optional automatic bring-forward behaviour;
• optional snapping;
• and persistent panel state.

At this stage everything was still simulated.

That was intentional.

Before connecting the prototype to an operating system, I wanted to know whether manipulating windows spatially actually felt understandable.

Separating application interaction from spatial manipulation

A normal desktop already gives the left mouse button an enormous amount of meaning.

Applications expect it.

Buttons, links, text selection, drawing tools, games, editors and browsers all depend on ordinary mouse behaviour.

Using that same input for moving the spatial panel would force the user to constantly switch modes or would make interacting with the application unreliable.

I therefore separated the two concepts.

The emerging interaction language became:

• left click: interact with the application;
• right-click hold: manipulate the panel;
• resize handles: change its dimensions;
• scroll while manipulating: move the panel forwards or backwards.

The distinction sounds small, but it became one of the foundational UX decisions in the project.

The application retains ownership of what happens inside its surface.

Desktop Diorama owns what happens to that surface in space.

That distinction also influenced the visual direction.

I moved away from recreating operating-system title bars, minimise controls and other conventional window furniture inside the scene. If the application already contains its controls, reproducing another operating system around it would add complexity without adding capability.

Desktop Diorama is not intended to become a fake operating system sitting inside the real one.

It is a spatial layer around the applications that already exist.

Phase 2 — Making the monitor feel like a window

Once panels could inhabit the scene, the next question was whether the depth illusion itself worked.

The first camera prototype used mouse position.

Moving the pointer changed the camera position slightly, producing enough parallax to simulate looking around objects.

This was useful as a development tool but poor as an interaction system. Moving a panel also moved the cursor, which in turn moved the camera. The thing being manipulated and the viewpoint used to manipulate it became coupled together.

I separated the camera behaviour into explicit modes:

• Static;
• Mouse Debug;
• Head Input.

Mouse movement could still be used when testing the scene, but the real experience could now use an independent input source.

Head tracking without specialist hardware

I wanted the effect to work using ordinary hardware rather than requiring a VR headset or dedicated tracking system.

A webcam was enough to test the idea.

I created a Python/OpenCV sidecar that detects the user's face and converts its position into normalised values. These values are sent to Godot over UDP and used to offset the camera.

Horizontal and vertical head movement therefore alter the viewpoint into the scene.

I later used the apparent size of the tracked face to estimate movement towards and away from the monitor, adding a third approximate depth value.

The result is closer to looking into a box than looking at a traditional desktop.

If the user moves left, their viewpoint shifts left.

If they lean closer, the virtual camera moves in.

Objects at different depths reveal slightly different parts of themselves.

Several small refinements were required before this felt stable:

• calibration;
• smoothing;
• deadzones for insignificant movements;
• preserving the last useful position when tracking is temporarily lost;
• and keeping head tracking independent of mouse interaction.

This phase proved the emotional part of the concept.

Even using ordinary consumer hardware, the display could begin to feel less like a flat image and more like an opening into a small space.

But everything inside that space was still fake.

The next question was much harder.

The difficult part: bringing real applications inside

Phase 3 — The first window capture experiment

The project would not be useful if every application had to be recreated specifically for Desktop Diorama.

Its value depended on compatibility with the software someone already uses.

The central technical question became:

Can a real application window become the content of one of these panels?

I deliberately paused most visual development at this point.

There were plenty of tempting things I could have built: scenery, animated backgrounds, weather, decorative objects, themes and polished UI.

None of those mattered if real applications could not survive inside the system.

The first experiment therefore used the simplest possible capture approach.

Python captured the region of the desktop occupied by another application, wrote the result to an image and allowed Godot to display that image on a panel.

It worked.

And it immediately demonstrated why it could not be the final solution.

When something moved in front of the source application, the capture showed the object covering it.

I was not capturing a window.

I was photographing part of the monitor.

That distinction became critical because Desktop Diorama itself would eventually cover the entire screen.

Phase 4 — Capturing a window instead of the screen

The next prototype used the Windows PrintWindow API.

This allowed the capture process to target a specific native window handle rather than a rectangle of desktop pixels.

For ordinary applications the result was significantly better.

Windows such as:

• Notepad;
• Calculator;
• File Explorer;
• and largely static browser content

could continue appearing inside Godot even when another application covered them on the real desktop.

That solved one problem while exposing another.

GPU-accelerated content behaved differently.

Video playing inside browsers or applications such as VLC could freeze, blur or stop updating correctly after the original source was occluded.

The application itself was still running.

The capture method simply was not receiving the correct rendered surface.

This was an important point in the project because it demonstrated that “window capture” is not one problem.

Different applications use different rendering paths. Static UI, accelerated browser surfaces and video playback cannot automatically be treated as equivalent sources.

PrintWindow remained useful evidence, but it was not a strong enough architectural foundation.

Phase 5 — Windows Graphics Capture

The next direction was Windows Graphics Capture.

I initially approached it through Python because Python made it possible to answer the technical question quickly without rebuilding the surrounding prototype.

Frames were captured and passed towards Godot using a temporary file-based bridge.

The capture itself was much more promising.

The bridge was not.

Each frame could involve:

1. capturing the image;
2. converting or encoding it;
3. writing it to disk;
4. updating information about the frame;
5. detecting the update in Godot;
6. reading the file again;
7. decoding it;
8. and replacing the texture.

For an occasional image, this would be perfectly reasonable.

For something expected to behave like a live desktop, it was the software equivalent of sending every frame of a film through the post.

File locking, encoding overhead and repeated disk access created exactly the kind of latency a spatial interface cannot tolerate.

This prototype had nevertheless done its job.

Python had allowed me to prove the direction without prematurely committing to a much larger native implementation.

The prototype was scaffolding.

Once the architecture it was supporting became clear, the scaffolding could come down.

Moving closer to the operating system

Phase 6 — Native C++ streaming

The project then moved its critical capture path from the Python prototype into native C++.

This was one of the largest technical milestones so far.

The goal was not to rewrite working code simply because C++ was available. It was to remove work that did not belong in the final frame path.

A desktop interface cannot feel spatial if its content visibly trails the real application. The virtual panel has to feel like the application itself, not like a delayed broadcast of it.

The native capture work substantially changed that behaviour.

In current testing, captured video can now be displayed through the Godot panel at effectively the same frame rate as the source application, with no visible delay.

That result moved Desktop Diorama across an important threshold.

Before native streaming, I knew that real windows could technically appear in the scene.

After native streaming, I knew they could begin to live there.

The architecture has consequently evolved into a deliberate division of responsibility:

Windows / native application
        ↓
Native capture and OS integration
        ↓
Live frame stream
        ↓
Godot texture
        ↓
3D panel
        ↓
Spatial transformation and presentation
        ↓
User

Godot remains responsible for the thing it is particularly good at:

• 3D presentation;
• cameras;
• materials;
• spatial interaction;
• scene management;
• and product UI.

Native code handles the parts closest to Windows:

• application capture;
• source window integration;
• input forwarding;
• and other operating-system-specific behaviour.

Rather than forcing one technology to solve every layer of the product, each part of the architecture has gradually moved towards the tool best suited to it.

Turning captured windows back into applications

Phase 7 — Input through a transformed surface

Displaying a real window was only half of the problem.

A live browser that cannot be clicked is still essentially a television.

For Desktop Diorama to become an interface rather than a visualiser, interaction on a spatial panel had to reach the original application.

That introduced another transformation problem.

A user is not clicking the real Windows window.

They are clicking a three-dimensional representation of that window after it has been:

• positioned in a scene;
• scaled;
• potentially moved in depth;
• viewed through a perspective camera;
• and projected back onto a two-dimensional monitor.

The system therefore needs to determine where the user clicked on the panel and translate that position back into the coordinate system of the original application.

The current prototype does this by interpolating the position on the panel and converting it into the corresponding source-window coordinates.

The result is passed back to the real application.

The loop therefore becomes:

Mouse position
      ↓
Godot 3D ray / panel intersection
      ↓
Panel-local position
      ↓
Normalised surface coordinates
      ↓
Source-window pixel coordinates
      ↓
Native input
      ↓
Real application

This was another major viability milestone.

A browser rendered inside the diorama is no longer only displaying a browser.

The user can click the browser.

That distinction transforms the project from passive capture into interactive spatial computing.

Resizing the representation was not enough

Early resizing changed the size of the Godot panel.

Visually, that worked.

Conceptually, it was wrong.

The captured application texture simply stretched to fill its new surface. Text, controls and content changed scale as though the user were resizing a photograph.

That broke the illusion immediately.

A normal application responds to resizing.

Its content reflows. Toolbars reorganise. Text wraps. Viewports change. Websites respond to their new dimensions.

The diorama therefore needed to resize the real source window, not merely the mesh displaying it.

That is now substantially working.

Changing the dimensions of the spatial panel can alter the dimensions of the original application, after which the application performs its own normal layout behaviour and the capture updates accordingly.

The difference is subtle in architecture but enormous in experience.

The panel stops behaving like a picture of a window.

It starts behaving like the window itself.

The current core loop

The project has now assembled the pieces that originally existed as separate experiments.

A real application can:

1. run normally in Windows;
2. be captured through the native integration;
3. stream into Godot;
4. appear on a panel inside the 3D environment;
5. exist at a particular position and depth;
6. be viewed through head-tracked perspective;
7. be moved spatially;
8. be resized while affecting the real source window;
9. receive clicks translated back through the panel;
10. and continue updating live.

Conceptually:

          ┌───────────────────────┐
          │ Real Windows app      │
          └───────────┬───────────┘
                      │
                 native capture
                      │
                      ▼
          ┌───────────────────────┐
          │ Live Godot texture    │
          └───────────┬───────────┘
                      │
                3D transform
                      │
                      ▼
          ┌───────────────────────┐
          │ Spatial app panel     │
          └───────────┬───────────┘
                      │
                   user input
                      │
                      ▼
          transformed coordinates
                      │
                      ▼
          ┌───────────────────────┐
          │ Original Windows app  │
          └───────────────────────┘

This feedback loop is the heart of Desktop Diorama.

Everything else now sits around making that loop reliable, comfortable and useful.

Product principles that emerged through development

1. Spatiality has to earn its existence

The project is not intended to make ordinary computing more complicated simply because 3D is available.

Depth should eventually help with organisation, context and personalisation.

If a conventional 2D interaction remains better, I would rather preserve it than force a spatial alternative.

That is part of why left-click behaviour continues to belong to the real application.

2. Existing applications remain first-class citizens

Desktop Diorama does not ask developers to create special “Diorama apps”.

A browser should remain the user's browser.

Their editor should remain their editor.

Their communication tools should remain their communication tools.

The product is an interface layer, not another ecosystem designed to trap applications inside a proprietary format.

3. Prototype the unknown, then replace the prototype

Python was extremely valuable precisely because it did not have to remain the final architecture.

The capture experiments progressed through increasingly appropriate approaches:

screen-region capture → `PrintWindow` → Windows Graphics Capture → native C++ streaming

Each solution answered a question.

Each failure reduced uncertainty.

Moving away from an earlier implementation was therefore not wasted work. The implementation had already delivered the information it existed to discover.

4. Performance is part of accessibility

Spatial interfaces are often associated with expensive hardware.

I want Desktop Diorama to move in the opposite direction.

Head tracking currently requires only an ordinary webcam. The interface runs on a conventional monitor. Godot was chosen partly because it gives me control over the rendering environment without requiring a heavyweight browser-based desktop shell.

As the project matures, performance will remain a product requirement rather than a final optimisation pass.

That includes investigating:

• independent capture rates for different windows;
• reducing work for unchanged or background content;
• scalable visual quality;
• lightweight scenes;
• and sensible low-power behaviour.

The ambition is not to demonstrate the most technologically extravagant desktop possible.

It is to make an unusual interface accessible to ordinary computers.

5. The desktop should ultimately feel personal rather than merely impressive

The head tracking and spatial effects are visually striking, but spectacle is not the end goal.

Desktop Diorama is ultimately interested in what happens after the novelty disappears.

Can somebody arrange their workspace in a way that feels familiar?

Can applications occupy places they remember?

Can a computer feel less like an interchangeable productivity surface and more like a small environment belonging to its user?

That is the larger product question behind the technical work.

Current state

Desktop Diorama is currently best described as a functional technical alpha moving towards MVP hardening.

The central concept has progressed beyond speculative prototyping.

Working or substantially proven

• reusable 3D application panels
• independent panel movement
• depth positioning
• resizing
• spatial selection and manipulation
• configurable focus and panel behaviour
• camera modes
• mouse-controlled debug parallax
• webcam-based head tracking
• approximate forward/back head movement
• tracking calibration and smoothing
• native capture of real Windows applications
• capture that survives normal desktop occlusion
• video streaming with source-like frame rate in current testing
• source-aware window resizing
• coordinate interpolation between a transformed panel and the real application
• click-through interaction
• persistent spatial state foundations

Most importantly, the central interaction loop exists.

The project is no longer asking whether a real application could be made spatial.

It is now asking how reliable and comfortable that experience can become.

The largest remaining technical areas

The next problems are less existential but more numerous:

• keyboard focus and typing behaviour;
• broader mouse/input forwarding;
• reliable behaviour when source applications close or minimise;
• reconnecting persistent panels to real applications;
• handling application title or process changes;
• managing several captured applications simultaneously;
• mixed-DPI displays;
• multi-monitor coordinate systems;
• native helper lifecycle and recovery;
• failure states;
• performance profiling;
• capture throttling;
• and installation/packaging.

This is a different type of development from the earlier phases.

The first half of the project has largely been about proving possibility.

The next is about earning trust.

What MVP means for this project

I do not consider “the demo works on my machine” to be the MVP.

The milestone I am working towards is simpler to describe:

I should be able to use Desktop Diorama as my working environment for an hour without having to think about Desktop Diorama.

That test changes the priorities considerably.

Beautiful scenery cannot compensate for broken keyboard focus.

A sophisticated background cannot compensate for a panel losing its application.

An impressive head-tracking effect cannot compensate for clicking the wrong coordinate.

The immediate roadmap therefore prioritises reliability over spectacle.

A useful first workspace might contain only:

• a browser;
• an editor or notes application;
• a reference or communication window;
• optional head tracking;
• a persistent spatial arrangement;
• and a simple environment.

If using those applications feels natural for an extended session, the core product hypothesis becomes much stronger.

Roadmap

Stage 1 — Harden the spatial application loop

The immediate work is concerned with making existing behaviour dependable.

Priorities include:

1. complete keyboard focus and typing support;
2. broaden and harden mouse input forwarding;
3. handle source-window lifecycle changes cleanly;
4. strengthen resizing and coordinate mapping edge cases;
5. reconnect saved spatial state to application windows reliably;
6. manage several live sources together;
7. test DPI scaling and different resolutions;
8. profile CPU, GPU and memory behaviour;
9. improve failure recovery;
10. package native components alongside the Godot application.

This stage moves the project from technical alpha towards something that can survive ordinary daily behaviour rather than curated demonstrations.

Stage 2 — Complete the first spatial workspace

The first intended product version expands the vocabulary of the workspace itself.

Panel tilt and rotation

Panels currently establish depth primarily through position.

Allowing them to tilt and rotate will make spatial arrangements substantially more expressive, but it also makes coordinate transformation and interaction more demanding.

The goal is not arbitrary 3D chaos.

Rotation should make it possible to create useful arrangements such as slightly angled side displays while preserving readable content and predictable input.

Multi-display support

Multi-monitor support is a first-version requirement rather than an afterthought.

The intended architecture treats each physical display as its own viewpoint into the shared workspace, with one virtual camera corresponding to each monitor.

This is important because perspective is viewer-dependent.

Simply stretching one Godot camera across several physical displays would create incorrect geometry. Treating displays as separate viewpoints allows the illusion to remain coherent as the workspace expands.

This also turns Desktop Diorama from an interesting single-screen effect into something capable of addressing serious multi-window workflows.

Stage 3 — Productise the developer prototype

Once reliability is strong enough, the product needs to become understandable without access to its source code.

That includes:

• onboarding;
• application/source selection;
• head-tracking calibration;
• sensitivity controls;
• panel management;
• layout recovery;
• clear capture states;
• input/focus settings;
• performance controls;
• accessible defaults;
• and graceful explanations when something cannot be captured.

The existing development options are useful because they let individual behaviours be tested independently.

A user-facing settings system should translate those capabilities into understandable choices rather than exposing implementation details.

Stage 4 — Privacy and creator-facing behaviour

A spatial desktop can potentially expose much more information visually than an ordinary application.

Future versions therefore need explicit privacy behaviour rather than relying solely on whatever happens to be visible in the scene.

Planned areas include:

• privacy blackout or hiding for selected panels;
• clear indication of what is currently being captured;
• safer handling when screen sharing;
• intentional capture exclusions;
• and behaviour designed with OBS and recorded demonstrations in mind.

The aim is for people to understand when an application exists only for them and when it may also be visible to an audience.

Stage 5 — Build the actual diorama

Only after the interaction layer is trustworthy does it make sense to invest heavily in the environmental side of the project.

Possible workspace directions include:

• open landscapes;
• sky and atmospheric backgrounds;
• gentle time-of-day changes;
• weather;
• ambient animation;
• more architectural environments;
• workspace presets;
• seasonal variations;
• and eventually user-created environments.

The visual layer is not unimportant.

It is deliberately late.

I want the scenery to express an already useful product rather than disguise a fragile one.

What has changed since the original idea?

The clearest way to describe the project's history is through the questions it has answered.

Could application-like panels exist convincingly in depth?

Yes.

The initial Godot panel system established spatial movement, resizing and selection.

Could the monitor feel like a window into that space?

Yes.

Head tracking demonstrated convincing perspective using a normal webcam.

Could real applications replace simulated panel content?

Yes.

Several generations of capture experiments proved increasingly robust ways of bringing Windows applications into Godot.

Could demanding content such as video remain live?

Yes, substantially.

Moving the critical capture path into native C++ produced source-like playback performance without visible delay in current testing.

Could a user interact with the real application through its 3D representation?

Yes.

Panel coordinates can now be transformed back into source-window coordinates and passed to the original application.

Could resizing feel like resizing a window rather than stretching a video?

Yes.

The spatial panel can now affect the underlying source dimensions and allow the real application to respond.

Is it ready to replace the desktop?

Not yet.

Reliability, input completeness, window lifecycle behaviour, multiple displays, packaging and sustained real-world use remain ahead.

That distinction is important.

The project has proven its central mechanism, not completed its productisation.

The largest lessons so far

Technical feasibility should be treated as product discovery

Much of the early work could easily be described as engineering research.

I consider it product discovery.

Whether Windows Graphics Capture behaves correctly behind an occluded GPU-rendered application directly determines what product can be built.

Whether transformed input remains reliable directly determines which interaction model is viable.

Technical uncertainty and user-value uncertainty are not always separate things.

In an experimental interface product, architecture can be part of discovery.

The quickest prototype is not always the architecture

The file-based Python bridge was not a mistake because it was later removed.

It cheaply answered the question I had at that stage.

Keeping it after the question had changed would have been the mistake.

That distinction has become one of the most useful lessons from the project:

Prototype for information, architect for responsibility.

Illusions are systems of agreement

The spatial effect does not depend on one spectacular feature.

It depends on several systems agreeing with each other.

The camera says the panel exists at a particular depth.

The collision system says the cursor touched a particular point.

The capture system says a particular application exists on that surface.

The input system says that point corresponds to a particular source pixel.

The source application says its content has a particular size.

If one part disagrees noticeably with the others, the illusion weakens.

Building Desktop Diorama has therefore been less like creating a 3D scene and more like keeping several coordinate systems in conversation.

Product polish is valuable only after the product survives contact with reality

There have been many opportunities to make Desktop Diorama prettier.

I have deliberately postponed most of them.

A polished environment surrounding an unreliable capture system would create excellent screenshots and a poor product.

That risk-first approach has allowed each major development phase to reduce a specific uncertainty before the project becomes more expensive to change.

Outcome so far

Desktop Diorama began as a visual thought experiment.

It has since moved through:

• spatial interaction prototyping;
• reusable 3D panel architecture;
• head-tracked camera experiments;
• screen-region capture;
• native-window capture experiments;
• GPU-aware capture investigation;
• Windows Graphics Capture;
• Python streaming prototypes;
• native C++ integration;
• source-window resizing;
• transformed coordinate mapping;
• and interactive click-through.

The most important outcome is not any one implementation.

It is that the original idea has survived progressively harsher tests.

The project started with:

Could a desktop have convincing physical depth?

The question now is:

Can that spatial desktop become dependable enough that somebody chooses to work inside it?

That is a considerably more interesting problem to have.

Skills demonstrated

Desktop Diorama combines product work with unusually deep technical prototyping.

It demonstrates:

• product discovery;
• product strategy;
• hypothesis-driven development;
• risk prioritisation;
• interaction design;
• spatial UX;
• Godot and GDScript development;
• 3D scene architecture;
• camera systems;
• computer vision prototyping;
• Python;
• OpenCV;
• UDP communication;
• native Windows integration;
• Windows capture APIs;
• C++ development;
• real-time streaming;
• coordinate transformation;
• input forwarding;
• performance thinking;
• iterative architecture;
• debugging across multiple processes and coordinate systems;
• accessibility-conscious technical planning;
• and roadmap development.

More importantly, it demonstrates the ability to move between product and engineering decisions without treating them as separate worlds.

The decision to postpone environmental polish was a product decision informed by technical risk.

The move from Python to C++ was a technical decision informed by the interaction experience.

The decision to preserve ordinary application controls was a UX decision informed by compatibility.

Each layer constrains and informs the others.

Reflection

Desktop Diorama is still unfinished, but it has reached the point where I no longer think of it primarily as a speculative interface.

It works.

Not yet reliably enough, not yet broadly enough and not yet elegantly enough to call finished—but enough of the complete loop exists that the remaining work can be discussed as a roadmap rather than a collection of unknowns.

That is probably the part of the project I value most.

The original concept contained a significant amount of uncertainty. It would have been easy to spend months building an attractive 3D world around fake application windows and discover much later that the operating-system integration was impractical.

Instead, each stage moved towards the next uncomfortable question.

The fake panels had to become real windows.

The captured windows had to survive occlusion.

The static windows had to become video.

The video had to become fast.

The display had to become interactive.

The resized picture had to become a resized application.

There are still difficult questions ahead, particularly around robust input, multi-display geometry, performance and product packaging.

But they are now the problems of building the product well rather than proving that the product can exist.

For me, Desktop Diorama represents the kind of technology I am most interested in creating: technically curious without being technology for its own sake, playful without becoming disposable, and interested in making the relationship between people and computers feel warmer, more tangible and more personal.

The long-term ambition is not simply to put windows in 3D.

It is to make the computer feel a little more like somewhere you are.