Creating 3D Desktop Shell
25 feb 2026
# OffAxis Box Short Milestone Sequence
Only a 3D box/prism scene was set up. There was no camera yet; the initial goal was to establish the feeling of an off-axis box aligned with the physical screen plane.
A Win32 + DirectX 11 engine skeleton was written. Instead of using a ready-made engine, the platform, renderer, and projection layer were built from scratch.
Head/face tracking was added, and the 3D camera was tied to head movement.
The desktop image was mapped onto the opposite wall of the box.
Selected windows were added to the scene as 3D panels.
The desktop image began to be made interactive; with raycasting, the system mapped from the 3D surface to 2D desktop coordinates.
A 3D cursor independent from the system cursor was designed.
Click offsets were reduced by working on raycast and input-mapping issues.
The logic for focused and background windows was established, and spring-like transitions were added.
A more direct cursor/forwarding system based on raw mouse data was designed for interaction with windows.
Since GDI-based window capture created a performance bottleneck, the project switched to a Windows Graphics Capture–based GPU-to-GPU window pipeline.
On the face-tracking side, the path evolved through classic OpenCV, YuNet, eye tracking, a separate FaceTracking project, and finally MediaPipe Face Mesh.
Multiple filtering strategies were tested for calibration, thresholds, smoothing, and Z-axis stability.
More advanced visual systems such as ocular defocus, motion blur, and hover sharpness were planned for the desktop and background windows.
The project was not canceled; it was paused due to my current CPU and GPU limitations. It is planned to continue developing on a more powerful system.
full developing story's link here "https://drive.google.com/file/d/1V-nHgRIp_yicRvjV8wrhF-Segtc_wxNe/view?usp=sharing"
"https://drive.google.com/drive/folders/1wEtyVhpBP1Xlyo44b_ktJ6TrW_vpyz4C?usp=sharing"
and give me your feedback over feedback@yusuftok516.com please.