Viewing desktop in-scene

Question

Is there a way to view my desktop or other apps "behind" the game in-game? I know there are browsers and even potentially remote connections to view a separate machine, but I'd like to do it on the same one. I'm thinking something like viewing a terminal in-scene (probably inside a Render Texture) that your character can walk up to, but be able to open Paint or start up a game from your Steam library and leave it running on that in-scene console. Or sit on a golden throne on top of a mountain while updating the source code for your newest game on a magical floating portal that can view your desktop.

Answer 1

 //Note: Inspired by and uses some code found here: http://forum.unity3d.com/threads/windows-api-calls.127719/
 
 using UnityEngine;
 using System.Collections;
 using System;
 using System.Runtime.InteropServices; // Pro and Free!!!
 
 //WARNING!! Running this code inside Unity when not in a build will make the entire development environment transparent
 //Restarting Unity would however resolve
 
 public class TransparentWindow : MonoBehaviour
 {
     [DllImport("user32.dll", EntryPoint = "SetWindowLongA")]
     static extern int SetWindowLong(int hwnd, int nIndex, long dwNewLong);
     [DllImport("user32.dll")]
     static extern bool ShowWindowAsync(int hWnd, int nCmdShow);
     [DllImport("user32.dll", EntryPoint = "SetLayeredWindowAttributes")]
     static extern int SetLayeredWindowAttributes(int hwnd, int crKey, byte bAlpha, int dwFlags);
     [DllImport("user32.dll", EntryPoint = "GetActiveWindow")]
     private static extern int GetActiveWindow();
     [DllImport("user32.dll", EntryPoint = "GetWindowLong")]
     private static extern long GetWindowLong(int hwnd, int nIndex);
     [DllImport("user32.dll", EntryPoint = "SetWindowPos")]
     private static extern int SetWindowPos(int hwnd, int hwndInsertAfter, int x, int y, int cx, int cy, int uFlags);
 
     void Start()
     {
         int handle = GetActiveWindow();
         int fWidth = Screen.width;
         int fHeight = Screen.height;
 
         //Remove title bar
         long lCurStyle = GetWindowLong(handle, -16);     // GWL_STYLE=-16
         int a = 12582912;   //WS_CAPTION = 0x00C00000L
         int b = 1048576;    //WS_HSCROLL = 0x00100000L
         int c = 2097152;    //WS_VSCROLL = 0x00200000L
         int d = 524288;     //WS_SYSMENU = 0x00080000L
         int e = 16777216;   //WS_MAXIMIZE = 0x01000000L
 
         lCurStyle &= ~(a | b | c | d);
         lCurStyle &= e;
         SetWindowLong(handle, -16, lCurStyle);// GWL_STYLE=-16
 
         // Transparent windows with click through
         SetWindowLong(handle, -20, 524288 | 32);//GWL_EXSTYLE=-20; WS_EX_LAYERED=524288=&h80000, WS_EX_TRANSPARENT=32=0x00000020L
         SetLayeredWindowAttributes(handle, 0, 51, 2);// Transparency=51=20%, LWA_ALPHA=2
 
         SetWindowPos(handle, 0, 0, 0, fWidth, fHeight, 32 | 64); //SWP_FRAMECHANGED = 0x0020 (32); //SWP_SHOWWINDOW = 0x0040 (64)
         ShowWindowAsync(handle, 3); //Forces window to show in case of unresponsive app    // SW_SHOWMAXIMIZED(3)
     }
 }

1) Add the above code to your project

2) Go to your camera and select it. In the Inspector, change the colors and the alpha to 0.

3) Build the (Windows) app - change the settings to windowed, then run

You now have a full screen window (without title bar) that allows the user to select things behind it, while at the same time still displaying anything you've rendered on top :) Note: this is a windows only solution.

Edit 6: Updated and improved for Aero compatibility.

Answer 2

Answer by krougeau · May 05, 2015 at 02:15 PM

I know it's been a couple of months since you folks had this discussion, but with the help of a terrific Unity admin, I have the answer to 100% window transparency with 100% visible game objects! :) LWA_COLORKEY is a red herring... For all the details, and an example, have a look at this thread on the Unity forums: http://forum.unity3d.com/threads/solved-windows-transparent-window-with-opaque-contents-lwa_colorkey.323057/

Add comment · Share

Answer 3

Thanks so much for letting me know! I got it working with click-through and always on top

using System; using System.Runtime.InteropServices; using UnityEngine;

public class TransparentWindow : MonoBehaviour { [SerializeField] private Material m_Material;

 private struct MARGINS
 {
     public int cxLeftWidth;
     public int cxRightWidth;
     public int cyTopHeight;
     public int cyBottomHeight;
 }

 [DllImport("user32.dll")]
 private static extern IntPtr GetActiveWindow();

 [DllImport("user32.dll")]
 private static extern int SetWindowLong(IntPtr hWnd, int nIndex, uint dwNewLong);

 [DllImport("user32.dll")]
 static extern bool ShowWindowAsync(IntPtr hWnd, int nCmdShow);

 [DllImport("user32.dll", EntryPoint = "SetLayeredWindowAttributes")]
 static extern int SetLayeredWindowAttributes(IntPtr hwnd, int crKey, byte bAlpha, int dwFlags);

 [DllImport("user32.dll", EntryPoint = "SetWindowPos")]
 private static extern int SetWindowPos(IntPtr hwnd, int hwndInsertAfter, int x, int y, int cx, int cy, int uFlags);

 [DllImport("Dwmapi.dll")]
 private static extern uint DwmExtendFrameIntoClientArea(IntPtr hWnd, ref MARGINS margins);

 const int GWL_STYLE = -16;
 const uint WS_POPUP = 0x80000000;
 const uint WS_VISIBLE = 0x10000000;
 const int HWND_TOPMOST = -1;

 void Start()
 {
     #if !UNITY_EDITOR   // You really don't want to enable this in the editor..
     
     int fWidth = Screen.width;
     int fHeight = Screen.height;
     var margins = new MARGINS() { cxLeftWidth = -1 };
     var hwnd = GetActiveWindow();

     SetWindowLong(hwnd, GWL_STYLE, WS_POPUP | WS_VISIBLE);

     // Transparent windows with click through
     SetWindowLong(hwnd, -20, 524288 | 32);//GWL_EXSTYLE=-20; WS_EX_LAYERED=524288=&h80000, WS_EX_TRANSPARENT=32=0x00000020L
     SetLayeredWindowAttributes(hwnd, 0, 255, 2);// Transparency=51=20%, LWA_ALPHA=2
     SetWindowPos(hwnd, HWND_TOPMOST, 0, 0, fWidth, fHeight, 32 | 64); //SWP_FRAMECHANGED = 0x0020 (32); //SWP_SHOWWINDOW = 0x0040 (64)
     DwmExtendFrameIntoClientArea(hwnd, ref margins);

endif

}

 void OnRenderImage(RenderTexture from, RenderTexture to)
 {
     Graphics.Blit(from, to, m_Material);
 }

}

Answer 4

Answer by emathew · Aug 18, 2017 at 12:10 PM

Hey all!

Any idea how to this on Mac OS?

Add comment · Share

Answer 5

I typed a big answer about how to do what you're trying to do, you want your PC and/or apps to run and be rendered in a running Unity build. Unfortunately, this 'help site' decided my post was an attack and erased what I wrote. So, I'm not going to type it all again, but it is easy to do (if you know what you're doing). The answers on this page seem to have nothing to do with your question, they are mostly for altering windows. You had it right when you mentioned Remote Desktop, basically. That is the way Remote Desktop works, it takes the video-memory stream, translates it to a mobile-friendly version, then feeds it over the internet to your device/PC. That's the simple part, ANY data going through your computer can be read, copied or altered. Graphics, video, sound, text, data from programs, input from the keyboard, etc. Unity would be even faster than Remote Desktop because it wouldn't have to send the data online, assuming Unity was running on the device you wanted to 'stream'. Also, instead of streaming to Unity or emulating an app inside Unity, sometimes it might just be easier to write a basic C#/java web-browser that runs inside Unity, or a 'fake Desktop' that interfaces with Windows/Android.

Viewing desktop in-scene

8 Replies

endif

Your answer

Welcome to Unity Answers

Follow this Question

Related Questions