• Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
  • Asset Store
  • Get Unity

UNITY ACCOUNT

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account
  • Blog
  • Forums
  • Answers
  • Evangelists
  • User Groups
  • Beta Program
  • Advisory Panel

Navigation

  • Home
  • Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
    • Blog
    • Forums
    • Answers
    • Evangelists
    • User Groups
    • Beta Program
    • Advisory Panel

Unity account

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account

Language

  • Chinese
  • Spanish
  • Japanese
  • Korean
  • Portuguese
  • Ask a question
  • Spaces
    • Default
    • Help Room
    • META
    • Moderators
    • Topics
    • Questions
    • Users
    • Badges
  • Home /
avatar image
3
Question by StephanM · Jan 05, 2015 at 03:06 PM · rendertexture

Viewing desktop in-scene

Is there a way to view my desktop or other apps "behind" the game in-game? I know there are browsers and even potentially remote connections to view a separate machine, but I'd like to do it on the same one. I'm thinking something like viewing a terminal in-scene (probably inside a Render Texture) that your character can walk up to, but be able to open Paint or start up a game from your Steam library and leave it running on that in-scene console. Or sit on a golden throne on top of a mountain while updating the source code for your newest game on a magical floating portal that can view your desktop.

Comment
Add comment · Show 14
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image richyrich · Jan 05, 2015 at 03:38 PM 0
Share

What's wrong with Alt+Tab?

avatar image StephanM · Jan 05, 2015 at 03:40 PM 0
Share

If I were sitting on golden throne on a mountain with a magical portal with a desktop IRL, alt+tab would be just fine ;) Until then, I need it to render in-scene in a unity project.

avatar image Mikenseer · Jan 12, 2015 at 10:48 PM 0
Share

If you could post a screenshot of the results, that would be spectacular.

Such as: is it possible to say recreate something like the spaceshooter tutorial with your desktop as the background ins$$anonymous$$d of a texture?

avatar image richyrich · Jan 13, 2015 at 02:24 AM 0
Share

Hi @$$anonymous$$ikenseer, @Stephan$$anonymous$$, @chrishtanaka. The code has been gutted and rebuilt with support for Windows 7 Classic and Aero styles. I've tested it on my Windows 7 machine. It needs testing on other Windows 7 and also Windows 8, in classic and Aero styles?

The code form as requested means that any mouse clicks made in game will actually go to the app behind and make it the active window (focus) for further inputs. While the game is in focus, unlike the mouse, keyboard keys will go to the game (I tested with a rotating cube with no problems).

avatar image chrishtanaka · Jan 13, 2015 at 06:26 AM 0
Share

@richyrich On Windows 8 setting the transparency level of the window seems to work fine. Your original code before your edit actually worked fine when setting the transparency of the entire window (LWA_ALPHA). However, using a color key (LWA_COLOR$$anonymous$$EY) still does not work.

Show more comments

8 Replies

· Add your reply
  • Sort: 
avatar image
6
Best Answer

Answer by richyrich · Jan 05, 2015 at 06:00 PM

 //Note: Inspired by and uses some code found here: http://forum.unity3d.com/threads/windows-api-calls.127719/
 
 using UnityEngine;
 using System.Collections;
 using System;
 using System.Runtime.InteropServices; // Pro and Free!!!
 
 //WARNING!! Running this code inside Unity when not in a build will make the entire development environment transparent
 //Restarting Unity would however resolve
 
 public class TransparentWindow : MonoBehaviour
 {
     [DllImport("user32.dll", EntryPoint = "SetWindowLongA")]
     static extern int SetWindowLong(int hwnd, int nIndex, long dwNewLong);
     [DllImport("user32.dll")]
     static extern bool ShowWindowAsync(int hWnd, int nCmdShow);
     [DllImport("user32.dll", EntryPoint = "SetLayeredWindowAttributes")]
     static extern int SetLayeredWindowAttributes(int hwnd, int crKey, byte bAlpha, int dwFlags);
     [DllImport("user32.dll", EntryPoint = "GetActiveWindow")]
     private static extern int GetActiveWindow();
     [DllImport("user32.dll", EntryPoint = "GetWindowLong")]
     private static extern long GetWindowLong(int hwnd, int nIndex);
     [DllImport("user32.dll", EntryPoint = "SetWindowPos")]
     private static extern int SetWindowPos(int hwnd, int hwndInsertAfter, int x, int y, int cx, int cy, int uFlags);
 
     void Start()
     {
         int handle = GetActiveWindow();
         int fWidth = Screen.width;
         int fHeight = Screen.height;
 
         //Remove title bar
         long lCurStyle = GetWindowLong(handle, -16);     // GWL_STYLE=-16
         int a = 12582912;   //WS_CAPTION = 0x00C00000L
         int b = 1048576;    //WS_HSCROLL = 0x00100000L
         int c = 2097152;    //WS_VSCROLL = 0x00200000L
         int d = 524288;     //WS_SYSMENU = 0x00080000L
         int e = 16777216;   //WS_MAXIMIZE = 0x01000000L
 
         lCurStyle &= ~(a | b | c | d);
         lCurStyle &= e;
         SetWindowLong(handle, -16, lCurStyle);// GWL_STYLE=-16
 
         // Transparent windows with click through
         SetWindowLong(handle, -20, 524288 | 32);//GWL_EXSTYLE=-20; WS_EX_LAYERED=524288=&h80000, WS_EX_TRANSPARENT=32=0x00000020L
         SetLayeredWindowAttributes(handle, 0, 51, 2);// Transparency=51=20%, LWA_ALPHA=2
 
         SetWindowPos(handle, 0, 0, 0, fWidth, fHeight, 32 | 64); //SWP_FRAMECHANGED = 0x0020 (32); //SWP_SHOWWINDOW = 0x0040 (64)
         ShowWindowAsync(handle, 3); //Forces window to show in case of unresponsive app    // SW_SHOWMAXIMIZED(3)
     }
 }

1) Add the above code to your project

2) Go to your camera and select it. In the Inspector, change the colors and the alpha to 0.

3) Build the (Windows) app - change the settings to windowed, then run

You now have a full screen window (without title bar) that allows the user to select things behind it, while at the same time still displaying anything you've rendered on top :) Note: this is a windows only solution.

Edit 6: Updated and improved for Aero compatibility.

Comment
Add comment · Show 15 · Share
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image StephanM · Jan 05, 2015 at 06:31 PM 0
Share

That's awesome!!! Not totally what I'm looking for, but it gives me hope that it can be done with the Windows API stuff. $$anonymous$$aybe it would be possible to turn the screen invisible, copy the screen contents, show the scene, and render the contents to a texture every frame. Thanks a ton!

avatar image richyrich · Jan 05, 2015 at 06:32 PM 0
Share

If using multiple platforms, the script will need pre-processor directives to avoid problems: http://docs.unity3d.com/$$anonymous$$anual/PlatformDependentCompilation.html

avatar image richyrich · Jan 05, 2015 at 06:39 PM 0
Share

If you follow the instructions underneath carefully, you will find that it is invisible (apart from the title bar and borders).

There is a Unity command that allows you to take a screenshot in game, so with the window invisible, you can use that (trim it) then render as required. If you only want part of the screen transparent, then you need only place a GameObject as background to half the screen.

avatar image StephanM · Jan 05, 2015 at 07:02 PM 0
Share

Great! I was just reading an edit of another answer of yours and wondering if that would be possible here, too. Although this would be great, it would be nice to be able to see the entire background making the game full-screen, or eliminating the borders. I wonder how this fits in with using an oculus rift. $$anonymous$$ight not even be applicable. That's my end goal, but I'm still waiting for the final version to come out before trying it out.

avatar image StephanM · Jan 06, 2015 at 12:19 PM 0
Share

Awesome! I have to give this a try! Thank you so much!!

Show more comments
avatar image
2

Answer by krougeau · May 05, 2015 at 02:15 PM

I know it's been a couple of months since you folks had this discussion, but with the help of a terrific Unity admin, I have the answer to 100% window transparency with 100% visible game objects! :) LWA_COLORKEY is a red herring... For all the details, and an example, have a look at this thread on the Unity forums: http://forum.unity3d.com/threads/solved-windows-transparent-window-with-opaque-contents-lwa_colorkey.323057/

Comment
Add comment · Share
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image
2

Answer by StephanM · May 10, 2015 at 06:46 PM

Thanks so much for letting me know! I got it working with click-through and always on top

using System; using System.Runtime.InteropServices; using UnityEngine;

public class TransparentWindow : MonoBehaviour { [SerializeField] private Material m_Material;

 private struct MARGINS
 {
     public int cxLeftWidth;
     public int cxRightWidth;
     public int cyTopHeight;
     public int cyBottomHeight;
 }

 [DllImport("user32.dll")]
 private static extern IntPtr GetActiveWindow();

 [DllImport("user32.dll")]
 private static extern int SetWindowLong(IntPtr hWnd, int nIndex, uint dwNewLong);

 [DllImport("user32.dll")]
 static extern bool ShowWindowAsync(IntPtr hWnd, int nCmdShow);

 [DllImport("user32.dll", EntryPoint = "SetLayeredWindowAttributes")]
 static extern int SetLayeredWindowAttributes(IntPtr hwnd, int crKey, byte bAlpha, int dwFlags);

 [DllImport("user32.dll", EntryPoint = "SetWindowPos")]
 private static extern int SetWindowPos(IntPtr hwnd, int hwndInsertAfter, int x, int y, int cx, int cy, int uFlags);

 [DllImport("Dwmapi.dll")]
 private static extern uint DwmExtendFrameIntoClientArea(IntPtr hWnd, ref MARGINS margins);

 const int GWL_STYLE = -16;
 const uint WS_POPUP = 0x80000000;
 const uint WS_VISIBLE = 0x10000000;
 const int HWND_TOPMOST = -1;

 void Start()
 {
     #if !UNITY_EDITOR   // You really don't want to enable this in the editor..
     
     int fWidth = Screen.width;
     int fHeight = Screen.height;
     var margins = new MARGINS() { cxLeftWidth = -1 };
     var hwnd = GetActiveWindow();

     SetWindowLong(hwnd, GWL_STYLE, WS_POPUP | WS_VISIBLE);

     // Transparent windows with click through
     SetWindowLong(hwnd, -20, 524288 | 32);//GWL_EXSTYLE=-20; WS_EX_LAYERED=524288=&h80000, WS_EX_TRANSPARENT=32=0x00000020L
     SetLayeredWindowAttributes(hwnd, 0, 255, 2);// Transparency=51=20%, LWA_ALPHA=2
     SetWindowPos(hwnd, HWND_TOPMOST, 0, 0, fWidth, fHeight, 32 | 64); //SWP_FRAMECHANGED = 0x0020 (32); //SWP_SHOWWINDOW = 0x0040 (64)
     DwmExtendFrameIntoClientArea(hwnd, ref margins);

endif

}

 void OnRenderImage(RenderTexture from, RenderTexture to)
 {
     Graphics.Blit(from, to, m_Material);
 }

}

Comment
Add comment · Show 2 · Share
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image niltoid · Jun 26, 2018 at 06:11 PM 0
Share

still works great on Unity 2017.1 - thank you, it gave me exactly the effect i was looking for

avatar image BigRot_ niltoid · Jan 27, 2019 at 08:54 PM 0
Share

Which material do i need to put in m_$$anonymous$$aterial? I can't understand. Thank you!

avatar image
1

Answer by emathew · Aug 18, 2017 at 12:10 PM

Hey all!

Any idea how to this on Mac OS?

Comment
Add comment · Share
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image
0

Answer by Honorsoft · Dec 21, 2017 at 07:47 AM

I typed a big answer about how to do what you're trying to do, you want your PC and/or apps to run and be rendered in a running Unity build. Unfortunately, this 'help site' decided my post was an attack and erased what I wrote. So, I'm not going to type it all again, but it is easy to do (if you know what you're doing). The answers on this page seem to have nothing to do with your question, they are mostly for altering windows. You had it right when you mentioned Remote Desktop, basically. That is the way Remote Desktop works, it takes the video-memory stream, translates it to a mobile-friendly version, then feeds it over the internet to your device/PC. That's the simple part, ANY data going through your computer can be read, copied or altered. Graphics, video, sound, text, data from programs, input from the keyboard, etc. Unity would be even faster than Remote Desktop because it wouldn't have to send the data online, assuming Unity was running on the device you wanted to 'stream'. Also, instead of streaming to Unity or emulating an app inside Unity, sometimes it might just be easier to write a basic C#/java web-browser that runs inside Unity, or a 'fake Desktop' that interfaces with Windows/Android.

Comment
Add comment · Share
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
  • 1
  • 2
  • ›

Your answer

Hint: You can notify a user about this post by typing @username

Up to 2 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

Welcome to Unity Answers

The best place to ask and answer questions about development with Unity.

To help users navigate the site we have posted a site navigation guide.

If you are a new user to Unity Answers, check out our FAQ for more information.

Make sure to check out our Knowledge Base for commonly asked Unity questions.

If you are a moderator, see our Moderator Guidelines page.

We are making improvements to UA, see the list of changes.



Follow this Question

Answers Answers and Comments

39 People are following this question.

avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image

Related Questions

How do you programmatically retrieve the rendering camera from its render texture? 2 Answers

what does Camera.targetTexture do? 1 Answer

Window Computer Lock causes a problem 1 Answer

Rendering screenshot using multiple cameras and one rendertexture 0 Answers

OnInspectorGUI Custom Drawing? 1 Answer

  • Anonymous
  • Sign in
  • Create
  • Ask a question
  • Spaces
  • Default
  • Help Room
  • META
  • Moderators
  • Explore
  • Topics
  • Questions
  • Users
  • Badges