Is it possible to rotate the entire screen orientation?

Certainly! If you don't want to change the screen resolution in Windows you could always just rotate your camera -90 on the z axis.

Your UI and controls would also have to be rotated accordingly.

One way to rotate your canvas would be to add a Grid Layout Group to it:

I set the cell size to be my screen resolution and made sure I set the canvas to Screen Space Overlay.

Next I added a layout object that could be set to fill the whole screen and contain all my UI items. That item should be set to fill the whole screen. I added all my UI elements under it.

Finally I added a script to my canvas. The script has a "tateMode" boolean to track the current state and rotates the camera and UI to match. The script I made is below, and I've zipped the whole project and attached it so you can test it out. I think using this the controls could remain the same.

 using System.Collections;
 using System.Collections.Generic;
 using UnityEngine;
 using UnityEngine.UI;
 
 public class FlipToTATE : MonoBehaviour {
 
     private bool tateMode;
     public GameObject myCamera;
     public GridLayoutGroup myGrid;
     public RectTransform layoutRotator;
 
     // Use this for initialization
     void Start () {
         tateMode = false;
 
     }
     
     // Update is called once per frame
     void Update () {
         if (Input.GetKeyUp(KeyCode.Space))
         {
             tateMode = !tateMode;
             if (tateMode)
             {
                 myCamera.transform.eulerAngles = new Vector3(0f, 0f, -90f);
                 layoutRotator.eulerAngles = new Vector3(0f,0f,90f);
                 myGrid.cellSize = new Vector2(Screen.currentResolution.height, Screen.currentResolution.width);
             }
             else
             {
                 myCamera.transform.eulerAngles = new Vector3(0f, 0f, 0f);
                 layoutRotator.eulerAngles = new Vector3(0f, 0f, 0f);
                 myGrid.cellSize = new Vector2(Screen.currentResolution.width, Screen.currentResolution.height);
             }
         }
     }
 }

[2]: /storage/temp/127100-tate-mode.zip