Change orthographic size with custom projection matrix in runtime

I am developing a two-player game. On one of client sides, I want to flip the camera. I used the script below to achieve that:

  void Start()
         {
             Matrix4x4 mat = _cam.projectionMatrix;
             mat *= Matrix4x4.Scale(new Vector3(-1, 1, 1));
             _cam.projectionMatrix = mat;
             Debug.Log("Camera is flipped");
         }

it works just fine. However, if I want to change zoom in the camera by changing the orthographicSize in runtime, the orthographicSize value didn't apply on the camera. I have read some documents, it says when a custom projection matrix is used, it will ignore the orthographicSize? Is there any way that I could perform camera zooming via modify the projectionMatrix? Thank you!,