Sniper scope zoom if crosshair is not in the middle of the screen

Hi, As you could understand I'm working on FPS game and I need to have a look into sniper scope but the thing is that my crosshair can be anywhere in the screen which makes some troubles for sniper aiming.

Here is my view before the sniper aim.

And my view after.

What I'm doing is decreasing camera's field of view but I get the unneeded offset (as you can see on the screenshot).

Could you please help me to figure out how to calculate new camera position or what do I need to do in this situation?

         var aimTransform = weapon.transform.Find("aim_pos");
         mainCamera.transform.position = aimTransform.position;
         //mainCamera.transform.rotation = aimTransform.rotation;
 
         //TODO: calculate "offset" to stay point at the same object as before zoom in
         // distance to first hit
         // mainCamera.ScreenPointToRay(new Vector3
         var position = new Vector3(crosshairPosition.x, Screen.height - crosshairPosition.y, 0);
         Ray rCast = mainCamera.ScreenPointToRay(position);
         //Debug.LogWarning("Ray param = " + rCast);
         Debug.DrawLine(rCast.origin, rCast.origin + rCast.direction * 10000, Color.yellow,600);
         
         RaycastHit hitInfo;
         bool wasHit = Physics.Raycast(rCast, out hitInfo);
         float distance;
         if (wasHit)
         {
             Debug.Log(string.Format("Hit {0} distance = {1}", hitInfo.collider.name, hitInfo.distance));
             distance = hitInfo.distance;
         }
         else
         {
             Debug.LogWarning("Not hit. take farClipPlane");
             distance = mainCamera.farClipPlane;
         }

         Vector3 wp1 = mainCamera.ScreenToWorldPoint(new Vector3(crosshairPosition.x, crosshairPosition.y, distance));
         Debug.Log(wp1);
         mainCamera.fieldOfView = zoomInFow;
         Vector3 wp2 = mainCamera.ScreenToWorldPoint(new Vector3(crosshairPosition.x, crosshairPosition.y, distance));
         Debug.Log("new " + wp2);
 
         var worldOffsetX = wp1.x - wp2.x;
         var worldOffsetY = wp1.y - wp2.y;
         var worldOffsetZ = wp1.z - wp2.z;
         Debug.LogWarning(string.Format("World offset x={0} y={1}", worldOffsetX, worldOffsetY));
 
 mainCamera.transform.localPosition = new Vector3(mainCamera.transform.localPosition.x + worldOffsetX, mainCamera.transform.localPosition.y + worldOffsetY, mainCamera.transform.localPosition.z);
 

Thank you, Ihor.