I only rotated X, but Y and Z are shifting as well when I play the game.

My question is in the title. My code is below.

using UnityEngine;
using System.Collections;

public class CameraController : MonoBehaviour {
public Transform target;
public float smoothing = 5f;
Vector3 offset;
public float rotateSpeed = 10f;

void Start() {
	offset = transform.position - target.position;
}

void FixedUpdate(){
	Vector3 targetCamPos = target.position + offset;
	transform.position = Vector3.Lerp (transform.position, targetCamPos, smoothing * Time.deltaTime);
	//transform.position = targetCamPos; Less smooth alternative method

	float h = Input.GetAxisRaw ("Horizontal");
	transform.Rotate(0f, h * 2, 0f);

}

}

You need to set the pivot point in the center of the object or else you would face something like this