1) The Slerp is probably doing you no good, because the next FixedUpdate you are just going to overwrite the rotation again.
In fact it might part of the problem, because Time.deltaTime is always going to be close to zero, so you are basically just going to set the rotation to something close to the current forward vector.
Try setting the rotation to the final rotation you desire directly, without any Slerp. Once that's correct you can work on making the transition look nice.
2) This is very minor, but note that the following line:
if (Vector3.Distance(relativePos, Vector3(0,0,0)) > 0 )
is equivalent to the following but not as clear:
if (relativePos.magnitude > 0)
If this was being used many times per frame you could replace that with
if (relativePos.sqrMagnitude > 0) // more efficient since you don't need the actual distance
or for that matter
if ( nowPos != orgPos ) // probably the most efficient
3) I'm wondering if the LookAt call and the setting of the Euler z rotation are interfering with each other. Try just one or the other.
4) General rotation tips:
Learn quaternions; they are generally
superior to Euler angles. You don't
have to understand how they work,
just treat them as a black box
representation of your object's
rotation.