3d camera not working as I want it to

Hello, first post! Hope I'm doing t$$anonymous$$s right and put it in the right place, if there's any info needed please ask!

.
So.. I'm trying to get a camera working in 3d although I'm having some issues. It's a free orbit camera when mouse button 0 is held down and when it's not the camera rotates with my character.

.
My problem is that after rotating the camera using the character inputs instead of the mouse when I click the mouse it snaps the camera to it's last position before I started moving the character with WASD.

.
Ideally I'd like it to start orbiting from where it is currently I just can't figure it out for the life of me though. I've singled out the line of code that seems to be the problem and it is as follows: .

.

t$$anonymous$$s.xForm_Parent.rotation = Quaternion.Lerp(t$$anonymous$$s.xForm_Parent.rotation, Qt, Time.deltaTime * orbitDamp);

.

xForm_Parent.rotation isn't changing when my character rotates causing the camera to snap back to where it was. Qt is another quaternion based on the x and y axis of the mouse input.
.

Here is Qt: Quaternion Qt = Quaternion.Euler(localRotation.y, localRotation.x, 0); localRotation.y & x is set by the mouse input.
.
Sorry for the horrible formating, I couldn't figure out how to put in line breaks without text so ignore the periods between paragraphs. Any help would be greatly appreciated as it's driving me nuts.