Camera jitters / lag behind when jumping or falling

It is usually inadvisable to directly modify velocity. It can make for some erratic behaviour. Jitter usually arises from differences of updates between FixedUpdate and Update. Timing of FixedUpdate is not really as Fixed as it may suggest and sometimes placing code in Update instead produces better results, creating better scaling of variables with respect to how the User views it from frame to frame.

Movement by direct Transform modification can also create some jittery results. A combination of the two could be asking for trouble. The results will be worsened at lower fps.

The reason why your character jitters in vertical whilst horizontal Velocity is being set is that Spamming a velocity component to 0f will cause that object to stop dead in that direction and then regain speed under the Physics Engines computational updates. This is why a lot of people end up with Player-characters that float like clouds when jumping.

When setting Velocity, make sure to first read off the Current Velocity x,y,z values and use those as a base for your next calculation, adding or subtracting a scaled amount, instead of hard setting values. This serves to add some (de)/(a)cceleration.