2D RigidBody slightly penetrates objects causing a bounce effect.

So I am trying to set up a simple 2D platformer. I have Box colliders on my character and walls/floors. The collision is handled, just not correctly. When you collide with a wall, the character partially penetrates that wall, then the wall forces them back out causes a bouncing effect, and I can't get the character to stop flush to a wall.

The effect is basically a vibration between the two pictures below. I also have a link to a video for further clarification.

Video of the Issue

***Note : amtToMove is a float set to 5;

 void fixedUpdate()

 {
     if(Input.GetKey("right"))
     { 
         amtToMove = Input.GetAxisRaw("Horizontal") * playerSpeed * Time.deltaTime;

         //rigidbody.AddForce(Vector3.right * amtToMove);

         rigidbody.AddForce(Vector3.right * amtToMove,ForceMode.Impulse);
     }

     if(Input.GetKey("left"))
     { 
         amtToMove = Input.GetAxisRaw("Horizontal") * playerSpeed * Time.deltaTime;
         rigidbody.AddForce(Vector3.right * amtToMove,ForceMode.Impulse);
     }

 }

I have tried moving the character using Impulses, basic forces, and using basic translation. I am also handling the movement in the FixedUpdate() function.

I appreciate any and all help.