Stop or cancel movement if the player hits wall.

hello, I'm having a pretty annoying problem that I've tried everything in my skill-set. from be it 2D ray-casting(which didn't work because i need the box collider for triggers) to collision detection even just really complex nested if statements but Im still at square one with this problem.

here is how its supposed to go

1: player presses direction

2:player moves in that direction(while avoiding the circle antagonist who is paroling)

3:players collider hits the trigger on the key

4: player unlocks door

5:player reaches goal

now everything else works but when the play hits a wall it will jitters back and forth here is my code and why it does that:

 //Inspector Variables

 public bool Key = false;

 Vector3 pos;                                // For movement
 float speed = 2.0f;                         // Speed of movement

 void Start()
 {
     pos = transform.position;          // Take the initial position
 }

 void Update()
 {

     //
     if (Key == true)
     {
         this.tag = "Key";
     }
     else
     {
         this.tag = "Player";

     }

    
 }

 void FixedUpdate()
 {
 
     if (Input.GetKey(KeyCode.A) && transform.position == pos)
     {        // Left
         pos += new Vector3(-1.5f, 0, 0);
     }
     if (Input.GetKey(KeyCode.D) && transform.position == pos)
     {        // Right
         pos += new Vector3(1.5f, 0, 0);
     }
     if (Input.GetKey(KeyCode.W) && transform.position == pos)
     {        // Up
         pos += new Vector3(0, 1.5f, 0);
     }
     if (Input.GetKey(KeyCode.S) && transform.position == pos)
     {        // Down
         pos += new Vector3(0, -1.5f, 0);
     }
     transform.position = Vector3.MoveTowards(transform.position, pos, Time.deltaTime * speed);

 }

as you can see i use grid based movement, and keep it at a nice steady pace when it hits a wall(tagged Wall). It will continually try to push through the wall causing that jitter. This stops all movement. sorry for asking for a simple thing i know its a little embarrassing but I'm still trying to learn code.

How can I have it realize its hitting a wall and return to the last position or just cancel out the movement command all together

thank you in advance

(I should add those white boxes aren't individual but one big sprite grid)