Three out of four of these raycasts aren't working

My goal is to have four raycasts protruding from my player that, if they collide with a certain object, stops the player in that direction.

 RaycastHit2DUpRayHit=(Physics2D.Raycast(transform.position,Vector2.up,distance));
 
 RaycastHit2DDownRayHit=(Physics2D.Raycast(transform.position,Vector2.down,distance));
 
 RaycastHit2DRightRayHit=(Physics2D.Raycast(transform.position,Vector2.right,distance));
 
 RaycastHit2DLeftRayHit=(Physics2D.Raycast(transform.position,Vector2.left,distance));
 
 
 
 if(UpRayHit.collider.gameObject.tag=="Building")
 {
 Debug.Log("Working");
 
 SpeedUp=0;
 
 }
 
 
 if(DownRayHit.collider.gameObject.tag=="Building")
 {
 Debug.Log("Working");
 
 SpeedDown=0;
 
 }
 
 if(RightRayHit.collider.gameObject.tag=="Building")
 {
 Debug.Log("Working");
 
 SpeedRight=0;
 
 }
 
 
 if(LeftRayHit.collider.gameObject.tag=="Building")
 {
 Debug.Log("Working");
 
 SpeedLeft=0;
 
 }

The up raycast works perfectly, however the other three from what I can tell have their distances stunted for some reason. They only trigger in the very middle of the player (and the if statements execute.) Commenting out the first makes the second work, but then the other two still don't. I'm not an expect at programming and this is my first time using raycasts, is there something obvious I'm missing?