Raycast does not seem to properly detect object it hit

Hello,

I have this function below that projects a number of rays in a angle based on the parameters. My issue is that, as my images below illustrate, when I draw this ray with Debug.DrawRay, it seems that the rays pass through the object. However, when I do a Physics.Raycast using the same position and direction, it does not return true that it has hit the cube as my output from the print of the string is always empty string. Am I missing something in my understanding of how Raycasts work? (Utils.VectorHalfUp is (0,.5,0) by the way)

     public void AngledAttack(float startAngle, float forwardMultiplier, int numPoints, float angleIncrement) {
         Vector3 direction = (Quaternion.AngleAxis(startAngle, Vector3.up) * transform.rotation) * ((Vector3.forward * forwardMultiplier));
         var temp = pos;
         Quaternion stepAngle = Quaternion.AngleAxis(angleIncrement, Vector3.up);
         string collidedObject = "";
         Ray ray = new Ray(pos+Utils.VectorHalfUp, direction);
         print(ray);
         RaycastHit hit;
         for(int i = 0; i < numPoints; i++) {
             Debug.DrawRay(pos+Utils.VectorHalfUp, direction, Color.red);
             
             if(Physics.Raycast(ray, out hit, Mathf.Infinity)) {
                 collidedObject += hit.collider.gameObject.name;// "Hit: " + Utils.FindTaggedParent(hit.collider.gameObject).name + "\n";
             }
             direction = stepAngle * direction;
 
         }
         print(collidedObject);
     }