ComputeBuffer stays empty in compute shader in URP

I'm trying to pass a ComputeBuffer to read on my ComputeShader with the Universal Render Pipeline, and everything works, except that the buffer remains empty in the shader.

I'm relatively new to the ScriptableRenderPipeline and shaders, tried to piece things together from various forum threads and SRP tutorials, so there might be some major concept I'm missing to get this to work. But I couldn't find any code example with ComputeShader in SRP, let alone using a ComputeBuffer.

I've created a separate project to isolate the problem, with only one ScriptableRenderPass, that creates a RenderTexture, dispatches the ComputeShader on it and blit it to the camera's texture. I send a float4 _Color for sanity check and draw it on every pixels, that works. I then send a StructuredBuffer _Colors with the exact same content, and draw it on every pixels, and I get nothing.

Here's the renderpass:

 class CustomRenderPass : ScriptableRenderPass {
 public ComputeShader computeShader;
 public string commandBufferName;
 public RenderTargetIdentifier source;

 RenderTexture target;
 RenderTargetIdentifier targetIdentifier;
 
 public override void Configure(CommandBuffer cmd, RenderTextureDescriptor cameraTextureDescriptor) {
     if (target != null)
         target.Release();
     
     target = new RenderTexture(cameraTextureDescriptor) {
         enableRandomWrite = true
     };
     target.Create();
     
     targetIdentifier = new RenderTargetIdentifier(target);
 }
 
 public override void Execute(ScriptableRenderContext context, ref RenderingData renderingData) {
     CommandBuffer cmd = CommandBufferPool.Get(commandBufferName);
     
     //Setting the compute shader's texture to target
     cmd.SetComputeTextureParam(computeShader, 0, "Result", targetIdentifier);
     
     //Setting the StructuredBuffer<float4> _Colors
     ComputeBuffer buffer = new ComputeBuffer(1, 4 * sizeof(float));
     buffer.SetData(new Vector4[] {new Vector4(1, 0, 0, 1)});
     cmd.SetComputeBufferParam(computeShader, 0, "_Colors", buffer);
     
     //Setting an example float4 _Color
     cmd.SetComputeVectorParam(computeShader, "_Color", new Vector4(1, 0, 0, 1));
     
     //Dispatching compute shader
     cmd.DispatchCompute(
         computeShader,
         0,
         Mathf.CeilToInt(Screen.width / 8f),
         Mathf.CeilToInt(Screen.height / 8f),
         1
     );
     
     //Pasting texture on the camera's
     cmd.Blit(targetIdentifier, source);
     
     context.ExecuteCommandBuffer(cmd);
     buffer.Release();
     CommandBufferPool.Release(cmd);
 }
 }

And here's the compute shader:

 #pragma kernel CSMain
 
 RWTexture2D<float4> Result;
 StructuredBuffer<float4> _Colors;
 float4 _Color;
 
 [numthreads(8,8,1)]
 void CSMain (uint3 id : SV_DispatchThreadID) {
     //This doesn't work
     Result[id.xy] = _Colors[0];
     
     //But this works
     //Result[id.xy] = _Color;
 }
 

I've removed everything else so I don't see how this could not work. I don't get any errors, it just draws black.

For context, I'm trying to render spheres with Ray Marching on top of my scene, and I was following GPU Ray Tracing in Unity for instructions on the Compute Shader as well as a Ray Marching tutorial, I've got hard coded spheres to render no issues, but I'm stuck when trying to pass a ComputeBuffer of sphere structs.