Tangent Space Normal Map to World Space Normal Vector in C# Script

I am writing a path tracer in C# using Physics.Raycast, and I want to implement normal maps. When an object is hit by a raycast, I can correctly sample its normal map at the proper uv, then I convert the rgb pixel into an xyz vector. Once I have this normal vector, which is measured in the object's tangent space, I need to transform it into world space. I know this involves finding the tangent and binormal vectors to the surface, but I am unsure how to do so. Is there any consistent way of determining these vectors so that the results will match a normal-mapped shader?