Prevent Render Texture clearing

In a nutshell, How can I prevent RenderTextures from being cleared after every frame?

I have a script that writes high intensity values to a render texture and I want to do an image effect with this over multiple frames while constantly drawing more to it; kinda like a camera with no clear flags. The problem with this is that Unity is damn dedicated to clear the RenderTexture on every frame with the camera content. Is it possible to prevent this somehow or make a workaround?

Here's my simple script for reference.

Unity 5.3.1p1

 using UnityEngine;
 
 [RequireComponent(typeof(Camera))]
 public class TemporalTest : MonoBehaviour {
 
     public Shader temporalTestShader;
     public RenderTexture rt;
     public Material temporalMaterial;
 
     void Start () {
         rt = new RenderTexture(Screen.width, Screen.height, 0);
         temporalMaterial = new Material(temporalTestShader);
     }
 
     void OnRenderImage(RenderTexture source, RenderTexture destination) {

         // rt gets filled correctly here
         // but it's cleared every frame for no reason causing nuclear headaches

         temporalMaterial.SetTexture("_TemporalMap", rt);
         Graphics.Blit(source, rt, temporalMaterial, 0);
     }
 }