Realtime color correction of video

I am working on a mobile IOS Augmented Reality app for a book. The tracking images will trigger video with have an alpha channel and the effect is that the characters on the page will come to life. So far it's fairly convincing, but what I'd really like to achieve is some level of color correction so the video file more accurately matches the real world lighting conditions.

Since I know the original color of the tracking image I'm thinking there must be a way to read the color change from the live image and apply that same value to the video.

Can anyone steer me in the right direction as to how I might accomplish this?