object cleanup when not relevant

hello guys!

I have a question regarding cleaning objects:

I'm working on a networked game where the server send each client states of objects that are nearby that player on each tick, which the client interpolates and visualizes said objects. I want to clean (hide or dispose) objects that are not updated for X time.

I came up with this solution: Create a "cleaning" which receives a CancelationTokenSource and will wait for a timeout and then call the cleaning action. if canceled (meaning an update for that object had arrived) then I'll recreate that task. Will performance be an issue for such a mechanism where I might end up with dozens or hundreds of tasks running at the same time? would it be better to simply check on every tick of all the synced objects, which is no longer updated (for more than a few ticks)?

If you could suggest a better approach for this problem it would be very helpful!