C# - How to get only Distinct values (with threshold) and a specific count?

I have the following Problem:

I have a list Vector3 positions of an object I tracked in Unity (e.g. an animated sphere flying some track). I am adding the position of it every frame to the List. Which leads to a sum of e.g. 500 values. When it's animation stopped i want to "clean" the recorded track and only keep the most distinct values (in the right order). But the List.Count has to be EXACTLY 100. So it has to check for "equal" values with a threshold.

What i've done so far: (see code below)

I am recording the position of "trackableObject" as Vector3 in every frame and cleaning it directly (only keep values that are further away than 'cleaningDiffTolerance' and/or have a greater angle difference then 'cleaningAngleMaxDiff'). This way i am getting only values for significant changes in direction/distance and get more points i curves/corners.

What i want:

Do the cleaning not every frame but do it after i stopped recording. Then I want to only keep the most distinct values in correct order and exactly 100 values.

 private List<Vector3> positions = new List<Vector3>();
 public Transform trackableObject;
 private bool recordPosition = false;    //Records position when true
 private int valuesRecorded = 0;    //Count of recorded values
 private bool firstValue = true;    //true if it's the first recorded value
 private bool deleteLastRecorded = false;
 private float cleaningDiffTolerance = 0.2f; //Default 0.2f = 20cm
 private float cleaningAngleMaxDiff = 5.0f;    //Default 5.0f = 5°

 void Update(){
 if (recordPosition) {
         if (deleteLastRecorded) {    //Check if the last recorded has been cleaned and delete the List element
             positions.RemoveAt (valuesRecorded - 1);
         }
         deleteLastRecorded = false;

         Vector3 tempVect = trackableObject.position;     //Record position

         positions.Add (tempVect);    //Add value to list

         if (firstValue) {    //Remove first recorded value
             if (valuesRecorded == 0) {
                 positions.RemoveAt (valuesRecorded);
                 if (showDebug) {
                     Destroy (debugPoint);
                     debugPoints.RemoveAt (valuesRecorded);
                 }
                 valuesRecorded--;
                 firstValue = false;
             }
         }
         if (valuesRecorded > 1) {    //CLEANING
             Vector3 valueLast = positions [valuesRecorded - 2];
             Vector3 valueCurrent = positions [valuesRecorded - 1];
             Vector3 valueNext = positions [valuesRecorded];

             float diffCurrLast = Vector3.Distance (valueCurrent, valueLast);    //Distance between CURRENT and LAST value
             float diffCurrNext = Vector3.Distance (valueCurrent, valueNext);     //Distance between CURRENT and NEXT value
             Vector3 lastDirection = valueLast - valueCurrent;    //Direction from LAST to CURRENT position
             Vector3 nextDirection = valueCurrent - valueNext;    //Direction from CURRENT to NEXT position
             float diffAngle = 0;

             if (diffCurrLast < cleaningDiffTolerance) {    //Check Distance between CURRENT and LAST value
                 diffAngle = Vector3.Angle (lastDirection, nextDirection);    //Get Diff Angle
                 if (diffAngle < cleaningAngleMaxDiff) {
                     deleteLastRecorded = true;
                     valuesRecorded--;    //Reset i for next item
                 }
             } else {
                 if (diffCurrNext < cleaningDiffTolerance) {    //Check Distance between CURRENT and NEXT value
                     diffAngle = Vector3.Angle (lastDirection, nextDirection);    //Get Diff Angle
                     if (diffAngle < cleaningAngleMaxDiff) {
                         deleteLastRecorded = true;
                         valuesRecorded--;    //Reset i for next item
                     }
                 }
             }
         }
         valuesRecorded++;
     }
 }