Remove Duplicates From a List

How do we remove duplicates from a list? One way is to go through the original list, pick up unique values, and append them to a new list.

Let’s prepare a simple test. I will use the randrange to generate 1 million random numbers between 0 and 99 (this will guarantee some duplicates):

# duplicates.py

from random import randrange

DUPLICATES = [randrange(100) for _ in range(1_000_000)]