In other programs such as InfoPath it's an option that you can choose. You would choose it if you were using the value to filter another list. In that instance you would only need the value and not the id so it wouldn't matter. As a matter of fact in a dropdown it would actually be pointless to see duplicate items unless there was some other piece of information attached that allowed you to select the correct one. Otherwise how would you know which one to pick?
Just as an example. If the first option in a cascading dropdown scenario has duplicates you can set it to only show unique names. Then you can use that to filter what is shown in the next dropdown.
So if a value in my first dropdown is 'All', I only need to see and select one to filter the next dropdown to show all items where a secondary column value is 'All'.
It's very common and works well. I used it for years in InfoPath.
Eliminating duplicates is a lot of extra work and significantly increases the size of the lists involved.
I totally agree with what you are saying but the SharePoint way of doing this is to make sure the lists are setup efficiently to avoid duplicates.