How to go backwards in a list in Haskell? - haskell

Question

I need to write a simple function for one of my assignments that should remove all the duplicates from a given list except for the first occurrence of the element in the list.
Here is what I wrote:
remDup :: [Int]->[Int]
remDup []=[]
remDup (x:xs)
| present x xs==True = remDup xs
| otherwise = x:remDup xs
where
present :: Int->[Int]->Bool
present x [] = False
present x (y:ys)
| x==y =True
| otherwise = present x ys
But this code removes the duplicates except for the last occurrence of the element.
That is, if the given list is [1,2,3,3,2], it produces [1,3,2] instead of [1,2,3].
How to do it the other way around?

Answer 1

How about this idea:
remDup [] = []
remDup (x:xs) = x : remDup ( remove x xs )
where remove x xs removes all occurrences of x from the list xs (implementation left as an exercise.)

Answer 2

For every element you encounter, you simply want to check if you have encountered it before; build up a Set of encountered elements and use that to check if an element should be deleted.
remDup :: [Int] -> [Int]
remDup xs = helper S.empty xs
where
helper s [] = []
helper s (x:xs) | S.elem x s = helper xs
| otherwise = x:helper (S.insert x s) xs

Answer 3

You could reverse it, run your current duplicate remover, and then reverse the result.

Answer 4

So this is what I finally came up with after following user5402's advice.
remDup1 [] = []
remDup1 (x:xs) = x:remDup1(remove x xs)
remove x []=[]
remove x (y:ys)
| x==y = remove x ys
| x/=y = y:(remove x ys)

Answer 5

If you care about efficiency, you should think about using HashSet as an auxiliary data structure. Doing that, we can get an average-case complexity of O(n log n) and actually O(n) in practice (source).
import Data.Hashable
import Data.HashSet (HashSet)
import qualified Data.HashSet as HashSet
remDupSet :: (Hashable a, Eq a) => [a] -> [a]
remDupSet l = remDupSetAux HashSet.empty l
where remDupSetAux :: (Hashable a, Eq a) => HashSet a -> [a] -> [a]
remDupSetAux _ [] = []
remDupSetAux s (x:xs) = if x `HashSet.member` s
then remDupSetAux s xs
else x : remDupSetAux (HashSet.insert x s) xs
I just quickly wrote a program to compare the performance of this solution with the top-voted one:
import Data.List
import Data.Hashable
import Data.HashSet (HashSet)
import qualified Data.HashSet as HashSet
import Data.Time.Clock
import Control.DeepSeq
main :: IO ()
main = do
let a = [1..20000] :: [Int]
putStrLn "Test1: 20000 different values"
test "remDup" $ remDup a
test "remDupSet" $ remDupSet a
putStrLn ""
let b = replicate 20000 1 :: [Int]
putStrLn "Test2: one value repeted 20000 times"
test "remDup" $ remDup b
test "remDupSet" $ remDupSet b
test :: (NFData a) => String -> a -> IO ()
test s a = do time1 <- getCurrentTime
time2 <- a `deepseq` getCurrentTime
putStrLn $ s ++ ": " ++ show (diffUTCTime time2 time1)
remDup :: (Eq a) => [a] -> [a]
remDup [] = []
remDup (x:xs) = x : remDup (delete x xs)
remDupSet :: (Hashable a, Eq a) => [a] -> [a]
remDupSet l = remDupSetAux HashSet.empty l
where remDupSetAux :: (Hashable a, Eq a) => HashSet a -> [a] -> [a]
remDupSetAux _ [] = []
remDupSetAux s (x:xs) = if x `HashSet.member` s
then remDupSetAux s xs
else x : remDupSetAux (HashSet.insert x s) xs
As expected, there is a huge difference mainly when there are many distinct values:
Test1: 20000 different values
remDup: 15.79859s
remDupSet: 0.007725s
Test2: one value repeted 20000 times
remDup: 0.001084s
remDupSet: 0.00064s