What is the core difference between strings and numbers in Common Lisp? - string

Question

Being new with CL, I play a lot with simple algorithms. For instance, I tried to implement a function for removing all unique elements in a list.
(1 2 2 3 3 4 5 3) -> (2 2 3 3 3)
First attempt lead to this code:
(defun remove-unique (items)
(let ((duplicates (set-difference items (remove-duplicates items :test #'equal))))
(append duplicates (remove-duplicates duplicates :test #'equal))))
This works ok with strings but does always return NIL for numbers. Reading a bit more about set-difference I've learned that it isn't suppose to work with duplicate populated lists at all, it just works somehow in my case, so I abandoned the approach as unportable and moved along.
Another attempt is:
(defun remove-unique (items)
(loop for item in items
when (member item (cdr (member item items)))
collect item))
And this works ok with numbers, but returns NIL for strings.
Apparently there is a core difference between strings and numbers I don't understand. How come list processing functions such as member and set-difference work differently on them?

Answer 1

The equality comparison for numbers, characters and strings is indeed different. Equal, which you should be wary to use because it is more expensive, does structure equality (so it descends on some objects). eq does object equality. And eql does object equality for most cases except for numbers (where they check type and value) and characters (where they check 'value')
See the hyperspec entries for equal, eql and eq for more information.

Answer 2

Strings are more related to lists than numbers since both lists and strings are sequences.
"Hello" is a sequence (compund data type) starting with the primitive character value #\H and ending with #\o.
'(1 2 3) is a sequence (compond data type) starting with the primitive numeric value 1 and ending with 3.
Characters are similar to numbers in that they are primitive values. Primitive values can be compared using eql while sequences, that are not the same object, can be compared using equal
(setq list1 (list 1 2 3))
(setq list2 (list 1 2 3))
(eql list1 list2)
;==> NIL
(equal list1 list2)
;==> T
;; comparing first element of both lists using eql
(eql (car list1) (car list2))
;==> T
(setq string1 "Hello")
(setq string2 "Hello")
(eql string1 string2)
;==> NIL
(equal string1 string2)
;==> T
;; comparing first character of both strings using eql
(eql (elt string1 0) (elt string2 0))
;==> T
Most (if not all) functions in Common Lisp that compares something usually has an optional named argument :test where you can supply how elements compare. the default usually is eql. To make them behave corretly with sequences you need to supply #'equal as :test.

Answer 3

(defun remove-unique (items &key (test 'eql))
(loop
:with table := (make-hash-table :test test)
:for element :in items :do
(setf (gethash element table)
(1+ (gethash element table 0)))
:finally
(return
(loop
:for k :being :the :hash-keys :of table
:using (:hash-value v)
:when (> v 1) :nconc (make-list v :initial-element k)))))
(defun remove-unique (items &key (test 'eql))
(loop
:with table := (make-hash-table :test test)
:for element :in items :do
(setf (gethash element table)
(1+ (gethash element table 0)))
:finally
(return
(loop
:for element :in items
:unless (= 1 (gethash element table))
:collect element))))
I'd probably use the first variant because it makes less reads from hash-table, but you'd need to check that items in the list aren't modified later in place.
(remove-unique '("1" "2" "2" "3" "3" "4" "5" "3") :test #'equal)
gives:
("2" "2" "3" "3" "3")
but
(remove-unique '("1" "2" "2" "3" "3" "4" "5" "3"))
gives:
NIL