4

Why does clojure.core/rest output a list when input is a vector?

This creates an unexpected effect:

(conj [1 2 3] 4)
; => [1 2 3 4]

(conj (rest [1 2 3]) 4)
; => (4 2 3)

I know that "it calls seq on its argument" from the docs which creates this effect. I don't understand why this is the desired effect. As a naïve user, I would expect (rest [1 2 3]) to behave like (subvec [1 2 3] 1). I know I could just use subvec for my use case. For the sake of learning, I would like to understand the rationale of rest, and use cases where outputting a list is desirable (even when the input is a vector).

  • the rationale of `rest` is: `returns a possibly empty seq of the items after the first`, the issue is that conj'ng to a seq is different than to a vector, since the rationale of `conj` is to add to the underlying data structure in the "most efficient way" (seq prepend, vector append), at least that's how I understand it. – birdspider Aug 21 '19 at 17:45
  • To go from `seq` (the thing, not the function) to vector, just use `(vec myseq)`. This may or may not involve computational cost. But it's a dynamic language, don't worry, be hacky. Comparions between vector and list works as expected, i.e. `[1 2] is the same as `(1 2)`. – David Tonhofer Aug 22 '19 at 10:56

4 Answers4

4

The output of rest is NOT a list, but a seq, which is an even lower level abstraction. From the official documentation for rest:

Returns a possibly empty seq of the items after the first. Calls seq on its argument.

The confusion arises from the fact that both are printed between parens, but if you look closely, they are different:

user=> (list? (rest [1 2 3]))
false

user=> (seq? (rest [1 2 3]))
true

How it's a seq different from a list? seqs are implemented with an Interface that requires implementing first, rest and cons, but details are up to the collection implementation. For instance, vectors use their own implementation:

user=> (class (rest [1 2 3]))
clojure.lang.PersistentVector$ChunkedSeq

user=> (class (rest '(1 2 3)))
clojure.lang.PersistentList

List are an implementation that at least extends a basic Seq interface, and builds on top. For instance, clojure.lang.PersistentList implements the Counted interface which requires a constant-time version of count.

For a detailed description of the differences between Seqs and Lists, check these links:

Denis Fuenzalida
  • 3,271
  • 1
  • 17
  • 22
  • Thanks for making that distinction and correcting me - but my question still stands. Why design `rest` to output a `seq` when the input is a `vector`? From my perspective, `rest` is unnecessarily converting my dtype. –  Aug 22 '19 at 15:56
  • 1
    There is a Gist in Github that tries to track some of Rich Hickey's design decisions and touches a bit on `rest` here ( https://gist.github.com/reborg/dc8b0c96c397a56668905e2767fd697f#why-there-is-a-rest-and-a-next ). It seems to me that the design went on implementing `rest` like the function of the same name from Common Lisp, however, the singly-linked list from CL matches to the Seq abstraction in Clojure. – Denis Fuenzalida Aug 23 '19 at 06:40
  • My abstractions get more general with height: so a sequence is a *higher* level abstraction than a list. That's the way class diagrams are usually drawn. – Thumbnail Aug 23 '19 at 13:24
3

You make a good case for rest on a vector returning a vector. The trouble is that rest is one of the fundamental operations on sequences, and a vector is not a sequence:

=> (seq? [1 2 3 4])
false

However, if rest can accept a seqable thing such as a vector, you could say that it ought to be able to return such.

What does it return?

=> (type (rest [1 2 3 4]))
clojure.lang.PersistentVector$ChunkedSeq

This gives every appearance of being a subvec wrapped in a seq call.

Thumbnail
  • 13,293
  • 2
  • 29
  • 37
0

I know that "it calls seq on its argument"

That is correct. Seqs are implemented with an Interface (ISeq) that requires implementing first, rest and cons.

rest takes any Seq'able (any collection that implements ISequable). The reason for using this is efficiency and simplicity.

The way different collection works, the most efficient way of getting the first and rest is different.

Which is why when you convert one collection into a seq, it will come with the most efficient implementation on rest and the others.

I hope this was clear

Amanuel Nega
  • 1,899
  • 1
  • 24
  • 42
-2

I agree that this behavior is unexpected and counterintuitive. As a workaround, I created the append and prepend functions in the Tupelo library.

From the docs, we see examples:

Clojure has the cons, conj, and concat functions, but it is not obvious how they should be used to add a new value to the beginning of a vector or list:

; Add to the end
> (concat [1 2] 3)    ;=> IllegalArgumentException
> (cons   [1 2] 3)    ;=> IllegalArgumentException
> (conj   [1 2] 3)    ;=> [1 2 3]
> (conj   [1 2] 3 4)  ;=> [1 2 3 4]
> (conj  '(1 2) 3)    ;=> (3 1 2)       ; oops
> (conj  '(1 2) 3 4)  ;=> (4 3 1 2)     ; oops

; Add to the beginning
> (conj     1  [2 3] ) ;=> ClassCastException
> (concat   1  [2 3] ) ;=> IllegalArgumentException
> (cons     1  [2 3] ) ;=> (1 2 3)
> (cons   1 2  [3 4] ) ;=> ArityException
> (cons     1 '(2 3) ) ;=> (1 2 3)
> (cons   1 2 '(3 4) ) ;=> ArityException

Do you know what conj does when you pass it nil instead of a sequence? It silently replaces it with an empty list: (conj nil 5) ⇒ (5) This can cause you to accumulate items in reverse order if you aren’t aware of the default behavior:

(-> nil
  (conj 1)
  (conj 2)
  (conj 3))
;=> (3 2 1)

These failures are irritating and unproductive, and the error messages don’t make it obvious what went wrong. Instead, use the simple prepend and append functions to add new elements to the beginning or end of a sequence, respectively:

(append [1 2] 3  )   ;=> [1 2 3  ]
(append [1 2] 3 4)   ;=> [1 2 3 4]

(prepend   3 [2 1])  ;=> [  3 2 1]
(prepend 4 3 [2 1])  ;=> [4 3 2 1]

Both prepend and append always return a vector result.

Alan Thompson
  • 29,276
  • 6
  • 41
  • 48
  • 1
    I was going to complain that your append and prepend have unpredictable performance, expecting vectors to have fast append and slow prepend. I guess I can't make that complaint, because they have predictably slow performance regardless of collection type. – amalloy Aug 21 '19 at 18:21
  • If you are accumulating 5 million items, you should definitely plan your data structures carefully. Perhaps even use a native Java object like ArrayList (preallocated, of course). If you are accumulating 5 items, the data structure doesn't matter for speed. In this case, correctness & ease-of-use are paramount. – Alan Thompson Aug 21 '19 at 18:25