As records are immutable i am unable to read in the data and parse it without it making a new instance of itself. Moreover, how would i be able to read my excel files from a number of specific columns instead of reading from column 0 to EOF. Is there anyway where i could read in the data from say column 1, column 3, column 5. Supposedly, Column 1 would be parsed as a string, Column 3 as an integer and Column 5 as a long.
(defrecord Record [Name Age Index])
(defn read-csv [fname count]
(with-open [file (io/reader fname)]
(doall (take count (map (comp first csv/read-csv)
(line-seq file))))))
(def records (map #(apply ->Record %) (read-csv "C:/Users/user/Documents/URECA/hi/lib/test.csv" 1)))
This is what i have but it seems to read the columns incrementally
To keep quotes of text fields you can parse csv file by regexp:
(defn read-csv [fname count]
(with-open [file (io/reader fname)]
(doall (map #(str/split % #",") ; doesn't work with commas in text fields
(take count (line-seq file))))))
(defn make-record [idxs types row]
(apply ->Record
(map (fn [idx t]
(let [value (nth row idx)]
(case t
:string value
:int (Integer/parseInt value)
:long (Long/parseLong value))))
idxs types)))
(def records (map (partial make-record
[0 2 4]
[:string :int :long])
(read-csv "/home/mobyte/test.csv" 3)))
(pprint records)
-> ({:Name "\"s1\"", :Age 1, :Index 111}
{:Name "\"s2\"", :Age 2, :Index 112}
{:Name "\"s3\"", :Age 3, :Index 113})
(type (:Age (first records)))
->java.lang.Integer
(type (:Index (first records)))
-> java.lang.Long
(type (:Name (first records)))
-> java.lang.String
Related
This is what I have so far:
(defn cities []
(with-open [rdr (reader)]
(doseq [line (next (line-seq rdr))]
(print line))))
I am able to print all the rows minus the first but am unable to return all the rows? How would I be able to do this
Do you see that (next (line-seq rdr)) part? That's returning the list without the first item. Instead of printing it, you can just return it.
Here, however, you have to make sure that it is fully realized before the reader is closed. You can do that with doall:
(defn cities []
(with-open [rdr (reader)]
(doall (next (line-seq rdr)))))
Being new with CL, I play a lot with simple algorithms. For instance, I tried to implement a function for removing all unique elements in a list.
(1 2 2 3 3 4 5 3) -> (2 2 3 3 3)
First attempt lead to this code:
(defun remove-unique (items)
(let ((duplicates (set-difference items (remove-duplicates items :test #'equal))))
(append duplicates (remove-duplicates duplicates :test #'equal))))
This works ok with strings but does always return NIL for numbers. Reading a bit more about set-difference I've learned that it isn't suppose to work with duplicate populated lists at all, it just works somehow in my case, so I abandoned the approach as unportable and moved along.
Another attempt is:
(defun remove-unique (items)
(loop for item in items
when (member item (cdr (member item items)))
collect item))
And this works ok with numbers, but returns NIL for strings.
Apparently there is a core difference between strings and numbers I don't understand. How come list processing functions such as member and set-difference work differently on them?
The equality comparison for numbers, characters and strings is indeed different. Equal, which you should be wary to use because it is more expensive, does structure equality (so it descends on some objects). eq does object equality. And eql does object equality for most cases except for numbers (where they check type and value) and characters (where they check 'value')
See the hyperspec entries for equal, eql and eq for more information.
Strings are more related to lists than numbers since both lists and strings are sequences.
"Hello" is a sequence (compund data type) starting with the primitive character value #\H and ending with #\o.
'(1 2 3) is a sequence (compond data type) starting with the primitive numeric value 1 and ending with 3.
Characters are similar to numbers in that they are primitive values. Primitive values can be compared using eql while sequences, that are not the same object, can be compared using equal
(setq list1 (list 1 2 3))
(setq list2 (list 1 2 3))
(eql list1 list2)
;==> NIL
(equal list1 list2)
;==> T
;; comparing first element of both lists using eql
(eql (car list1) (car list2))
;==> T
(setq string1 "Hello")
(setq string2 "Hello")
(eql string1 string2)
;==> NIL
(equal string1 string2)
;==> T
;; comparing first character of both strings using eql
(eql (elt string1 0) (elt string2 0))
;==> T
Most (if not all) functions in Common Lisp that compares something usually has an optional named argument :test where you can supply how elements compare. the default usually is eql. To make them behave corretly with sequences you need to supply #'equal as :test.
(defun remove-unique (items &key (test 'eql))
(loop
:with table := (make-hash-table :test test)
:for element :in items :do
(setf (gethash element table)
(1+ (gethash element table 0)))
:finally
(return
(loop
:for k :being :the :hash-keys :of table
:using (:hash-value v)
:when (> v 1) :nconc (make-list v :initial-element k)))))
(defun remove-unique (items &key (test 'eql))
(loop
:with table := (make-hash-table :test test)
:for element :in items :do
(setf (gethash element table)
(1+ (gethash element table 0)))
:finally
(return
(loop
:for element :in items
:unless (= 1 (gethash element table))
:collect element))))
I'd probably use the first variant because it makes less reads from hash-table, but you'd need to check that items in the list aren't modified later in place.
(remove-unique '("1" "2" "2" "3" "3" "4" "5" "3") :test #'equal)
gives:
("2" "2" "3" "3" "3")
but
(remove-unique '("1" "2" "2" "3" "3" "4" "5" "3"))
gives:
NIL
I'm making a function that generates a .wav file. I have the header all set, but I'm running into trouble with the data itself. I have a function for creating a sine wave at 880Hz (at least I think that's what it does, but that's not my question)--the question is, how do I convert a collection of byte arrays to just one byte array with their contents? This was my best try:
(defn lil-endian-bytes [i]
(let [i (int i)]
(byte-array
(map #(.byteValue %)
[(bit-and i 0x000000ff)
(bit-shift-right (bit-and i 0x0000ff00) 8)
(bit-shift-right (bit-and i 0x00ff0000) 16)
(bit-shift-right (bit-and i 0xff000000) 24)]))))
(def leb lil-endian-bytes)
(let [wr (io/output-stream (str name ".wav") :append true)]
(.write wr
(byte-array (flatten (concat (map
(fn [sample] (leb (* 0xffff (math/sin (+ (* 2 3.14 880) sample)))))
(range (* duration s-rate)) )))))
but it doesn't do what I want it to do: concat all of the byte-arrays into one vector and then into a single byte array. And it makes sense to me why it can't: it can't concat/flatten a byte[] because it's not a vector; it's a byte[]. And it can't cast a byte[] into a byte. But what do I need to do to get this working?
You might be looking for something like:
(byte-array (mapcat seq my-sequence-of-byte-arrays))
Converting the byte arrays to be concatenated to sequences and back to byte array may be a little inefficient if you are dealing with large arrays. Here's how to concatenate byte arrays making java.nio.ByteBuffer do the heavy lifting:
(defn concat-byte-arrays [& byte-arrays]
(when (not-empty byte-arrays)
(let [total-size (reduce + (map count byte-arrays))
result (byte-array total-size)
bb (java.nio.ByteBuffer/wrap result)]
(doseq [ba byte-arrays]
(.put bb ba))
result)))
It will be as simple as:
Your byte arrays that needs to be combined in a single byte-array:
(def byte-arrays [(byte-array 10 (byte 1))
(byte-array 10 (byte 2))
(byte-array 10 (byte 3))])
Combine:
(byte-array (for [ar byte-arrays
i ar] i))
I'll start with my test using the speclj framework.
(it "turns the string into a hash-map"
(should= {1 "1" 2 "2" 3 "3"}
(format-string "1=1 2=2 3=3")))
Then my code:
(:use [clojure.string :only (split)])
(defn format-string [string]
(split string #"\s+"))
Right now, the format-string function returns ["1=1" "2=2" "3=3"] and the test fails. As you can see in my test, I want it to return a hash-map with key-value pairs indicated by the = sign.
I've tried a few things and I've gotten close, but can't quite understand how to make this transformation.
EDIT
Figured out one solution, although the keys are strings instead of integers.
My code:
(defn format-board [route]
(let [[first second third] (split route #"\s+")]
(merge
(apply hash-map (split-at-equals first))
(apply hash-map (split-at-equals second))
(apply hash-map (split-at-equals third))
This returns {"1" "1" "2" "2" "3" "3"}.
You have split at the spaces, but then you need to split again at the = delimiter. You can use regular expressions to do the parsing. Once you have your pairs you can assoc into a hash-map. Here I've used reduce to effect the transformation.
user=> (reduce #(assoc % (read-string (nth %2 1)) (nth %2 2)) {}
#_> (re-seq #"([^=\s]+)=([^=\s]+)" "1=1 2=2 3=3") )
{3 "3", 2 "2", 1 "1"}
Note key order is not applicable to hash-maps
user=> (= {1 "1", 2 "2", 3 "3"} *1)
true
Here is the potential parallel version using clojure.core.reducers:
(require '[clojure.core.reducers :as r])
(require '[clojure.string :as s])
(def string-of-pairs "1=1 2=2 3=3 4=4")
; reducing fn to convert seq of (key, value) to hash-map
(defn rf ([] {}) ([acc [k v]] (assoc acc k v)))
; for large colls, fold will parallelize execution
(r/fold merge rf (r/map #(s/split % #"=") (s/split string-of-pairs #"\s+")))
For better understanding of reducers, watch this video where
Rich explains motivation behind reducers and demonstrates some usage.
Is it possible in racket to get all fields of an object at the same time?
I would like basically to convert an object to a hash table with field names as keys and field values as values.
I found a function (field-names obj), but then I don't know how to use the returned field names to get the values from the obj. The function get-field can be used to get the value of a field, but I dont know how to use it with a value:
> (define x% (class object% (init-field x y) (super-new)))
> (define obj (make-object x% 1 2))
> (get-field x obj)
1
> (field-names obj)
'(y x)
> (define field-name (second (field-names obj)))
> field-name
'x
> (get-field field-name obj)
get-field: given object does not have the requested field
field name: field-name
object: (object:x% ...)
errortrace...:
context...:
/usr/lib/racket/collects/racket/private/class-internal.rkt:4906:0: obj-error29
/usr/lib/racket/collects/racket/private/misc.rkt:87:7
Here's some code to get you started
#lang racket
> (define x% (class object% (inspect #f) (init-field x y) (super-new)))
> (define obj (make-object x% 1 2))
> (let-values (((name field-cnt field-name-list field-accessor field-mutator super-class skipped)
(class-info x%)))
(for/hash ((name field-name-list)
(idx field-cnt))
(values name (field-accessor obj idx))))
'#hash((x . -1) (y . 0))
You may want to change the inspector from #f to something less vulnerable, but open enough for your needs. Read up on class-info and inspectors in general.