FoldList like primitive in J - j

Mathematica has a built-in function called FoldList FoldList function description. Is there a similar primitive verb in J?
(I know that J has a ^: verb, which is like Nest and FixedPoint.)
To clarify my question, J has dyadic verb, so usually u / x1 x2 x3 becomes x1 u (x2 u x3), which works just like FoldList, with reverse order.
Except if the function u takes y, in a different shape from x. In FoldList there is an initial x. In J, if x3 is a different shape, one has to rely on < to pack it together. For example, one has to pack and unpack
[list =. (;/ 3 3 4 3 3 34),(< 1 2)
+-+-+-+-+-+--+---+
|3|3|4|3|3|34|1 2|
+-+-+-+-+-+--+---+
tf =: 4 : '<((> x) , >y)'
tf/ list
+----------------+
|1 2 3 3 4 3 3 34|
+----------------+
tf/\ |. list
+---+------+--------+----------+------------+--------------+----------------+
|1 2|1 2 34|1 2 34 3|1 2 34 3 3|1 2 34 3 3 4|1 2 34 3 3 4 3|1 2 34 3 3 4 3 3|
+---+------+--------+----------+------------+--------------+----------------+
which is kind of inconvenient. Any better solutions?

u/\ comes very close (if you don't mind the right folding):
+/\ 1 2 3 4
1 3 6 10
*/\1+i.10
1 2 6 24 120 720 5040 ...
(+%)/\7#1. NB. continued fraction of phi
1 2 1.5 1.66667 1.6 1.625 1.61538
edit on your edit:
The first two elements of FoldList are x and f(x,a). In J those two have to be of the same "kind" (shape+type) if you want them on the same list. The inconvenience comes from J's data structures not from the lack of a FoldList verb. If you exclude x from the list, things are easier:
FoldListWithout_x =: 1 : 'u/ each }.<\y'
; FoldListWithout_x 1 2 3 4
┌─────┬───────┬─────────┐
│┌─┬─┐│┌─┬─┬─┐│┌─┬─┬─┬─┐│
││1│2│││1│2│3│││1│2│3│4││
│└─┴─┘│└─┴─┴─┘│└─┴─┴─┴─┘│
└─────┴───────┴─────────┘
>+ FoldListWithout_x 1 2 3 4
3 6 10
(+%) FoldListWithout_x 7#1
┌─┬───┬───────┬───┬─────┬───────┐
│2│1.5│1.66667│1.6│1.625│1.61538│
└─┴───┴───────┴───┴─────┴───────┘
The next logical step is to include a boxed x after making the folds, but that will either require more complex code or a case-by-case construction. Eg:
FoldList =: 1 :'({.y) ; u FoldListWithout_x y'
+ FoldList 1 2 3 4
┌─┬─┬─┬──┐
│1│3│6│10│
└─┴─┴─┴──┘
; FoldList 1 2 3 4
┌─┬─────┬───────┬─────────┐
│1│┌─┬─┐│┌─┬─┬─┐│┌─┬─┬─┬─┐│
│ ││1│2│││1│2│3│││1│2│3│4││
│ │└─┴─┘│└─┴─┴─┘│└─┴─┴─┴─┘│
└─┴─────┴───────┴─────────┘
vs
FoldList =: 1 :'(<{.y) ; u FoldListWithout_x y'
+ FoldList 1 2 3 4
┌───┬─┬─┬──┐
│┌─┐│3│6│10│
││1││ │ │ │
│└─┘│ │ │ │
└───┴─┴─┴──┘
; FoldList 1 2 3 4
┌───┬─────┬───────┬─────────┐
│┌─┐│┌─┬─┐│┌─┬─┬─┐│┌─┬─┬─┬─┐│
││1│││1│2│││1│2│3│││1│2│3│4││
│└─┘│└─┴─┘│└─┴─┴─┘│└─┴─┴─┴─┘│
└───┴─────┴───────┴─────────┘

I guess #Dan Bron's comment deserves an answer. It is discussed with some solutions in http://www.jsoftware.com/pipermail/programming/2006-May/002245.html
if we define an adverb (modified from the link above)
upd =: 1 : 0
:
u&.> /\ ( <"_ x),<"0 y
)
then
1 2 , upd |. 3 3 4 3 3 34
┌───┬──────┬────────┬──────────┬────────────┬──────────────┬────────────────┐
│1 2│1 2 34│1 2 34 3│1 2 34 3 3│1 2 34 3 3 4│1 2 34 3 3 4 3│1 2 34 3 3 4 3 3│
└───┴──────┴────────┴──────────┴────────────┴──────────────┴────────────────┘

Related

Print values and it's associated index when working with vectors / matrix in J

If I want to check how many values in a vector or matrix are less than a given value
I can use +/ (a < 20). But what if I wanted to know both the specific value and it's index.
Something like (2(value) 5(index)) as a table. I looked at i., i: (which give first and last position) and I. Does sorting first help?
A very common pattern in J is the creation of a mask from a filter and applying an action on and/or using the masked data in a hook or fork:
((actions) (filter)) (data)
For example:
NB. Random array
a =: ? 20 $ 10
6 3 9 0 3 3 0 6 2 9 2 4 6 8 7 4 6 1 7 1
NB. Filter and mask
f =: 5 < ]
m =: f a
1 0 1 0 0 0 0 1 0 1 0 0 1 1 1 0 1 0 1 0
NB. Values of a on m
m # a
6 9 6 9 6 8 7 6 7
NB. Indices of a on m
I. m
0 2 7 9 12 13 14 16 18
NB. Joint results
(I.m) ,: (m # a)
0 2 7 9 12 13 14 16 18
6 9 6 9 6 8 7 6 7
In other words, in this case you have m&# and f acting on a and I. acting on m. Notice that the final result can be derived from an action on m alone by commuting the arguments of copy #~:
(I. ,: (a #~ ]) m
0 2 7 9 12 13 14 16 18
6 9 6 9 6 8 7 6 7
and a can be pulled out from the action on m like so:
a ( (]I.) ,: (#~ ])) m
But since m itself is derived from an action (f) on a, we can write:
a ( (]I.) ,: (#~ ])) (f a)
which is a simple monadic hook y v (f y) → (v f) y.
Therefore:
action =: (]I.) ,: (#~ ])
filter =: 5 < ]
data =: a
(action filter) data
0 2 7 9 12 13 14 16 18
6 9 6 9 6 8 7 6 7

Use # (Copy) as selection or filter on 2-d array

The J primitive Copy (#) can be used as a filter function, such as
k =: i.8
(k>3) # k
4 5 6 7
That's essentially
0 0 0 0 1 1 1 1 # i.8
The question is if the right-hand side of # is 2-d or higher rank shaped array, how to make a selection using #, if possible. For example:
k =: 2 4 $ i.8
(k > 3) # k
I got length error
What is the right way to make such a selection?
You can use the appropriate verb rank to get something like a 2d-selection:
(2 | k) #"1 1 k
1 3
5 7
but the requested axes have to be filled with 0s (or !.) to keep the correct shape:
(k > 3) #("1 1) k
0 0 0 0
4 5 6 7
(k > 2) #("1 1) k
3 0 0 0
4 5 6 7
You have to better define select for dimensions > 1 because now you have a structure. How do you discard values? Do you keep empty "cells"? Do you replace with 0s? Is structure important for the result?
If, for example, you only need the "values where" then just ravel , the array:
(,k > 2) # ,k
3 4 5 6 7
If you need to "replace where", then you can use amend }:
u =: 5 :'I. , 5 > y' NB. indices where 5 > y
0 u } k
0 0 0 0
0 5 6 7
z =: 3 2 4 $ i.25
u =: 4 :'I. , (5 > y) +. (0 = 3|y)' NB. indices where 5>y or 3 divides y
_999 u } z
_999 _999 _999 _999
_999 5 _999 7
8 _999 10 11
_999 13 14 _999
16 17 _999 19
20 _999 22 23

The concept of Rank in J

If I have one dimension data such as
v1 =: 5 $ i.5
v1
0 1 2 3 4
then v1 -"1 0 v1 gives me the Euclidean vectors like
v1 -"1 0 v1
0 1 2 3 4
_1 0 1 2 3
_2 _1 0 1 2
_3 _2 _1 0 1
_4 _3 _2 _1 0
We can also find Euclidean distance matrix easily.
This is how I find the Euclidean vectors and distance matrix of a 2D vector
v2=: <"1 ? 5 2 $ 10
v2
┌───┬───┬───┬───┬───┐
│4 0│4 5│5 7│8 3│6 0│
└───┴───┴───┴───┴───┘
direction_vector=: <"1 #: (-"0 #:(-/"2 #: (>"0 #: (diff))))
distance =: +/"1 #: *: #: (>"2 #:(direction_vector))
m =: 3 : '(i.(#y)) distance"0 _ y'
m v2
0 25 50 25 4
25 0 5 20 29
50 5 0 25 50
25 20 25 0 13
4 29 50 13 0
However, my problem is that I am not sure how to find the Euclidean vectors and distance of 2D data in a smart and clean way
As you can see from the table below, my algorithm took more than 1/3 of time to calculate the direction vectors of data. 14.5 second is not bad, but the problem arises when I have a bigger dataset.
Time (seconds)
+----------------+------+--------+---------+-----+----+------+
|name |locale|all |here |here%|cum%|rep |
+----------------+------+--------+---------+-----+----+------+
|direction_vector|base |6.239582| 5.105430| 35.2| 35 |773040|
|move |base |9.741510| 1.753868| 12.1| 47 | 3390|
|script |base |1.969949| 1.443148| 9.9| 57 | 18|
|distance |base |5.650358| 1.318022| 9.1| 66 |579780|
|enclosestrings |pcsv |1.491832| 1.255603| 8.6| 75 | 1|
|diff |base |1.134585| 1.134585| 7.8| 83 |773186|
|makedsv |pcsv |1.728721| 0.236883| 1.6| 84 | 4|
|norm |base |0.221794| 0.221794| 1.5| 86 | 3390|
|xpt |base |0.194896| 0.194896| 1.3| 87 | 3390|
|ypt |base |0.193579| 0.193579| 1.3| 89 | 3390|
|writedsv |pcsv |2.067408| 0.186687| 1.3| 90 | 4|
|cosd |base |0.172359| 0.172359| 1.2| 91 | 113|
|[rest] | | | 1.300733| 9.0|100 | |
|[total] | | |14.517587|100.0|100 | |
+----------------+------+--------+---------+-----+----+------+
I think I can definitely simplify direction_vector one by using rank, but I got stuck. I tried "2 1 "1 1 "1 2 "_ 1 "1 0 ..., but none of them gave me a clear result.
Can anyone help me on this issue? Thank you!
I would begin with noticing that v2 u/ v2 makes a table out of the items of v2 by applying u between those items. Also, you can simplify this a bit by using u/~ v2 which is the same as v2 u/ v2. The next question is what is u, but before we go there, boxing really slows things down and you don't actually have to box the vector for this to work since the items can already be written like this:
[ v2=: 5 2 $ 4 0 4 5 5 7 8 3 6 0
4 0
4 5
5 7
8 3
6 0
and this makes the items the vectors, which is what you want to be able to use u/~ v2
Now, back to the question of what we want u to be. We are going to be working on the items of v2 Since u is being fed the items of v2 to make the table, you would like to subtract the items from each other as vectors (rank 1) and then square them and then add them together. Translating this into J you get +/#:*:#:-"1 as u
+/#:*:#:-"1/~ v2
0 25 50 25 4
25 0 5 20 29
50 5 0 25 50
25 20 25 0 13
4 29 50 13 0
If you time this I hope that you will find it much faster than your solution because it does not require boxing. The key area that you were concerned about with rank was that it be applied after the Table adverb /
Hope this helps even though it is a slightly different approach and let me know what your timings look like.
New answer so that I could format the result properly.
Looking at your result I think that there is further room for improvement at least by my measurements. Getting away from the tacit approach and moving to a sequential application with the change of +/ to +/"1 gives roughly twice the speed.
[ v2=: 5 2 $ 4 0 4 5 5 7 8 3 6 0
4 0
4 5
5 7
8 3
6 0
(10000) 6!:2 '+/#:*:#:-"1/~ v2'
8.4647e_6
(10000) 6!:2 '+/"1 *: -"1/~ v2'
3.1289e_6
(+/"1 *: -"1/~ v2) -: (+/#:*:#:-"1/~ v2)
1
+/"1 *: -"1/~ v2
0 25 50 25 4
25 0 5 20 29
50 5 0 25 50
25 20 25 0 13
4 29 50 13 0

Seeming inconsistency in the way transpose |: works

Consider:
|: 2 3 $ 1 2 3
1 1
2 2
3 3
|: 1 2 3
1 2 3
The first one makes sense to me: the rows are now columns. But, by analogy, I expected the output of the 2nd one to be:
|: 1 2 3
1
2
3
Why is it still a row, rather than a column?
|:
reverses the order of the axes of its argument
So
$ |: 2 3 $ 1 2 3
3 2
$ |: 1 2 3 $ 1 2 3
3 2 1
and naturally
$ |: 1 2 3
3
which is the list 1 2 3
The result that you expected has axes 3 1; you would get this for the transpose of the list 1 3 $ 1 2 3
] l =: 1 3 $ 1 2 3
1 2 3
|: l
1
2
3
($ l);($ |: l)
┌───┬───┐
│1 3│3 1│
└───┴───┘

How can I implement a grouping algorithm in J?

I'm trying to implement A006751 in J. It's pretty easy to do in Haskell, something like:
concat . map (\g -> concat [show $ length g, [g !! 0]]) . group . show
(Obviously that's not complete, but it's the basic heart of it. I spent about 10 seconds on that, so treat it accordingly.) I can implement any of this fairly easily in J, but the part that eludes me is a good, idiomatic J algorithm that corresponds to Haskell's group function. I can write a clumsy one, but it doesn't feel like good J.
Can anyone implement Haskell's group in good J?
Groups are usually done with the /. adverb.
1 1 2 1 </. 'abcd'
┌───┬─┐
│abd│c│
└───┴─┘
As you can see, it's not sequential. Just make your key sequential like so (essentially determining if an item is different from the next, and do a running sum of the resulting 0's and 1's):
neq =. 13 : '0, (}. y) ~: (}: y)'
seqkey =. 13 : '+/\neq y'
(seqkey 1 1 2 1) </. 'abcd'
┌──┬─┬─┐
│ab│c│d│
└──┴─┴─┘
What I need then is a function which counts the items (#), and tells me what they are ({. to just pick the first). I got some inspiration from nubcount:
diffseqcount =. 13 : ',(seqkey y) (#,{.)/. y'
diffseqcount 2
1 2
diffseqcount 1 2
1 1 1 2
diffseqcount 1 1 1 2
3 1 1 2
If you want the nth result, just use power:
diffseqcount(^:10) 2 NB. 10th result
1 3 2 1 1 3 2 1 3 2 2 1 1 3 3 1 1 2 1 3 2 1 2 3 2 2 2 1 1 2
I agree that /. ( Key ) is the best general method for applying verbs to groups in J. An alternative in this case, where we need to group consecutive numbers that are the same, is dyadic ;. (Cut):
1 1 0 0 1 0 1 <(;.1) 3 1 1 1 2 2 3
┌─┬─────┬───┬─┐
│3│1 1 1│2 2│3│
└─┴─────┴───┴─┘
We can form the frets to use as the left argument as follows:
1 , 2 ~:/\ 3 1 1 1 2 2 3 NB. inserts ~: in the running sets of 2 numbers
1 1 0 0 1 0 1
Putting the two together:
(] <;.1~ 1 , 2 ~:/\ ]) 3 1 1 1 2 2 3
┌─┬─────┬───┬─┐
│3│1 1 1│2 2│3│
└─┴─────┴───┴─┘
Using the same mechanism as suggested previously:
,#(] (# , {.);.1~ 1 , 2 ~:/\ ]) 3 1 1 1 2 2 3
1 3 3 1 2 2 1 3
If you are looking for a nice J implementation of the look-and-say sequence then I'd suggest the one on Rosetta Code:
las=: ,#((# , {.);.1~ 1 , 2 ~:/\ ])&.(10x&#.inv)#]^:(1+i.#[)
5 las 1 NB. left arg is sequence length, right arg is starting number
11 21 1211 111221 312211

Resources