I would like to count the number of sequences in a dataset.
A new sequence is started whenever a the value changes from zero to positive or from positive to zero.
The data represent mm rain one entry for every hour of the day.
If it starts raining a new sequence starts, if it stops raining a new sequence starts as well.
On some days it keeps raining (1 sequence of rain, 0 sequences of dry).
On others days it will be dry (0 sequence of rain, 1 sequence of dry).
The other days will have X sequences of rain and Y sequences of dry (X and Y differ 0 or 1).
Example:
Range (N1:N25) contains
RH
0
0
0
0
0
0
1
3
9
2
0
0
0
0
0
0
2
0
0
0
0
0
0
0
Using Matrix formulas I can calculate
The longest sequence of non-zero's, being 4, by
=MAX(FREQUENCY(IF($N$2:$N$25>0,ROW($N$2:$N$25)),IF($N$2:$N$25<=0,ROW($N$2:$N$25)))
but i cannot calculate the number of sequences of non-zero's, being 2, nor the total number of sequences, being 5.
How can I do that?
I looked for the answer on Stackoverflow but couldn't find it
To get the count of non 0 groupings:
=SUMPRODUCT(--(FREQUENCY(IF($N$2:$N$25<>0,ROW($N$2:$N$25)),IF($N$2:$N$25=0,ROW($N$2:$N$25)))>0))
To get the count of 0 groupings:
=SUMPRODUCT(--(FREQUENCY(IF($N$2:$N$25=0,ROW($N$2:$N$25)),IF($N$2:$N$25<>0,ROW($N$2:$N$25)))>0))
then you can add them together.
Related
I have a daily log with four different value types. Positive, Negative, Zero, or Blank.
I am interested in the consecutive Zeroes, where Positive and Negative values end the streak, and Blanks are ignored.
Each record is on 1 row and contains a name and values
Example:
Day 1 2 3 4 5 6 7 8
Fred 1 -2 0 _ 0 0 0 0 calculate current streak of 5
Jane 0 -5 _ 0 0 _ _ _ calculate current streak of 2
John 4 -6 3 0 0 0 3 0 calculate current streak of 1
Amie 2 0 0 0 0 0 0 -3 current streak 0
I can use COUNTIF to get the total zeroes, but not the streak
From this similar question, calculate consecute streak in excel row
I've tried variations of =IFERROR(MATCH(TRUE,INDEX(B2:I2<>0,0),0)-1,COUNT(B2:I2))
I'm getting wonky results, and my supervisor wants me to be done with this already. Any help or ideas would be greatly appreciated!
Formula in K2:
=IFERROR(SUM(INDEX((COLUMN(B2:I2)>LOOKUP(2,1/((B2:I2<>0)*(B2:I2<>"")),COLUMN(B2:I2)))*(B2:I2=0)*(B2:I2<>""),)),COUNTIF(B2:I2,0))
Drag down..
There probably is a neater way.
I am trying to understand the the MCTAL output of a spherical TMESH tally. What I want is to create one tally bin that has the following boundaries 1.9 cm and 2.1 cm in the radial direction, 88 to 92 degrees in theta and 180 to 360 degrees in the phi direction. my input for the tally is
C tally card spherical mesh energy tally
TMESH
SMESH1:p DOSE 1 1 1 1.0 PEDEP MFACT 1 1 0 1.0
CORA1 1.9 2.1
CORB1 88 92
CORC1 180 360
Now what I expect is one result for that volume what I get are eight values as shown below.
ntal 1
1
tally 1 -1 -3
0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
f 4 0 1 2 2
1.90000E+00 2.10000E+00
0.00000E+00 8.80000E+01 9.20000E+01
0.00000E+00 1.80000E+02 3.60000E+02
d 1
u 1
s 2
m 1
c 1
e 1
t 1
vals
5.57481E-04 0.0067 7.68088E-09 0.0493 8.24471E-03 0.0046 1.38395E-07 0.0639
5.53931E-04 0.0046 7.44313E-09 0.0287 8.24244E-03 0.0042 1.27868E-07 0.0553
I am assuming that these eight vals correspond to the eight points that that are listed under f. Does TMESH only give one values for individual points on a grid or can it be used to create a volume within which to obtain a result? lastly to what points do what vals correspond to ?
The matrix bellow the vals is true value of your meshtally result.
but
you must load data to Matlab and reshape it to your mesh tally matrix
With your SMESH setup you score both dose and energy deposition. This causes two bins along the segment axis (the "s 2" record in your mctal). Then, you have only 1 bin along the radial direction (1.9-2.1 cm) and actually TWO bins along each of the angular directions (0-88, 88-92, and 0-180, 180-360) which sums up to 2^3 = 8 bins. The mctal file format is described in the manual: it'a 11-dimension loop. In your case only the s, j and k axes are divided, so it's actully a 3D loop (in this exact order: s being the outer, k - the inner loop). Therefore the value for your volume is either the 4th (1.38395E-07 0.0639) or last (1.27868E-07 0.0553) record depending on whether you need dose or energy deposition.
I had a J program I wrote in 1985 (on vax vms). One section was creating a diagonal matrix from a vector.
a=(n,n)R1,nR0
b=In
a=bXa
Maybe it wasn't J but APL in ascii, but these lines work in current J (with appropriate changes in the primitive functions). But not in APL (gnu , NARS2000 or ELI). I get domain error in the last line.
Is there an easy way to do this without looping?
Your code is an ASCII transliteration of APL. The corresponding J code is:
a=.(n,n)$1,n$0
b=.i.n
a=.b*a
Try it online! However, no APL (as of yet — it is being considered for Dyalog APL) has major cell extension which is required on the last line. You therefore need to specify that the scalars of the vector b should be multiplied with the rows of the matrix a using bracket axis notation:
a←(n,n)⍴1,n⍴0
b←⍳n
a←b×[1]a
Try it online! Alternatively, you can use the rank operator (where available):
a←(n,n)⍴1,n⍴0
b←⍳n
a←b(×⍤0 1)a
Try it online!
A more elegant way to address diagonals is ⍉ with repeated axes:
n←5 ◊ z←(n,n)⍴0 ◊ (1 1⍉z)←⍳n ◊ z
1 0 0 0 0
0 2 0 0 0
0 0 3 0 0
0 0 0 4 0
0 0 0 0 5
Given an input vector X, the following works in all APLs, (courtesy of #Adám in chat):
(2⍴S)⍴((2×S)⍴1,-S←⍴X)\X
And here's a place where you can run it online.
Here are my old, inefficient versions that use multiplication and the outer product (the latter causes the inefficiency):
((⍴Q)⍴X)×Q←P∘.=P←⍳⍴X
((⍴Q)⍴X)×Q←P Pρ1,(P←≢X)ρ0
Or another way:
(n∘.=n)×(2ρρn)ρn←⍳5
should give you the following in most APLs
1 0 0 0 0
0 2 0 0 0
0 0 3 0 0
0 0 0 4 0
0 0 0 0 5
This solution works in the old ISO Apl:
a←(n,n)⍴v,(n,n)⍴0
I'm dabbling my feet with J and, to get the ball rolling, decided to write a function that:
gets integer N;
spits out a table that follows this pattern:
(example for N = 4)
1
0 1
0 0 1
0 0 0 1
i.e. in each row number of zeroes increases from 0 up to N - 1.
However, being newbie, I'm stuck. My current labored (and incorrect) solution for N = 4 case looks like:
(4 # ,: 0 1) #~/"1 1 (1 ,.~/ i.4)
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
And the problem with it is twofold:
it's not general enough and looks kinda ugly (parens and " usage);
trailing zeroes - as I understand, all arrays in J are homogeneous, so in my case every row should be boxed.
Like that:
┌───────┐
│1 │
├───────┤
│0 1 │
├───────┤
│0 0 1 │
├───────┤
│0 0 0 1│
└───────┘
Or I should use strings (e.g. '0 0 1') which will be padded with spaces instead of zeroes.
So, what I'm kindly asking here is:
please provide an idiomatic J solution for this task with explanation;
criticize my attempt and point out how it could be finished.
Thanks in advance!
Like so many challenges in J, sometimes it is better to keep your focus on your result and find a different way to get there. In this case, what your initial approach is doing is creating an identity matrix. I would use
=/~#:i. 4
1 0 0 0
0 1 0 0
0 0 1 0
0 0 0 1
You have correctly identified the issue with the trailing 0's and the fact that J will pad out with 0's to avoid ragged arrays. Boxing avoids this padding since each row is self contained.
So create your lists first. I would use overtake to get the extra 0's
4{.1
1 0 0 0
The next line uses 1: to return 1 as a verb and boxes the overtakes from 1 to 4
(>:#:i. <#:{."0 1:) 4
+-+---+-----+-------+
|1|1 0|1 0 0|1 0 0 0|
+-+---+-----+-------+
Since we want this as reversed and then made into strings, we add ":#:|.#: to the process.
(>:#:i. <#:":#:|.#:{."0 1:) 4
+-+---+-----+-------+
|1|0 1|0 0 1|0 0 0 1|
+-+---+-----+-------+
Then we unbox
>#:(>:#:i. <#:":#:|.#:{."0 1:) 4
1
0 1
0 0 1
0 0 0 1
I am not sure this is the way everyone would solve the problem, but it works.
An alternative solution that does not use boxing and uses the dyadic j. (Complex) and the fact that
1j4 # 1
1 0 0 0 0
(1 j. 4) # 1
1 0 0 0 0
(1 #~ 1 j. ]) 4
1 0 0 0 0
So, I create a list for each integer in i. 4, then reverse them and make them into strings. Since they are now strings, the extra padding is done with blanks.
(1 ":#:|.#:#~ 1 j. ])"0#:i. 4
1
0 1
0 0 1
0 0 0 1
Taking this step by step as to hopefully explain a little better.
i.4
0 1 2 3
Which is then applied to (1 ":#:|.#:#~ 1 j. ]) an atom at a time, hence the use of "0
Breaking down what is going on within the parenthesis. I first take the right three verbs which form a fork.
( 1 j. ])"0#:i.4
1 1j1 1j2 1j3
Now, effectively that gives me
1 ":#:|.#:#~ 1 1j1 1j2 1j3
The middle tine of the fork becomes the verb acting on the two noun arguments.The ~ swaps the arguments. so it becomes equivalent to
1 1j1 1j2 1j3 ":#:|.#:# 1
which because of the way #: works is the same as
": |. 1 1j1 1j2 1j3 # 1
I haven't shown the results of these components because using the "0 on the fork changes how the arguments that are sent to the middle tine and assembled later. I'm hoping that there is enough here that with some hand waving the explanation may suffice
The jump from tacit to explicit can be a big one, so it may be a better exercise to write the same verb explicitly to see if it makes more sense.
lowerTriangle =: 3 : 0
rightArg=. i. y
complexCopy=. 1 j. rightArg
1 (":#:|.#:#~)"0 complexCopy
)
lowerTriangle 4
1
0 1
0 0 1
0 0 0 1
lowerTriangle 5
1
0 1
0 0 1
0 0 0 1
0 0 0 0 1
See what happens when you 'get the ball rolling'? I guess the thing about J is that the ball goes down a pretty steep slope no matter where you begin. Exciting, eh?
Assume I have two lists, one is the text t, one is a list of characters c. I want to count how many times each character appears in the text.
This can be done easily with the following APL code.
+⌿t∘.=c
However it is slow. It take the outer product, then sum each column.
It is a O(nm) algorithm where n and m are the size of t and c.
Of course I can write a procedural program in APL that read t character by character and solve this problem in O(n+m) (assume perfect hashing).
Are there ways to do this faster in APL without loops(or conditional)? I also accept solutions in J.
Edit:
Practically speaking, I'm doing this where the text is much shorter than the list of characters(the characters are non-ascii). I'm considering where text have length of 20 and character list have length in the thousands.
There is a simple optimization given n is smaller than m.
w ← (∪t)∩c
f ← +⌿t∘.=w
r ← (⍴c)⍴0
r[c⍳w] ← f
r
w contains only the characters in t, therefore the table size only depend on t and not c. This algorithm runs in O(n^2+m log m). Where m log m is the time for doing the intersection operation.
However, a sub-quadratic algorithm is still preferred just in case someone gave a huge text file.
NB. Using "key" (/.) adverb w/tally (#) verb counts
#/.~ 'abdaaa'
4 1 1
NB. the items counted are the nub of the string.
~. 'abdaaa'
abd
NB. So, if we count the target along with the string
#/.~ 'abc','abdaaa'
5 2 1 1
NB. We get an extra one for each of the target items.
countKey2=: 4 : '<:(#x){.#/.~ x,y'
NB. This subtracts 1 (<:) from each count of the xs.
6!:2 '''1'' countKey2 10000000$''1234567890'''
0.0451088
6!:2 '''1'' countKey2 1e7$''1234567890'''
0.0441849
6!:2 '''1'' countKey2 1e8$''1234567890'''
0.466857
NB. A tacit version
countKey=. [: <: ([: # [) {. [: #/.~ ,
NB. appears to be a little faster at first
6!:2 '''1'' countKey 1e8$''1234567890'''
0.432938
NB. But repeating the timing 10 times shows they are the same.
(10) 6!:2 '''1'' countKey 1e8$''1234567890'''
0.43914
(10) 6!:2 '''1'' countKey2 1e8$''1234567890'''
0.43964
Dyalog v14 introduced the key operator (⌸):
{⍺,⍴⍵}⌸'abcracadabra'
a 5
b 2
c 2
r 2
d 1
The operand function takes a letter as ⍺ and the occurrences of that letter (vector of indices) as ⍵.
I think this example, written in J, fits your request. The character list is longer than the text (but both are kept short for convenience during development.) I have not examined timing but my intuition is that it will be fast. The tallying is done only with reference to characters that actually occur in the text, and the long character set is looked across only to correlate characters that occur in the text.
c=: 80{.43}.a.
t=: 'some {text} to examine'
RawIndicies=: c i. ~.t
Mask=: RawIndicies ~: #c
Indicies=: Mask # RawIndicies
Tallies=: Mask # #/.~ t
Result=: Tallies Indicies} (#c)$0
4 20 $ Result
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 4 0
0 0 1 0 0 0 2 1 2 0 0 0 1 3 0 0 0 2 0 0
4 20 $ c
+,-./0123456789:;<=>
?#ABCDEFGHIJKLMNOPQR
STUVWXYZ[\]^_`abcdef
ghijklmnopqrstuvwxyz
As noted in other answers, the key operator does this directly. However the classic APL way of solving this problem is still worth knowing.
The classic solution is "sort, shift, and compare":
c←'missippi'
t←'abcdefghijklmnopqrstuvwxyz'
g←⍋c
g
1 4 7 0 5 6 2 3
s←c[g]
s
iiimppss
b←s≠¯1⌽s
b
1 0 0 1 1 0 1 0
n←b/⍳⍴b
n
0 3 4 6
k←(1↓n,⍴b)-n
k
3 1 2 2
u←b/s
u
imps
And for the final answer:
z←(⍴t)⍴0
z
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
z[t⍳u]←k
z
0 0 0 0 0 0 0 0 3 0 0 0 1 0 0 2 0 0 2 0 0 0 0 0 0 0
This code is off the top of my head, not ready for production. Have to look for empty cases - the boolean shift is probably not right for all cases....
"Brute force" in J:
count =: (i.~~.) ({,&0) (]+/"1#:=)
Usage:
'abc' count 'abdaaa'
4 1 0
Not sure how it's implemented internally, but here are the timings for different input sizes:
6!:2 '''abcdefg'' count 100000$''abdaaaerbfqeiurbouebjkvwek''' NB: run time for #t = 100000
0.00803909
6!:2 '''abcdefg'' count 1000000$''abdaaaerbfqeiurbouebjkvwek'''
0.0845451
6!:2 '''abcdefg'' count 10000000$''abdaaaerbfqeiurbouebjkvwek''' NB: and for #t = 10^7
0.862423
We don't filter input date prior to 'self-classify' so:
6!:2 '''1'' count 10000000$''1'''
0.244975
6!:2 '''1'' count 10000000$''1234567890'''
0.673034
6!:2 '''1234567890'' count 10000000$''1234567890'''
0.673864
My implementation in APL (NARS2000):
(∪w),[0.5]∪⍦w←t∩c
Example:
c←'abcdefg'
t←'abdaaaerbfqeiurbouebjkvwek'
(∪w),[0.5]∪⍦w←t∩c
a b d e f
4 4 1 4 1
Note: showing only those characters in c that exist in t
My initial thought was that this was a case for the Find operator:
T←'ABCDEFGHIJKLMNOPQRSTUVWXYZ'
C←'MISSISSIPPI'
X←+/¨T⍷¨⊂C
The used characters are:
(×X)/T
IMPS
Their respective frequencies are:
X~0
4 1 2 4
I've only run toy cases so I have no idea what the performance is, but my intuition tells me it should be cheaper that the outer product.
Any thoughts?