J code seems to me like a big "math function" that maps input to output (very functional).
I wonder if J (maybe inside jqt) also supports the code pattern of asynchronous/callback function kind of
async_run(read_from_serial_port, call_back_fun(data){
display(after_processing(data))
})
keep_on_doing_something_else
The idea is to make a gui (real-time, non-blocking) that plots data reading back from the serial port in J.
Related
I'm new to python3 and I've been practicing by doing some basic programs but now I'm stuck with a little snake type game I wrote. The code I wrote so far have the following parts in order:
Imported modules
Classes to get a key from the user without the need to press "enter"
A function that is called by the module threading (named move_alt()).
t = threading.Thread(target=move_alt) # this is immediately after the portion of code
described in point 3)
4)A move_forward() function to create the map using a list of characters and to move a "position character" starting at the center of the map to the right till it crashes into a "wall caracter". This function is named move_forward().
t.setDaemon(True)
t.start()
move_forward()
The idea is to start a thread to run simultaneously with move_forward function, which targets the function described in 2) to take a key from the user and depending on what key was pressed make some action over move_forward function, for example (and to be more specific) move_up or move_down, etc.
I don't know how to use the threading module correctly so I'm not getting my code to work as expected. Instead what is happening is the program would run the thread without running simultaneously the move_forward function and until I don't press a key (which is what it's done inside the thread) it doesn't do anything. Actually, at the beginning when i just run the program it tries to draw the map (list of characters) but only draws some characters and then gets stuck.
Sorry if my explanation wasn't efficient but English is not my first language and I tend to write too much.
P.S= I didn't put the code above because it is too long due to comments and stuff I made to learn (and not forget)
After a lot of thinking I was finally able to make the program work as expected. First of all, the threading module wasn't the best option but for some reason I was confusing threading (which is used for concurrency purposes) with whatever I was trying to do. So after a little while I stopped using this idea.
The way I managed to make it work was actually using another module called "keyboard" (for linux users you have to use the module as root for some reason). Here is a little bit of a code to use that module:
def onkeypressw(event): this function is used as a condition in the "while loop" for the move_up condition
global stopw
if event.name == 's' or event.name == "a" or event.name == "d":
stopw = True
keyboard.on_press(onkeypressw)
def onkeypressw1(event): # this function is used as condition in the if sentence in the main function to call move_up() function
global runw1
if event.name == "w":
runw1 = True
else:
runw1 = False
keyboard.on_press(onkeypressw1)
Then in the main function i have a while True loop running, where i have some if sentences that run one of four possibles move function: move_up, move_down, move_left, move_right depending of which key i pressed (w-s-d-a). Inside these "move functions" there are while loops to draw the map and move the character position(let's say the snake head) using a list of characters, until I press another button. So for example if in the main function I pressed "w" the program is going to run move_up function and move the character position until either it crashes against "the wall character" or I press "s, d or a". The conditions I put for the "while loop" are: 1)one for the wall character and 2)"not stopw" so it draws the list until I crash against the wall or stopw = True.
I know you can make this code using others modules and get the same result with less work but i wanted to use as much basic code as possible to learn python3 faster because I had previous experience programming with C.
P.S: Here is one of the moves function, just in case someone is curious:
def move_up():
global c, f
global map1
global stopw
while f> 0 and not stopw:
map1[f, c] = " o " # f represents the rows and c the columns, I replaced the initial position character "_" for "o"
f = f - 1 # up one row
map1[f, c] = " _ " # I put the position character "_" at the new position
cls() # clear the screen
time.sleep(0.4) # add a little delay just to modify the snake speed
stopw = False # reset the boolean
The problem
I am producing data files with Fortran 90 program and then analyzing it with Python 3.7. The data files are just many numbers, written in scientific notation.
My problem is that at some point the scientific notation inside files is wrong. The E just disappears. See an example from the Fortran output :
...
0.2693606328922176223E+99
0.3769504465049542893E+99
0.5275144982939579450E+99
0.7382178439909821718E+99
0.1033081719932198510+100
0.1445722084267474381+100
0.2023182004494124187+100
...
Infinity
Infinity
Infinity
...
This leads to the following error on my Python analysis program
ValueError: could not convert string to float: '0.1383817626772070210+100'
Kind of solution I'm looking for
I suppose this problem is inherent to Fortran. I am looking for a Python solution that could handle this weird notation, or at least way to just skip those files that may give weird results after all.
Program sample
I give you a little sample from my Python code, this may help. The following code opens a file, reads a column, and load the whole column inside the PMvar. The for loop is here because I read many files.
for rf in range(elem_2):
# load fluorescence and Temperature
PMvar[tr] = some_function(filepath[rf])[0]
The some_function takes as a parameter filepath[rf] which is the file path. I only retrieve the first column thus the [0] .
def some_function(filepath):
out0 = loadtxt('{}.dat'.format(filepath),comments='%')
out1 = loadtxt('{}.dat'.format(filepath),comments='%')
return out0, out1
I can't see any explicit conversions from string to float.
Is there a way to adapt this code in order to handle the weird scientific notation ? Or should I just rewrite some stuff to comply with this case ?
EDIT 1
The Fortran code is not from me, but I think I found where the data is saved.
open(10, file = trim(adjustl(str_file_Temp))//'.dat', status='replace', access='sequential', action='write')
print*, 'j_save_temp Temp',j_save_temp
do i = 1, j_save_temp
write(10,221) save_temperature(i,0),&
m_kb_x_inv_n_ions2(1)*save_temperature(i,1:3), &
m_kb_x_inv_n_ions(1) *save_temperature(i,4:6), &
save_temperature(i,7)/real(n_dt)
enddo
221 format( 8(1X,e26.19))
close(10)
deallocate(save_temperature)
with double precision, allocatable, dimension(:,:) :: save_temperature
I found H2O has the function h2o.deepfeatures in R to pull the hidden layer features
https://www.rdocumentation.org/packages/h2o/versions/3.20.0.8/topics/h2o.deepfeatures
train_features <- h2o.deepfeatures(model_nn, train, layer=3)
But I didn't find any example in Python? Can anyone provide some sample code?
Most Python/R API functions are wrappers around REST calls. See http://docs.h2o.ai/h2o/latest-stable/h2o-py/docs/_modules/h2o/model/model_base.html#ModelBase.deepfeatures
So, to convert an R example to a Python one, move the model to be the this, and all other args should shuffle along. I.e. the example from the manual becomes (with dots in variable names changed to underlines):
prostate_hex = ...
prostate_dl = ...
prostate_deepfeatures_layer1 = prostate_dl.deepfeatures(prostate_hex, 1)
prostate_deepfeatures_layer2 = prostate_dl.deepfeatures(prostate_hex, 2)
Sometimes the function name will change slightly (e.g. h2o.importFile() vs. h2o.import_file() so you need to hunt for it at http://docs.h2o.ai/h2o/latest-stable/h2o-py/docs/index.html
I am new to Spark and reasonably new to Clojure (although I really like what Clojure can do so far). I am currently trying to parse JSON in Clojure using sparkling, and I am having trouble with the basics of transforming data and getting it back in a form I can understand and debug. I use dummy data in the example below, but my actual data is over 400GB.
As an example, I first tried splitting each line of my JSON input (each line is a full record) by commas so that I would have a list of keys and values (for eventual conversion to keyword and value maps). In Scala (for which it is easier to find Spark examples) with dummy data, this works fine:
val data = sc.parallelize(Array ("a:1,b:2","a:3,b:4"))
val keyVals = data.map(line => line.split(","))
keyVals.collect()
This returns Array[Array[String]] = Array(Array(a:1, b:2), Array(a:3, b:4)), which is at least a reasonable starting point for key-value mapping.
However, when I run the following in clojure with sparkling:
(def jsony-strings (spark/parallelize sc ["a:1,b:2","a:3,b:4"]))
(def jsony-map (->> jsony-strings
(spark/map (fn [l] (string/split l #",")))
))
(spark/collect jsony-map)
I get the usual concurrency spaghetti from the JVM, the crux of which seems to be:
2018-08-24 18:49:55,796 WARN serialization.Utils:55 - Error deserializing object (clazz: gdelt.core$fn__7437, namespace: gdelt.core)
java.lang.ClassNotFoundException: gdelt.core$fn__7437
Which is an error I seem to get pretty much anything I try to do something more complex than counts.
Can someone please point me in the right direction?
I guess I should note that my Big Problem is processing lots and lots of lines of JSON in a bigger-than-memory (400G) dataset. I will be using the JSON keys to filter, sort, calculate, etc., and the Spark pipelines looked good for both rapid parallel processing and convenient for these functions. But I am certainly open to considering other alternatives for processing this dataset.
You should use Cheshire for this:
;; parse some json
(parse-string "{\"foo\":\"bar\"}")
;; => {"foo" "bar"}
;; parse some json and get keywords back
(parse-string "{\"foo\":\"bar\"}" true)
;; => {:foo "bar"}
I like to use a shortcut for the 2nd case, since I always want to convert string keys into clojure keywords:
(is= {:a 1 :b 2} (json->edn "{\"a\":1, \"b\":2}"))
It is just a simple wrapper with (I think) an easier-to-remember name:
(defn json->edn [arg]
"Shortcut to cheshire.core/parse-string"
(cc/parse-string arg true)) ; true => keywordize-keys
(defn edn->json [arg]
"Shortcut to cheshire.core/generate-string"
(cc/generate-string arg))
Update: Note that Cheshire can work with lazy streams:
;; parse a stream lazily (keywords option also supported)
(parsed-seq (clojure.java.io/reader "/tmp/foo"))
I am conducting linearK function for the observed point pattern on a linear network and I get the following error
Error in retainedges[dat$seg] : invalid subscript type 'list'
I do not understand what it means and how should I correct it.
On the traceback call, I get the following information
> traceback()
4: thinNetwork(x, retainvertices = subi)
3: countends(L, X[-j], D[-j, j], toler = toler)
2: linearKengine(X, r = r, ..., denom = denom, correction = correction,
ratio = ratio)
1: linearK(sl2)
Could someone help me on what this error means and how I can correct it.
Thank you.
Your network is a bit problematic since it is disconnected. It has one very big component with 3755 vertices and 5593 lines and then 5 small components with only 2 or 3 vertices and 1 or 2 lines that are not connected to anything else. In your example you have only two points in this big network (both occurring in the big component as far as I can tell). We might be able to handle this in future versions of spatstat, but for now I suggest you simply discard the small empty components. Then I think linearK works as expected for your example (although I doubt you find interesting information from a pattern of 2 points!).
To identify connected components of a linear network use connected.linnet with argument what = "components" then you get a list of connected components and you can use the big connected component to define a new lpp on a connected linnet. With your example you could do something like (noting that component number 1 is the main component):
comp <- connected(as.linnet(sl2), what = "comp")
sl2new <- lpp(as.ppp(sl2), comp[[1]])