I have a SpatVector with 10 points and 2 attributes (i_1, i_2). I need to summarise my SpatVector over one grid cell covering those 10 points and thought terra::rasterize was the most appropriate function to do just that. The output of rasterize needs to take into account both attributes of the SpatVector. However, I just can't figure it out how to pass the attributes to the function. E.g., using this odd function :
fn <- function(i_1, i_2,...) {
out <- mean(i_1*i_2, na.rm = T)
return(out)
}
terra::rasterize(points, raster, fun = fn(i_1, i_2))
returns a raster with 1 line and 1 column (what I wanted) but with value 1, whereas if I do:
fn(points$i_1, points$i_2)
I get what I'm supposed to be getting.
Either I'm missing how to pass the attributes of the SpatVector to the function in terra::rasterize or this function can't handle this analysis.
It would seem to me that you can combine the two fields prior to using rasterize
points$i_12 <- points$i_1 * points$i_2
out <- rasterize(points, raster, "i_12", fun = mean, na.rm=TRUE)
The same principle applies for more complex functions. First apply the function, then rasterize. Unless the function is complex in the sense that it, for example, takes the mean of each variable and adds these. In that case, you need to rasterize in steps.
Related
This question has somehow to do with an earlier post from me. See here overlap-of-nested-lists-creates-unwanted-gap
I think that I have found a solution but i can't figure out how to implement it.
First the relevant code since I think it is easier to explain my problem that way. I have prepared a fiddle to show the code:
PYFiddle here
Each iteration fills a nested list in ag depending on the axis. The next iteration is supposed to fill the next nested list in ag but depending on the length of the list filled before.
The generell idea to realise this is as follows:
First I would assign each nested list within the top for-loop to a variable like that:
x = ag[0]
y = ag[1]
z = ag[2]
In order to identify that first list I need to access data_j like that. I think the access would work that way.
data_j[i-1]['axis']
data_j[i-1]['axis'] returns either x,y or z as string
Now I need to get the length of the list which corresponds to the axis returned from data_j[i-1]['axis'].
The problem is how do I connect the "value" of data_j[i-1]['axis'] with its corresponding x = ag[0], y = ag[1] or z = ag[2]
Since eval() and globals() are bad practice I would need a push into the right direction. I couldn't find a solution
EDIT:
I think I figured out a way. Instead of taking the detour of using the actual axis name I will try to use the iterator i of the parent loop (See the fiddle) since it increases for each element from data_j it kinda creates an id which I think I can use to create a method to use it for the index of the nest to address the correct list.
I managed to solve it using the iterator i. See the fiddle from my original post in order to comprehend what I did with the following piece of code:
if i < 0:
cond = 0
else:
cond = i
pred_axis = data_j[cond]['axis']
if pred_axis == 'x':
g = 0
elif pred_axis == 'y':
g = 1
elif pred_axis == 'z':
g = 2
calc_size = len(ag[g])
n_offset = calc_size+offset
I haven't figured yet why cond must be i and not i-1 but it works. As soon as I figure out the logic behind it I will post it.
EDIT: It doesn't work for i it works for i-1. My indices for the relevant list start at 1. ag[0] is reserved for a constant which can be added if necessary for further calculations. So since the relevant indices are moved up by the value of 1 from the beginning already i don't need to decrease the iterator in each run.
This might be a simple question. However, I wanted to get some clarifications of how the following code works.
a = np.arange(8)
a
array([1,2,3,4,5,6,7])
Example Function = a[0:-1]+a[1:]/2.0
In the Example Function, I want to draw your attention to the plus sign between the array a[0:-1]+a[1:]. How does that work? What does that look like?
For instance, is the plus sign (addition) adding the first index of each array? (e.g 1+2) or add everything together? (e.g 1+2+2+3+3+4+4+5+5+6+6+7)
Then, I assume /2.0 is just dividing it by 2...
A numpy array uses vector algebra in that you can only add two arrays if they have the same dimensions as you are adding element by element
a = [1,2,3,4,5]
b = [1,1,1]
a+b # will throw an error
whilst
a = [1,2,3,4,5]
b = [1,1,1,1,1]
a+b # is ok
The division is also element by element.
Now to your question about the indexing
a = [1,2,3,4,5]
a[0:-1]= [1,2,3,4]
a[1:] = [2,3,4,5]
or more generally a[index_start: index_end] is inclusive at the start_index but exclusive at the end_index - unless you are given a a[start_index:]where it includes everything up to and including the last element.
My final tip is just to try and play around with the structures - there is no harm in trying different things, the computer will not explode with a wrong value here or there. Unless you trying to do so of course.
If arrays have identical shapes, they can be added:
new_array = first_array.__add__(second_array)
This simple operation adds each value from first_array to each value in second_array and puts result into new_array.
Coming from a C# background, I would say that the ref keyword is very useful in certain situations where changes to a method parameter are desired to directly influence the passed value for value types of for setting a parameter to null.
Also, the out keyword can come in handy when returning a multitude of various logically unconnected values.
My question is: is it possible to pass a parameter to a function by reference in Haskell? If not, what is the direct alternative (if any)?
There is no difference between "pass-by-value" and "pass-by-reference" in languages like Haskell and ML, because it's not possible to assign to a variable in these languages. It's not possible to have "changes to a method parameter" in the first place in influence any passed variable.
It depends on context. Without any context, no, you can't (at least not in the way you mean). With context, you may very well be able to do this if you want. In particular, if you're working in IO or ST, you can use IORef or STRef respectively, as well as mutable arrays, vectors, hash tables, weak hash tables (IO only, I believe), etc. A function can take one or more of these and produce an action that (when executed) will modify the contents of those references.
Another sort of context, StateT, gives the illusion of a mutable "state" value implemented purely. You can use a compound state and pass around lenses into it, simulating references for certain purposes.
My question is: is it possible to pass a parameter to a function by reference in Haskell? If not, what is the direct alternative (if any)?
No, values in Haskell are immutable (well, the do notation can create some illusion of mutability, but it all happens inside a function and is an entirely different topic). If you want to change the value, you will have to return the changed value and let the caller deal with it. For instance, see the random number generating function next that returns the value and the updated RNG.
Also, the out keyword can come in handy when returning a multitude of various logically unconnected values.
Consequently, you can't have out either. If you want to return several entirely disconnected values (at which point you should probably think why are disconnected values being returned from a single function), return a tuple.
No, it's not possible, because Haskell variables are immutable, therefore, the creators of Haskell must have reasoned there's no point of passing a reference that cannot be changed.
consider a Haskell variable:
let x = 37
In order to change this, we need to make a temporary variable, and then set the first variable to the temporary variable (with modifications).
let tripleX = x * 3
let x = tripleX
If Haskell had pass by reference, could we do this?
The answer is no.
Suppose we tried:
tripleVar :: Int -> IO()
tripleVar var = do
let times_3 = var * 3
let var = times_3
The problem with this code is the last line; Although we can imagine the variable being passed by reference, the new variable isn't.
In other words, we're introducing a new local variable with the same name;
Take a look again at the last line:
let var = times_3
Haskell doesn't know that we want to "change" a global variable; since we can't reassign it, we are creating a new variable with the same name on the local scope, thus not changing the reference. :-(
tripleVar :: Int -> IO()
tripleVar var = do
let tripleVar = var
let var = tripleVar * 3
return()
main = do
let x = 4
tripleVar x
print x -- 4 :(
I am interested in creating a list / array of functions "G" consisting of many small functions "g". This essentially should correspond to a series of functions 'evolving' in time.
Each "g" takes-in two variables and returns the product of these variables with an outside global variable indexed at the same time-step.
Assume obs_mat (T x 1) is a pre-defined global array, and t corresponds to the time-steps
G = []
for t in range(T):
# tried declaring obs here too.
def g(current_state, observation_noise):
obs = obs_mat[t]
return current_state * observation_noise * obs
G.append(g)
Unfortunately when I test the resultant functions, they do not seem to pick up on the difference in the obs time-varying constant i.e. (Got G[0](100,100) same as G[5](100,100)). I tried playing around with the scope of obs but without much luck. Would anyone be able to help guide me in the right direction?
This is a common "gotcha" to referencing variables from an outer scope when in an inner function. The outer variable is looked up when the inner function is run, not when the inner function is defined (so all versions of the function see the variable's last value). For each function to see a different value, you either need to make sure they're looking in separate namespaces, or you need to bind the value to a default parameter of the inner function.
Here's an approach that uses an extra namespace:
def make_func(x):
def func(a, b):
return a*b*x
return func
list_of_funcs = [make_func(i) for i in range(10)]
Each inner function func has access to the x parameter in the enclosing make_func function. Since they're all created by separate calls to make_func, they each see separate namespaces with different x values.
Here's the other approach that uses a default argument (with functions created by a lambda expression):
list_of_funcs = [lambda a, b, x=i: a*b*x for i in range(10)]
In this version, the i variable from the list comprehension is bound to the default value of the x parameter in the lambda expression. This binding means that the functions wont care about the value of i changing later on. The downside to this solution is that any code that accidentally calls one of the functions with three arguments instead of two may work without an exception (perhaps with odd results).
The problem you are running into is one of scoping. Function bodies aren't evaluated until the fuction is actually called, so the functions you have there will use whatever is the current value of the variable within their scope at time of evaluation (which means they'll have the same t if you call them all after the for-loop has ended)
In order to see the value that you would like, you'd need to immediately call the function and save the result.
I'm not really sure why you're using an array of functions. Perhaps what you're trying to do is map a partial function across the time series, something like the following?
from functools import partial
def g(current_state, observation_noise, t):
obs = obs_mat[t]
return current_state * observation_noise * obs
g_maker = partial(g, current, observation)
results = list(map(g_maker, range(T)))
What's happening here is that partial creates a partially-applied function, which is merely waiting for its final value to be evaluated. That final value is dynamic (but the first two are fixed in this example), so mapping that partially-applied function over a range of values gets you answers for each value.
Honestly, this is a guess because it's hard to see what else you are trying to do with this data and it's hard to see what you're trying to achieve with the array of functions (and there are certainly other ways to do this).
The issue (assuming that your G.append call is mis-indented) is simply that the name t is mutated when you loop over the iterator returned by range(T). Since every function g you create stores returns the same name t, they wind up all returning the same value, T - 1. The fix is to de-reference the name (the simplest way to do this is by sending t into your function as a default value for an argument in g's argument list):
G = []
for t in range(T):
def g(current_state, observation_noise, t_kw=t):
obs = obs_mat[t_kw]
return current_state * observation_noise * obs
G.append(g)
This works because it creates another name that points at the value that t references during that iteration of the loop (you could still use t rather than t_kw and it would still just work because tg is bound to the value that tf is bound to - the value never changes, but tf is bound to another value on the next iteration, while tg still points at the "original" value.
There are some other questions on here that are similar but sufficiently different that I need to pose this as a fresh question:
I have created an empty class, lets call it Test. It doesn't have any properties or methods. I then iterate through a map of key/value pairs, dynamically creating properties named for the key and containing the value... like so:
def langMap = [:]
langMap.put("Zero",0)
langMap.put("One",1)
langMap.put("Two",2)
langMap.put("Three",3)
langMap.put("Four",4)
langMap.put("Five",5)
langMap.put("Six",6)
langMap.put("Seven",7)
langMap.put("Eight",8)
langMap.put("Nine",9)
langMap.each { key,val ->
Test.metaClass."${key}" = val
}
Now I can access these from a new method created like this:
Test.metaClass.twoPlusThree = { return Two + Three }
println test.twoPlusThree()
What I would like to do though, is dynamically load a set of instructions from a String, like "Two + Three", create a method on the fly to evaluate the result, and then iteratively repeat this process for however many strings containing expressions that I happen to have.
Questions:
a) First off, is there simply a better and more elegant way to do this (Based on the info I have given) ?
b) Assuming this path is viable, what is the syntax to dynamically construct this closure from a string, where the string references variable names valid only within a method on this class?
Thanks!
I think the correct answer depends on what you're actually trying to do. Can the input string be a more complicated expression, like '(Two + Six) / Four'?
If you want to allow more complex expressions, you may want to directly evaluate the string as a Groovy expression. Inside the GroovyConsole or a Groovy script, you can directly call evaluate, which will evaluate an expression in the context of that script:
def numNames = 'Zero One Two Three Four Five Six Seven Eight Nine'.split()
// Add each numer name as a property to the script.
numNames.eachWithIndex { name, i ->
this[name] = i
}
println evaluate('(Two + Six) / Four') // -> 2
If you are not in one of those script-friendly worlds, you can use the GroovyShell class:
def numNames = 'Zero One Two Three Four Five Six Seven Eight Nine'.split()
def langMap = [:]
numNames.eachWithIndex { name, i -> langMap[name] = i }
def shell = new GroovyShell(langMap as Binding)
println shell.evaluate('(Two + Six) / Four') // -> 2
But, be aware that using eval is very risky. If the input string is user-generated, i would not recommend you going this way; the user could input something like "rm -rf /".execute(), and, depending on the privileges of the script, erase everything from wherever that script is executed. You may first validate that the input string is "safe" (maybe checking it only contains known operators, whitespaces, parentheses and number names) but i don't know if that's safe enough.
Another alternative is defining your own mini-language for those expressions and then parsing them using something like ANTLR. But, again, this really depends on what you're trying to accomplish.