I have several 100 *.mat files with matrices with same shapes. But over time this matrices were saved with different names. Lets say for this example a or b.
How can I now load and process this data without knowing the name using a new name instead?
An if condition is not an option, because there are too many different names.
I already tryed:
data = load('example_file.mat')
but then I need again the old variable names to access the matrices with data.a or data.b ...
What I need is something like:
load('example_file.mat') as matrix
Is this possible in Octave?
Your attempt
data = load('example_file.mat')
is the right start. You can examine data to find out what variables it contains. Assuming there’s always a single matrix in the MAT-file,
names = fieldnames(data);
data = data.(names{1});
Related
The title is really all there is to the question: how would you compute the values of a Fibonacci sequence (first N values, where N is an input variable) and store them in a Terraform local variable?
This could, of course, be done with an external data source, but I'm looking for a way to do it in pure Terraform.
There's no real need to actually do this, but the Fibonacci sequence is a representation of a problem I need to solve in Terraform (where values in a list depend on previous values of that same list).
I think the easiest way would be to create your own external data source for that, as in TF you can't access existing list elements during iteration when you create the lists itself.
And specific to Fibonacci sequence. I would just per-compute its values, and then just read any number of values I need from a list in TF or a file. Usually you would know a possible maximum number of those elements your app requires. Thus there is no reason to recalculate it every single time.
I am using the Neo library for linear algebra in Nim, and I would like to extract arbitrary rows from a matrix.
I can explicitly select a continuous sequence of rows as per the examples in the README, but can't select a disjoint subset of rows.
import neo
let x = randomMatrix(10, 4)
let some_rows = #[1,3,5]
echo x[2..4, All] # works fine
echo x[some_rows, All] ## error
The first echo works because you are creating a Slice object, which neo has defined a proc for. The second echo uses a sequence of integers, and that kind of access is not defined in the neo library. Unfortunately Slices define contiguous closed ranges, you can't even specify steps to iterate in bigger increments than one, so there is no way to accomplish what you want.
Looking at the structure of a Matrix, it seems that it is highly optimised to avoid copying data. Matrix transformation operations seem to reuse the data of the previous matrix and change the access/dimensions. As such, a matrix transformation with arbitrary random would not be possible, the indexes in your example specifically access non contiguos data and this would need to be encoded somehow in the new structure. Plus if you wrote #[1,5,3] that would defeat any kind of normal iterative looping.
An alternative of course is to write a proc which accepts a sequence instead of a slice and then builds a new matrix copying data from the old one. This implies a performance penalty, but if you think this is a good addition to the library please request it in the issue tracker of the project. If it is not accepted, then you will need to write yourself such a proc for personal use in your programs.
I have data in hadoop that looks like this:
DUMP EmailData; (Email, StateName) (allen#a.com, California)
(bobby#b.com, Arizona) (cindy#c.com, New York)
DUMP StateData; (StateCode, Lon, Lat) (AZ, -111.93248, 34.17163) (NY,
-75.810280, 42.75633) (CA, -119.25700, 37.26842)
I would like to do something along the lines of:
Locations = JOIN EmailData BY StateName, StateData BY StateCode;
But obviously you can't match names to codes.
If it were SQL, I would either use case statements or define a table and insert values that can bridge the relationship I want.
What is the best way to do this in Pig?
The simplest, fastest way would just be to quickly put together a 50-line (or more, depending on the presence of DC or any territories) file translating codes to names and JOIN it in. If you are feeling ambitious and don't want to deal with an extra file, you could write a UDF to do the translation, with the data hard-coded in. Since this data is very slow to change, hard-coding it is not a big deal.
I am writing a query which should detect certain loops within a graph, which means that I need I need to assign names to certain nodes within the path so that I can compare nodes later in the path with the saved ones. for example A -> B -> C -> A. Is this possible within gremlin?
It sounds like you're looking for something like this:
https://github.com/tinkerpop/gremlin/wiki/Except-Retain-Pattern
where you keep a list of previously traversed vertices and then utilize that list later in the traversal.
Suppose I have a list of numbers and I've computed the q-quantile (using Quantile).
Now a new datapoint comes along and I want to update my q-quantile, without having stored the whole list of previous datapoints.
What would you recommend?
Perhaps it can't be done exactly without, in the worst case, storing all previous datapoints.
In that case, can you think of something that would work well enough?
One idea I had, if you can assume normality, is to use the inverse CDF instead of the q-quantile.
Keep track of the sample variance as you go and then you can compute InverseCDF[NormalDistribution[sampleMean,sampleVariance], q] which should be the value such that a fraction q of the values are smaller, which is what the q-quantile is.
(I see belisarius was thinking along the same lines.
Here's the link he pointed to: http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#On-line_algorithm )
Unless you know that your underlying data comes from some distribution, it is not possible to update arbitrary quantiles without retaining the original data. You can, as others suggested, assume that the data has some sort of distribution and store the quantiles this way, but this is a rather restrictive approach.
Alternately, have you thought of programming this somewhere besides Mathematica? For example, you could create a class for your datapoints that contains (1) the Double value and (2) some timestamp for when the data came in. In a SortedList of these datapoints classes (which compares based on value), you could get the quantile very fast by simply referencing the index of the datapoints. Want to get a historical quantile? Simply filter on the timestamps in your sorted list.