Name variable based on string MATLAB - string

I have a variable that is created by a loop. The variable is large enough and in a complicated enough form that I want to save the variable each time it comes out of the loop with a different name.
PM25 is my variable. But I want to save it as PM25_year in which the year changes based on `str = fname(13:end)'
PM25 = permute(reshape(E',[c,r/nlay,nlay]),[2,1,3]); % Reshape and permute to achieve the right shape. Each face of the 3D should be one day
str = fname(13:end); % The year
% Third dimension is organized so that the data for each site is on a face
save('PM25_str', 'PM25_Daily_US.mat', '-append')
The str would be a year, like 2008. So the variable saved would be PM25_2008, then PM25_2009, etc. as it is created.

Defining new variables based on data isn't considered best practice, but you can store your data more efficiently using a cell array. You can store even a large, complicated variable like your PM25 variable within a single cell. Here's how you could go about doing it:
Place your PM25 data for each year into the cell array C using your loop:
for i = 1:numberOfYears
C{i} = PM25;
end
Resulting in something like this:
C = { PM25_2005, PM25_2006, PM25_2007 };
Now let's say you want to obtain your variable for the year 2006. This is easy (assuming you aren't skipping years). The first year of your data will correspond to position 1, the second year to position 2, etc. So to find the index of the year you want:
minYear = 2005;
yearDesired = 2006;
index = yearDesired - minYear + 1;
PM25_2006 = C{index};

You can do this using eval, but note that it's often not considered good practice. eval may be a security risk, as it allows user input to be executed as code. A better way to do this may be to use a cell array or an array of objects.
That said, I think this will do what you want:
for year = 2008:2014
eval(sprintf('PM25_%d = permute(reshape(E',[c,r/nlay,nlay]),[2,1,3]);',year));
save('PM25_Daily_US.mat',sprintf('PM25_%d',year),'-append');
end

I do not recommend to set variables like this since there is no way to track these variables and completely prevents all kind of error checking that MATLAB does beforehand. This kind of code is handled completely in runtime.
Anyway in case you have a really good reason for doing this I recommend that you use the function assignin for this.
assignin('caller', ['myvar',num2str(1)], 63);

Related

Changes in a temporary variable are affecting the variable that feeds from

I'm designing a Mastermind game, which basically compares 2 lists and marks the similarities. When a colour is found at the right place, a flag making the correct position is added and the item found on the reference list is marked off. The reference list is feeding off an array from another function. The problem is at the mark off, as any changes done to the reference list is changing also the original array, which i don't want it to happen
tempCode = mCode #mCode is the array combination randomly generated from another function
for i in range (len(uCode)): #user input array
for j in range (len(tempCode)): #temp array
if uCode[i] == tempCode[j]: # compare individual chars
if i == j: #compare position
flagMark = "*"
tempCode.insert(j+1, "x") #problem starts here
tempCode.remove(tempCode[j])
fCode.append(flagMark)
When the insert is reached both the tempCode and mCode change which it is not intended.
The code is written in a way should the user enter a combination of the same colours, thus checking the chras(the colours are just letters) and the position, and then mark them of with "x"
As it stands, when it gets to
tempCode.insert(j+1, "x")
the arrays will change to
mCode = ["B","R","x","G","Y"]
tempCode = ["B","R","x","G","Y"]
when I would just want
mCode = ["B","R","G","Y"]
tempCode = ["B","R","x","G","Y"]
See also this answer, which is a different presentation of the same problem.
Essentially, when you do tempCode = mCode, you're not making a copy of mCode, you're actually making another reference to it. Anything you do to tempCode thereafter affects the original as well, so at any given time the condition tempCode == mCode will be true (as they're the same object).
You probably want to make a copy of mCode, which could be done in either of the following ways:
tempCode = mCode.copy()
tempCode = mCode[:]
which produces a different list with the same elements, rather than the same list

PowerShell on CSV file - looking for string depending on string

I need your help regarding PowerShell programming on CSV file.
I've made some searches but cannot find what I'm looking for (or perhaps I don't know the technical terms). Basically, I have an Excel workbook with large amount of data (more or less 38 columns x 350.000 rows), and there are a couple of formulas that take hours to calculate.
I was first wondering if PowerShell could speed up a bit the calculation compared to Excel. The calculations taking most of my time are in fact not that complex (at least at first glance). My data is more or less constructed like this:
Ref Title
----- --------------------------
A/001 "free_text"
A/002 "free_text A/001 free_text"
... ...
A/005 "free_text A/004 free_text"
A/006 "free_text"
B/001 "free_text"
B/002 "free_text"
C/001 "free_text"
C/002 "free_text"
...
C/050 "free_text C/047 free_text"
... ...
C/103 "free_text"
D/001 "free_text"
D/002 "free_text D/001 free_text"
... ....
Basically the data is as follows:
the Ref field contains unique values, in {letter}/{incremental value} format.
In some rows, the Title field may call up one of the Ref data. For example, in line 2, the Title calls for the A/001 Ref. In the last row, the Title calls for the D/001 Ref, etc.
There is no logic pattern defining when this ref could be called up in a title. This is random.
However, what I'm 100% sure of is the following:
The Ref called in the Title is always belonging to the same {letter} block. For example: the string 'C/047' in the Title field can only be found in the block where the Ref {letter} is C.
The Ref called in the Title will always be located 'after' (or in a lower row) than the Ref it refers to. In other words, I cannot have a line with following pattern:
Ref Title
------------ -----------------------------------------
{letter/i} {free_text {letter/j} free_text} with j<i
→ This is not possible.
→ j is always > i
I've used these characteristics in Excel to minimize my lookup arrays. But it still takes an hour to calculate everything.
I've therefore looked into PowerShell, and started to 'play' a bit with the CSV, and looping with the ForEach-Object hoping I would have quicker results. Up to now I basically ended-up looping twice on my CSV file.
$CSV1 = myfile.csv
$CSV2 = myfile.csv
$CSV1 | ForEach-Object {
# find Title
$TitSearch = $_.$Ref
$CSV2 | ForEach-Object {
if ($_.$Title -eq $TitSearch) {
myinstructions
}
}
}
It works but it's really really really long. So I then tried the following instead of using the $CSV2 | ForEach...:
$CSV | where {$_.$Title -eq $TitleSearch} | % $Ref
In either case, it's too long and not efficient at all. Additionally with these 2 solutions, I'm not using above characteristics which could reduce the lookup array and as already stated, it seems I end up looping twice on the CSV file from its beginning up to the end.
Questions:
Is there a leaner way to do this?
Am I wasting my time with PowerShell?
I though about creating 1 file per Ref {letter} block (1 file for block A, 1 for B, etc...). However I have about 50.000 blocks to create. Or create them one by one, carry out the analysis, put the results in a new file, and delete them. Would that be quicker?
Note: this is for work, to be used by other colleagues, and Excel and PowerShell are really the only softwares we may use. I know VBA but ok... At the end I'm curious about how and if this can be solved in a simple manner using PowerShell.
As far as I can see your base algorithm do N^2 iteration (~120 billion). There is a standard way to make it efficient - you need to build a hashtable first. Hashtable is a key/value storage, and look up is pretty much instantaneous, so algorithm's time complexity will become ~N.
Powershell has built-in data type for that. In your case the key would be ref, and the value an array of cell data (assuming your table is smth like: ref, title, col1, ..., colN)
$hash = #{}
foreach($row in $table} {$hash.Add($row.ref, #($row.title, $row.col1, ...)}
#it will take 350K steps to generate it
#then you can iterate over it again
foreach($key in $hash.Keys) {
$key # access current ref
$rowData = $hash.$key # access to current row elements (by index)
$refRowData = $hash[$rowData[$j]] # lookup from other rows, assuming lookup reference is in some column
}
So it's a general idea how to solve the time issue. To be honest I don't believe you need to recreate a wheel and code it yourself. What you need is a relational database. Since you have excel, you should have MS ACCESS too. Just import your data in there, make ref and title an index, then all you need to do is self join. MS Access suck, but I'm sure it will handle 350K row just fine.
Ideally you'd need to get a database on some corporate MSSQL server (open a ticket, talk to your manger, etc). It will calculate all that in seconds, and then you can link the output to a spreadsheet as well.

Lua: Parsing and Manipulating Input with Loops - Looking for Guidance

I am currently attempting to parse data that is sent from an outside source serially. An example is as such:
DATA|0|4|7x5|1|25|174-24|7x5|1|17|TERW|7x5|1|9|08MN|7x5|1|1|_
This data can come in many different lengths, but the first few pieces are all the same. Each "piece" originally comes in with CRLF after, so I've replaced them with string.gsub(input,"\r\n","|") so that is why my input looks the way it does.
The part I would like to parse is:
4|7x5|1|25|174-24|7x5|1|17|TERW|7x5|1|9|08MN|7x5|1|1|_
The "4" tells me that there will be four lines total to create this file. I'm using this as a means to set the amount of passes in the loop.
The 7x5 is the font height.
The 1 is the xpos.
The 25 is the ypos.
The variable data (172-24 in this case) is the text at these parameters.
As you can see, it should continue to loop this pattern throughout the input string received. Now the "4" can actually be any variable > 0; with each number equaling a set of four variables to capture.
Here is what I have so far. Please excuse the loop variable, start variable, and print commands. I'm using Linux to run this function to try to troubleshoot.
function loop_input(input)
var = tonumber(string.match(val, "DATA|0|(%d*).*"))
loop = string.match(val, "DATA|0|")
start = string.match(val, loop.."(%d*)|.*")
for obj = 1, var do
for i = 1, 4 do
if i == 1 then
i = "font" -- want the first group to be set to font
elseif i == 2 then
i = "xpos" -- want the second group to be set to xpos
elseif i == 3 then
i = "ypos" -- want the third group to be set to ypos
else
i = "txt" -- want the fourth group to be set to text
end
obj = font..xpos..ypos..txt
--print (i)
end
objects = objects..obj -- concatenate newly created obj variables with each pass
end
end
val = "DATA|0|4|7x5|1|25|174-24|7x5|1|17|TERW|7x5|1|9|08MN|7x5|1|1|_"
print(loop_input(val))
Ideally, I want to create a loop that, depending on the var variable, will plug in the captured variables between the pipe deliminators and then I can use them freely as I wish. When trying to troubleshoot with parenthesis around my four variables (like I have above), I receive the full list of four variables four times in a row. Now I'm having difficulty actually cycling through the input string and actually grabbing them out as the loop moves down the data string. I was thinking that using the pipes as a means to delineate variables from one another would help. Am I wrong? If it doesn't matter and I can keep the [/r/n]+ instead of each "|" then I am definitely all for that.
I've searched around and found some threads that I thought would help but I'm not sure if tables or splitting the inputs would be advisable. Like these threads:
Setting a variable in a for loop (with temporary variable) Lua
How do I make a dynamic variable name in Lua?
Most efficient way to parse a file in Lua
I'm fairly new to programming and trying to teach myself. So please excuse my beginner thread. I have both the "Lua Reference Manual" and "Programming in Lua" books in paperback which is how I've tried to mock my function(s) off of. But I'm having a problem making the connection.
I thank you all for any input or guidance you can offer!
Cheers.
Try this:
val = "DATA|0|4|7x5|1|25|174-24|7x5|1|17|TERW|7x5|1|9|08MN|7x5|1|1|_"
val = val .. "|"
data = val:match("DATA|0|%d+|(.*)$")
for fh,xpos,ypos,text in data:gmatch("(.-)|(.-)|(.-)|(.-)|") do
print(fh,xpos,ypos,text)
end

fminsearch is overwriting in loop while storing data

I have the code mentioned below in matlab. I want to write all the 162 rows and 4 columns calculated into an excel file.
When i use xlswrite in the code i get only one row and 4 columns as the value of P gets overwritten in each iterative step.
If i use another loop inside the for loop the execution time increase drastically. Please help to least write the values of P into an array which i can later write into excel file(when i tried 'In an assignment A(I) = B, the number of elements in B and I must be the same' error appeared.)
please help
function FitSMC_BC
clc
% Parameters: P(1)=theta_S; P(2)=theta_r; P(3)=psib; P(4)=lamda;
smcdata=xlsread('asimdata');
nn=length(smcdata)-1;
for i=1:nn
psi=smcdata(:,1);
thetaObs=smcdata(:,i+1);
%Make an initial guess:
Pini=[0.5 0.1 -1 1.5];
P=fminsearch(#ObFun,Pini,[],psi,thetaObs);
disp(['result',num2str(i),': P=',num2str(P)]);
theta=Gettheta(P,psi);
end
function OF=ObFun(P,psi,thetaObs)
theta=Gettheta(P,psi);
OF=sqrt(mean((theta - thetaObs).^2));
function theta=Gettheta(P,psi)
SoilPars.theta_S=P(1);
SoilPars.theta_r=P(2);
SoilPars.psib=P(3);
SoilPars.lamda=P(4);
[theta]=thetaFun(psi,SoilPars);
function [theta]=thetaFun(psi,SoilPars)
theta_S=SoilPars.theta_S;
theta_r=SoilPars.theta_r;
psib=SoilPars.psib;
lamda=SoilPars.lamda;
theta=theta_r+((theta_S-theta_r)*((psib./psi).^lamda));
theta(psi>psib)=theta_S;
You can modify the P line with
P(i,:) = fminsearch(#ObFun,Pini,[],psi,thetaObs);
P will store each calculation (4 element vector) in a new line.
You may also initialise P before the for loop with P = nan(nn, 4);
Then write P in an Excel file using xlswrite.
I haven't studied your code in-depth, but as far as I can tell, you have two options:
Create a matrix P and use xlswrite on the entire matrix. This seems to me like the most reasonable approach.
Use xlswrite1 from the fileexchange in a loop. This will increase execution time a bit, but not nearly as much as using regular xlswrite as it is specially deigned to be used inside loops. The reason why it is so much faster is because it only opens and closes the Excel-file once, whereas the regular xlswrite opens and closes it every time you call the function.
You seem to know how to use indexing so I'm not sure why you're simply doing something like this:
P = zeros(size(smcdata,1),nn)
for i=1:nn
...
P(:,i) = fminsearch(#ObFun,Pini,[],psi,thetaObs);
disp(['result',num2str(i),': P=',num2str(P(:,i))]);
theta = Gettheta(P(:,i),psi); % Why is this here? Are you writing it to file too?
end
xlswrite('My_FileName.xls',P);
Or you could call xlswrite on each iteration of the loop (probably slower) and append the new data using something like this:
for i=1:nn
...
P = fminsearch(#ObFun,Pini,[],psi,thetaObs);
disp(['result',num2str(i),': P=',num2str(P)]);
theta = Gettheta(P,psi); % Why is this here? Are you writing it to file too?
xlswrite('My_FileName.xls',P,1,['A' int2str((i-1)*size(P,2)+1)]);
end
Of course your code isn't runnable so you'll have to debug any other little errors. Also, since smcdata seems to be a matrix rather than a vector, you should be careful using length with it. You probably should use size.

Using load with data from cells

In my code I'm trying to use load with entries from a cell, but it is not working. The portion of my code below produces a 3 dimensional array of strings. The strings represent the paths to file names.
for i = 1:Something
for j = 1:Something Different
for k = 1: Yet Something Something Different
DataPath{j,k,i} = 'F:\blah\blah\blah\fileijk %file changes based on i,j,and k
end
end
end
In the next part of the code I want to use load to open the files using the path names defined in the code above. I do this using the code below.
Dummy = DataPath{l,(k-1)*TSRRange+m};
Data = load(Dummy);
The idea is for Dummy to take the string content out of DataPath so I can use it in load. By doing this I thought that Dummy would be defined as a string and not a cell, but this is not the case. How do I pull the string out of DataPath so I can use it with load? Thanks.
I have to load the data this way because the data is located in multiple folders. I can post more of the code if needed, but it is complex.
Dummy is a cell because you assigned a 3D cell array but are accessing a 2D cell with Dummy = Datapath{1,(k-1)*TSRRange+m}
I don't believe that you can expect to access all cell elements I this way. Instead, use three indices just as you did when creating it.

Resources