Matlab Data Preprocessing and Dynamic Struct Assignments - excel

I'm quite new to Matlab and I'm struggling trying to figure out how to properly preprocess my data in order to make some calculations with it.
I have an Excel table with financial log returns of many companies such that every row is a day and every column is a company:
I imported everything correctly into Matlab like this:
Now I have to create what's caled "rolling windows". To do this I use the following code:
function [ROLLING_WINDOWS] = setup_returns(RETURNS)
bandwidth = 262;
[rows, columns] = size(RETURNS);
limit_rows = rows - bandwidth;
for i = 1:limit_rows
ROLLING_WINDOWS(i).SYS = RETURNS(i:bandwidth+i-1,1);
end
end
Well if I run this code for the first column of returns everything works fine... but my aim is to produce the same thing for every column of log returns. So basically I have to add a second for loop... but what I don't get is which syntax I need to use in order to make that ".SYS" dynamic and based on my array of string cells containing company names so that...
ROLLING_WINDOWS(i)."S&P 500" = RETURNS(i:bandwidth+i-1,1);
ROLLING_WINDOWS(i)."AIG" = RETURNS(i:bandwidth+i-1,2);
and so on...
Thanks for your help guys!
EDIT: working function
function [ROLLING_WINDOWS] = setup_returns(COMPANIES, RETURNS)
bandwidth = 262;
[rows, columns] = size(RETURNS);
limit_rows = rows - bandwidth;
for i = 1:limit_rows
offset = bandwidth + i - 1;
for j = 1:columns
ROLLING_WINDOWS(i).(COMPANIES{j}) = RETURNS(i:offset, j);
end
end
end
Ok everything is perfect... just one question... matlab intellissense tells me "ROLLING_WINDOWS appears to change size on every loop iteration bla bla bla consider preallocating"... how can I perform this?

You're almost there. Use dynamic field names by building strings for fields. Your fields are in a cell array called COMPANIES and so:
function [ROLLING_WINDOWS] = setup_returns(COMPANIES, RETURNS)
bandwidth = 262;
[rows, columns] = size(RETURNS);
limit_rows = rows - bandwidth;
%// Preallocate to remove warnings
ROLLING_WINDOWS = repmat(struct(), limit_rows, 1);
for i = 1:limit_rows
offset = bandwidth + i - 1;
for j = 1:columns
%// Dynamic field name referencing
ROLLING_WINDOWS(i).(COMPANIES{j}) = RETURNS(i:offset, j);
end
end
end
Here's a great article by Loren Shure from MathWorks if you want to learn more: http://blogs.mathworks.com/loren/2005/12/13/use-dynamic-field-references/ ... but basically, if you have a string and you want to use this string to create a field, you would do:
str = '...';
s.(str) = ...;
s is your structure and str is the string you want to name your field.

Related

How to use Autofill/Filldown with a range of values

I have been trying to get Excel to apply a formula over a set of columns and then extend the pattern across the entire set of rows.
This has led to the following code:
For i = 0 To avgsheetNames.Count - 1
If Contains(CStr(avgsheetNames(i)), "Scores") = True Then
With mainWorkBook.Worksheets(avgsheetNames(i))
strFormulas(1) = "=SUM(Aggregated_Internal_Scores!I2:I7)/6"
strFormulas(2) = "=SUM(Aggregated_Internal_Scores!J2:J7)/6"
strFormulas(3) = "=SUM(Aggregated_Internal_Scores!K2:K7)/6"
strFormulas(4) = "=SUM(Aggregated_Internal_Scores!L2:L7)/6"
strFormulas(5) = "=SUM(Aggregated_Internal_Scores!M2:M7)/6"
strFormulas(6) = "=SUM(Aggregated_Internal_Scores!N2:N7)/6"
strFormulas2(1) = "=SUM(Aggregated_Internal_Scores!I8:I13)/6"
strFormulas2(2) = "=SUM(Aggregated_Internal_Scores!J8:J13)/6"
strFormulas2(3) = "=SUM(Aggregated_Internal_Scores!K8:K13)/6"
strFormulas2(4) = "=SUM(Aggregated_Internal_Scores!L8:L13)/6"
strFormulas2(5) = "=SUM(Aggregated_Internal_Scores!M8:M13)/6"
strFormulas2(6) = "=SUM(Aggregated_Internal_Scores!N8:N13)/6"
mainWorkBook.Worksheets(avgsheetNames(i)).Range("C2:H2").Formula = strFormulas
mainWorkBook.Worksheets(avgsheetNames(i)).Range("C3:H3").Formula = strFormulas2
mainWorkBook.Worksheets(avgsheetNames(i)).Range("C2:H3").AutoFill Destination:=mainWorkBook.Worksheets(avgsheetNames(i)).Range("C2:H32")
End With
End If
As you can see I have tried to provide the pattern I am going for where the values extracted from the "Aggregated_Internal_Scores" sheet should follow the pattern I2:I7 > I8:I13 > I14:I19 and so on.
However, when the macro has been executed what I get is I2:I7 > I8:I13 > I4:I9 > I10:I15?
It seems Excel is taking the block C2:H3 as the pattern and just incrementing by 2 at the start of every block.
Can you anyone explain where I have gone wrong and how I can specify that I want the extraction of sheet values to follow a certain pattern?
Thank you in advance!
Use:
mainWorkBook.Worksheets(avgsheetNames(i)).Range("C2:H32").Formula = "=SUM(INDEX(Aggregated_Internal_Scores!I:I,(ROW($ZZ1)-1)*6+2):INDEX(Aggregated_Internal_Scores!I:I,(ROW($ZZ1)-1)*6+7))/6"
Replace everything inside the If with that.
If one has Office 365 with dynamic array formula then use:
mainWorkBook.Worksheets(avgsheetNames(i)).Range("C2:H32").Formula2 = "=SUM(INDEX(Aggregated_Internal_Scores!I:I,SEQUENCE(6,,(ROW($ZZ1)-1)*6+2))/6"

Using MATLAB to write String values into Excel spreadsheet via ActiveX protocol.. found a problem

I'm using MATLAB 2017a and have been using xlswrite in the past to perform this operation. The problem I ran into was with execution speed and I was looking for a better way. So, I decided to use actxserver and write data using get(obj) from MATLAB and Range.Value from ActiveX. Here's what the code looks like:
e = actxserver('Excel.Application);
eWorkbook = e.Workbooks.Add;
e.Visible = 1;
eSheets = e.ActiveWorkbook.Sheets;
eSheet1 = eSheets.get('Item',1);
eSheet1.Activate;
A = ["Str1";"Str2";"Str3";];
eActivesheetRange = get(e.Activesheet, 'Range', 'A1:A3');
eActivesheetRange.Value = A;
This inocuous bit of code does not execute, nor does it throw a warning or error message.. Nothin'. In my mind, the eActivesheetRange evaluates to: Range("A1:A3") on the ActiveX side. Interestingly, if I replace
A = ["Str1";"Str2";"Str3";];
with
A = char(["Str1";"Str2";"Str3";]);
then the program writes the A char array to each cell in the eActivesheetRange Range.
Is there a way to call cells() using the MATLAB Range.Value connection? Would cells().Value be able to solve this problem?
I don't think writing to Excel using ActiveX is able to handle string types properly. In this case, you can make it work by simply converting your string array into a cell array of character vectors using cellstr. Changing your last line of code to the following works for me (in R2016b):
eActivesheetRange.Value = cellstr(A);
Replacing the last two lines with the following also works:
e.Activesheet.Range('A1:A3').Value = cellstr(A);
The solution to this is of course, a for loop.
alphacolumn=char(97:117);
% iterate through data array
for i=1:21
str=string(alphacolumn(i))+2;
str2=string(alphacolumn(i))+202;
write1=char(str+":"+str2);
if ~isreal(tsc{i,1})
T = (tsc{i,1});
for j = 1:length(T)
rrange = xl.ActiveWorkbook.Activesheet.Range(char(string(alphacolumn(i)) + string(j+1)));
xlcompatiblestring1 = char(string(T(j,:,:)));
rrange.Value= xlcompatiblestring1;
end
else
tsci=tsc{i,1};
% write data to xl target file
%xlswrite(xlfilepath,tsci,write1);
xlActivesheetRange = get(xl.Activesheet,'Range',write1);
xlActivesheetRange.Value = tsci;
end
end

Matlab Justify Strings

I'm using Matlab R2014b (that's why I cannot use strings, but only char vectors). Working inside a class, I have to take data from a table variable, format it following my needs, and then insert it into a GUI table (an instance of uitable, to be exact):
function UpdateTable(this)
siz = size(mydata);
tab = cell(siz);
tab(:,1) = num2cell(this.Data.ID);
tab(:,2) = cellstr(datestr(this.Data.Date,'dd/mm/yyyy'));
tab(:,3) = arrayfun(#(x){MyClass.TypeDef1{x,1}},this.Data.Type1);
tab(:,4) = arrayfun(#(x){MyClass.TypeDef2{x,1}},this.Data.Type2);
tab(:,5) = arrayfun(#(x){MyClass.FormatNumber(x)},this.Data.Value);
this.UITable.Data = tab;
end
Where:
properties (Access = private, Constant)
TypeDef1 = {
'A1' 'Name A1';
'B1' 'Name B1';
'C1' 'Name C1';
'D1' 'Name D1';
...
}
TypeDef2 = {
'A2' 'Name A2';
'B2' 'Name B2';
'C2' 'Name C2';
'D2' 'Name D2';
...
}
end
methods (Access = private, Static)
function str = FormatNumber(num)
persistent df;
if (isempty(df))
dfs = java.text.DecimalFormatSymbols();
dfs.setDecimalSeparator(',');
dfs.setGroupingSeparator('.');
df = java.text.DecimalFormat();
df.setDecimalFormatSymbols(dfs);
df.setMaximumFractionDigits(2);
df.setMinimumFractionDigits(2);
end
str = char(df.format(num));
end
end
Everything is working fine. Now I would like to right justify the strings to be inserted in columns 1 and 5, to improve the table readability. I found the Matlab function that suits my needs, strjust. Reading the documentation, I saw that it can be used with cell arrays of char vectors, so I modified part of my UpdateTable code as follows:
tab(:,1) = cellstr(num2str(this.Data.ID));
tab(:,5) = strjust(arrayfun(#(x){MyClass.FormatNumber(x)},this.Data.Value));
TThe second one produces no changes (strings are still not justified). Should the strings already contain enough whitespace to be all the same length?
Ok, I solved the problem by myself using the following code:
function UpdateTable(this)
siz = size(this.Data);
los = arrayfun(#(x){MyClass.FormatNumber(x)},this.Data.Value);
los_lens = cellfun(#(x)numel(x),los);
pad = cellfun(#blanks,num2cell(max(los_lens) - los_lens),'UniformOutput',false);
tab = cell(siz);
tab(:,1) = cellstr(num2str(this.Data.ID));
tab(:,2) = cellstr(datestr(this.Data.Date,'dd/mm/yyyy'));
tab(:,3) = arrayfun(#(x){MyClass.TypeDef1{x,1}},this.Data.Type1);
tab(:,4) = arrayfun(#(x){MyClass.TypeDef2{x,1}},this.Data.Type2);
tab(:,5) = cellstr(strcat(pad,los));
this.UITable.Data = tab;
end
It's probably not the most elegant solution, but it works. Starting from Matlab 2016, the padding can be performed using the built-in pad function.

How to loop through excel sheets, perform calculations, and compile results

I have roughly 70,000 sheets that all have to have calculations done, and then all results compiled into a new sheet (which would be 70,000 lines long).
It needs to be sorted by date.
I'm VERY very very poor at matlab, but I've what I need the script to do for each excel sheet, I'm just unsure how to make it do them for all.
Thank you!!! (I took out some of the not important code)
%Reading in excel sheet
B = xlsread('24259893-008020361800.TorqueData.20160104.034602AM.csv');
%Creating new matrix
[inYdim, inXdim] = size(B);
Ydim = inYdim;
[num,str,raw]=xlsread('24259893-008020361800.TorqueData.20160104.034602AM.csv',strcat('A1:C',num2str(Ydim)));
%Extracting column C
C=raw(:,3);
for k = 1:numel(C)
if isnan(C{k})
C{k} = '';
end
end
%Calculations
TargetT=2000;
AvgT=mean(t12);
TAcc=((AvgT-TargetT)/TargetT)*100 ;
StdDev=std(B(ind1:ind2,2));
ResTime=t4-t3;
FallTime=t6-t5;
DragT=mean(t78);
BreakInT=mean(t910);
BreakInTime=(t10-t9)/1000;
BreakInE=BreakInT*BreakInTime*200*.1047;
%Combining results
Results=[AvgT TAcc StdDev ResTime FallTime DragT BreakInT BreakInTime BreakInE]
I think I need to do something along the lines of:
filenames=dir('*.csv')
and I found this that may be useful:
filenames=dir('*.csv');
for file=filenames'
csv=load(file.name);
with stuff in here
end
You have the right idea, but you need to index your file names in order to be able to step through them in the for loop.
FileDir = 'Your Directory';
FileNames = {'Test1';'Test2';'Test3'};
for k=1:length(FileNames)
file=[FileDir,'/',FileNames{k}]);
[outputdata]=xlsread(file,sheet#, data locations);
THE REST OF YOUR LOOP, Indexed by k
end
How you choose to get the file names and directory is up to you.

How to compare matlab array with entries in a data structure

I am trying to write code in Matlab that will allow me to do the following. There is a part of the code that generates an array D and uses an input file to create this structure called EEG which contains a lot of information. Specifically I am interested in a "labels" field of the chanlocs field of the EEG structure. It contains entries like 'F7', 'F8', 'FP1'... and 17 such entries. The array D that is generated also contains entries like this but in a different order.
So for e.g. D = ['F7','F8', 'FP1'] and EEG.chanlocs.labels = ['FP1','F7','F8']
they contain the same entries but they are in a different order and for what I am trying to do the order is important.
What I basically want to do is to have Matlab scan all entries of D and find that particular index of EEG.chanlocs.labels to which that entry corresponds.
Example: If D(1) = 'F7' I want it to return for e.g. i = 2 because F7 is the 2nd entry in EEG.chanlocs.labels. In this way I want it to scan all of D and return the indices in EEG.chanlocs.labels.
What I have tried so far is:
for i=1:17
if any(strcmp(D(:),[EEG.chanlocs(i).labels]))
msgbox(sprintf('i is: %d',i));
else
msgbox(sprintf('Error'));
end
end
But it does not work and it returns weird things... I am not entirely sure what to try...
Can anybody help? Any help would be greatly appreciated!!
Thanks.
Edited:
The following code shows how I obtain D. I give the user 3 prompt windows to input certain data. I then store the inputs from each of these in "data" or "data2" or "data3" and then I put all of them together in D.
uiwait(msgbox(sprintf('Please enter your new references for each electrode.\nFor FP1, FP2, O1 and O2 provide two references.')));
prompt = {'Fp1','F7','T3','T5','O1'};
prompt2 = {'FP2','F8','T4','T6','O2'};
prompt3 = {'C3','CP3','Cz','CPz','C4','CP4'};
dlg_title = 'Input references';
num_lines = 1;
%def = {'20','hsv'};
answer = inputdlg(prompt,dlg_title,num_lines );
answer2 = inputdlg(prompt2,dlg_title,num_lines );
answer3 = inputdlg(prompt3,dlg_title,num_lines );
for i=1:5
data(i,:) = answer(i,:);
data2(i,:) = answer2(i,:);
end
for i=1:6
data3(i,:) = answer3(i,:);
end
D(1:5)=data(:);
D(6:10)=data2(:);
D(11:16)=data3(:);
D=D';

Resources