How to do sequence classification using LSTM? - keras

Question

I'm working on a human activity recognition problem using depth stream.
Each sample is a Matlab file of size (20,3,no_of_frames), i.e., there are 20 rows and 3 cols in every frame and the number of frames can vary for different samples.
I have padded all the samples with 0's so that all of them contains same no of frames (say 100).
So now all samples are of size (20,3,100).
Also let the total no of samples are 400 and no of classes be 10.
How can I arrange my dataset so as to use LSTM in Keras. Would you also suggest some basic LSTM model for the classification purpose?

Answer 1

Here is my code to create dataset:
clc;
clear all;
ctr=0;
longest=326; % length of longest sequence
gyroData = zeros(431,longest*3);
gyroLabel=zeros(431,1);
for a=1:27
for s=1:2:7
for t=1:4
fname = strcat('a', int2str(a), '_s', int2str(s), '_t', int2str(t),'_inertial.mat');
if exist(fname)
load(fname);
d_iner = d_iner(:,1:3);
r = size(d_iner,1);
d_iner = cat(1,d_iner,zeros(longest-r,3)); % do zero padding to longest sequence
ctr = ctr+1;
d_iner = d_iner';
gyroData(ctr,:,:) = d_iner(:);
gyroLabel(ctr,1)=a-1;
end
end
end
end
n1 = randperm(ctr);
for i=1:ctr
if i==1
x1=gyroData(n1(i),:);
y1=gyroLabel(n1(i),1);
else
x1=cat(1,x1,gyroData(n1(i),:));
y1=cat(1,y1,gyroLabel(n1(i),1));
end
end
%%
ctr=0;
gyroData = zeros(430,longest*3);
gyroLabel=zeros(430,1);
for a=1:27
for s=2:2:8
for t=1:4
fname = strcat('a', int2str(a), '_s', int2str(s), '_t', int2str(t),'_inertial.mat');
if exist(fname)
load(fname);
d_iner = d_iner(:,1:3);
r = size(d_iner,1);
d_iner = cat(1,d_iner,zeros(longest-r,3)); % do zero padding to longest sequence
ctr = ctr+1;
d_iner = d_iner';
gyroData(ctr,:,:) = d_iner(:);
gyroLabel(ctr,1)=a-1;
end
end
end
end
n1 = randperm(ctr);
for i=1:ctr
if i==1
x2=gyroData(n1(i),:);
y2=gyroLabel(n1(i),1);
else
x2=cat(1,x2,gyroData(n1(i),:));
y2=cat(1,y2,gyroLabel(n1(i),1));
end
end
save('inertial_padded.mat', 'x1', 'y1', 'x2', 'y2');
**And this is my LSTM code in Keras**
import numpy as np
import keras
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import LSTM
import scipy.io
# fix random seed for reproducibility
np.random.seed(7)
mat = scipy.io.loadmat('inertial_padded.mat')
x_train = mat['x1']
y_train = mat['y1']
x_test = mat['x2']
y_test = mat['y2']
data_dim = 3
timesteps = 326
num_classes = 27
x_train = x_train.reshape(x_train.shape[0], 326,3)
x_test = x_test.reshape(x_test.shape[0], 326, 3)
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)
print('x_train shape:', x_train.shape)
print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')
model = Sequential()
model.add(LSTM(32, input_shape=(timesteps, data_dim), activation='sigmoid'))
#model.add(LSTM(32, activation='tanh')) # returns a sequence of vectors of dimension 32
#model.add(LSTM(32)) # return a single vector of dimension 32
model.add(Dense(27, activation='softmax'))
model.compile(loss='categorical_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
print(model.summary())
model.fit(x_train, y_train,
batch_size=16, epochs=25,
validation_data=(x_test, y_test))
# Final evaluation of the model
scores = model.evaluate(X_test, y_test, verbose=0)
print("Accuracy: %.2f%%" % (scores[1]*100))