Is it possible to provide multivariate input to the linear layer)
I am working on a stock forecasting model and want to provide open, close and volume data as input for the past 10 days. And want to predict the closing price of the next day?
if it is possible can someone share an example, it will be really helpful.
Thanks
Related
I have the following LSTM network(Fig 1) for predicting the Bitcoin Price. The input is every hour close price of Bitcoin. I am facing some issues and any advice is appreciated.
Earlier on the same network, my RMSE on testing and training set was 6.71 and 7.41 RMSE. I recompiled the whole code and there was an abrupt increase to 233.51 for the training set and 345.56 for the testing set. Can anyone help me with finding out the reason behind this?
Also, How to improve the accuracy of my network as it very low in every iteration?
How should I decide the parameters for my LSTM network. (units, epochs, batch_size, time_steps to input)
Thank you in advance for any help extended.
Your question requires a lot more information. For example, data size, timesteps lookback, data preprocessing procedure, etc. But I would recommend you to debug your problem with the following method. First, check whether your input/output data are processed properly or not. Then, try to train the simpler model apart from LSTM as it could result in overfitting. But sometimes, if the input signal is too random, it is normal that your model results would highly fluctuate as there's no correlation in data.
PS. never use the Machine Learning model to predict stock price. It never works.
I am currently working on my grade project. The main goal of this project is to build a AI aided inventory system. My current approach is to use a neural network to "teach" the system how to behave according to its current state (the current inventory) and a pseudorandom demand. The inputs of the net should be the current inventory and the network should output how much the company should order of a given product to minimize both sales losses due to lacking inventory and inventory level (the model should not output crazy amounts because keeping inventory costs money). The outputs from one run should be used to compute the inputs of the next run based on current inventory, current demand and orders(the outputs themselves). The problem I am facing is that I only need an initial state (inventory at time 0) to train the model, but I do not know any ways to set up Keras like this. Maybe someone could point me to a useful resource to take on this problem. To me it looks like I need to define a custom loss function and maybe create a custom layer to the system (I know to to define the function and the inventory update rule, buy I don't know how to tell Keras what to do with them).
I don't think that you need to create custom layers or custom losses.
IMO the steps you shoul follow are:
Create a dataset where this time correlation between inventory at time t and t - 1 inventory is present.
Decalare a lookback hyperparameter (How many inventories you want to analyse in a single sequence?)
Then you can feed this data to an RNN in order to solve the regression problem (For example a loss for regression is mse).
Check this explainatory notebook written by François Chollet on how to work with RNNs.
I have a machine learning model built that tries to predict weather data, and in this case I am doing a prediction on whether or not it will rain tomorrow (a binary prediction of Yes/No).
In the dataset there is about 50 input variables, and I have 65,000 entries in the dataset.
I am currently running a RNN with a single hidden layer, with 35 nodes in the hidden layer. I am using PyTorch's NLLLoss as my loss function, and Adaboost for the optimization function. I've tried many different learning rates, and 0.01 seems to be working fairly well.
After running for 150 epochs, I notice that I start to converge around .80 accuracy for my test data. However, I would wish for this to be even higher. However, it seems like the model is stuck oscillating around some sort of saddle or local minimum. (A graph of this is below)
What are the most effective ways to get out of this "valley" that the model seems to be stuck in?
Not sure why exactly you are using only one hidden layer and what is the shape of your history data but here are the things you can try:
Try more than one hidden layer
Experiment with LSTM and GRU layer and combination of these layers together with RNN.
Shape of your data i.e. the history you look at to predict the weather.
Make sure your features are scaled properly since you have about 50 input variables.
Your question is little ambiguous as you mentioned RNN with a single hidden layer. Also without knowing the entire neural network architecture, it is tough to say how can you bring in improvements. So, I would like to add a few points.
You mentioned that you are using "Adaboost" as the optimization function but PyTorch doesn't have any such optimizer. Did you try using SGD or Adam optimizers which are very useful?
Do you have any regularization term in the loss function? Are you familiar with dropout? Did you check the training performance? Does your model overfit?
Do you have a baseline model/algorithm so that you can compare whether 80% accuracy is good or not?
150 epochs just for a binary classification task looks too much. Why don't you start from an off-the-shelf classifier model? You can find several examples of regression, classification in this tutorial.
I have 5 time series that I want a neural network to predict. The time series are related to each other. Each time series consists of numbers between 0 and 100. I want to predict the next number for each time series. I already have a model to train one time series using a GRU and that works reasonably well. I have tried two strategies:
I normalized the numbers and made the problem a regression problem. The best validation accuracy so far is 0.38.
I one-hot-encoded the time series, and this works significantly better (an added accuracy of 0.15) but it costs 100 times as much memory.
For 5 time series, I tried 5 independent models but in that case the relationship between the 5 time series was lost. I am wondering what an efficient strategy to proceed might be. I can think of two myself but I might be missing something:
I can stack the input so that I have a five-hot-encoded input instead of 5 one-hot-encoded. Can this be done?
I can create 5 models and merge them. I am not sure what to do with the output. Should I split the model, one for each time series?
Is there a strategy I have overlooked? Memory is a problem. With thousands of time series, with sample lengths of 100, the data uses a lot of memory and processing time. I Googled around but could not find an efficient strategy. Could someone suggest how to implement this problem efficiently in Keras?
I am trying to develop an Artificial Neural Network using PyBrain to model biological data. My ANN compiles and runs, but its accuracy value is very low, never surpassing ~62%. From a coding perspective, how can I improve the ANN's accuracy? Something I noticed was that each time, the outputs of the ANN are not the same, either, even though the test data set doesn't change--is there a reason the ANN is acting to unstably, and how can I improve this?
Thank you! :)
If you creating new network each time you run your script then it is normal that outputs are different.
Each time you create ANN pybrain initialize weights of connections with random values (range 0 to 1).
You can save your ANN with NetworkWriter and read it with NetworkReader in pybrain.tools.customxml (see code documentation for reference, pybrain API is missing few things).
You can adjust training process with learning rate and momentum. Also you could apply more training epoch to your network.
If you provide your code I could say more.