After fitting GARCH model in R and obtain the output, how do I know whether there is any evidence of ARCH effect?
I am not toosure whether I have to check in optimal parameters, Information criteria, Q-statistics on standardized residuals, ARCM LM Tests, Nyblom stability test, Sign Bias Test or Adjusted Pearson Goodness-of-fit test?
I assume I have to check under ARCH LM Tests, and if the p-value is rather high, there is an ARCH effect, am I right?
Thank you
You need to start by looking for second order persistence in the return series itself before going on to fit a GARCH model. Lets work through a quick example of how this will work
Start by getting the return series. Here I will use the quantmod library to load in the data for SPDR S&P 500 ETF or SPY
library(quantmod)
library(PerformanceAnalytics)
rtn<-getSymbols(c('SPY'),return.class='ts')
Next, calculate the return series either yourself or using the Return.calculate function as provided by the PerformanceAnalytics library
Rtn <- diff(log(SPY[,"SPY.Close"])) * 100
#OR
Rtn <- Return.calculate(SPY[,"SPY.Close"], method = c("compound","simple")[2]) * 100
Now, lets have a look at the persistence of the first and second order moments of the series. For second order moments, lets use the squared return series as a proxy.
Plotdata<-cbind(Rtn, Rtn^2)
plot.zoo(Plotdata)
There remains strong first persistence in returns and there is clearly periods of strong second order persistence as seen in the squared returns.
We can now formally start testing for ARCH-effects. A formal test for ARCH effects is LBQ stats on squared returns:
Box.test(coredata(Rtn^2), type = "Ljung-Box", lag = 12)
Box-Ljung test
data: coredata(Rtn^2)
X-squared = 2001.2, df = 12, p-value < 2.2e-16
We can clearly reject the null hypothesis of independence in a given time series. (ARCH-effects)
Fin.Ts also provides the ARCH-LM test for conditional heteroskedasticity in the returns:
library(FinTS)
ArchTest(Rtn)
ARCH LM-test; Null hypothesis: no ARCH effects
data: Rtn
Chi-squared = 722.19, df = 12, p-value < 2.2e-16
This supports the conclusion of the LBQ test that ARCH-effects are present.
Related
I am interested in which processes/activities contribute most to the Life Cycle Impact Assessment (LCIA) that I am conducting. For this, I run a contribution analysis (see code below). To crosscheck the results of my contribution analysis and to ensure that I get everything right, I wanted to compare the returned contributions with the impact assessment result (lca.score).
The documentation of ca.annotated_top_processes(lca) says: "Returns a list of tuples: (lca score, supply, activity)."
In my understanding, lca.score should be the same value as the sum of all the first values in the tuples that are returned by ca.annotated_top_processes(lca) (the printed values). However, this is not the case. What am I missing? Is there some sort of cut-off applied or did I misunderstand something?
import bw2analyzer as bwa
random_act = db_ei381.random()
lca = bw2data.LCA(
{random_act: 1},
('ReCiPe Midpoint (H) V1.13', 'water depletion', 'WDP')
)
lca.lci()
lca.lcia()
print(lca.score)
# %% Contribution analysis
ca = bwa.ContributionAnalysis()
contributions = ca.annotated_top_processes(lca)
print(sum([i[0] for i in contributions]))
It is not well documented, but you can introduce an argument limit that specifies the number of activities that are considered in the contribution analysis. The default value I think is 25. It is sorted so the most important activities come first. If you write something like this you should see how the result converges to the total score as the number of activities increase:
import matplotlib.pyplot as plt
cutoff = [25,50,100,500,1000,1200]
scores = []
for n in cutoff:
contributions = ca.annotated_top_processes(lca,limit=n)
contr_sum = sum([i[0] for i in contributions])
scores.append(contr_sum)
plt.plot(cutoff,scores)
plt.axhline(lca.score,ls='--',color='r');
I have a tensorflow model like this
def input_spec():
return(
tf.TensorSpec([None, 122], tf.float64),
tf.TensorSpec([None, 5],tf.uint8))
def model_fn():
model=tf.keras.models.Sequential([
tf.keras.layers.Dense(64,input_shape=(122,)),
tf.keras.layers.Dense(32,activation='relu'),
tf.keras.layers.Dropout(.15),
tf.keras.layers.Dense(32,activation='relu'),
tf.keras.layers.Dropout(.15),
tf.keras.layers.Dense(32,activation='relu'),
tf.keras.layers.Dropout(.15),
tf.keras.layers.Dense(5,activation='softmax')])
return tff.learning.from_keras_model(
model,
input_spec=input_spec(),
loss=tf.keras.losses.CategoricalCrossentropy(),
metrics=[tf.keras.metrics.CategoricalAccuracy()])
I set the iterative_process in the following
iterative_process=tff.learning.algorithms.build_weighted_fed_avg(
model_fn,
client_optimizer_fn=lambda: tf.keras.optimizers.Adam(),
server_optimizer_fn=lambda: tf.keras.optimizers.Adam())
I have learnt that we can obtain the aggregated weight by model_weights=iterative_process.get_model_weights(state), but I still need to know how to obtain the aggregated gradients.
While running the training procedure, the aggregated (pseudo) gradients can in some cases be computed by subtracting the state at the beginning of the round from that at the end. In the code snippet above, this will not quite literally be true since the server optimizer is Adam (which performs some rescaling of the pseudo-gradients, as well as the addition of a momentum accumulator, if I recall correctly).
If you are simply using gradient-descent with a learning rate of 1 on the server (traditionally the default setting for FedAvg), code like the following should give you this aggregated pseudogradient:
pseudo_grad = tf.nest.map_structure(
lambda x, y: x - y, previous_state.global_model_weights.trainable,
state.global_model_weights.trainable)
Some helpful measurements for debugging can alternatively be accessed by wrapping the aggregator parameter to your build_weighted_fed_avg call in an aggregator that adds these debug measurements, if this is the underlying goal here. You would additionally be able to read these values directly if you implemented a tff.templates.AggregationProcess which output the averaged pseudogradient in the measurements field of its result; these should be passed through directly by the rest of the FedAvg implementation.
I am running a Montecarlo simulation on ecoinvent v 3.8 consequential system model and when randomly sampling the activity market for waste paper, sorted' (kilogram, GLO, None) I get very unrealistic and overestimated results.
myact = bw.Database('ecoinvent 3.8_conseq').get('aae12a8b0ba521d60af5341c75cc9d3c') # waste paper sorted
mymethod = ('IPCC 2013', 'climate change', 'GWP 100a')
lca = bw.LCA({myact : 1}, mymethod)
lca.lci()
lca.lcia()
lca.score
Returns a value of -2.768 kg CO2-eq while
mc = bw.MonteCarloLCA({myact: 1}, mymethod)
mc_results = [next(mc) for x in range(20)]
Returns values with a median over 100 kg CO2-eq. Which not only seems absurd but also skews all results in when sampling any foreground or background activity downstream, i.e. having this activity as input (e.g. cellulose fibre production '48506ab8ea444c5e826cc079ff0d4c11')
I have tried removing all uncertainties for the exchanges in the activity but the result did not change.
for exc in list(myact.exchanges()):
exc['uncertainty type'] = 0
exc['loc'], exc['scale'] = np.log(1), np.log(1)
exc.save()
My question is: how can I figure out if this is an ecoinvent problem or a brightway problem and how to fix it?
An excellent question, not easy to answer, but you can find my workings here:
https://github.com/brightway-lca/brightway2/blob/master/notebooks/Investigating%20interesting%20Monte%20Carlo%20results.ipynb
As of bw2analzyer 0.11.4, the modified recurisive function is included in the library.
As it is long and now included in the Brightway docs, I don't think it makes sense to adopt and add to the SO format.
Here is one approach to reduce these large uncertainty intervals.
I’am trying to use the tune.svm-function and since I don’t really know which parameters will produce a good model (as training data will be picked by a user) I need to cover a wide range of values. Currently I've got this behavior
tune(svm, value ~ . , data= data_l, ranges=list(cost = 10^(0:5), epsilon = 10^(-1:0)))
Parameter tuning of ‘svm’:
- sampling method: 10-fold cross validation
- best parameters:
cost epsilon
100 0.1
- best performance: 277.5491
and
tune(svm, value ~ . , data= data_l, ranges=list(cost = 10^(0:5), epsilon = 10^(-1:1)))
Error in predict.svm(ret, xhold, decision.values = TRUE) : Model is empty!
(Difference in the highest value of epsilon)
I know that svm won’t work with epsilon =10 but my intuition for this tuning function would be, that it can handle parameters that will not produce a model.
Why wouldn't it pick the models that could be generated?
Is there an “easy” way to omit this error-behavior? (I tried tryCatch(tune()) and a lot of other stuff I found, but I guess I would have to dig deep into the tune/svm/predict-codes which doesn’t sound “easy” anymore)
My understanding of the "Model is empty!" error is that it indicates a singular training matrix was fed into the SVM. See Salvy's answer to this related post and Oldrich Kruza's related message in the R mailing list.
I'm trying to predict stock prices using sklearn. I'm new to prediction. I tried the example from sklearn for stock prediction with gaussian hmm. But predict gives states sequence which overlay on the price and it also takes points from given input close price. My question is how to generate next 10 prices?
You will always use the last state to predict the next state, so let's add 10 days worth of inputs by changing the end date to the 23rd:
date2 = datetime.date(2012, 1, 23)
You can double check the rest of the code to make sure I am not actually using future data for the prediction. The rest of these lines can be added to the bottom of the file. First we want to find out what the expected return is for a given state. The model.means_ array has returns, both those were the returns that got us to this state, not the future returns which is what you want. To get the future returns, we consider the probability of going to any one of the 5 states, and what the return of those states is. We get the probability of going to any particular state from the model.transmat_ matrix, the for the return of each state we use the model.means_ values. We take the dot product to get the expected return for a particular state. Then we remove the volume data (you can leave it in if you want, but you seemed to be most interested in future prices).
expected_returns_and_volumes = np.dot(model.transmat_, model.means_)
returns_and_volumes_columnwise = zip(*expected_returns_and_volumes)
returns = returns_and_volumes_columnwise[0]
If you print the value for returns[0], you'll see the expected return in dollars for state 0, returns[1] for state 1 etc. Now, given a day and a state, we want to predict the price for tomorrow. You said 10 days so let's use that for lastN.
predicted_prices = []
lastN = 10
for idx in xrange(lastN):
state = hidden_states[-lastN+idx]
current_price = quotes[-lastN+idx][2]
current_date = datetime.date.fromordinal(dates[-lastN+idx])
predicted_date = current_date + datetime.timedelta(days=1)
predicted_prices.append((predicted_date, current_price + returns[state]))
print(predicted_prices)
If you were running this in "production" you would set date2 to the last date you have and then lastN would be 1. Note that I don't take into account weekends for the predicted_date.
This is a fun exercise but you probably wouldn't run this in production, hence the quotes. First, the time series is the raw price; this should really be percentage returns or log returns. Plus there is no justification for picking 5 states for the HMM, or that a HMM is even good for this kinda problem, which I doubt. They probably just picked it as an example. I think the other sklearn example using PCA is much more interesting.