Overflow error subclassing a distribution using scipy.stats.rv_continuous - python-3.x

In the documentation page of rv_continuous we can find a 'custom' gaussian being subclassed as follows.
from scipy.stats import rv_continuous
import numpy as np
class gaussian_gen(rv_continuous):
"Gaussian distribution"
def _pdf(self, x):
return np.exp(-x**2 / 2.) / np.sqrt(2.0 * np.pi)
gaussian = gaussian_gen(name='gaussian')
In turn, I attempted to create a class for an exponential distribution with base 2, to model some nuclear decay:
class time_dist(rv_continuous):
def _pdf(self, x):
return 2**(-x)
random_var = time_dist(name = 'decay')
This had the purpose of then calling random_var.rvs() in order to generate a randomly distributed sample of values according to the pdf I defined. However, when I run this, I receive an OverflowError, and I don't quite understand why. Initially I thought it had to do with the fact that the function was not normalized. However, I keep making changes to the _pdf definition to no avail. Is there anything wrong with the code or is this method just ill-advised for defining functions of this sort?

According to wikipedia, the pdf of an exponential distribution would be:
lambda * exp(-lambda*x) for x >= 0
0 for x < 0
So, probably the function should be changed as follows:
from scipy.stats import rv_continuous
import numpy as np
import matplotlib.pyplot as plt
class time_dist(rv_continuous):
def _pdf(self, x):
return np.log(2) * 2 ** (-x) if x >= 0 else 0
random_var = time_dist(name='decay')
plt.hist(random_var.rvs(size=500))
plt.show()

Related

Using scipy's solve_ivp with event

I reproduced the issue I have with my more complex code with the following example:
import numpy as np
from scipy.integrate import solve_ivp
def funct(t, y):
return np.sqrt(100-y)
def event(t, y):
return y-100
event.terminal = True
sol = solve_ivp(funct, [0, 50], np.array([0]), events=event, rtol=1e-9, atol=1e-12)
As you can see, the integration should stop when y = 100. For this reason, I created an event in order to detect that and stop the integration.
However, the function is being computed with values y > 100 because I get the warning:
RuntimeWarning: invalid value encountered in sqrt return np.sqrt(100-y)
I guess this happens because according to Scipy documentation
The solver will find an accurate value of t at which event(t, y(t)) = 0 using a root-finding algorithm.
Is there a way to avoid this from happening?

Is there any equivalent of hyperopts lognormal in Optuna?

I am trying to use Optuna for hyperparameter tuning of my model.
I am stuck in a place where I want to define a search space having lognormal/normal distribution. It is possible in hyperopt using hp.lognormal. Is it possible to define such a space using a combination of the existing suggest_ api of Optuna?
You could perhaps make use of inverse transforms from suggest_float(..., 0, 1) (i.e. U(0, 1)) since Optuna currently doesn't provide suggest_ variants for those two distributions directly. This example might be a starting point https://gist.github.com/hvy/4ef02ee2945fe50718c71953e1d6381d
Please find the code below
import matplotlib.pyplot as plt
import numpy as np
from scipy.stats import norm
from scipy.special import erfcinv
import optuna
def objective(trial):
# Suggest from U(0, 1) with Optuna.
x = trial.suggest_float("x", 0, 1)
# Inverse transform into normal.
y0 = norm.ppf(x, loc=0, scale=1)
# Inverse transform into lognormal.
y1 = np.exp(-np.sqrt(2) * erfcinv(2 * x))
return y0, y1
if __name__ == "__main__":
n_objectives = 2 # Normal and lognormal.
study = optuna.create_study(
sampler=optuna.samplers.RandomSampler(),
# Could be "maximize". Does not matter for this demonstration.
directions=["minimize"] * n_objectives,
)
study.optimize(objective, n_trials=10000)
fig, axs = plt.subplots(n_objectives)
for i in range(n_objectives):
axs[i].hist(list(t.values[i] for t in study.trials), bins=100)
plt.show()

Why is the gradient computed with GradientTape wrong in this case (using tfp.vi.monte_carlo_variational_loss)

The gradients from tf.GradientTape seem not to match the correct minimum in the function I'm trying to minimise.
I'm trying to use tensorflowprobability's black-box variational inference (using tf2), with the tf.GradientTape, a keras optimizer, calling the apply_gradients function. The surrogate posterior is a simple 1d Normal. I'm trying to approximate a pair of normals, see pdist function. For simplicity I just try to optimise the scale parameter.
Current code:
from scipy.special import erf
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
import tensorflow as tf
import tensorflow_probability as tfp
from tensorflow_probability import distributions as tfd
def pdist(x):
return (.5/np.sqrt(2*np.pi)) * np.exp((-(x+3)**2)/2) + (.5/np.sqrt(2*np.pi)) * np.exp((-(x-3)**2)/2)
def logpdist(x):
logp = np.log(1e-30+pdist(x))
assert np.all(np.isfinite(logp))
return logp
optimizer = tf.keras.optimizers.Adam(learning_rate=0.1)
mu = tf.Variable(0.0,dtype=tf.float64)
scale = tf.Variable(1.0,dtype=tf.float64)
for it in range(100):
with tf.GradientTape() as tape:
surrogate_posterior = tfd.Normal(mu,scale)
elbo_loss = tfp.vi.monte_carlo_variational_loss(logpdist,surrogate_posterior,sample_size=10000)
gradients = tape.gradient(elbo_loss, [scale])
optimizer.apply_gradients(zip(gradients, [scale]))
if it%10==0: print(scale.numpy(),gradients[0].numpy(),elbo_loss.numpy())
Output (showing every 10th iteration):
SCALE GRAD ELBO_LOSS
1.100, -1.000, 2.697
2.059, -0.508, 1.183
2.903, -0.354, 0.859 <<< (right answer about here)
3.636, -0.280, 1.208
4.283, -0.237, 1.989
4.869, -0.208, 3.021
5.411, -0.187, 4.310
5.923, -0.170, 5.525
6.413, -0.157, 7.250
6.885, -0.146, 8.775
For some reason the gradient doesn't reflect the true gradient, which should be about zero around scale=2.74.
Why does the gradient not relate to the actual elbo_loss?
Hopefully someone can elaborate on why the previous implementation failed (and also why it doesn't except, but instead just has the wrong answer). Anyway, I found I could fix it by ensuring that key expressions used the tensorflow maths library and not numpy's. Specifically replacing the two methods above with;
def pdist(x):
return (.5/np.sqrt(2*np.pi)) * tf.exp((-(x+3)**2)/2) + (.5/np.sqrt(2*np.pi)) * tf.exp((-(x-3)**2)/2)
def logpdist(x):
return tf.math.log(pdist(x))
The stochastic optimisation now works.
Output:
2.020, -0.874, 1.177
2.399, -0.393, 0.916
2.662, -0.089, 0.857
2.761, 0.019, 0.850
2.765, 0.022, 0.843
2.745, -0.006, 0.851
2.741, 0.017, 0.845
2.752, 0.005, 0.852
2.744, 0.015, 0.852
2.747, 0.013, 0.862
I'm not going to accept my own answer as I'd be grateful if some answers could be given that give intuition about why this now works and why it failed previously (and why the failure mode wasn't an exception or similar but instead an incorrect gradient).

How do I calculate the global efficiency of graph in igraph (python)?

I am trying to calculate the global efficiency of a graph in igraph but I am not sure if I using the module correctly. I think there is a solution that might make a bit of sense but it is in r, and I wasn't able to decipher what they were saying.
I have tried writing the code in a networkx fashion trying to emulate the way they calculate global efficiency but I have been unsuccessful thus far. I am using igraph due to the fact that I am dealing with large graphs. Any help would be really appreciated :D
This is what I have tried:
import igraph
import pandas as pd
import numpy as np
from itertools import permutations
datasafe = pd.read_csv("b1.csv", index_col=0)
D = datasafe.values
g = igraph.Graph.Adjacency((D > 0).tolist())
g.es['weight'] = D[D.nonzero()]
def efficiency_weighted(g):
weights = g.es["weight"][:]
eff = (1.0 / np.array(g.shortest_paths_dijkstra(weights=weights)))
return eff
def global_efficiecny_weighted(g):
n=180.0
denom=n*(n-1)
g_eff = sum(efficiency_weighted(g) for u, v in permutations(g, 2))
return g_eff
global_efficiecny_weighted(g)
The error message I am getting says:- TypeError: 'Graph' object is not iterable
Assuming that you want the nodal efficiency for all nodes, then you can do this:
import numpy as np
from igraph import *
np.seterr(divide='ignore')
# Example using a random graph with 20 nodes
g = Graph.Erdos_Renyi(20,0.5)
# Assign weights on the edges. Here 1s everywhere
g.es["weight"] = np.ones(g.ecount())
def nodal_eff(g):
weights = g.es["weight"][:]
sp = (1.0 / np.array(g.shortest_paths_dijkstra(weights=weights)))
np.fill_diagonal(sp,0)
N=sp.shape[0]
ne= (1.0/(N-1)) * np.apply_along_axis(sum,0,sp)
return ne
eff = nodal_eff(g)
print(eff)
#[0.68421053 0.81578947 0.73684211 0.76315789 0.76315789 0.71052632
# 0.81578947 0.81578947 0.81578947 0.73684211 0.71052632 0.68421053
# 0.71052632 0.81578947 0.84210526 0.76315789 0.68421053 0.68421053
# 0.78947368 0.76315789]
To get the global just do:
np.mean(eff)

Solving the Lorentz model using Runge Kutta 4th Order in Python without a package

I wish to solve the Lorentz model in Python without the help of a package and my codes seems not to work to my expectation. I do not know why I am not getting the expected results and Lorentz attractor. The main problem I guess is related to how to store the various values for the solution of x,y and z respectively.Below are my codes for the Runge-Kutta 45 for the Lorentz model with 3D plot of solutions:
import numpy as np
import matplotlib.pyplot as plt
#from scipy.integrate import odeint
#a) Defining the Runge-Kutta45 method
def fx(x,y,z,t):
dxdt=sigma*(y-z)
return dxdt
def fy(x,y,z,t):
dydt=x*(rho-z)-y
return dydt
def fz(x,y,z,t):
dzdt=x*y-beta*z
return dzdt
def RungeKutta45(x,y,z,fx,fy,fz,t,h):
k1x,k1y,k1z=h*fx(x,y,z,t),h*fy(x,y,z,t),h*fz(x,y,z,t)
k2x,k2y,k2z=h*fx(x+k1x/2,y+k1y/2,z+k1z/2,t+h/2),h*fy(x+k1x/2,y+k1y/2,z+k1z/2,t+h/2),h*fz(x+k1x/2,y+k1y/2,z+k1z/2,t+h/2)
k3x,k3y,k3z=h*fx(x+k2x/2,y+k2y/2,z+k2z/2,t+h/2),h*fy(x+k2x/2,y+k2y/2,z+k2z/2,t+h/2),h*fz(x+k2x/2,y+k2y/2,z+k2z/2,t+h/2)
k4x,k4y,k4z=h*fx(x+k3x,y+k3y,z+k3z,t+h),h*fy(x+k3x,y+k3y,z+k3z,t+h),h*fz(x+k3x,y+k3y,z+k3z,t+h)
return x+(k1x+2*k2x+2*k3x+k4x)/6,y+(k1y+2*k2y+2*k3y+k4y)/6,z+(k1z+2*k2z+2*k3z+k4z)/6
sigma=10.
beta=8./3.
rho=28.
tIn=0.
tFin=10.
h=0.05
totalSteps=int(np.floor((tFin-tIn)/h))
t=np.zeros(totalSteps)
x=np.zeros(totalSteps)
y=np.zeros(totalSteps)
z=np.zeros(totalSteps)
for i in range(1, totalSteps):
x[i-1]=1. #Initial condition
y[i-1]=1. #Initial condition
z[i-1]=1. #Initial condition
t[0]=0. #Starting value of t
t[i]=t[i-1]+h
x,y,z=RungeKutta45(x,y,z,fx,fy,fz,t[i-1],h)
#Plotting solution
from mpl_toolkits.mplot3d import Axes3D
from matplotlib import cm
fig=plt.figure()
ax=fig.gca(projection='3d')
ax.plot(x,y,z,'r',label='Lorentz 3D Solution')
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.set_zlabel('z')
ax.legend()
I changed the integration step (btw., classical 4th order Runge-Kutta, not any adaptive RK45) to use the python core concept of lists and list operations extensively to reduce the number of places where the computation is defined. There were no errors there to correct, but I think the algorithm itself is more concentrated.
You had an error in the system that changed it into a system that rapidly diverges. You had fx = sigma*(y-z) while the Lorenz system has fx = sigma*(y-x).
Next your main loop has some strange assignments. In every loop you first set the previous coordinates to the initial conditions and then replace the full arrays with the RK step applied to the full arrays. I replaced that completely, there are no small steps to a correct solution.
import numpy as np
import matplotlib.pyplot as plt
#from scipy.integrate import odeint
def fx(x,y,z,t): return sigma*(y-x)
def fy(x,y,z,t): return x*(rho-z)-y
def fz(x,y,z,t): return x*y-beta*z
#a) Defining the classical Runge-Kutta 4th order method
def RungeKutta4(x,y,z,fx,fy,fz,t,h):
k1x,k1y,k1z = ( h*f(x,y,z,t) for f in (fx,fy,fz) )
xs, ys,zs,ts = ( r+0.5*kr for r,kr in zip((x,y,z,t),(k1x,k1y,k1z,h)) )
k2x,k2y,k2z = ( h*f(xs,ys,zs,ts) for f in (fx,fy,fz) )
xs, ys,zs,ts = ( r+0.5*kr for r,kr in zip((x,y,z,t),(k2x,k2y,k2z,h)) )
k3x,k3y,k3z = ( h*f(xs,ys,zs,ts) for f in (fx,fy,fz) )
xs, ys,zs,ts = ( r+kr for r,kr in zip((x,y,z,t),(k3x,k3y,k3z,h)) )
k4x,k4y,k4z =( h*f(xs,ys,zs,ts) for f in (fx,fy,fz) )
return (r+(k1r+2*k2r+2*k3r+k4r)/6 for r,k1r,k2r,k3r,k4r in
zip((x,y,z),(k1x,k1y,k1z),(k2x,k2y,k2z),(k3x,k3y,k3z),(k4x,k4y,k4z)))
sigma=10.
beta=8./3.
rho=28.
tIn=0.
tFin=10.
h=0.01
totalSteps=int(np.floor((tFin-tIn)/h))
t = totalSteps * [0.0]
x = totalSteps * [0.0]
y = totalSteps * [0.0]
z = totalSteps * [0.0]
x[0],y[0],z[0],t[0] = 1., 1., 1., 0. #Initial condition
for i in range(1, totalSteps):
x[i],y[i],z[i] = RungeKutta4(x[i-1],y[i-1],z[i-1], fx,fy,fz, t[i-1], h)
Using tFin = 40 and h=0.01 I get the image
looking like the typical image of the Lorenz attractor.

Resources