Setting the RNG seed from the time - j

When you open up a new J console and execute:
? 1000
You'll always see the same result, 689 (Try it online!).
This is because the RNG will always start with the same initial seed.
I'm looking for a way to change the seed semi-randomly. It doesn't have to be secure (eg, using milliseconds of time or some similar method is fine).
I assume I would do this using 9!:45 (documented on this page) but I'm unsure how to proceed.
I also don't care which of the 4 RNGs I use. I simply want a way to produce different random-ish results even in a fresh console.

The foreign that you need to set the random seed is 9!:1 (Random Seed).
You can use the randomize verb available from the general/misc addon to randomly set the random seed using guids, or current time if the guid script is not available. For example:
9!:0 '' NB. Query the random seed/link
16807
require 'numeric'
randomize '' NB. Set a new seed for the session
_1672920848 1179844600 923541917 _282857428
9!:0 '' NB. Query the random seed/link
_1672920848 1179844600 923541917 _282857428
The definition of randomize is:
randomize
3 : 0
try.
require 'guid'
tmp=. _2 (3!:4) , guids 1
catch. NB. general/misc/guid.ijs not available
tmp=. >:<.0.8*0 60 60 24 31#.0 0 0 0 _1+|.<.}.6!:0 ''
end.
([ 9!:1) {.^:(2~:9!:42'') tmp NB. set random initial random seed
)

Related

PuLP solvers do not respond to options fed to them

So I've got a fairly large optimization problem and I'm trying to solve it within a sensible amount of time.
Ive set it up as:
import pulp as pl
my_problem = LpProblem("My problem",LpMinimize)
# write to problem file
my_problem.writeLP("MyProblem.lp")
And then alternatively
solver = CPLEX_CMD(timeLimit=1, gapRel=0.1)
status = my_problem .solve(solver)
solver = pl.apis.CPLEX_CMD(timeLimit=1, gapRel=0.1)
status = my_problem .solve(solver)
path_to_cplex = r'C:\Program Files\IBM\ILOG\CPLEX_Studio1210\cplex\bin\x64_win64\cplex.exe' # and yes this is the actual path on my machine
solver = pl.apis.cplex_api.CPLEX_CMD(timeLimit=1, gapRel=0.1, path=path_to_cplex)
status = my_problem .solve(solver)
solver = pl.apis.cplex_api.CPLEX_CMD(timeLimit=1, gapRel=0.1, path=path_to_cplex)
status = my_problem .solve(solver)
It runs in each case.
However, the solver does not repond to the timeLimit or gapRel instructions.
If I use timelimit it does warn this is depreciated for timeLimit. Same for fracgap: it tells me I should use relGap. So somehow I am talking to the solver.
However, nor matter what values i pick for timeLimit and relGap, it always returns the exact same answer and takes the exact same amount of time (several minutes).
Also, I have tried alternative solvers, and I cannot get any one of them to accept their variants of time limits or optimization gaps.
In each case, the problem solves and returns an status: optimal message. But it just ignores the time limit and gap instructions.
Any ideas?
out of the zoo example:
import pulp
import cplex
bus_problem = pulp.LpProblem("bus", pulp.LpMinimize)
nbBus40 = pulp.LpVariable('nbBus40', lowBound=0, cat='Integer')
nbBus30 = pulp.LpVariable('nbBus30', lowBound=0, cat='Integer')
# Objective function
bus_problem += 500 * nbBus40 + 400 * nbBus30, "cost"
# Constraints
bus_problem += 40 * nbBus40 + 30 * nbBus30 >= 300
solver = pulp.CPLEX_CMD(options=['set timelimit 40'])
bus_problem.solve(solver)
print(pulp.LpStatus[bus_problem.status])
for variable in bus_problem.variables():
print ("{} = {}".format(variable.name, variable.varValue))
Correct way to pass solver option as dictionary
pulp.CPLEX_CMD(options={'timelimit': 40})
#Alex Fleisher has it correct with pulp.CPLEX_CMD(options=['set timelimit 40']). This also works for CBC using the following syntax:
prob.solve(COIN_CMD(options=['sec 60','Presolve More','Multiple 15', 'Node DownFewest','HEUR on', 'Round On','PreProcess Aggregate','PassP 10','PassF 40','Strong 10','Cuts On', 'Gomory On', 'CutD -1', 'Branch On', 'Idiot -1', 'sprint -1','Reduce On','Two On'],msg=True)).
It is important to understand that the parameters, and associated options, are specific to a solver. PuLP seems to be calling CBC via the command line so an investigation of those things is required. Hope that helps

Numerical integration of a numpy array in incremental time steps

I have two arrays. The first one is time in terms of Age (yrs) and the second one is a parameter that needs to be integrated with respect to time.
age = [5.00000e+08, 5.60322e+08, 6.27922e+08, 7.03678e+08, 7.88572e+08,
8.83709e+08, 9.90324e+08, 1.10980e+09, 1.24369e+09, 1.39374e+09,
1.56188e+09, 1.75032e+09, 1.96148e+09, 2.19813e+09, 2.46332e+09,
2.76050e+09, 3.09354e+09, 3.46676e+09, 3.88501e+09, 4.35371e+09,
4.87897e+09, 5.46759e+09, 6.12722e+09, 6.86644e+09, 7.69484e+09,
8.62318e+09, 9.66352e+09, 1.08294e+10, 1.21359e+10, 1.36000e+10]
sfr = [1.86120543e-02, 1.46680445e-02, 1.07275184e-02, 8.56960274e-03,
6.44041855e-03, 4.93194263e-03, 3.69203448e-05, 2.69813985e-04,
6.17644783e-04, 1.00780427e-02, 1.20645391e-02, 3.05009362e-02,
3.91535011e-02, 5.35479858e-02, 7.36489068e-02, 9.63931263e-02,
1.11108326e-01, 1.47781221e-01, 1.63057763e-01, 2.27429626e-01,
2.20941333e-01, 2.74413180e-01, 2.72010867e-01, 4.32215233e-01,
5.79654549e-01, 7.39362218e-01, 9.41168727e-01, 1.18868347e+00,
1.42839043e+00, 1.91326333e+00]
I want to perform integration of sfr array with respect to age array, but in steps.
For example, the first integration should contain only the first elements of both arrays, the second integration should contain the first 2 elements of both arrays, the third should have first 3 elements of both arrays and so on and so forth. And save the integration result for each step in a single output array.
The exact form of your desired result is not so clear. So, here are 2 posibilities:
age = [5.00000e+08, 5.60322e+08, 6.27922e+08, 7.03678e+08, 7.88572e+08,
8.83709e+08, 9.90324e+08, 1.10980e+09, 1.24369e+09, 1.39374e+09,
1.56188e+09, 1.75032e+09, 1.96148e+09, 2.19813e+09, 2.46332e+09,
2.76050e+09, 3.09354e+09, 3.46676e+09, 3.88501e+09, 4.35371e+09,
4.87897e+09, 5.46759e+09, 6.12722e+09, 6.86644e+09, 7.69484e+09,
8.62318e+09, 9.66352e+09, 1.08294e+10, 1.21359e+10, 1.36000e+10]
sfr = [1.86120543e-02, 1.46680445e-02, 1.07275184e-02, 8.56960274e-03,
6.44041855e-03, 4.93194263e-03, 3.69203448e-05, 2.69813985e-04,
6.17644783e-04, 1.00780427e-02, 1.20645391e-02, 3.05009362e-02,
3.91535011e-02, 5.35479858e-02, 7.36489068e-02, 9.63931263e-02,
1.11108326e-01, 1.47781221e-01, 1.63057763e-01, 2.27429626e-01,
2.20941333e-01, 2.74413180e-01, 2.72010867e-01, 4.32215233e-01,
5.79654549e-01, 7.39362218e-01, 9.41168727e-01, 1.18868347e+00,
1.42839043e+00, 1.91326333e+00]
integr_pairs = [[(a, s) for a, s in zip(age[:i], sfr[:i])] for i in range(1, len(age))]
print(integr_pairs)
# [[(500000000.0, 0.0186120543)], [(500000000.0, 0.0186120543), (560322000.0, 0.0146680445)], ....
integr_list = [[item for t in [(a, s) for a, s in zip(age[:i], sfr[:i])] for item in t ]for i in range(1, len(age))]
print(integr_list)
# [[500000000.0, 0.0186120543], [500000000.0, 0.0186120543, 560322000.0, 0.0146680445],

PACF function in statsmodels.tsa.stattools gives numbers greater than 1 when using ywunbiased?

I have a dataframe which is of length 177 and I want to calculate and plot the partial auto-correlation function (PACF).
I have the data imported etc and I do:
from statsmodels.tsa.stattools import pacf
ys = pacf(data[key][array].diff(1).dropna(), alpha=0.05, nlags=176, method="ywunbiased")
xs = range(lags+1)
plt.figure()
plt.scatter(xs,ys[0])
plt.grid()
plt.vlines(xs, 0, ys[0])
plt.plot(ys[1])
The method used results in numbers greater than 1 for very long lags (90ish) which is incorrect and I get a RuntimeWarning: invalid value encountered in sqrtreturn rho, np.sqrt(sigmasq) but since I can't see their source code I don't know what this means.
To be honest, when I search for PACF, all the examples only carry out PACF up to 40 lags or 60 or so and they never have any significant PACF after lag=2 and so I couldn't compare to other examples either.
But when I use:
method="ols"
# or
method="ywmle"
the numbers are corrected. So it must be the algo they use to solve it.
I tried importing inspect and getsource method but its useless it just shows that it uses another package and I can't find that.
If you also know where the problem arises from, I would really appreciate the help.
For your reference, the values for data[key][array] are:
[1131.130005, 1144.939941, 1126.209961, 1107.300049, 1120.680054, 1140.839966, 1101.719971, 1104.23999, 1114.579956, 1130.199951, 1173.819946, 1211.920044, 1181.27002, 1203.599976, 1180.589966, 1156.849976, 1191.5, 1191.329956, 1234.180054, 1220.329956, 1228.810059, 1207.01001, 1249.47998, 1248.290039, 1280.079956, 1280.660034, 1294.869995, 1310.609985, 1270.089966, 1270.199951, 1276.660034, 1303.819946, 1335.849976, 1377.939941, 1400.630005, 1418.300049, 1438.23999, 1406.819946, 1420.859985, 1482.369995, 1530.619995, 1503.349976, 1455.27002, 1473.98999, 1526.75, 1549.380005, 1481.140015, 1468.359985, 1378.550049, 1330.630005, 1322.699951, 1385.589966, 1400.380005, 1280.0, 1267.380005, 1282.829956, 1166.359985, 968.75, 896.23999, 903.25, 825.880005, 735.090027, 797.869995, 872.8099980000001, 919.1400150000001, 919.320007, 987.4799800000001, 1020.6199949999999, 1057.079956, 1036.189941, 1095.630005, 1115.099976, 1073.869995, 1104.48999, 1169.430054, 1186.689941, 1089.410034, 1030.709961, 1101.599976, 1049.329956, 1141.199951, 1183.26001, 1180.550049, 1257.640015, 1286.119995, 1327.219971, 1325.829956, 1363.609985, 1345.199951, 1320.640015, 1292.280029, 1218.890015, 1131.420044, 1253.300049, 1246.959961, 1257.599976, 1312.410034, 1365.680054, 1408.469971, 1397.910034, 1310.329956, 1362.160034, 1379.319946, 1406.579956, 1440.670044, 1412.160034, 1416.180054, 1426.189941, 1498.109985, 1514.680054, 1569.189941, 1597.569946, 1630.73999, 1606.280029, 1685.72998, 1632.969971, 1681.550049, 1756.540039, 1805.810059, 1848.359985, 1782.589966, 1859.449951, 1872.339966, 1883.949951, 1923.569946, 1960.22998, 1930.6700440000002, 2003.369995, 1972.290039, 2018.050049, 2067.560059, 2058.899902, 1994.9899899999998, 2104.5, 2067.889893, 2085.51001, 2107.389893, 2063.110107, 2103.840088, 1972.180054, 1920.030029, 2079.360107, 2080.409912, 2043.939941, 1940.2399899999998, 1932.22998, 2059.73999, 2065.300049, 2096.949951, 2098.860107, 2173.600098, 2170.949951, 2168.27002, 2126.149902, 2198.810059, 2238.830078, 2278.8701170000004, 2363.639893, 2362.719971, 2384.199951, 2411.800049, 2423.409912, 2470.300049, 2471.649902, 2519.360107, 2575.26001, 2584.840088, 2673.610107, 2823.810059, 2713.830078, 2640.8701170000004, 2648.050049, 2705.27002, 2718.3701170000004, 2816.290039, 2901.52002, 2913.97998]
Your time series is pretty clearly not stationary, so that Yule-Walker assumptions are violated.
More generally, PACF is usually appropriate with stationary time series. You might difference your data first, before considering the partial autocorrelations.

How do I search and replace in vi to eliminate a random number of random characters preceding a known string?

I have text files which look like this:
0 298047498 /directory1/app/20170417/file1.blob 0 f191
e 6569844 /directory1/app/20170417/file2.blob 0 f191
344 /directory1/app/20170417/file3.blob 0
8946 /directory1/app/20170417/file4.blob 0
196496 /directory1/app/20170417/file5.blob 0
9 182340752 /directory1/app/20170417/file6.blob 0 f191
68802 /directory1/app/20170417/file7.blob 0
I want to remove everything prior to the first / and everything after the file extension.
Results should look like this:
/directory1/app/20170417/file1.blob
/directory1/app/20170417/file2.blob
/directory1/app/20170417/file3.blob
Is there a way to do this using vi search and replace?
This type of question may be better placed here: https://vi.stackexchange.com/
But for now:
Yout can e.g. use a simple vim-macro, in which you collect all the key-strokes you need to edit one line and repeat this macro as many times as you need it.
Here are simply the key-strokes for one line:
dt/WD
d = delete..
t = ..till the first "/"
W = [shift]+[w] jumps to the next Word (after the "file-location-string")
D = [shift]+[d] deletes till the end of the current line
If you want to record this as a macro, do the following, with the keystrokes from above, inbetween - like this:
qmdt/WD[home][down]q
qm = start the recording of a macro in buffer "m"
... key-strokes from above
[home][down] = key [home] followed by [arrow down]-key, to move into the next line (for convenince)
q = end up the macro-recording
Now execute that macro with:
#m
And if you added the [down] key, you can do something like:
7#m
with which you fire your macro 7 times, for all your 7 lines.

Generate a random number from normal distribution in J

In J programming: I know how to get a linear random number.
? 5#10
1 3 3 4 7
But how to get a random number from normal distribution, e.g. N(0,1)? Thanks!
I think you're looking for normalrand in the stats package:
load 'stats'
normalrand 5
_0.514477 1.23645 _0.353373 _0.522193 1.23505
See also the stats/distribs addon
load 'stats/distrib'
rnorm 4
_0.486091 _0.339021 1.50653 0.19308
10 2 rnorm 4 NB. from distribution N(10,2)
10.0588 11.0472 13.6208 8.78888

Resources