I've been trying to implement the fast powering algorithm into python code, but it turned out to be slower than the native approach, where's the problem?
import timeit
test = """
def Power(base, power, mod):
base_to_the_powers = [base]
temp = list(bin(power)[2:])
temp.reverse()
for i in range(len(temp)-1):
base_to_the_powers.append(base_to_the_powers[i]**2%mod)
values_to_multiply = [base_to_the_powers[index] for index,i in enumerate(temp) if i == '1' ]
x=1
for value in values_to_multiply:
x = (x* value) % 1000
return x
Power(3,218,1000)
"""
test2 ="""
result = (3**218)%1000
"""
print(timeit.timeit(test),"\n", timeit.timeit(test2))
Related
I'm writing code to find median orders of tournaments, given a tournament T with n vertices, a median order is an ordering of the vertices of T such that it maximices the number of edges pointing in the 'increasing' direction with respect to the ordering.
In particular if the vertex set of T is {0,...,n-1}, the following integer problem (maximizing over the set of permutations) yields an optimal answer to the problem where Q is also a boolean n by n matrix.
I've implemented a linealization of this problem, noting that Q is a permutation, the following python code works, but my computer can't handle graphs that are as small as 10 vertices, where i would expect to have fast answers, is there any relatively easy way to speed up this computation?.
import numpy as np
from pyscipopt import Model,quicksum
from networkx.algorithms.tournament import random_tournament as rt
import math
# Some utilities to define the adjacency matrix of an oriented graph
def adjacency_matrix(t,order):
n = len(order)
adj_t = np.zeros((n,n))
for e in t.edges:
adj_t[order.index(e[0]),order.index(e[1])] = 1
return adj_t
# Random tournaments to instanciate the problem
def random_tournament(n):
r_t = rt(n)
adj_t = adjacency_matrix(r_t,list(range(n)))
return r_t, adj_t
###############################################################
############# PySCIPOpt optimization Routine ##################
###############################################################
n = 5 # some arbitrary size parameter
t,adj_t = random_tournament(n)
model = Model()
p,w,r = {},{},{}
# Defining model variables and weights
for k in range(n):
for l in range(n):
p[k,l] = model.addVar(vtype='B')
for i in range(n):
for j in range(i,n):
r[i,k,l,j] = model.addVar(vtype='C')
w[i,k,l,j] = adj_t[k][l]
for i in range(n):
# Forcing p to be a permutation
model.addCons(quicksum(p[s,i] for s in range(n))==1)
model.addCons(quicksum(p[i,s] for s in range(n))==1)
for k in range(n):
for j in range(i,n):
for l in range(n):
# Setting r[i,k,l,j] = min(p[i,k],p[l,j])
model.addCons(r[i,k,l,j] <= p[k,i])
model.addCons(r[i,k,l,j] <= p[l,j])
# Set the objective function
model.setObjective(quicksum(r[i,k,l,j]*w[i,k,l,j] for i in range(n) for j in range(i,n) for k in range(n) for l in range(n)), "maximize")
model.data = p,r
model.optimize()
sol = model.getBestSol()
# Print the solution on a readable format
Q = np.array([math.floor(model.getVal(model.data[0][key])) for key in model.data[0].keys()]).reshape([n,n])
print(f'\nOptimization ended with status {model.getStatus()} in {"{:.2f}".format(end_optimization-end_setup)}s, with {model.getObjVal()} increasing edges and optimal solution:')
print('\n',Q)
order = [int(x) for x in list(np.matmul(Q.T,np.array(range(n))))]
new_adj_t = adjacency_matrix(t,order)
print(f'\nwhich induces the ordering:\n\n {order}')
print(f'\nand induces the following adjacency matrix: \n\n {new_adj_t}')
Right now I've run it for n=5 taking between 5 and 20 seconds, and have ran it succesfully for small integers 6,7 with not much change in time needed.
For n=10, on the other hand, it has been running for around an hour with no solution yet, I suppose the linearization having O(n**4) variables hurts, but I don't understand why it blows up so fast. Is this normal? How would a better implementation be in case there is one?.
I am learning some Python basics and we had this code about wrapper functions and how they can be used to add functionality to wrapped functions. So we calculate time that was used to compute two different functions:
import time
def timer(fn):
def wrapper():
t0 = time.time()
result = fn()
dt = time.time() - t0
return dt
return wrapper
def pow_2():
return 10000000 ** 2
def in_build_pow():
return pow(10000000, 2)
pow_2 = timer(pow_2)
in_build_pow = timer(in_build_pow)
a, b = 0, 0
N = 100
for i in range(N):
a += pow_2()
b += in_build_pow()
print("a = ", a)
print("b = ", b)
print(f"Average time pow_2 = {a / N:.10f}")
print(f"Average time in_build_pow = {b / N:.10f}")
But my result looks like this:
Unless I increase the iteration count N to more than 1000. The higher N, the more often results are not 0.0.
This is N = 1000:
And this is N = 10,000:
Does anyone knows why it is like this? Why I cannot count time for fewer iterations? The code works fine in online compilers, though. But VSCode, VS2022, and PyCharm all have same issues for me.
Some OS has problems calculating very short time intervals with those functions. So its better to use perf_counter_ns() method of time module. Me personally had problems on Win10, while MacOS worked perfectly fine with time() method.
I'm trying to automate data collection from an SR245 Boxcar using Python 3.6 and the PyVisa library (version 1.11.1). 9/10 times, it works great. However, three times over the course of two days it has caused the entire computer to crash and reboot (running on Windows 10). This has resulted in a lot of data loss, and I'm trying to figure out what I'm doing wrong that is leading to the whole system crashing. Code is below (it is part of a larger program, but I also run this piece of code by itself, and it has caused crashes). The data_processing file is not shown, but the functions there are simple calculations (e.g. divide the values in a list by the values in another list, return the average value from a list of integers, etc.)
import pyvisa
from pyvisa.constants import SerialTermination
import time
import numpy as np
from data_processing import *
def connect_boxcar(pNum):
rm = pyvisa.ResourceManager()
port = "COM"+pNum
sr = rm.open_resource(port)
return sr
def config_boxcar(boxcar):
#Configure the boxcar settings
boxcar.write_termination = '\r'
boxcar.read_termination='\r'
boxcar.baud_rate=19200
boxcar.end_output = SerialTermination.termination_char
def preset_scan(boxcar):
#Reset boxcar settings
boxcar.write('MR')
boxcar.write('MS;ET;T1;I2;W0')
def scan(boxcar, num):
#Send the SCAN command to the boxcar, set to the specified number of data points
command = 'SC1,2:' + str(num)
boxcar.write(command)
def read_data(boxcar, num):
#Read the stored scan data and return it as a value list
data_list = []
for x in range(num * 2):
data_list.append(float(boxcar.query('N')))
return data_list
def collect_baseline(boxcar, n):
#Get a baseline signal for later processing
config_boxcar(boxcar)
preset_scan(boxcar)
scan(boxcar, n)
raw_data = read_data(boxcar, n)
chan1 = raw_data[::2]
chan2 = raw_data[1::2]
normal_data = normalize(chan1, chan2, n)
return average_list(normal_data)
def main():
rm = pyvisa.ResourceManager()
n = 10
sleep_timer = 0.1 * n + 0.5
sr245 = rm.open_resource('COM5')
#Configure/preset
config_boxcar(sr245)
preset_scan(sr245)
#Set a timer to measure scanning time
t0 = time.time()
scan(sr245, n)
time.sleep(sleep_timer)
raw_data = read_data(sr245, n)
t1 = time.time()
#Breakdown data by channel and normalize
chan1 = raw_data[::2]
chan2 = raw_data[1::2]
normal_data = normalize(chan1, chan2, n)
elapsed_time = t1 - t0
print('Elapsed time: ', elapsed_time)
print('Channel 1: ', chan1)
print('Channel 2: ', chan2)
print('Normalized Data: ', normal_data)
print('Average Normalized Data: ', average_list(normal_data))
print('Standard Deviation: ', np.std(normal_data))
if __name__ == '__main__':
main()
I'm trying to extract the orientation of a wiimote using cwiid in python. I've managed to get the accelerometer values but there doesn't seem to be any object attributes relating to the purely gyroscopic data.
This guy managed to do it in python, but to the best of my knowledge there's no python code online with an example.
https://www.youtube.com/watch?v=cUjh0xQO6eY
There is information on wiibrew about the controller data but again this seems to be excluded from any python library.
Has anyone got any suggestions? This link has an example of getting gyro data but the packages used don't seem available.
I was actually looking for this a few days ago, found this post: https://ofalcao.pt/blog/2014/controlling-the-sbrick-with-a-wiimote. More specifically, I think the code you're looking for is:
# roll = accelerometer[0], standby ~125
# pitch = accelerometer[1], standby ~125
...
roll=(wm.state['acc'][0]-125)
pitch=(wm.state['acc'][1]-125)
I'm assuming you can use the z-axis (index 2) for the yaw
So this question has a few parts, firstly how to extract the gyro data from the motion plus sensor. To do this, the motion plus will first need to be enabled.
The gyro provides the angular rotation vectors, but due to drift caused by integration errors you can't simply use a combination of these two things to get Eular angles. The second part of the question is how to use this data to give position, and that is done by using a Kalman filter, a highly complex matrix sequence, or a complementary filter, a less complex mathematical operation. Both of these filters are essentially combining the gyro and accelerometer data, so as mentioned in a comment above, resulting in more stable measurements, less drift and a system not prone to breaking when the remote is shaken.
Kalman filter:
http://blog.tkjelectronics.dk/2012/09/a-practical-approach-to-kalman-filter-and-how-to-implement-it/
Using PyKalman on Raw Acceleration Data to Calculate Position
Complementary filter
https://www.instructables.com/Angle-measurement-using-gyro-accelerometer-and-Ar/
Still currently developing the core but will post when I'm finished, hopefully tomorrow.
The foundation code I am using to test the measurements is found here:
http://andrew-j-norman.blogspot.com/2010/12/more-code.html. Very handy, as it plots the sensor readings automatically after recording. You can see by doing this that even when still, the position estimate by using simple integration of the angular velocities results in a drift in the position vector.
EDIT:
Testing this allows for the gyro sensor to accurately calculate the angle changed over time, however there remains drift in the acceleration - which I believe is unavoidable.
Here is an image demonstrating the gyro motion sensor:
Just finished up the code:
#!/usr/bin/python
import cwiid
from time import time, asctime, sleep, perf_counter
from numpy import *
from pylab import *
import math
import numpy as np
from operator import add
HPF = 0.98
LPF = 0.02
def calibrate(wiimote):
print("Keep the remote still")
sleep(3)
print("Calibrating")
messages = wiimote.get_mesg()
i=0
accel_init = []
angle_init = []
while (i<1000):
sleep(0.01)
messages = wiimote.get_mesg()
for mesg in messages:
# Motion plus:
if mesg[0] == cwiid.MESG_MOTIONPLUS:
if record:
angle_init.append(mesg[1]['angle_rate'])
# Accelerometer:
elif mesg[0] == cwiid.MESG_ACC:
if record:
accel_init.append(list(mesg[1]))
i+=1
accel_init_avg = list(np.mean(np.array(accel_init), axis=0))
print(accel_init_avg)
angle_init_avg = sum(angle_init)/len(angle_init)
print("Finished Calibrating")
return (accel_init_avg, angle_init_avg)
def plotter(plot_title, timevector, data, position, n_graphs):
subplot(n_graphs, 1, position)
plot(timevector, data[0], "r",
timevector, data[1], "g",
timevector, data[2], "b")
xlabel("time (s)")
ylabel(plot_title)
print("Press 1+2 on the Wiimote now")
wiimote = cwiid.Wiimote()
# Rumble to indicate a connection
wiimote.rumble = 1
print("Connection established - release buttons")
sleep(0.2)
wiimote.rumble = 0
sleep(1.0)
wiimote.enable(cwiid.FLAG_MESG_IFC | cwiid.FLAG_MOTIONPLUS)
wiimote.rpt_mode = cwiid.RPT_BTN | cwiid.RPT_ACC | cwiid.RPT_MOTIONPLUS
accel_init, angle_init = calibrate(wiimote)
str = ""
print("Press plus to start recording, minus to end recording")
loop = True
record = False
accel_data = []
angle_data = []
messages = wiimote.get_mesg()
while (loop):
sleep(0.01)
messages = wiimote.get_mesg()
for mesg in messages:
# Motion plus:
if mesg[0] == cwiid.MESG_MOTIONPLUS:
if record:
angle_data.append({"Time" : perf_counter(), \
"Rate" : mesg[1]['angle_rate']})
# Accelerometer:
elif mesg[0] == cwiid.MESG_ACC:
if record:
accel_data.append({"Time" : perf_counter(), "Acc" : [mesg[1][i] - accel_init[i] for i in range(len(accel_init))]})
# Button:
elif mesg[0] == cwiid.MESG_BTN:
if mesg[1] & cwiid.BTN_PLUS and not record:
print("Recording - press minus button to stop")
record = True
start_time = perf_counter()
if mesg[1] & cwiid.BTN_MINUS and record:
if len(accel_data) == 0:
print("No data recorded")
else:
print("End recording")
print("{0} data points in {1} seconds".format(
len(accel_data), perf_counter() - accel_data[0]["Time"]))
record = False
loop = False
else:
pass
wiimote.disable(cwiid.FLAG_MESG_IFC | cwiid.FLAG_MOTIONPLUS)
if len(accel_data) == 0:
sys.exit()
timevector = []
a = [[],[],[]]
v = [[],[],[]]
p = [[],[],[]]
last_time = 0
velocity = [0,0,0]
position = [0,0,0]
for n, x in enumerate(accel_data):
if (n == 0):
origin = x
else:
elapsed = x["Time"] - origin["Time"]
delta_t = x["Time"] - last_time
timevector.append(elapsed)
for i in range(3):
acceleration = x["Acc"][i] - origin["Acc"][i]
velocity[i] = velocity[i] + delta_t * acceleration
position[i] = position[i] + delta_t * velocity[i]
a[i].append(acceleration)
v[i].append(velocity[i])
p[i].append(position[i])
last_time = x["Time"]
n_graphs = 3
if len(angle_data) == len(accel_data):
n_graphs = 5
angle_accel = [(math.pi)/2 if (j**2 + k**2)==0 else math.atan(i/math.sqrt(j**2 + k**2)) for i,j,k in zip(a[0],a[1],a[2])]
ar = [[],[],[]] # Angle rates
aa = [[],[],[]] # Angles
angle = [0,0,0]
for n, x in enumerate(angle_data):
if (n == 0):
origin = x
else:
delta_t = x["Time"] - last_time
for i in range(3):
rate = x["Rate"][i] - origin["Rate"][i]
angle[i] = HPF*(np.array(angle[i]) + delta_t * rate) + LPF*np.array(angle_accel)
ar[i].append(rate)
aa[i].append(angle[i])
last_time = x["Time"]
plotter("Acceleration", timevector, a, 1, n_graphs)
if n_graphs == 5:
plotter("Angle Rate", timevector, ar, 4, n_graphs)
plotter("Angle", timevector, aa, 5, n_graphs)
show()
I was doing a benchmark for myself that I encountered this interesting thing. I am trying to get the first 30 keys of a dictionary, and I have written three ways to get it as follows:
import time
dic = {str(i): i for i in range(10 ** 6)}
start_time = time.time()
x = list(dic.keys())[0:30]
print(time.time() - start_time)
start_time = time.time()
y = list(dic.keys())
x = y[0:30]
print(time.time() - start_time)
start_time = time.time()
z = dic.keys()
y = list(z)
x = y[0:30]
print(time.time() - start_time)
The results are:
0.015970945358276367
0.010970354080200195
0.01691460609436035
Surprisingly, the second method is much faster! Any thoughts on this?
Using Python's timeit module to measure various alternatives. I added mine which doesn't convert the keys to list:
from timeit import timeit
dic = {str(i): i for i in range(10 ** 6)}
def f1():
x = list(dic.keys())[0:30]
return x
def f2():
y = list(dic.keys())
x = y[0:30]
return x
def f3():
z = dic.keys()
y = list(z)
x = y[0:30]
return x
def f4():
x = [k for _, k in zip(range(30), dic.keys())]
return x
t1 = timeit(lambda: f1(), number=10)
t2 = timeit(lambda: f2(), number=10)
t3 = timeit(lambda: f3(), number=10)
t4 = timeit(lambda: f4(), number=10)
print(t1)
print(t2)
print(t3)
print(t4)
Prints:
0.1911074290110264
0.20418328599771485
0.18727918600779958
3.5186996683478355e-05
Maybe this is due to inaccuracies in your measure of time. You can use timeit for doing this kind of things:
import timeit
dic = {str(i): i for i in range(10 ** 6)}
# 27.5125/29.0836/26.8525
timeit.timeit("x = list(dic.keys())[0:30]", number=1000, globals={"dic": dic})
# 28.6648/26.4684/30.9534
timeit.timeit("y = list(dic.keys());x=y[0:30]", number=1000)
# 31.7345/29.5301/30.7541
timeit.timeit("z=dic.keys();y=list(z);x=y[0:30]", number=1000, globals={'dic': dic})
The comments show the times I got when running the same code 3 different times. As you can see, even by performing a large number of repetitions, it is possible to obtain quite large variations in time measured. This can be due to several different things:
An item can be in the cache of your processor or not.
Your processor can be occupied doing several other things.
Etc...
As stated by #Andrej Kesely, your bottleneck is due to the fact that you cast your dictionary keys into a list. By doing so, Python goes through the entire dictionary keys, because that's how it converts something to a list generally. Hence, by avoiding this, you can get much better results.