Misunderstanding about Julia threading race conditions - multithreading

This question is a follow up of one of my previous questions: Fastest way to compare lists with threshold in Julia
I am trying to parallelise the code using threads, here is an before/after
for i = 1:l
B[i] || continue
for j = i + 1:l
B[j] || continue
if all(abs(P[i, k] - P[j, k]) <= 0.001 for k=1:1540)
B[j] = false
end
end
end
after:
for i = 1:l
B[i] || continue
Threads.#threads for j = i + 1:l
B[j] || continue
if all(abs(P[i, k] - P[j, k]) <= 0.001 for k=1:1540)
B[j] = false
end
end
end
Where B is a BitVector of size m and P is matrix of size (m, 1540)
Just threading the loop actually improved performance and seemed to work fine with large array, (shape of P = (400000, 1540)) but doing some tests on smaller arrays with higher number of threads (m = 1056, nthreads > 8) I realised that I wasn't getting the same answer anymore. Some false in B weren't set correctly.
Now what I don't understand is that there shouldn't be a race condition here: no threads will access the same j value?
Changing to the code below actually fix the problem without hurting the performance but I don't really understand why?
for i = 1:l
B[i] || continue
o = ReentrantLock()
Threads.#threads for j = i + 1:l
B[j] || continue
if all(abs(P[i, k] - P[j, k]) <= 0.001 for k=1:1540)
Threads.lock(o) do
B[j] = false
end
end
end
end

Related

PuLP Optimization problem - how to add constraints?

I am trying to solve a linear programming problem with the following constraints:
for some given values of N and T. So suppose N={1,2} and T={1,2}
It is easy to write out for small |N|, but is impossible to write as N becomes large.
Im confused as to how to actually code these sums by indexing on my objective variables.
I used the following to create my objective variables:
for t in range(1,T+1):
for i in range(1,n+1):
for j in range(1,n+1):
# Skip the x_i,i,t entries
if j == i:
continue
element = "x"+str(i)+','+str(j)+','+str(t)
x_ijt_holding.append(element)
x_ijt =[]
for i in x_ijt_holding:
x_ijt.append(pulp.LpVariable(i, cat = "Binary"))
Initially I thought I could just define each constraint with LpVariable, but I realized the solver doesn't like that.
For example I did:
# Constraint 2 enter entries
x_ijt_2_holding = []
for j in range(1,n+1):
for i in range(1,j):
equations_2 = []
equations_3 = []
for t in range(1, T+1):
equations_2.append("x"+str(i)+','+str(j)+','+str(t))
equations_2.append("x"+str(j)+','+str(i)+','+str(t))
x_ijt_2_holding.append(equations_2)
# Constraint 2 as LpVariable:
for i in x_ijt_2_holding:
temp = []
for sublist in i:
temp.append(pulp.LpVariable(sublist, cat = "Binary"))
x_ijt_con2.append(temp)
So how would I then code the constraints into the problem?
Constraints defines relationships on previously defined variable written as equations. When you are building your constraints you need to reference the variables you defined in your first part. Saving your variables in dicts makes it much easier to reference your variables later.
Take a look at this
"""
Optimizing
sum(x[i][j][t] + x[j][i][t]) ==1 where j in N \ {i} for each i in N and for each t in T
sum(x[i][j][t] + x[j][i][t]) ==2 where t in T for each i,j in N, i < j
programmer Michael Gibbs
"""
import pulp
N = [1,2]
T = [1,2]
model = pulp.LpProblem("basis")
# variables
# x[i][j][t]
x = {
i:{
j:{
t:pulp.LpVariable(
'x_' + str(i) + '_' + str(j) + '_' + str(t),
cat=pulp.LpBinary
)
for t in T
}
for j in N if j != i
}
for i in N
}
# constraints
#sum(x[i][j][t] + x[j][i][t]) ==1 where j in N \ {i} for each i in N and for each t in T
for i in N:
for t in T:
c = pulp.lpSum([x[i][j][t] + x[j][i][t] for j in N if j != i]) == 1
model += c
#sum(x[i][j][t] + x[j][i][t]) ==2 where t in T for each i,j in N, i < j
for i in N:
for j in N:
if i < j:
c = pulp.lpSum([x[i][j][t] + x[j][i][t] for t in T]) == 2
model += c
# no objective
model.solve()
print("i","j","t","value")
print('-------------------')
[print(i,j,t, pulp.value(x[i][j][t])) for i in N for j in N if i != j for t in T]

How to implement quick sort on python?

maybe I’m the next person who asks how to release quick sort on python correctly. But it’s important for me to know if I wrote this algorithm correctly by reading the pseudocode from the textbook Essential Algorithms: A Practical Approach to Computer Algorithms.
When I run the code, I get this message. RecursionError: maximum recursion depth exceeded in comparison
import random
def quickSort(arr, start, end):
if start >= end: # if len(arr) < 2
return arr
else:
divideIndex = partition(arr, start, end)
quickSort(arr, start, divideIndex - 1)
quickSort(arr, divideIndex, end)
def partition(arr, head, tail):
left = head
right = tail
pivot = arr[(head + tail) // 2] # mid
while right >= left:
# looking through the array from the left
while arr[left] <= pivot:
left = left + 1
# looking through the array from the right
while arr[right] > pivot:
right = right - 1
# found a couple of elements that can be exchanged.
if left <= right:
swap(arr[right], arr[left])
# move the left and right wall
left = left + 1
right = right - 1
# return one elements if not found a couple
return left
def swap(arr1, arr2):
temp = arr1
arr1 = arr2
arr2 = temp
# Generator random variables
deck = list(range(50))
random.shuffle(deck)
start = 0
end = len(deck) - 1
print(quickSort(deck, start, end))
Try this:
def partition(arr,low,high):
i = ( low-1 )
pivot = arr[high]
for j in range(low , high):
if arr[j] <= pivot:
i = i+1
arr[i],arr[j] = arr[j],arr[i]
arr[i+1],arr[high] = arr[high],arr[i+1]
return ( i+1 )
def quickSort(arr,low,high):
if low < high:
pi = partition(arr,low,high)
quickSort(arr, low, pi-1)
quickSort(arr, pi+1, high)

Count the number of possible ways to solve an equation

This question was asked in a challenge in HackerEarth:
Mark is solving an interesting question. He needs to find out number
of distinct ways such that
(i + 2*j+ k) % (x + y + 2*z) = 0, where 1 <= i,j,k,x,y,z <= N
Help him find it.
Constraints:
1<= T<= 10
1<=N<= 1000
Input Format:
First line contains T, the number of test cases. Each of the test case
contains a single integer,N in a separate line.
Output Format:
For each test case , output in a separate line, the number of distinct
ways.
Sample Input
2
1
2
Sample Output
1
15
Explanation
In the first case, the only possible way is i = j = k = x =y = z = 1
I am not getting any way how to solve this problem, I have tried one and I know it's not even close to the question.
import random
def CountWays (N):
# Write your code here
i = random.uniform(1,N)
j = random.uniform(1,N)
k = random.uniform(1,N)
x = random.uniform(1,N)
y = random.uniform(1,N)
z = random.uniform(1,N)
d = 0
for i in range(N):
if (i+2*j+k)%(x+y+2*z)==0:
d += 1
return d
T = int(input())
for _ in range(T):
N = int(input())
out_ = CountWays(N)
print (out_)
My Output
0
0
Instead it should give the output
1
15
The value of the numerator (num) can range from 4 to 4N. The value of the denominator (dom) can range from 4 to num. You can split your problem into two smaller problems: 1) How many values of the denominator is a given value of the numerator divisible by? 2) How many ways can a given denominator and numerator be constructed?
To answer 1) we can simply loop through all the possible values of the numerator, then loop over all the values of the denominator where numerator % denominator == 0. To answer 2) we can find all the partitions of the numerator and denominator that satisfies the equality and constraints. The number of ways to construct a given numerator and denominator will be the product of the number of partitions of each.
import itertools
def divisible_numbers(n):
"""
Get all numbers with which n is divisible.
"""
for i in range(1,n+1):
if n % i == 0:
yield i
if i >= n:
break
def get_partitions(n):
"""
Generate ALL ways n can be partitioned into 3 integers.
Modified from http://code.activestate.com/recipes/218332-generator-for-integer-partitions/#c9
"""
a = [1]*n
y = -1
v = n
while v > 0:
v -= 1
x = a[v] + 1
while y >= 2 * x:
a[v] = x
y -= x
v += 1
w = v + 1
while x <= y:
a[v] = x
a[w] = y
if w == 2:
yield a[:w + 1]
x += 1
y -= 1
a[v] = x + y
y = a[v] - 1
if w == 3:
yield a[:w]
def get_number_of_valid_partitions(num, N):
"""
Get number of valid partitions of num, given that
num = i + j + 2k, and that 1<=i,j,k<=N
"""
n = 0
for partition in get_partitions(num):
# This can be done a bit more cleverly, but makes
# the code extremely complicated to read, so
# instead we just brute force the 6 combinations,
# ignoring non-unique permutations using a set
for i,j,k in set(itertools.permutations(partition)):
if i <= N and j <= N and k <= 2*N and k % 2 == 0:
n += 1
return n
def get_number_of_combinations(N):
"""
Get number of ways the equality can be solved under the given constraints
"""
out = 0
# Create a dictionary of number of valid partitions
# for all numerator values we can encounter
n_valid_partitions = {i: get_number_of_valid_partitions(i, N) for i in range(1,4*N+1)}
for numerator in range(4,4*N+1):
numerator_permutations = n_valid_partitions[numerator]
for denominator in divisible_numbers(numerator):
denominator_permutations = n_valid_partitions[denominator]
if denominator < 4:
continue
out += numerator_permutations * denominator_permutations
return out
N = 2
out = get_number_of_combinations(N)
print(out)
The scaling of the code right now is very poor due to the way the get_partitions and the get_number_of_valid_partitions functions interact.
EDIT
The following code is much faster. There's a small improvement to divisible_numbers, but the main speedup lies in get_number_of_valid_partitions not creating a needless amount of temporary lists as it has now been joined with get_partitions in a single function. Other big speedups comes from using numba. The code of get_number_of_valid_partitions is all but unreadable now, so I've added a much simpler but slightly slower version named get_number_of_valid_partitions_simple so you can understand what is going on in the complicated function.
import numba
#numba.njit
def divisible_numbers(n):
"""
Get all numbers with which n is divisible.
Modified from·
"""
# We can save some time by only looking at
# values up to n/2
for i in range(4,n//2+1):
if n % i == 0:
yield i
yield n
def get_number_of_combinations(N):
"""
Get number of ways the equality can be solved under the given constraints
"""
out = 0
# Create a dictionary of number of valid partitions
# for all numerator values we can encounter
n_valid_partitions = {i: get_number_of_valid_partitions(i, N) for i in range(4,4*N+1)}
for numerator in range(4,4*N+1):
numerator_permutations = n_valid_partitions[numerator]
for denominator in divisible_numbers(numerator):
if denominator < 4:
continue
denominator_permutations = n_valid_partitions[denominator]
out += numerator_permutations * denominator_permutations
return out
#numba.njit
def get_number_of_valid_partitions(num, N):
"""
Get number of valid partitions of num, given that
num = i + j + 2l, and that 1<=i,j,l<=N.
"""
count = 0
# In the following, k = 2*l
#There's different cases for i,j,k that we can treat separately
# to give some speedup due to symmetry.
#i,j can be even or odd. k <= N or N < k <= 2N.
# Some combinations only possible if num is even/odd
# num is even
if num % 2 == 0:
# i,j odd, k <= 2N
k_min = max(2, num - 2 * (N - (N + 1) % 2))
k_max = min(2 * N, num - 2)
for k in range(k_min, k_max + 1, 2):
# only look at i<=j
i_min = max(1, num - k - N + (N + 1) % 2)
i_max = min(N, (num - k)//2)
for i in range(i_min, i_max + 1, 2):
j = num - i - k
# if i == j, only one permutations
# otherwise two due to symmetry
if i == j:
count += 1
else:
count += 2
# i,j even, k <= N
# only look at k<=i<=j
k_min = max(2, num - 2 * (N - N % 2))
k_max = min(N, num // 3)
for k in range(k_min, k_max + 1, 2):
i_min = max(k, num - k - N + N % 2)
i_max = min(N, (num - k) // 2)
for i in range(i_min, i_max + 1, 2):
j = num - i - k
if i == j == k:
# if i == j == k, only one permutation
count += 1
elif i == j or i == k or j == k:
# if only two of i,j,k are the same there are 3 permutations
count += 3
else:
# if all differ, there are six permutations
count += 6
# i,j even, N < k <= 2N
k_min = max(N + 1 + (N + 1) % 2, num - 2 * N)
k_max = min(2 * N, num - 4)
for k in range(k_min, k_max + 1, 2):
# only look for i<=j
i_min = max(2, num - k - N + 1 - (N + 1) % 2)
i_max = min(N, (num - k) // 2)
for i in range(i_min, i_max + 1, 2):
j = num - i - k
if i == j:
# if i == j, only one permutation
count += 1
else:
# if all differ, there are two permutations
count += 2
# num is odd
else:
# one of i,j is even, the other is odd. k <= N
# We assume that j is odd, k<=i and correct the symmetry in the counts
k_min = max(2, num - 2 * N + 1)
k_max = min(N, (num - 1) // 2)
for k in range(k_min, k_max + 1, 2):
i_min = max(k, num - k - N + 1 - N % 2)
i_max = min(N, num - k - 1)
for i in range(i_min, i_max + 1, 2):
j = num - i - k
if i == k:
# if i == j, two permutations
count += 2
else:
# if i and k differ, there are four permutations
count += 4
# one of i,j is even, the other is odd. N < k <= 2N
# We assume that j is odd and correct the symmetry in the counts
k_min = max(N + 1 + (N + 1) % 2, num - 2 * N + 1)
k_max = min(2 * N, num - 3)
for k in range(k_min, k_max + 1, 2):
i_min = max(2, num - k - N + (N + 1) % 2)
i_max = min(N, num - k - 1)
for i in range(i_min, i_max + 1, 2):
j = num - i - k
count += 2
return count
#numba.njit
def get_number_of_valid_partitions_simple(num, N):
"""
Simpler but slower version of 'get_number_of_valid_partitions'
"""
count = 0
for k in range(2, 2 * N + 1, 2):
for i in range(1, N + 1):
j = num - i - k
if 1 <= j <= N:
count += 1
return count
if __name__ == "__main__":
N = int(sys.argv[1])
out = get_number_of_combinations(N)
print(out)
The current issue with your code is that you've picked random numbers once, then calculate the same equation N times.
I assume you wanted to generate 1..N for each individual variable, which would require 6 nested loops from 1..N, for each variable
Now, that's the brute force solution, which probably fails on large N values, so as I commented, there's some trick to find the multiples of the right side of the modulo, then check if the left side portion is contained in that list. That would only require two triple nested lists, I think
(i + 2*j+ k) % (x + y + 2*z) = 0, where 1 <= i,j,k,x,y,z <= N
(2*j + i + k) is a multiple of (2*z + x + y)
N = 2
min(2*j + i + k) = 4
max(2*j + i + k) = 8
ways to make 4: 1 * 1 = 1
ways to make 5: 2 * 2 = 4
ways to make 6: 2 * 2 = 4
ways to make 7: 2 * 2 = 4
ways to make 8: 1 * 1 = 1
Total = 14
But 8 is a multiple of 4 so we add one more instance for a total of 15.

How do i make these two code snippets run in parallel

I would like to run these two parts of the code in parallel. Is this possible in python? How would i have to modify the code to accommodate this?
def smo(self, X, y):
iterations = 0
n_samples = X.shape[0]
# Initial coefficients
alpha = numpy.zeros(n_samples)
# Initial gradient
g = numpy.ones(n_samples)
while True:
yg = g * y
# KKT Conditions
y_pos_ind = (y == 1)
y_neg_ind = (numpy.ones(n_samples) - y_pos_ind).astype(bool)
alpha_pos_ind = (alpha >= self.C)
alpha_neg_ind = (alpha <= 0)
indices_violating_Bi_1 = y_pos_ind * alpha_pos_ind
indices_violating_Bi_2 = y_neg_ind * alpha_neg_ind
indices_violating_Bi = indices_violating_Bi_1 + indices_violating_Bi_2
yg_i = yg.copy()
yg_i[indices_violating_Bi] = float('-inf')
# First of the maximum violating pair
i = numpy.argmax(yg_i)
Kik = self.kernel_matrix(X, i)
indices_violating_Ai_1 = y_pos_ind * alpha_neg_ind
indices_violating_Ai_2 = y_neg_ind * alpha_pos_ind
indices_violating_Ai = indices_violating_Ai_1 + indices_violating_Ai_2
yg_j = yg.copy()
yg_j[indices_violating_Ai] = float('+inf')
# Second of the maximum violating pair
j = numpy.argmin(yg_j)
Kjk = self.kernel_matrix(X, j)
# Optimality criterion
if(yg_i[i] - yg_j[j]) < self.tol or (iterations >= self.max_iter):
break
min_term_1 = (y[i] == 1) * self.C - y[i] * alpha[i]
min_term_2 = y[j] * alpha[j] + (y[j] == -1) * self.C
min_term_3 = (yg_i[i] - yg_j[j]) / (Kik[i] + Kjk[j] - 2 * Kik[j])
# Direction search
lamda = numpy.min([min_term_1, min_term_2, min_term_3])
# Gradient update
g += lamda * y * (Kjk - Kik)
# Update coefficients
alpha[i] = alpha[i] + y[i] * lamda
alpha[j] = alpha[j] - y[j] * lamda
iterations += 1
print('{} iterations to arrive at the minimum'.format(iterations))
return alpha
I would like to run this line
Kik = self.kernel_matrix(X, i)
and this line
Kjk = self.kernel_matrix(X, j)
in parallel. How do i change the code to accommodate this?
Giving you a response with just the finished multi threading code probably wouldn't be that helpful to you and is hard given I don't know what the functions themselves do but check out this link: https://realpython.com/intro-to-python-threading/
The general idea is you will have to start a thread for each task you want to run in parallel like this:
thread1 = threading.Thread(target=kernel_matrix,args=(X,j))
thread1.start()
If you want to wait for a thread to finish you call thread.join()
You'll need to watch out for race conditions too good thread on that here: What is a race condition?

List rotation to the left in Python 3

I'm trying to create a list (b) that is list (a) rotating a's members k times to the left. I came up with this on Python 3:
n = 5
k = 4
a = [1,2,3,4,5]
b = []
for i in a:
if (i + k) <= (n - 1):
b.append(a[i+k])
elif (i+k-n) < (n-1):
b.append(a[i+k-n])
print(b)
But for some reason, it doesn't work since when I tell print(b) it returns a list that is exactly like list a
What am I missing here?
A simple solution:
k = k % len(a) #we don't care about shifting x*len(a) times since it does not have any effect
b = a[k:] + a[:k]
Thanks to #RemcoGerlich for helping me see my mistake! Here's how I quickly fixed my code:
n = 5
k = 4
a = [1,2,3,4,5]
b = []
i = 0
while (i < n):
if (i + k) <= (n - 1):
b.append(a[i+k])
elif (i+k-n) < (n-1):
b.append(a[i+k-n])
i = i+1
print(b)

Resources