I am trying to make clusters of sites based on their distance from each other. I am using or tools cp solver to achieve this. The program runs fine for 40 to 50 number of sites but when i try to run for about 200 sites with each cluster containing 10 sites the program gets stuck and does not give any output. I am using the cp_model for this.
Please below the code I am using :
from ortools.sat.python import cp_model
import pandas as pd,sys,os,requests,re
matrix_data = pd.read_csv(''.join([filepath,'//matrix.csv']),
matrix = matrix_data.values.tolist()
#data = {}
#data['distance_matrix'] = []
distance_matrix = []
for i in matrix:
distance_matrix.append(i)
def main():
"""Entry point of the program."""
num_nodes = len(distance_matrix)
print('Num nodes =', num_nodes)
# Number of groups to split the nodes, must divide num_nodes.
num_groups = 10
# Model.
model = cp_model.CpModel()
# Variables.
neighbors = {}
obj_vars = []
obj_coeffs = []
for n1 in range(num_nodes - 1):
for n2 in range(n1 + 1, num_nodes):
same = model.NewBoolVar('neighbors_%i_%i' % (n1, n2))
neighbors[n1, n2] = same
obj_vars.append(same)
obj_coeffs.append(distance_matrix[n1][n2] + distance_matrix[n2][n1])
# Number of neighborss:
for n in range(num_nodes):
model.Add(sum(neighbors[m, n] for m in range(n)) +
sum(neighbors[n, m] for m in range(n + 1, num_nodes)) ==
group_size - 1)
# Enforce transivity on all triplets.
for n1 in range(num_nodes - 2):
for n2 in range(n1 + 1, num_nodes - 1):
for n3 in range(n2 + 1, num_nodes):
model.Add(
neighbors[n1, n3] + neighbors[n2, n3] + neighbors[n1, n2] != 2)
# Redundant constraints on total sum of neighborss.
model.Add(sum(obj_vars) == num_groups * group_size * (group_size - 1) // 2)
# Minimize weighted sum of arcs.
model.Minimize(
sum(obj_vars[i] * obj_coeffs[i] for i in range(len(obj_vars))))
# Solve and print out the solution.
solver = cp_model.CpSolver()
solver.parameters.log_search_progress = True
solver.parameters.num_search_workers = 6
status = solver.Solve(model)
print(solver.ResponseStats())
visited = set()
for g in range(num_groups):
flonglist = []
flatlist = []
for n in range(num_nodes):
if not n in visited:
visited.add(n)
output = str(n)
for o in range(n + 1, num_nodes):
if solver.BooleanValue(neighbors[n, o]):
visited.add(o)
output += ' ' + str(o)
print('Group', g, ':', output)
print(site_list)
break
if __name__ == '__main__':
main()
print("-- %s seconds ---" % (time.time() - start_time))
The matrix file contain the distance of sites from each other in matrix format.
need some help.
algorithm question:
Given n points on a 2D plane, find the maximum number of points that lie on the same straight line.
Example 1:
Input: [[1,1],[2,2],[3,3]]
Output: 3
Explanation:
^
|
| o
| o
| o
+------------->
0 1 2 3 4
Example 2:
Input: [[1,1],[3,2],[5,3],[4,1],[2,3],[1,4]]
Output: 4
the working python 3 code is below:
wondering
snippet 1 d[slope] = d.get(slope, 1) + 1 is working
but why this snippet 2 is not working correctly for example 2 even though snippet 1 and 2 are the same
if slope in d:
d[slope] += 1
else:
d[slope] = 1
def gcd(self, a, b):
if b == 0:
return a
return self.gcd(b, a%b)
def get_slope(self, p1, p2):
dx = p1[0] - p2[0]
dy = p1[1] - p2[1]
c = self.gcd(dx, dy)
dx /= c
dy /= c
return str(dy) + "/" + str(dx)
def is_same_points(self, p1:List[int], p2:List[int]):
return p1[0] == p2[0] and p1[1] == p2[1]
def maxPoints(self, points: List[List[int]]) -> int:
if not points:
return 0
n = len(points)
count = 1
for i in range(0, n):
d = {}
duped = 0
localmax = 1
p1 = points[i]
for j in range(i+1, n):
p2 = points[j]
if self.is_same_points(p1, p2):
duped += 1
else:
slope = self.get_slope(p1, p2)
# 1) not work: output is 3 in example 2
# if slope in d:
# d[slope] += 1
# else:
# d[slope] = 1
# 2) works: correct output 4 for example 2
d[slope] = d.get(slope, 1) + 1
localmax = max(localmax, d[slope]);
count = max(count, localmax + duped)
return count
Interesting problem and nice solution.
The reason why the commented out code doesn't work is because of that:
else:
d[slope] = 1 ## correct would be d[slope] = 2
Every 2 points are on the same line, you are counting only one point for the first two p1 p2, thus you get one less in the final answer.
This question was asked in a challenge in HackerEarth:
Mark is solving an interesting question. He needs to find out number
of distinct ways such that
(i + 2*j+ k) % (x + y + 2*z) = 0, where 1 <= i,j,k,x,y,z <= N
Help him find it.
Constraints:
1<= T<= 10
1<=N<= 1000
Input Format:
First line contains T, the number of test cases. Each of the test case
contains a single integer,N in a separate line.
Output Format:
For each test case , output in a separate line, the number of distinct
ways.
Sample Input
2
1
2
Sample Output
1
15
Explanation
In the first case, the only possible way is i = j = k = x =y = z = 1
I am not getting any way how to solve this problem, I have tried one and I know it's not even close to the question.
import random
def CountWays (N):
# Write your code here
i = random.uniform(1,N)
j = random.uniform(1,N)
k = random.uniform(1,N)
x = random.uniform(1,N)
y = random.uniform(1,N)
z = random.uniform(1,N)
d = 0
for i in range(N):
if (i+2*j+k)%(x+y+2*z)==0:
d += 1
return d
T = int(input())
for _ in range(T):
N = int(input())
out_ = CountWays(N)
print (out_)
My Output
0
0
Instead it should give the output
1
15
The value of the numerator (num) can range from 4 to 4N. The value of the denominator (dom) can range from 4 to num. You can split your problem into two smaller problems: 1) How many values of the denominator is a given value of the numerator divisible by? 2) How many ways can a given denominator and numerator be constructed?
To answer 1) we can simply loop through all the possible values of the numerator, then loop over all the values of the denominator where numerator % denominator == 0. To answer 2) we can find all the partitions of the numerator and denominator that satisfies the equality and constraints. The number of ways to construct a given numerator and denominator will be the product of the number of partitions of each.
import itertools
def divisible_numbers(n):
"""
Get all numbers with which n is divisible.
"""
for i in range(1,n+1):
if n % i == 0:
yield i
if i >= n:
break
def get_partitions(n):
"""
Generate ALL ways n can be partitioned into 3 integers.
Modified from http://code.activestate.com/recipes/218332-generator-for-integer-partitions/#c9
"""
a = [1]*n
y = -1
v = n
while v > 0:
v -= 1
x = a[v] + 1
while y >= 2 * x:
a[v] = x
y -= x
v += 1
w = v + 1
while x <= y:
a[v] = x
a[w] = y
if w == 2:
yield a[:w + 1]
x += 1
y -= 1
a[v] = x + y
y = a[v] - 1
if w == 3:
yield a[:w]
def get_number_of_valid_partitions(num, N):
"""
Get number of valid partitions of num, given that
num = i + j + 2k, and that 1<=i,j,k<=N
"""
n = 0
for partition in get_partitions(num):
# This can be done a bit more cleverly, but makes
# the code extremely complicated to read, so
# instead we just brute force the 6 combinations,
# ignoring non-unique permutations using a set
for i,j,k in set(itertools.permutations(partition)):
if i <= N and j <= N and k <= 2*N and k % 2 == 0:
n += 1
return n
def get_number_of_combinations(N):
"""
Get number of ways the equality can be solved under the given constraints
"""
out = 0
# Create a dictionary of number of valid partitions
# for all numerator values we can encounter
n_valid_partitions = {i: get_number_of_valid_partitions(i, N) for i in range(1,4*N+1)}
for numerator in range(4,4*N+1):
numerator_permutations = n_valid_partitions[numerator]
for denominator in divisible_numbers(numerator):
denominator_permutations = n_valid_partitions[denominator]
if denominator < 4:
continue
out += numerator_permutations * denominator_permutations
return out
N = 2
out = get_number_of_combinations(N)
print(out)
The scaling of the code right now is very poor due to the way the get_partitions and the get_number_of_valid_partitions functions interact.
EDIT
The following code is much faster. There's a small improvement to divisible_numbers, but the main speedup lies in get_number_of_valid_partitions not creating a needless amount of temporary lists as it has now been joined with get_partitions in a single function. Other big speedups comes from using numba. The code of get_number_of_valid_partitions is all but unreadable now, so I've added a much simpler but slightly slower version named get_number_of_valid_partitions_simple so you can understand what is going on in the complicated function.
import numba
#numba.njit
def divisible_numbers(n):
"""
Get all numbers with which n is divisible.
Modified from·
"""
# We can save some time by only looking at
# values up to n/2
for i in range(4,n//2+1):
if n % i == 0:
yield i
yield n
def get_number_of_combinations(N):
"""
Get number of ways the equality can be solved under the given constraints
"""
out = 0
# Create a dictionary of number of valid partitions
# for all numerator values we can encounter
n_valid_partitions = {i: get_number_of_valid_partitions(i, N) for i in range(4,4*N+1)}
for numerator in range(4,4*N+1):
numerator_permutations = n_valid_partitions[numerator]
for denominator in divisible_numbers(numerator):
if denominator < 4:
continue
denominator_permutations = n_valid_partitions[denominator]
out += numerator_permutations * denominator_permutations
return out
#numba.njit
def get_number_of_valid_partitions(num, N):
"""
Get number of valid partitions of num, given that
num = i + j + 2l, and that 1<=i,j,l<=N.
"""
count = 0
# In the following, k = 2*l
#There's different cases for i,j,k that we can treat separately
# to give some speedup due to symmetry.
#i,j can be even or odd. k <= N or N < k <= 2N.
# Some combinations only possible if num is even/odd
# num is even
if num % 2 == 0:
# i,j odd, k <= 2N
k_min = max(2, num - 2 * (N - (N + 1) % 2))
k_max = min(2 * N, num - 2)
for k in range(k_min, k_max + 1, 2):
# only look at i<=j
i_min = max(1, num - k - N + (N + 1) % 2)
i_max = min(N, (num - k)//2)
for i in range(i_min, i_max + 1, 2):
j = num - i - k
# if i == j, only one permutations
# otherwise two due to symmetry
if i == j:
count += 1
else:
count += 2
# i,j even, k <= N
# only look at k<=i<=j
k_min = max(2, num - 2 * (N - N % 2))
k_max = min(N, num // 3)
for k in range(k_min, k_max + 1, 2):
i_min = max(k, num - k - N + N % 2)
i_max = min(N, (num - k) // 2)
for i in range(i_min, i_max + 1, 2):
j = num - i - k
if i == j == k:
# if i == j == k, only one permutation
count += 1
elif i == j or i == k or j == k:
# if only two of i,j,k are the same there are 3 permutations
count += 3
else:
# if all differ, there are six permutations
count += 6
# i,j even, N < k <= 2N
k_min = max(N + 1 + (N + 1) % 2, num - 2 * N)
k_max = min(2 * N, num - 4)
for k in range(k_min, k_max + 1, 2):
# only look for i<=j
i_min = max(2, num - k - N + 1 - (N + 1) % 2)
i_max = min(N, (num - k) // 2)
for i in range(i_min, i_max + 1, 2):
j = num - i - k
if i == j:
# if i == j, only one permutation
count += 1
else:
# if all differ, there are two permutations
count += 2
# num is odd
else:
# one of i,j is even, the other is odd. k <= N
# We assume that j is odd, k<=i and correct the symmetry in the counts
k_min = max(2, num - 2 * N + 1)
k_max = min(N, (num - 1) // 2)
for k in range(k_min, k_max + 1, 2):
i_min = max(k, num - k - N + 1 - N % 2)
i_max = min(N, num - k - 1)
for i in range(i_min, i_max + 1, 2):
j = num - i - k
if i == k:
# if i == j, two permutations
count += 2
else:
# if i and k differ, there are four permutations
count += 4
# one of i,j is even, the other is odd. N < k <= 2N
# We assume that j is odd and correct the symmetry in the counts
k_min = max(N + 1 + (N + 1) % 2, num - 2 * N + 1)
k_max = min(2 * N, num - 3)
for k in range(k_min, k_max + 1, 2):
i_min = max(2, num - k - N + (N + 1) % 2)
i_max = min(N, num - k - 1)
for i in range(i_min, i_max + 1, 2):
j = num - i - k
count += 2
return count
#numba.njit
def get_number_of_valid_partitions_simple(num, N):
"""
Simpler but slower version of 'get_number_of_valid_partitions'
"""
count = 0
for k in range(2, 2 * N + 1, 2):
for i in range(1, N + 1):
j = num - i - k
if 1 <= j <= N:
count += 1
return count
if __name__ == "__main__":
N = int(sys.argv[1])
out = get_number_of_combinations(N)
print(out)
The current issue with your code is that you've picked random numbers once, then calculate the same equation N times.
I assume you wanted to generate 1..N for each individual variable, which would require 6 nested loops from 1..N, for each variable
Now, that's the brute force solution, which probably fails on large N values, so as I commented, there's some trick to find the multiples of the right side of the modulo, then check if the left side portion is contained in that list. That would only require two triple nested lists, I think
(i + 2*j+ k) % (x + y + 2*z) = 0, where 1 <= i,j,k,x,y,z <= N
(2*j + i + k) is a multiple of (2*z + x + y)
N = 2
min(2*j + i + k) = 4
max(2*j + i + k) = 8
ways to make 4: 1 * 1 = 1
ways to make 5: 2 * 2 = 4
ways to make 6: 2 * 2 = 4
ways to make 7: 2 * 2 = 4
ways to make 8: 1 * 1 = 1
Total = 14
But 8 is a multiple of 4 so we add one more instance for a total of 15.
I've written a Timsort sorting algorithm for a computer science class, I would like to be able to compare the runtime to other similar algorithms, such as merge sort for instance. However, I am not sure where I should put the count (ie: count +=1)within the code to have an accurate run time. Any help would be much appreciated.
RUN = 32
def insertion_sort(arr, left, right):
for i in range(left + 1, right + 1):
temp = arr[i]
j = i - 1
while (arr[j] > temp and j >= left):
arr[j + 1] = arr[j]
arr[j] = temp
j -= 1
def merge(arr, left, right, count):
c = 0
index = count
length = len(left) + len(right)
while left and right:
if left[0] < right[0]:
arr[index] = left.pop(0)
c += 1
index += 1
else:
arr[index] = right.pop(0)
c += 1
index += 1
if len(left) == 0:
while c < length:
arr[index] = right.pop(0)
c += 1
index += 1
elif len(right) == 0:
while c < length:
arr[index] = left.pop(0)
c += 1
index += 1
def tim_sort(arr):
n = len(arr)
for i in range(0, n, RUN):
insertion_sort(arr, i, min((i + (RUN - 1)), (n - 1)))
size = RUN
while size < n:
for left in range(0, n, 2 * size):
if (left + size > n):
merge(arr, arr[left:n], [], left)
else:
left_sub_arr = arr[left:(left + size)]
right_sub_arr = arr[(left + size):min((left + 2 * size), n)]
merge(arr, left_sub_arr, right_sub_arr, left)
size *= 2
return arr
Is there a way to rewrite this code using only one loop to increase the speed while the input is very large numbers?
The code is used to count how many integers in a list are greater than all integers to the right of it.
count = 0
for i,x in enumerate(items):
d = True
for y in items[i+1:]:
if x <= y:
d = False
if d:
count = count+1
return count
The current value is larger than all the ones to the right if and only it is larger than the maximum one.
This code implements the above idea by iterating from right to left:
count = 0
max = None
for val in items[::-1]:
if max is None or val > max:
max = val
count += 1
return count
I timed some options to compare them:
The code from the question:
def f1(item_list):
count = 0
for i, x in enumerate(item_list):
d = True
for y in item_list[i+1:]:
if x <= y:
d = False
if d:
count = count+1
return count
The code from this answer from qwertyman:
def f2(item_list):
max_elem = None
count = 0
for val in item_list[::-1]:
if max_elem is None or val > max_elem:
max_elem = val
count += 1
return count
My improved version (just used reversed() instead of [::-1]):
def f3(item_list):
max_elem = None
count = 0
for val in reversed(item_list):
if max_elem is None or max_elem < val:
max_elem = val
count += 1
return count
The comparison code:
if __name__ == '__main__':
func_list = [f1, f2, f3]
print('{:>8s} {:15s} {:>10s} {:>10s} {:>10s}'.format(
'n', 'items', 'f1', 'f2', 'f3'))
for n in (100, 1000, 5000):
items_1 = [random.randint(1, 1000) for _ in range(n)]
items_2 = list(sorted(items_1))
items_3 = list(sorted(items_1, reverse=True))
for label, items in [
('random', items_1),
('sorted', items_2),
('sorted-reverse', items_3),
]:
# assure that all functions return the same result
assert len(set([func(items) for func in func_list])) == 1
t_list = []
for func in func_list:
t_list.append(
timeit.timeit(
'func(items)',
'from __main__ import func, items',
number=100))
print('{:8d} {:15s} {:10.6f} {:10.6f} {:10.6f}'.format(
n, label, *t_list))
The results (using Python 3.6 on Ubuntu 18.04):
n items f1 f2 f3
100 random 0.016022 0.000348 0.000370
100 sorted 0.015840 0.000339 0.000326
100 sorted-reverse 0.014122 0.000572 0.000505
1000 random 1.502731 0.003212 0.003077
1000 sorted 1.496299 0.003332 0.003089
1000 sorted-reverse 1.256896 0.005412 0.005196
5000 random 36.812474 0.015695 0.014762
5000 sorted 36.902378 0.015983 0.015067
5000 sorted-reverse 31.218129 0.019741 0.018419
Clearly, the proposal by qwertyman is orders of magnitude faster than the original code, and it can be sped up a little bit by using reversed() (obviously, for more speed one could use another language instead of Python).