Related
I'm trying to solve a Constraint Satisfaction Optimisation Problem that assigns agents to tasks. However, different then the basic Assignment Problem, a agent can be assigned to many tasks if the tasks do not overlap.
Each task has a fixed start_time and end_time.
The agents are assigned to the tasks according to some unary&binary constraints.
Variables = set of tasks
Domain = set of compatible agents (for each variable)
Constraints = unary&binary
Optimisation fct = some liniar function
An example of the problem: the allocation of parking space (or teams) for trucks for which we know the arrival and departure time.
I'm interested if there is in the literature a precise name for these type of problems. I presume it is some kind of assignment problem. Also, if you ever approach the problem, how do you solve it?
Thank you.
I would interpret this as: rectangular assignment-problem with conflicts
which is arguably much more hard (NP-hard in general) than the polynomially-solvable assignment-problem.
The demo shown in the other answer might work and ortools' cp-sat is great, but i don't see a good reason to use discrete-time based reasoning here like it's done: interval-variables, edge-finding and co. based scheduling constraints (+ conflict-analysis / explanations). This stuff is total overkill and the overhead will be noticable. I don't see any need to reason about time, but just about time-induced conflicts.
Edit: One could label those two approaches (linked + proposed) as compact formulation and extended formulation. Extended formulations usually show stronger relaxations and better (solving) results as long as scalability is not an issue. Compact approaches might become more viable again with bigger data (bit it's hard to guess here as scheduling-propagators are not that cheap).
What i would propose:
(1) Formulate an integer-programming model following the basic assignment-problem formulation + adaptions to make it rectangular -> a worker is allowed to tackle multiple tasks while all tasks are tackled (one sum-equality dropped)
see wiki
(2) Add integrality = mark variables as binary -> because the problem is not satisfying total unimodularity anymore
(3) Add constraints to forbid conflicts
(4) Add constraints: remaining stuff (e.g. compatibility)
Now this is all straightforward, but i would propose one non-naive improvement in regards to (3):
The conflicts can be interpreted as stable-set polytope
Your conflicts are induced by a-priori defined time-windows and their overlappings (as i interpret it; this is the core assumption behind this whole answer)
This is an interval graph (because of time-windows)
All interval graphs are chordal
Chordal graphs allow enumeration of all max-cliques in poly-time (implying there are only polynomial many)
python: networkx.algorithms.chordal.chordal_graph_cliques
The set (enumeration) of all maximal cliques define the facets of the stable-set polytope
Those (a constraint for each element in the set) we add as constraints!
(The stable-set polytope on the graph in use here would also allow very very powerful semidefinite-relaxations but it's hard to foresee in which cases this would actually help due to SDPs being much more hard to work with: warmstart within tree-search; scalability; ...)
This will lead to a poly-size integer-programming problem which should be very very good when using a good IP-solver (commercials or if open-source needed: Cbc > GLPK).
Small demo about (3)
import itertools
import networkx as nx
# data: inclusive, exclusive
# --------------------------
time_windows = [
(2, 7),
(0, 10),
(6, 12),
(12, 20),
(8, 12),
(16, 20)
]
# helper
# ------
def is_overlapping(a, b):
return (b[1] > a[0] and b[0] < a[1])
# raw conflicts
# -------------
binary_conflicts = []
for a, b in itertools.combinations(range(len(time_windows)), 2):
if is_overlapping(time_windows[a], time_windows[b]):
binary_conflicts.append( (a, b) )
# conflict graph
# --------------
G = nx.Graph()
G.add_edges_from(binary_conflicts)
# maximal cliques
# ---------------
max_cliques = nx.chordal_graph_cliques(G)
print('naive constraints: raw binary conflicts')
for i in binary_conflicts:
print('sum({}) <= 1'.format(i))
print('improved constraints: clique-constraints')
for i in max_cliques:
print('sum({}) <= 1'.format(list(i)))
Output:
naive constraints: raw binary conflicts
sum((0, 1)) <= 1
sum((0, 2)) <= 1
sum((1, 2)) <= 1
sum((1, 4)) <= 1
sum((2, 4)) <= 1
sum((3, 5)) <= 1
improved constraints: clique-constraints
sum([1, 2, 4]) <= 1
sum([0, 1, 2]) <= 1
sum([3, 5]) <= 1
Fun facts:
Commercial integer-programming solvers and maybe even Cbc might even try to do the same reasoning about clique-constraints to some degree although without the assumption of chordality where it's an NP-hard problem
ortools' cp-sat solver has also a code-path for this (again: general NP-hard case)
Should trigger when expressing the conflict-based model (much harder to decide on this exploitation on general discrete-time based scheduling models)
Caveats
Implementation / Scalability
There are still open questions like:
duplicating max-clique constraints over each worker vs. merging them somehow
be more efficient/clever in finding conflicts (sorting)
will it scale to the data: how big will the graph be / how many conflicts and constraints from those do we need
But those things usually follow instance-statistics (aka "don't decide blindly").
I don't know a name for the specific variant you're describing - maybe others would. However, this indeed seems a good fit for a CP/MIP solver; I would go with the OR-Tools CP-SAT solver, which is free, flexible and usually works well.
Here's a reference implementation with Python, assuming each vehicle requires a team assigned to it with no overlaps, and that the goal is to minimize the number of teams in use.
The framework allows to directly model allowed / forbidden assignments (check out the docs)
from ortools.sat.python import cp_model
model = cp_model.CpModel()
## Data
num_vehicles = 20
max_teams = 10
# Generate some (semi-)interesting data
interval_starts = [i % 9 for i in range(num_vehicles)]
interval_len = [ (num_vehicles - i) % 6 for i in range(num_vehicles)]
interval_ends = [ interval_starts[i] + interval_len[i] for i in range(num_vehicles)]
### variables
# t, v is true iff vehicle v is served by team t
team_assignments = {(t, v): model.NewBoolVar("team_assignments_%i_%i" % (t, v)) for t in range(max_teams) for v in range(num_vehicles)}
#intervals for vehicles. Each interval can be active or non active, according to team_assignments
vehicle_intervals = {(t, v): model.NewOptionalIntervalVar(interval_starts[v], interval_len[v], interval_ends[v], team_assignments[t, v], 'vehicle_intervals_%i_%i' % (t, v))
for t in range(max_teams) for v in range(num_vehicles)}
team_in_use = [model.NewBoolVar('team_in_use_%i' % (t)) for t in range(max_teams)]
## constraints
# non overlap for each team
for t in range(max_teams):
model.AddNoOverlap([vehicle_intervals[t, v] for v in range(num_vehicles)])
# each vehicle must be served by exactly one team
for v in range(num_vehicles):
model.Add(sum(team_assignments[t, v] for t in range(max_teams)) == 1)
# what teams are in use?
for t in range(max_teams):
model.AddMaxEquality(team_in_use[t], [team_assignments[t, v] for v in range(num_vehicles)])
#symmetry breaking - use teams in-order
for t in range(max_teams-1):
model.AddImplication(team_in_use[t].Not(), team_in_use[t+1].Not())
# let's say that the goal is to minimize the number of teams required
model.Minimize(sum(team_in_use))
solver = cp_model.CpSolver()
# optional
# solver.parameters.log_search_progress = True
# solver.parameters.num_search_workers = 8
# solver.parameters.max_time_in_seconds = 5
result_status = solver.Solve(model)
if (result_status == cp_model.INFEASIBLE):
print('No feasible solution under constraints')
elif (result_status == cp_model.OPTIMAL):
print('Optimal result found, required teams=%i' % (solver.ObjectiveValue()))
elif (result_status == cp_model.FEASIBLE):
print('Feasible (non optimal) result found')
else:
print('No feasible solution found under constraints within time')
# Output:
#
# Optimal result found, required teams=7
EDIT:
#sascha suggested a beautiful approach for analyzing the (known in advance) time window overlaps, which would make this solvable as an assignment problem.
So while the formulation above might not be the optimal one for this (although it could be, depending on how the solver works), I've tried to replace the no-overlap conditions with the max-clique approach suggested - full code below.
I did some experiments with moderately large problems (100 and 300 vehicles), and it seems empirically that on smaller problems (~100) this does improve by some - about 15% on average on the time to optimal solution; but I could not find a significant improvement on the larger (~300) problems. This might be either because my formulation is not optimal; because the CP-SAT solver (which is also a good IP solver) is smart enough; or because there's something I've missed :)
Code:
(this is basically the same code from above, with the logic to support using the network approach instead of the no-overlap one copied from #sascha's answer):
from timeit import default_timer as timer
from ortools.sat.python import cp_model
model = cp_model.CpModel()
run_start_time = timer()
## Data
num_vehicles = 300
max_teams = 300
USE_MAX_CLIQUES = True
# Generate some (semi-)interesting data
interval_starts = [i % 9 for i in range(num_vehicles)]
interval_len = [ (num_vehicles - i) % 6 for i in range(num_vehicles)]
interval_ends = [ interval_starts[i] + interval_len[i] for i in range(num_vehicles)]
if (USE_MAX_CLIQUES):
## Max-cliques analysis
# for the max-clique approach
time_windows = [(interval_starts[i], interval_ends[i]) for i in range(num_vehicles)]
def is_overlapping(a, b):
return (b[1] > a[0] and b[0] < a[1])
# raw conflicts
# -------------
binary_conflicts = []
for a, b in itertools.combinations(range(len(time_windows)), 2):
if is_overlapping(time_windows[a], time_windows[b]):
binary_conflicts.append( (a, b) )
# conflict graph
# --------------
G = nx.Graph()
G.add_edges_from(binary_conflicts)
# maximal cliques
# ---------------
max_cliques = nx.chordal_graph_cliques(G)
##
### variables
# t, v is true iff point vehicle v is served by team t
team_assignments = {(t, v): model.NewBoolVar("team_assignments_%i_%i" % (t, v)) for t in range(max_teams) for v in range(num_vehicles)}
#intervals for vehicles. Each interval can be active or non active, according to team_assignments
vehicle_intervals = {(t, v): model.NewOptionalIntervalVar(interval_starts[v], interval_len[v], interval_ends[v], team_assignments[t, v], 'vehicle_intervals_%i_%i' % (t, v))
for t in range(max_teams) for v in range(num_vehicles)}
team_in_use = [model.NewBoolVar('team_in_use_%i' % (t)) for t in range(max_teams)]
## constraints
# non overlap for each team
if (USE_MAX_CLIQUES):
overlap_constraints = [list(l) for l in max_cliques]
for t in range(max_teams):
for l in overlap_constraints:
model.Add(sum(team_assignments[t, v] for v in l) <= 1)
else:
for t in range(max_teams):
model.AddNoOverlap([vehicle_intervals[t, v] for v in range(num_vehicles)])
# each vehicle must be served by exactly one team
for v in range(num_vehicles):
model.Add(sum(team_assignments[t, v] for t in range(max_teams)) == 1)
# what teams are in use?
for t in range(max_teams):
model.AddMaxEquality(team_in_use[t], [team_assignments[t, v] for v in range(num_vehicles)])
#symmetry breaking - use teams in-order
for t in range(max_teams-1):
model.AddImplication(team_in_use[t].Not(), team_in_use[t+1].Not())
# let's say that the goal is to minimize the number of teams required
model.Minimize(sum(team_in_use))
solver = cp_model.CpSolver()
# optional
solver.parameters.log_search_progress = True
solver.parameters.num_search_workers = 8
solver.parameters.max_time_in_seconds = 120
result_status = solver.Solve(model)
if (result_status == cp_model.INFEASIBLE):
print('No feasible solution under constraints')
elif (result_status == cp_model.OPTIMAL):
print('Optimal result found, required teams=%i' % (solver.ObjectiveValue()))
elif (result_status == cp_model.FEASIBLE):
print('Feasible (non optimal) result found, required teams=%i' % (solver.ObjectiveValue()))
else:
print('No feasible solution found under constraints within time')
print('run time: %.2f sec ' % (timer() - run_start_time))
total number of ways to reach the nth floor with following types of moves:
Type 1 in a single move you can move from i to i+1 floor – you can use the this move any number of times
Type 2 in a single move you can move from i to i+2 floor – you can use this move any number of times
Type 3 in a single move you can move from i to i+3 floor – but you can use this move at most k times
i know how to reach nth floor by following step 1 ,step 2, step 3 any number of times using dp like dp[i]=dp[i-1]+dp[i-2]+dp[i-3].i am stucking in the condition of Type 3 movement with atmost k times.
someone tell me the approach here.
While modeling any recursion or dynamic programming problem, it is important to identify the goal, constraints, states, state function, state transitions, possible state variables and initial condition aka base state. Using this information we should try to come up with a recurrence relation.
In our current problem:
Goal: Our goal here is to somehow calculate number of ways to reach floor n while beginning from floor 0.
Constraints: We can move from floor i to i+3 at most K times. We name it as a special move. So, one can perform this special move at most K times.
State: In this problem, our situation of being at a floor could be one way to model a state. The exact situation can be defined by the state variables.
State variables: State variables are properties of the state and are important to identify a state uniquely. Being at a floor i alone is not enough in itself as we also have a constraint K. So to identify a state uniquely we want to have 2 state variables: i indicating floor ranging between 0..n and k indicating number of special move used out of K (capital K).
State functions: In our current problem, we are concerned with finding number of ways to reach a floor i from floor 0. We only need to define one function number_of_ways associated with corresponding state to describe the problem. Depending on problem, we may need to define more state functions.
State Transitions: Here we identify how can we transition between states. We can come freely to floor i from floor i-1 and floor i-2 without consuming our special move. We can only come to floor i from floor i-3 while consuming a special move, if i >=3 and special moves used so far k < K.
In other words, possible state transitions are:
state[i,k] <== state[i-1,k] // doesn't consume special move k
state[i,k] <== state[i-2,k] // doesn't consume special move k
state[i,k+1] <== state[i-3, k] if only k < K and i >= 3
We should now be able to form following recurrence relation using above information. While coming up with a recurrence relation, we must ensure that all the previous states needed for computation of current state are computed first. We can ensure the order by computing our states in the topological order of directed acyclic graph (DAG) formed by defined states as its vertices and possible transitions as directed edges. It is important to note that it is only possible to have such ordering if the directed graph formed by defined states is acyclic, otherwise we need to rethink if the states are correctly defined uniquely by its state variables.
Recurrence Relation:
number_of_ways[i,k] = ((number_of_ways[i-1,k] if i >= 1 else 0)+
(number_of_ways[i-2,k] if i >= 2 else 0) +
(number_of_ways[i-3,k-1] if i >= 3 and k < K else 0)
)
Base cases:
Base cases or solutions to initial states kickstart our recurrence relation and are sufficient to compute solutions of remaining states. These are usually trivial cases or smallest subproblems that can be solved without recurrence relation.
We can have as many base conditions as we require and there is no specific limit. Ideally we would want to have a minimal set of base conditions, enough to compute solutions of all remaining states. For the current problem, after initializing all not computed solutions so far as 0,
number_of_ways[0, 0] = 1
number_of_ways[0,k] = 0 where 0 < k <= K
Our required final answer will be sum(number_of_ways[n,k], for all 0<=k<=K).
You can use two-dimensional dynamic programming:
dp[i,j] is the solution value when exactly j Type-3 steps are used. Then
dp[i,j]=dp[i-1,j]+dp[i-2,j]+dp[i-3,j-1], and the initial values are dp[0,0]=0, dp[1,0]=1, and dp[3*m,m]=m for m<=k. You can build up first the d[i,0] values, then the d[i,1] values, etc. Or you can do a different order, as long as all necessary values are already computed.
Following #LaszloLadanyi approach ,below is the code snippet in python
def solve(self, n, k):
dp=[[0 for i in range(k+1)]for _ in range(n+1)]
dp[0][0]=1
for j in range(k+1):
for i in range(1,n+1):
dp[i][j]+=dp[i-1][j]
if i>1:
dp[i][j]+=dp[i-2][j]
if i>2 and j>0:
dp[i][j]+=dp[i-3][j-1]
return sum(dp[n])
While reading the syntax of Verilog, I came across the four logic values: 0 1 x z.
After searching the web, seeking to find the difference between x and z, I found only that x is unknown value and z is high impedance (tristate). I think that I understand the definition of x but didn't quite understood the one of z - what does it mean "high impedance (tristate)"?
I would like to see an example for each logic value out of the two: x z
Z means the signal is in a high-impedance state also called tri-state. Another signal connected to it can change the value: a 0 will pull it low, a 1 will pull it high.
To understand impedance (and thus high impedance) you should have some understanding of resistance, voltage and current and their relations as defined by Ohms law.
I can't give you an example of 'X' or 'Z', just as I can't give you an example of '1' or '0'. These are just definitions of signal states. In fact in Verilog there are more then four states. There are seven strengths.
(See this webpage).
Here is a principle diagram of how a chip output port makes a zero, one or Z. In reality the switches are MOSFETs.
Tri-state signals are no longer used inside chips or inside FPGA's. They are only used outside for connecting signals together.
x, as you had already found describes an unknown state. By default verilog simulation starts with all variables initialized to this value. One of the task of the designer is to provide correct reset sequences to bring the model into a known state, without 'x', i.e.
always #(posedge clk)
if (rst)
q <= 0;
In the above example initial value of q which was x is replaced by a known value of 0.
The difference between 'x' and 'z' is that 'z' is a known state of high impedance, meaning actually disconnected. As such, it could be driven to any other value with some other driver. It is used to express tri-state buses or some other logic.
wire bus;
assign bus = en1 ? value1 : 1'bz;
...
assign bus = en2 ? value2 : 1'bz;
In the above example the bus is driven by 2 different drivers. If 'en1' or 'en2' is high, the bus is driven with a real 'value1' or 'value2'. Otherwise its state is 'z'.
verilog has truth tables for every operator for all the values. You can check how they are used. i.e. for '&'
& 0 1 x z
0 0 0 0 0
1 0 1 x x
x 0 x x x
z 0 x x x
you can find for every other gate as well. Note that there are no 'z' in the result, just 'x's.
In system verilog X is treated like unconnected wire and Z is Weak HIGH.
Suppose a situation where you have wire connecting 2 modules m1 and m2.
If you are driving Z onto that wire from m1 then you can pull down this wire by assigning it to zero by m2.
As I figured out :
"tristate" or "high impedance" In transistors occures when you have "nothing" in the output.
that may occur, for example :
In a situation that you have an nMOS transistor let's call that T1:
the gate value of T1 is for example 0
so T1 would not conduct and there is no conduction path between your supply (probably 0 ) and the drain(output)
-that may occur a "Z" or tristate
--
It may occur for PMOS transistors with value -> 1 too.
It seems that Promela initialises each variable (by default, to 0, or to the value that is given in the declaration).
How can I declare a variable that is initialised by an unknown value?
The documentation suggests if :: p = 0 :: p = 1 fi but I don't think
that it works: Spin still verifies this claim
bit p
init { if :: p = 0 :: p = 1 fi }
ltl { ! p }
(and falsifies p)
So what exactly is the semantics of init? There still is some
"pre-initial" state? How can I work around this - and not confuse my students?
This is an interesting question.
The documentation says that each and every variable is initialised to 0, unless the model specifies otherwise.
As with all variable declarations, an explicit initialization field is optional. The default initial value for all variables is zero. This applies both to scalar variables and to array variables, and it applies to both global and to local variables.
In your model, you don't initialise the variable when you declare it, therefore it is subsequently assigned to the value 0 in the initial state, which is located before your assignment:
bit p
init {
// THE INITIAL STATE IS HERE
if
:: p = 0
:: p = 1
fi
}
ltl { ! p }
Some Experiment.
A "naive" idea for dodging this limitation would be to modify the c source code of pan.c that is generated by spin when you invoke ~$ spin -a test.pml, so that the variable is initialised at random.
Instead of this initialisation function:
void
iniglobals(int calling_pid)
{
now.p = 0;
#ifdef VAR_RANGES
logval("p", now.p);
#endif
}
one could try writing this:
void
iniglobals(int calling_pid)
{
srand(time(NULL));
now.p = rand() % 2;
#ifdef VAR_RANGES
logval("p", now.p);
#endif
}
and adding an #include <time.h> in the header part.
However, once you compile that into a verifier with gcc pan.c, and you attempt to run it, you obtain non-deterministic behaviour depending on the initialization value of the variable p.
It can both determine that the property is violated:
~$ ./a.out -a
pan:1: assertion violated !( !( !(p))) (at depth 0)
pan: wrote test.pml.trail
(Spin Version 6.4.3 -- 16 December 2014)
Warning: Search not completed
+ Partial Order Reduction
Full statespace search for:
never claim + (ltl_0)
assertion violations + (if within scope of claim)
acceptance cycles + (fairness disabled)
invalid end states - (disabled by never claim)
State-vector 28 byte, depth reached 0, errors: 1
1 states, stored
0 states, matched
1 transitions (= stored+matched)
0 atomic steps
hash conflicts: 0 (resolved)
Stats on memory usage (in Megabytes):
0.000 equivalent memory usage for states (stored*(State-vector + overhead))
0.291 actual memory usage for states
128.000 memory used for hash table (-w24)
0.534 memory used for DFS stack (-m10000)
128.730 total actual memory usage
pan: elapsed time 0 seconds
or print that the property is satisfied:
~$ ./a.out -a
(Spin Version 6.4.3 -- 16 December 2014)
+ Partial Order Reduction
Full statespace search for:
never claim + (ltl_0)
assertion violations + (if within scope of claim)
acceptance cycles + (fairness disabled)
invalid end states - (disabled by never claim)
State-vector 28 byte, depth reached 0, errors: 0
1 states, stored
0 states, matched
1 transitions (= stored+matched)
0 atomic steps
hash conflicts: 0 (resolved)
Stats on memory usage (in Megabytes):
0.000 equivalent memory usage for states (stored*(State-vector + overhead))
0.291 actual memory usage for states
128.000 memory used for hash table (-w24)
0.534 memory used for DFS stack (-m10000)
128.730 total actual memory usage
unreached in init
test.pml:8, state 5, "-end-"
(1 of 5 states)
unreached in claim ltl_0
_spin_nvr.tmp:8, state 8, "-end-"
(1 of 8 states)
pan: elapsed time 0 seconds
Clearly, the initial state of a promela model verified by spin is assumed to be unique. Afterall, that's a reasonable assumption, since it would needlessly complicate things: you can always replace N different initial states S_i with an initial state S s.t. S allows to reach each S_i with an epsilon-transition. In this context, what you get is not truly an epsilon-transition, but in practice it makes little difference.
EDIT (from comments):
In principle, it is possible to make this work by modifying pan.c a little bit further:
transform the initial state initialiser into a generator of initial states
modify the verification routine to take into account that more than one initial state might exist, and that the property must hold for each initial state
Having said this, it is likely not worth the hassle, unless this is done by patching Spin's source code.
Workaround.
If you want to state that something is true in the initial state, or starting from the initial state, and take into account some non-deterministic behaviour, then you should write something as follows:
bit p
bool init_state = false
init {
if
:: p = 0
:: p = 1
fi
init_state = true // TARGET STATE
init_state = false
}
ltl { init_state & ! p }
with which you get:
~$ ./a.out -a
pan:1: assertion violated !( !((initialised& !(p)))) (at depth 0)
pan: wrote 2.pml.trail
(Spin Version 6.4.3 -- 16 December 2014)
Warning: Search not completed
+ Partial Order Reduction
Full statespace search for:
never claim + (ltl_0)
assertion violations + (if within scope of claim)
acceptance cycles + (fairness disabled)
invalid end states - (disabled by never claim)
State-vector 28 byte, depth reached 0, errors: 1
1 states, stored
0 states, matched
1 transitions (= stored+matched)
0 atomic steps
hash conflicts: 0 (resolved)
Stats on memory usage (in Megabytes):
0.000 equivalent memory usage for states (stored*(State-vector + overhead))
0.291 actual memory usage for states
128.000 memory used for hash table (-w24)
0.534 memory used for DFS stack (-m10000)
128.730 total actual memory usage
pan: elapsed time 0 seconds
Init Semantics.
Init is simply guaranteed to be the first process to spawn, and is meant to be used for spawning other processes when, for-example, the other routines take as input some parameters, e.g. some resources are shared. More info here.
I believe that this fragment of documentation is a bit misleading:
The init process is most commonly used to initialize global variables,
and to instantiate other processes, through the use of the run
operator, before system execution starts. Any process, not just the
init process, can do so, though
Since it is possible to guarantee that the init process executes all of its code before any other process using the atomic { } statement, one could say that it can be used to initialize variables before they are used by other processes from the programming point of view. But that is just a rough approximation, because the init process does not correspond to a unique state in the execution model, but rather to the tree of states at the root and the root itself is given only by the global environment as it is before any process starts.
Purely Functional Data Structures has the following exercise:
-- 2.5 Sharing can be useful within a single object, not just between objects.
-- For example, if the two subtress of a given node are identical, then they can
-- be represented by the same tree.
-- Part a: make a `complete a Int` function that creates a tree of
-- depth Int, putting a in every leaf of the tree.
complete :: a -> Integer -> Maybe (Tree a)
complete x depth
| depth < 0 = Nothing
| otherwise = Just $ complete' depth
where complete' d
| d == 0 = Empty
| otherwise = let copiedTree = complete' (d-1)
in Node x copiedTree copiedTree
Does this implementation run in O(d) time? Could you please say why or why not?
The interesting part of the code is the complete' function:
complete' d
| d == 0 = Empty
| otherwise = let copiedTree = complete' (d-1)
in Node x copiedTree copiedTree
As Cirdec's answer suggests, we should be careful to analyze each part of the implementation to make sure our assumptions are valid. As a general rule, we can assume that the following take 1 unit of time each*:
Using a data constructor to construct a value (e.g., using Empty to make an empty tree or using Node to turn a value and two trees into a tree).
Pattern matching on a value to see what data constructor it was built from and what values the data constructor was applied to.
Guards and if/then/else expressions (which are implemented internally using pattern matching).
Comparing an Integer to 0.
Cirdec mentions that the operation of subtracting 1 from an Integer is logarithmic in the size of the integer. As they say, this is essentially an artifact of the way Integer is implemented. It is possible to implement integers so that it takes only one step to compare them to 0 and also takes only one step to decrement them by 1. To keep things very general, it's safe to assume that there is some function c such that the cost of decrementing an Integer is c(depth).
Now that we've taken care of those preliminaries, let's get down to work! As is generally the case, we need to set up a system of equations and solve it. Let f(d) be the number of steps needed to calculate complete' d. Then the first equation is very simple:
f(0) = 2
That's because it costs 1 step to compare d to 0, and another step to check that the result is True.
The other equation is the interesting part. Think about what happens when d > 0:
We calculate d == 0.
We check if that is True (it's not).
We calculate d-1 (let's call the result dm1)
We calculate complete' dm1, saving the result as copiedTree.
We apply a Node constructor to x, copiedTree, and copiedTree.
The first part takes 1 step. The second part takes one step. The third part takes c(depth) steps, and the fifth step takes 1 step. What about the fourth part? Well, that takes f(d-1) steps, so this will be a recursive definition.
f(0) = 2
f(d) = (3+c(depth)) + f(d-1) when d > 0
OK, now we're cooking with gas! Let's calculate the first few values of f:
f(0) = 2
f(1) = (3+c(depth)) + f(0) = (3+c(depth)) + 2
f(2) = (3+c(depth)) + f(1)
= (3+c(depth)) + ((3+c(depth)) + 2)
= 2*(3+c(depth)) + 2
f(3) = (3+c(depth)) + f(2)
= (3+c(depth)) + (2*(3+c(depth)) + 2)
= 3*(3+c(depth)) + 2
You should be starting to see a pattern by now:
f(d) = d*(3+c(depth)) + 2
We generally prove things about recursive functions using mathematical induction.
Base case:
The claim holds for d=0 because 0*(3+c(depth))+2=0+2=2=f(0).
Suppose that the claim holds for d=D. Then
f(D+1) = (3+c(depth)) + f(D)
= (3+c(depth)) + (D*(3+c(depth))+2)
= (D+1)*(3+c(depth))+2
So the claim holds for D+1 as well. Thus by induction, it holds for all natural numbers d. As a reminder, this gives the conclusion that complete' d takes
f(d) = d*(3+c(depth))+2
time. Now how do we express that in big O terms? Well, big O doesn't care about the constant coefficients of any of the terms, and only cares about the highest-order terms. We can safely assume that c(depth)>=1, so we get
f(d) ∈ O(d*c(depth))
Zooming out to complete, this looks like O(depth*c(depth))
If you use the real cost of Integer decrement, this gives you O(depth*log(depth)). If you pretend that Integer decrement is O(1), this gives you O(depth).
Side note: As you continue to work through Okasaki, you will eventually reach section 10.2.1, where you will see a way to implement natural numbers supporting O(1) decrement and O(1) addition (but not efficient subtraction).
* Haskell's lazy evaluation keeps this from being precisely true, but if you pretend that everything is evaluated strictly, you will get an upper bound for the true value, which will be good enough in this case. If you want to learn how to analyze data structures that use laziness to get good asymptotic bounds, you should keep reading Okasaki.
Theoretical Answer
No, it does not run in O(d) time. Its asymptotic performance is dominated by the the Integer subtraction d-1, which takes O(log d) time. This is repeated O(d) times, giving an asymptotic upper bound on time of O(d log d).
This upper bound can improve if you use an Integer representation with an asymptotically optimal O(1) decrement. In practice we don't, since the asymptotically optimal Integer implementations are slower even for unimaginably large values.
Practically the Integer arithmetic will be a small part of the running time of the program. For practical "large" depths (smaller than a machine word) the program's running time will be dominated by allocating and populating memory. For larger depths you will exhaust the resources of the computer.
Practical Answer
Ask the run time system's profiler.
In order to profile your code, we first need to make sure it is run. Haskell is lazily evaluated, so, unless we do something to cause the tree to be completely evaluated, it might not be. Unfortunately, completely exploring the tree will take O(2^d) steps. We could avoid forcing nodes we had already visited if we kept track of their StableNames. Fortunately, traversing a structure and keeping track of visited nodes by their memory locations is already provided by the data-reify package. Since we will be using it for profiling, we need to install it with profiling enabled (-p).
cabal install -p data-reify
Using Data.Reify requires the TypeFamilies extension and Control.Applicative.
{-# LANGUAGE TypeFamilies #-}
import Data.Reify
import Control.Applicative
We reproduce your Tree code.
data Tree a = Empty | Node a (Tree a) (Tree a)
complete :: a -> Integer -> Maybe (Tree a)
complete x depth
| depth < 0 = Nothing
| otherwise = Just $ complete' depth
where complete' d
| d == 0 = Empty
| otherwise = let copiedTree = complete' (d-1)
in Node x copiedTree copiedTree
Converting data to a graph with data-reify requires that we have a base functor for the data type. The base functor is a representation of the type with explicit recursion removed. The base functor for Tree is TreeF. An additional type parameter is added for the representation of recursive occurrence of the type, and each recursive occurrence is replaced by the new parameter.
data TreeF a x = EmptyF | NodeF a x x
deriving (Show)
The MuRef instance required by reifyGraph requires that we provide a mapDeRef to traverse the structure with an Applicative and convert it to the base functor . The first argument provided to mapDeRef, which I have named deRef, is how we can convert the recursive occurrences of the structure.
instance MuRef (Tree a) where
type DeRef (Tree a) = TreeF a
mapDeRef deRef Empty = pure EmptyF
mapDeRef deRef (Node a l r) = NodeF a <$> deRef l <*> deRef r
We can make a little program to run to test the complete function. When the graph is small, we'll print it out to see what's going on. When the graph gets big, we'll only print out how many nodes it has.
main = do
d <- getLine
let (Just tree) = complete 0 (read d)
graph#(Graph nodes _) <- reifyGraph tree
if length nodes < 30
then print graph
else print (length nodes)
I put this code in a file named profileSymmetricTree.hs. To compile it, we need to enable profiling with -prof and enable the run-time system with -rtsopts.
ghc -fforce-recomp -O2 -prof -fprof-auto -rtsopts profileSymmetricTree.hs
When we run it, we'll enable the time profile with the +RTS option -p. We'll give it the depth input 3 for the first run.
profileSymmetricTree +RTS -p
3
let [(1,NodeF 0 2 2),(2,NodeF 0 3 3),(3,NodeF 0 4 4),(4,EmptyF)] in 1
We can already see from the graph that the nodes are being shared between the left and right sides of the tree.
The profiler makes a file, profileSymmetricTree.prof.
individual inherited
COST CENTRE MODULE no. entries %time %alloc %time %alloc
MAIN MAIN 43 0 0.0 0.7 100.0 100.0
main Main 87 0 100.0 21.6 100.0 32.5
...
main.(...) Main 88 1 0.0 4.8 0.0 5.1
complete Main 90 1 0.0 0.0 0.0 0.3
complete.complete' Main 92 4 0.0 0.2 0.0 0.3
complete.complete'.copiedTree Main 94 3 0.0 0.1 0.0 0.1
It shows in the entries column that complete.complete' was executed 4 times, and the complete.complete'.copiedTree was evaluated 3 times.
If you repeat this experiment with different depths, and plot the results, you should get a good idea what the practical asymptotic performance of complete is.
Here are the profiling results for a much greater depth, 300000.
individual inherited
COST CENTRE MODULE no. entries %time %alloc %time %alloc
MAIN MAIN 43 0 0.0 0.0 100.0 100.0
main Main 87 0 2.0 0.0 99.9 100.0
...
main.(...) Main 88 1 0.0 0.0 2.1 5.6
complete Main 90 1 0.0 0.0 2.1 5.6
complete.complete' Main 92 300001 1.3 4.4 2.1 5.6
complete.complete'.copiedTree Main 94 300000 0.8 1.3 0.8 1.3