Pyomo: Param and Var not constructed when placed in a list - python-3.x

Question

While modeling an optimisation problem using pyomo I noticed a weird behaviour when using a list of Var or Param: I always get the following error ValueError: Evaluating the numeric value of parameter 'SimpleParam' before the Param has been constructed (there is currently no value to return).
The following code (minise 4*x+1 such that x >= 0) runs exactly as expected:
import pyomo.environ as pyo
from pyomo.opt import SolverFactory
def _obj(model):
return model.c*model.x + 1
model = pyo.ConcreteModel()
model.x = pyo.Var(domain=pyo.NonNegativeReals)
model.c = pyo.Param(initialize=lambda model: 4, domain=pyo.NonNegativeReals)
model.obj = pyo.Objective(rule=_obj, sense=pyo.minimize)
opt = SolverFactory('glpk')
opt.solve(model)
but as I set model.x and model.c in lists, the program crashes when creating the objective function:
import pyomo.environ as pyo
from pyomo.opt import SolverFactory
def _obj(model):
return model.c[0]*model.x[0] + 1
model = pyo.ConcreteModel()
model.x = [pyo.Var(domain=pyo.NonNegativeReals)]
model.c = [pyo.Param(initialize=lambda model: 4, domain=pyo.NonNegativeReals)]
model.obj = pyo.Objective(rule=_obj, sense=pyo.minimize)
opt = SolverFactory('glpk')
opt.solve(model)
What is causing this error? Is this a desired behaviour for a reason that I don't understand or is this a bug? Anyway, how can I use lists of Params and Vars in a problem? I know that I can theoretically flatten all of my parameters and variables into a single IndexedVar or IndexedParam and handle the new indices myself, but that would be tedious since the range of the 3rd and 4th indices of my x and c depend on the 1st and 2nd index, therefore it would be a lot clearer in my code if I could use lists.
More precisely: I have a code looking like this (though I am still interested in knowning why the MWE above does not work):
# I, J are lists of indices and N is a list of integer values
model.Vs = [pyo.RangeSet(N[i]) for i in range(len(N))]
model.xs = [[pyo.Var(model.Vs[i], model.Vs[j]) for j in J] for i in I]
model.cs = [[pyo.Param(model.Vs[i], model.Vs[j]) for j in J] for i in I]
def _obj(model):
sum(model.xs[i][j][k,ell] * model.xs[i][j][k,ell] \\
for i in I for j in J \\
for k in model.Vs[i] for ell in model.Vs[j])
model.obj = Objective(rule=_obj, sense=pyo.minimize)
model.constraints = [
[pyo.Constraint(model.Vs[i], model.Vs[j], rule=...) for j in J]
for i in I
]
opt = SolverFactory('glpk')
opt.solve(model)

Answer 1

Your minimal example
import pyomo.environ as pyo
from pyomo.opt import SolverFactory
def _obj(model):
return model.c[0]*model.x[0] + 1
model = pyo.ConcreteModel()
model.x = [pyo.Var(domain=pyo.NonNegativeReals)]
model.c = [pyo.Param(initialize=lambda model: 4, domain=pyo.NonNegativeReals)]
model.obj = pyo.Objective(rule=_obj, sense=pyo.minimize)
opt = SolverFactory('glpk')
opt.solve(model)
generates the following error:
ValueError: Evaluating the numeric value of parameter 'SimpleParam' before
the Param has been constructed (there is currently no value to return).
The reason is because you are not directly attaching the Var and Param you are generating to the model. A lot happens when you attach a Pyomo modeling component to a Block (ConcreteModel objects are instances of constructed Blocks):
The component is assigned a name (that matches the Block attribute name)
The component is inserted into the hierarchy (basically, pointers are set so that methods can walk up and down the model hierarchy)
The component is categorized so that writers, solvers, and transformations can find it later
(If the Block has already been constructed), the component is automatically constructed.
By placing the component in a list, you are effectively "hiding" its existence from Pyomo. The first error you get has to do with this last bullet (the Param hasn't been constructed). However, simply constructing the Param and Var as you build the list will be insufficient, as the other actions won't take place and you will just hit a different error later (the next error would be an obsure one when the LP writer comes across a Var in the Objective that it had not found when it first walked the model hierarchy).

Answer 2

Perhaps this will help. I'm not sure I can answer why your example fails other than to say that pyomo is a modeling language that passes a structured math problem to a solver and the sets need to be discretely defined, not in lists of objects. Maybe somebody else can pitch in and explain it more clearly.
In your modeling, it appears you want to construct some kind of ragged set for x[i,j] where the range of j can vary based on i. You typically want to make sets for both I and J in order to support various constraint constructs. Then you can make a subset of "valid" (i, j) tuples for whatever model component needs to be indexed by this ragged set. You can either use this subset as the basis of iteration or use it to check membership if you are constructing things on-the-fly.
Here is an example using your list N:
import pyomo.environ as pyo
N = [1, 4, 3]
m = pyo.ConcreteModel()
m.I = pyo.Set(initialize=range(len(N)))
m.J = pyo.Set(initialize=range(max(N)))
m.IJ = pyo.Set(within=m.I * m.J, initialize =
[(i, j) for i in range(len(N)) for j in range(N[i])])
m.x = pyo.Var(m.IJ, domain=pyo.NonNegativeReals)
def _obj(model):
return sum(m.x[t] for t in m.IJ)
m.obj = pyo.Objective(rule=_obj)
def constrain_x2(model):
return sum(m.x[2, j] for j in m.J if (2, j) in m.IJ) >=1
m.c1 = pyo.Constraint(rule=constrain_x2)
m.pprint()
Yields:
4 Set Declarations
I : Dim=0, Dimen=1, Size=3, Domain=None, Ordered=False, Bounds=(0, 2)
[0, 1, 2]
IJ : Dim=0, Dimen=2, Size=8, Domain=IJ_domain, Ordered=False, Bounds=None
[(0, 0), (1, 0), (1, 1), (1, 2), (1, 3), (2, 0), (2, 1), (2, 2)]
IJ_domain : Dim=0, Dimen=2, Size=12, Domain=None, Ordered=False, Bounds=None
Virtual
J : Dim=0, Dimen=1, Size=4, Domain=None, Ordered=False, Bounds=(0, 3)
[0, 1, 2, 3]
1 Var Declarations
x : Size=8, Index=IJ
Key : Lower : Value : Upper : Fixed : Stale : Domain
(0, 0) : 0 : None : None : False : True : NonNegativeReals
(1, 0) : 0 : None : None : False : True : NonNegativeReals
(1, 1) : 0 : None : None : False : True : NonNegativeReals
(1, 2) : 0 : None : None : False : True : NonNegativeReals
(1, 3) : 0 : None : None : False : True : NonNegativeReals
(2, 0) : 0 : None : None : False : True : NonNegativeReals
(2, 1) : 0 : None : None : False : True : NonNegativeReals
(2, 2) : 0 : None : None : False : True : NonNegativeReals
1 Objective Declarations
obj : Size=1, Index=None, Active=True
Key : Active : Sense : Expression
None : True : minimize : x[0,0] + x[1,0] + x[1,1] + x[1,2] + x[1,3] + x[2,0] + x[2,1] + x[2,2]
1 Constraint Declarations
c1 : Size=1, Index=None, Active=True
Key : Lower : Body : Upper : Active
None : 1.0 : x[2,0] + x[2,1] + x[2,2] : +Inf : True
7 Declarations: I J IJ_domain IJ x obj c1