I have something like this:
x=T.matrix('x')
params = [self.W, self.b1, self.b2]
hidden = self.activation_function(T.dot(x, self.W)+self.b1)
output = T.dot(hidden,T.transpose(self.W))+self.b2
output = self.output_function(output)
L = -T.sum(x*T.log(output) + (1-x)*T.log(1-output), axis=1)
cost=L.mean()
th_train = th.function(inputs=[index], outputs=[cost], updates=updates,
givens={x:self.X[index:index+mini_batch_size,:]})
This is working fine. I would now like to see what the mean of the hidden units is. I tried adding this before the line where L = -T.sum... is declared:
hm = T.mean(hidden)
hidden_mean_func = th.function(inputs=[hm], outputs=[hm], name="hidden_mean_function_printer")
print hidden_mean_func(hm)
I get the following error:
TypeError: ('Bad input argument to theano function with name "hidden_mean_function_printer" at index 0(0-based)', 'Expected an array-like object, but found a Variable: maybe you are trying to call a function on a (possibly shared) variable instead of a numeric array?')
I really have two questions: 1) Why can't I do this? 2) What is the correct way to achieve what I want?
Thank you
As far as I can see, you are giving him the function as input. If you use your array/matrix of hidden units instead the code should work.
hidden_mean_func = th.function(inputs=[hidden], outputs=[hm], name="hidden_mean_function_printer")
print hidden_mean_func(hidden)
Related
I'm working on creating a dictionary of constraints for a large SCED power problem for minimization. However, I'm being given a ValueError saying an unknown type is passed despite only using Optimize.LinearConstraints at present. When I change to NonlinearConstraints (shown below), indicating that 'NonlinearConstraint' object has no attribute 'A'.
I have a feeling it's due to recursive elements, as even using a single constraint as I've defined them returns the same error
Any idea how I can create the recursive linear constraints?
##EDIT
I've been told to copy the code and provide a bit more context. "gen_supply_seg" is a three dimensional array that, depending on different points in time, has different constraints
def con2a():
for t in range(len(LOAD)):
for g in range(len(GEN)):
nlc2a = optimize.NonlinearConstraint(gen_supply_seg[t,g,1],lb=0,ub=P2Max[g])
return(nlc2a)
def con2b():
for t in range(len(LOAD)):
for g in range(len(GEN)):
nlc2b = optimize.NonlinearConstraint(gen_supply_seg[t,g,2],lb=0,ub=P3Max[g])
return (nlc2b)
def con2c():
for t in range(len(LOAD)):
for g in range(len(GEN)):
nlc2c = optimize.NonlinearConstraint(gen_supply_seg[t,g,3],lb=0,ub=P4Max[g])
return (nlc2c)
con2a = con2a()
con2b = con2b()
con2c = con2c()
These constraints are then added to a set like shown
cons = (con2a,
con2b,
con2c)
I'm trying to remove all values with index vals 1, 2, and 3, from a string list like
['1:1', '2:100.0', '3:100.0',...]. The data is in sparse vector format and was loaded as a pandas dataframe. I used an online regex tester to match the first three positions of this list with success.
But as it exists in my program, the same regex doesn't work. On running:
data = pd.read_csv("c:\data.csv")
for index, row in data.itterrows():
line = parseline(row)
def parseline(line):
line = line.values.flatten() # data like: ['1:1 2:100.0 3:100.0...']
stringLine = listToString(line) # data like: 1:1 2:100.0 3:100.0...
splitLine = stringLine.split(" ") # data like: ['1:1', '2:100.0', '3:100.0',...]
remove = re.findall(r"'1:1'|'[2,3]:\d+.\d+'")
splitLine.remove(remove)
print(splitLine)
I get the following error:
TypeError: findall() missing 1 required positional argument: 'string'
Does anyone have any ideas? Thanks in advance.
The splitLine object was actually a list, but the re.findall() method (and re.sub() method, which was what was actually used) requires a string, instead of a list. Was just operating on the wrong data structure. Ultimately:
def parseline(line):
line = line.values.flatten().tolist()
stringLine = listToString(line)
stringLine = re.sub(r"1:1 |2:\d+.\d+ ", "", stringLine)
...
did the trick.
I have read almost every similar question but none of them seems to solve my current problem.
In my python code, I am importing a string from my bashrc and in the following, I am defining the same name as a variable to index my dictionary. Here is the simple example
obs = os.environ['obs']
>> obs = 'id_0123'
id_0123 = numpy.where(position1 == 456)
>> position1[id_0123] = 456
>> position2[id_0123] = 789
But of course, when I do positions[obs], it throws an error since it is a string rather than an index (numpy.int64). So I have tried to look for a solution to convert my string into a variable but all solution suggesting to either convert into a dictionary or something else and assign the string to an integer, But I can not do that since my string is dynamic and will constantly change. In the end, I am going to have about 50 variables and I need to check the current obs corresponding to which variable, so I could use it as indices to access the parameters.
Edit:
Position1 and Position 2 are just bumpy arrays, so depending on the output of os.environ (which is 'id_0123' in this particular case), they will print an array element. So I can not assign 'id_0123' another string or number since I am using that exact name as a variable.
The logic is that there are many different arrays, I want to use the output of os.environ as an input to access the element of these arrays.
If you wanted to use a dictionary instead, this would work.
obs = 'id_0123'
my_dict = {obs: 'Tester'}
print (my_dict [obs])
print (my_dict ['id_0123'])
You could use the fact that (almost) everything is a dictionary in Python to create storage container that you index with obs:
container = lambda:None
container.id_0123 = ...
...
print(positions[container.__dict__[obs]])
Alternatively, you can use globals() or locals() to achieve the desired behavior:
import numpy
import os
def environment_to_variable(environment_key='obs', variable_values=globals()):
# Get the name of the variable we want to reference from the bash environment
variable_name = os.environ[environment_key]
# Grab the variable value that matches the named variable from the environment. Variable you want to reference must be visible in the global namespace by default.
variable_value = variable_values[variable_name]
return variable_value
positions = [2, 3, 5, 7, 11, 13]
id_0123 = 2 # could use numpy.where here
id_0456 = 4
index = environment_to_variable(variable_values=locals())
print(positions[index])
If you place this in a file called a.py, you can observe the following:
$ obs="id_0123" python ./a.py
5
$ obs="id_0456" python ./a.py
11
This allows you to index the array differently at runtime, which is what it seems like your intention is.
I created my own version of GridSearchCV module from sklearn.model_selection library. My version includes iterating through each parameter one by one instead of looking for all possible combinations. For example for a SVR model, if we have three parameters defined as follows:
{
'gamma' : np.arange(0.0, 1.0, 0.1),
'C': np.arange(1, 10, 1),
'epsilon': np.arange(0.0, 1.0, 0.1)
}
The algorithm would in the first turn find one best gamma coefficient (out of ten). Then it moves to assigning C parameter with given value of gamma. After ten iterations it moves to epsilon and assigns optimal epsilon value with given set of [gamma, C] parameters. This gives us in total 30 combinations to check instead of 1000 (10*10*10).
I'd like to import my opt_grid_search object into my projects, like below:
from own_udf_functions import show_description, opt_grid_search
The code of the object begins with dynamic statement that creates object that is going to be optimized:
exec_string = 'opt_object = ' + object_name + '(' + def_params + ')'
which returns for example:
opt_object = SVR(kernel = 'rbf')
However, when I try to use the code in another script as below:
opt_grid_search(object_name, params_set, X_train, y_train, X_test, y_test,
cross_val = 2, def_params = def_params)
following error appears:
*File "C:\Users\Marek\Desktop\Python\Github\Kernele\Kaggle Competitions\own-udf-
functions\own_udf_functions.py", line 40, in opt_grid_search
opt_object.fit(X_train,y_train)
NameError: name 'opt_object' is not defined*
It seems that opt_grid_search function didn't execute the following line of code:
opt_object = SVR(kernel = 'rbf')
and the object named opt_object wasn't actually created.
I think it has to do with classes, but I would like to ask you to help me better understand what actually happened in this error. I think it is a crucial knowledge that would help me a lot write more 'pythonic' codes instead of defining all of the functions in every single code.
Secondly, please let me know if such optimization makes sense as well or is it needed for the GridSearch to go through all possible combinations.
I tried to keep this description as short as possible, however if you would like to see / need it for the reference, my code is accessible below:
https://github.com/markoo26/own-udf-functions
The issue here is the exec function and the namespace/scope in which it operates. I'm struggling to get my head around this myself but essentially exec() doesn't work for assignment used inside a function in this way. The easiest workaround is to use eval() instead which explcitly returns an object. So end up with something like:
exec_string = object_name + '(' + def_params + ')'
opt_object = eval(exec_string)
I have a loop i would like to build a dictionary from. The part of the code I'm having trouble with is that both the key and the value are strings. I cannot convert the IP variable back string into a int nor is it a float.
Here is the method from my class I'm attempting to build the dictionary with. There is a loop elsewhere walking the IP range I'm interested in feeding the method parameter 'ip'.
def dictbuild(self,ip):
s = pxssh.pxssh()
s.force_password = True
try:
s.login(str(ip), 'root', 'password')
s.sendline('hostname') # run a command
s.prompt() # match the prompt
out = s.before # print everything before the prompt, this returns a byte object and could need decode(utf-8)
out = out.decode('utf-8')
out = out.split()
out = out[1]
print(type(out)) #These type function give us an easy output of data types in the traceback
print(type(ip))
ipdict = dict(out=ip) ###Getting stuck here on two string types.
print(ipdict)
s.logout()
except (pxssh.ExceptionPxssh, pexpect.EOF) as e:
pass
Instead of passing the string i want (it would actually be the hostname of the box) we get an output like this...
<class 'str'>
<class 'str'>
{'out': '10.20.234.3'}
The python documentation only gives examples of key as string and value as int. Am i using this wrong?
https://docs.python.org/2/tutorial/datastructures.html
Thanks in advance.
Use a dict literal instead:
ipdict = { out: ip }
Calling dict() in that way is just passing named arguments; dict literals take expressions for keys and expressions for values.