I am trying to create custom parameters for a Haiku hk.Module.
I know that hk.get_parameter() does create new parameters/returns existing parameters, however, I also want to modify the values of those parameters before they are returned when calling the .init() function. Modifying the parameters after the return from the .init() is straightforward as mentioned in the following StackOverflow thread: How does one get a parameter from a params (pytree) in haiku? (jax framework) I have tried using hk.custom_getter(), however, it only modifies the return value to the user (when calling hk.get_parameter()); the original parameters remain unmodified.
Is there a way to achieve this?
Here is a code snippet of what I am trying to achieve:
class HaikuModule(hk.Module):
def __call__(self):
w = hk.get_parameter("w", shape=[5,5], init=jnp.zeros) # create parameter w intialized to 0s
# modify the value of w
???
def forward_fn():
model = HaikuModule()
return model()
fn = hk.transform(forward_fn)
rng= jax.random.PRNGKey(10)
params = fn.init(rng)
# params should return modified params, not 0s(initialized value)
Related
I have a method called complete in my model, how can i use that in my class view, in my model method there is one parameter called person which is being passed i do not want my
overriden method to use that parameter how can i acheive that.
class Mymodel(models.Model):
is_done = model.BooleanField()
def complete(self, person):
self.is_done = True
self.save(update_fields=['is_done'])
self.done_by.add(person)
class MyView(SomeView):
def complete_record(self):
return Mymodel.complete(here it expects two arguments i need only self)
and i want to get rid of self.done_by.add(person) in model's complete
method
The complete() method can be called for a single model instance (a single object of the queryset).
In the View, maybe you want to do this if you have a id param in url:
instance = Mymodel.objects.get(pk=self.kwargs['id'])
instance.complete()
or the scenario can be this if you are using DetailView:
self.get_object().complete()
-- EDIT --
If you want to set is_done = True without add person and
if you can't edit model class, you can put the logic in the view:
class MyView(SomeView):
def complete_record(self):
record = Mymodel.objects.get(pk=id)
record.is_done = True
record.save()
return record
I have a BaseClass and two classes (Volume and testing) which inherits from the BaseClass. The class "Volume" use a method "driving_style" from another python module. I am trying to write another method "test_Score" which wants to access variables computed in the method "driving_style" which I want to use to compute further. These results will be accessed to the class "testing" as shown.
from training import Accuracy
import ComputeData
import model
class BaseClass(object):
def __init__(self, connections):
self.Type = 'Stock'
self.A = connections.A
self.log = self.B.log
def getIDs(self, assets):
ids = pandas.Series(assets.ids, index=assets.B)
return ids
class Volume(BaseClass):
def __init__(self, connections):
BaseClass.__init__(self, connections)
self.daystrade = 30
self.high_low = True
def learning(self, data, rootClass):
params.daystrade = self.daystrade
params.high_low = self.high_low
style = Accuracy.driving_style()
return self.Object(data.universe, style)
class testing(BaseClass):
def __init__(self, connections):
BaseClass.__init__(self, connections)
def learning(self, data, rootClass):
test_score = Accuracy.test_score()
return self.Object(data.universe, test_score)
def driving_style(date, modelDays, params):
daystrade = params.daystrade
high_low = params.high_low
DriveDays = model.DateRange(date, params.daystrade)
StopBy = ComputeData.instability(DriveDays)
if high_low:
style = ma.average(StopBy)
else:
style = ma.mean(StopBy)
return style
def test_score(date, modelDays, params):
"want to access the following from the method driving_style:"
DriveDays =
StopBy =
return test_score ("which i compute using values DriveDays and StopBy and use test_score in the method learning inside
the 'class - testing' which inherits some params from the BaseClass")
You can't use locals from a call to a function that was made elsewhere and has already returned.
A bad solution is to store them as globals that you can read from later (but that get replaced on every new call). A better solution might to return the relevant info to the caller along with the existing return values (return style, DriveDays, StopBy) and somehow get it to where it needs to go. If necessary, you could wrap the function into a class and store the computed values as attributes on an instance of the class, while keeping the return type the same.
But the best solution is probably to refactor, so the stuff you want is computed by dedicated methods that you can call directly from test_score and driving_style independently, without duplicating code or creating complicated state dependencies.
In short, basically any time you think you need to access locals from another function, you're almost certainly experiencing an XY problem.
I tried to do some argument modification on a decorator which decorates a function.
The original code looks like
#original_decorator(arg=some_object)
def calculate(a, b):
# complex business logic
raise Exception()
where original_decorator is responsible for exception handling.
What I want to achieve is to do some temporary modification on some_object and restore it's property after function returned.
And I've tried the following
def replace_arg(arg, add_some_property):
def decorator_wrapper(decorator_func):
def decorator_inner(*decorator_args, **decorator_kwargs):
def actual_wrapper(actual_func):
def actual_inner(*actual_args, **actual_kwargs):
original = arg['func']
arg['func'] = add_some_property
decorator_kwargs['arg'] = arg
result = actual_func(*actual_args, **actual_kwargs)
arg['func'] = original
return result
return actual_inner
return actual_wrapper
return retry_inner
return retry_wrapper
Also tried to place the modification logic in decorator_inner, but neither worked.
My Questions:
Is it possible to modify a decorator's argument?
If true, then how can I achieve it?
def fixed_given(self):
return #given(
test_df=data_frames(
columns=columns(
["float_col1"],
dtype=float,
),
rows=tuples(
floats(allow_nan=True, allow_infinity=True)),
)
)(self)
#pytest.fixture()
#fixed_given
def its_a_fixture(test_df):
obj = its_an_class(test_df)
return obj
#pytest.fixture()
#fixed_given
def test_1(test_df):
#use returned object from my fixture here
#pytest.fixture()
#fixed_given
def test_2(test_df):
#use returned object from my fixture here
Here, I am creating my test dataframe in a seperate function to use it commonly across all functions.
And then creating a pytest fixture to instantiate a class by passing the test dataframe generated by a fixed given function.
I am finding a way to get a return value from this fixture.
But the problem i am using a given decorator, its doesn't allow return values.
is there a way to return even after using given decorator?
It's not clear what you're trying to acheive here, but reusing inputs generated by Hypothsis gives up most of the power of the framework (including minimal examples, replaying failures, settings options, etc.).
Instead, you can define a global variable for your strategy - or write a function that returns a strategy with #st.composite - and use that in each of your tests, e.g.
MY_STRATEGY = data_frames(columns=[
column(name="float_col1", elements=floats(allow_nan=True, allow_infinity=True))
])
#given(MY_STRATEGY)
def test_foo(df): ...
#given(MY_STRATEGY)
def test_bar(df): ...
Specifically to answer the question you asked, you cannot get a return value from a function decorated with #given.
Instead of using fixtures to instantiate your class, try using the .map method of a strategy (in this case data_frames(...).map(its_an_class)), or the builds() strategy (i.e. builds(my_class, data_frames(...), ...)).
I am trying to add a custom field in my logging using LogRecordFactory. I am repeatedly calling a class and every time I do that, I want to set the custom_attribute in the init module so the remainder of the code within the class will have this attribute. But I cannot get this to work. I found the following which works, but its static.
import logging
old_factory = logging.getLogRecordFactory()
def record_factory(*args, **kwargs):
record = old_factory(*args, **kwargs)
record.custom_attribute = "whatever"
return record
logging.basicConfig(format="%(custom_attribute)s - %(message)s")
logging.setLogRecordFactory(record_factory)
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
logging.debug("test")
This will output correctly:
whatever - test
However, my use case is that the custom_attribute will vary. Every time I call a specific function, I want to change this. So it seems like record_factory needs another parameter passed to it so it can then return the correct record with the new parameter. But I cant figure it out. I have tried adding a parameter to the function, but when I make the call I get:
TypeError: __init__() missing 7 required positional arguments: 'name', 'level', 'pathname', 'lineno', 'msg', 'args', and 'exc_info'
I think this has something to do with the *args and **kwargs but I don't really know. Also, why are there no parenthesis after record_factory when its called by logging.setLogRecordFactory? I have never seen a function work like this.
You can try to use closure:
import logging
old_factory = logging.getLogRecordFactory()
def record_factory_factory(context_id):
def record_factory(*args, **kwargs):
record = old_factory(*args, **kwargs)
record.custom_attribute = context_id
return record
return record_factory
logging.basicConfig(format="%(custom_attribute)s - %(message)s")
logging.setLogRecordFactory(record_factory_factory("whatever"))
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
logging.debug("test")
logging.setLogRecordFactory(record_factory_factory("whatever2"))
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
logging.debug("test")
result:
$ python3 log_test.py
whatever - test
whatever2 - test
I stumbled upon this question while I was trying to do something similar. This is how I solved it, assuming that you want to add something called xyz to every log line (further explanation below):
import logging
import threading
thread_local = threading.local()
def add_xyz_to_logrecords(xyz):
factory = logging.getLogRecordFactory()
if isinstance(factory, XYZLogFactory):
factory.set_xyz(xyz)
else:
logging.setLogRecordFactory(XYZLogFactory(factory, xyz))
class XYZLogFactory():
def __init__(self, original_factory, xyz):
self.original_factory = original_factory
thread_local.xyz = xyz
def __call__(self, *args, **kwargs):
record = self.original_factory(*args, **kwargs)
try:
record.xyz = thread_local.xyz
except AttributeError:
pass
return record
def set_xyz(self, xyz):
thread_local.xyz = xyz
Here I've created a callable class XYZLogFactory, that remembers what the current value of xyz is, and also remembers what the original LogRecordFactory was. When called as a function, it creates a record using the original LogRecordFactory, and adds an xyz attribute with the current value.
The thread_local is to make it thread-safe, but for an easier version, you could just use an attribute on the XYZLogFactory:
class XYZLogFactory():
def __init__(self, original_factory, xyz):
self.original_factory = original_factory
self.xyz = xyz
def __call__(self, *args, **kwargs):
record = self.original_factory(*args, **kwargs)
record.xyz = self.xyz
return record
def set_xyz(self, xyz):
self.xyz = xyz
In my very first attempt (not shown here), I did not store the original factory, but stored it implicitly in the new LogRecordFactury using a closure. However, after a while that led to a RecursionError, because it kept calling the previous factory, which called the previous factory, etc.
Regarding your last question: there are no parentheses because the function is not called here. Instead it's passed to the logging.setLogRecordFactory, which saves it in a variable somewhere, and then calls that someplace else. If you want more information you can google something like 'functions as first class citizens'.
Easy example:
x = str # Assign to x the function that gives string representation of object
x(1) # outputs the string representation of 1, same as if you'd called str(1)
> '1'