I came across the notebook Contribution analysis and comparison which provides a function print_recursive_calculation.
def print_recursive_calculation(activity, lcia_method, lca_obj=None, total_score=None, amount=1, level=0, max_level=3, cutoff=1e-2):
if lca_obj is None:
lca_obj = bc.LCA({activity: amount}, lcia_method)
lca_obj.lci()
lca_obj.lcia()
total_score = lca_obj.score
elif total_score is None:
raise ValueError
else:
lca_obj.redo_lcia({activity: amount})
if abs(lca_obj.score) <= abs(total_score * cutoff):
return
print("{}{:4.3f} ({:06.4f}): {:.70}".format(" " * level, lca_obj.score / total_score, lca_obj.score, str(activity)))
if level < max_level:
for exc in activity.technosphere():
print_recursive_calculation(
activity=exc.input,
lcia_method=lcia_method,
lca_obj=lca_obj,
total_score=total_score,
amount=amount * exc['amount'],
level=level + 1,
max_level=max_level,
cutoff=cutoff
)
If I understand correctly, print_recursive_calculation returns the activities' contributions (lca_obj.score?) including the downstream contributions of other activities. Is it also possible to return only the direct contributions of each activity excluding the downstream contributions of other activities? If so, how?
Once you have an lca object you can access to the direct impacts of an activity using the lca.characterized_inventory matrix. Each column represents an activity in the product system and each row the elementary flows of that activity multiplied by the characterization factors. You can find the column corresponding to the activity you are looking for in the lca.activity_dict using the key of the activity as key. So for example:
lca = bwc.LCA({act:1},gwp100)
lca.lci()
lca.lcia()
lca.characterized_inventory[:,lca.activity_dict.get(act.key)].sum()
will return the direct impact of an activity. You can use the same logic to modify the function you posted (which by the way is now part of bw2analyzer).
If you are interested in the direct contributions of each activity in the product system you can also use the following:
import bw2analyzer as bwa
ca = bwa.ContributionAnalysis()
ca.annotated_top_processes(lca)
Related
I have read this article that explains how to set the level of a floor without moving it. The article refers to the Building Coder where the BuiltInParameter.LEVEL_PARAM is used. However this method no longer works due to updates in the API. I am able to find the new ForgeTypeId of the parameter, but I am told that the LevelId is a Read-Only parameter when I try to run my code. How do I change the level of a floor? In the GUI it's easy, how can this be so hard in the API and so easy in the GUI?
Doing this in RevitPythonShell, my code is the following:
typeid = s0.LookupParameter("Level").GetTypeId()
floorid = ElementId(5873761)
with Transaction(revit.doc,"change level") as t:
p = s0.GetParameter(typeid)
t.Start()
p.Set(floorid)
t.Commit()
Grateful for any help!
You shouldnt have to make a new floor - you can change the level of a floor just like any other Parameter:
levels = list(FilteredElementCollector(doc).OfClass(Level))
newLevelName = 'Level 2'
newLevel = [i for i in levels if i.Name == newLevelName][0]
floor = s0 # your selected floor here
levelParam = floor.LookupParameter('Level')
t = Transaction(doc, 'Changing Floor Level to '+newLevelName)
t.Start()
try:
levelParam.Set(newLevel.Id)
print 'changed level of floor to',level.Name
except Exception as e:
print '!!!',e
t.Commit()
Interestingly, the UserModifiable value of the levelParam is False - turns out users can still modify it though!
I'm exploring Prefect's map-reduce capability as a powerful idiom for writing massively-parallel, robust importers of external data.
As an example - very similar to the X-Files tutorial - consider this snippet:
#task
def retrieve_episode_ids():
api_connection = APIConnection(prefect.context.my_config)
return api_connection.get_episode_ids()
#task(max_retries=2, retry_delay=datetime.timedelta(seconds=3))
def download_episode(episode_id):
api_connection = APIConnection(prefect.context.my_config)
return api_connection.get_episode(episode_id)
#task(trigger=all_finished)
def persist_episodes(episodes):
db_connection = DBConnection(prefect.context.my_config)
...store all episodes by their ID with a success/failure flag...
with Flow("import_episodes") as flow:
episode_ids = retrieve_episode_ids()
episodes = download_episode.map(episode_ids)
persist_episodes(episodes)
The peculiarity of my flow, compared with the simple X-Files tutorial, is that I would like to persist results for all the episodes that I have requested, even for the failed ones. Imagine that I'll be writing episodes to a database table as the episode ID decorated with an is_success flag. Moreover, I'd like to write all episodes with a single task instance, in order to be able to perform a bulk insert - as opposed to inserting each episode one by one - hence my persist_episodes task being a reduce task.
The trouble I'm having is in being able to gather the episode ID for the failed downloads from that reduce task, so that I can store the failed information in the table under the appropriate episode ID. I could of course rewrite the download_episode task with a try/catch and always return an episode ID even in the case of failure, but then I'd lose the automatic retry/failure functionality which is a good deal of the appeal of Prefect.
Is there a way for a reduce task to infer the argument(s) of a failed mapped task? Or, could I write this differently to achieve what I need, while still keeping the same level of clarity as in my example?
Mapping over a list preserves the order. This is a property you can use to link inputs with the errors. Check the code I have below, will add more explanation after.
from prefect import Flow, task
import prefect
#task
def retrieve_episode_ids():
return [1,2,3,4,5]
#task
def download_episode(episode_id):
if episode_id == 5:
return ValueError()
return episode_id
#task()
def persist_episodes(episode_ids, episodes):
# Note the last element here will be the ValueError
prefect.context.logger.info(episodes)
# We change that ValueError into a "fail" message
episodes = ["fail" if isinstance(x, BaseException) else x for x in episodes]
# Note the last element here will be the "fail"
prefect.context.logger.info(episodes)
result = {}
for i, episode_id in enumerate(episode_ids):
result[episode_id] = episodes[i]
# Check final results
prefect.context.logger.info(result)
return
with Flow("import_episodes") as flow:
episode_ids = retrieve_episode_ids()
episodes = download_episode.map(episode_ids)
persist_episodes(episode_ids, episodes)
flow.run()
The handling will largely happen in the persist_episodes. Just pass the list of inputs again and then we can match the inputs with the failed tasks. I added some handling around identifying errors and replacing them with what you want. Does that answer the question?
Always happy to chat more. You can reach out in the Prefect Slack or Discourse as well.
We have a feature A with several scenarios. And we need one scenario from that file. Can we call it in our feature B?
No. You need to extract that Scenario into a separate*.feature file and then re-use it using the call keyword.
EDIT: Karate 0.9.0 onwards will support being able to call by tag as follows:
* def result = call read('some.feature#tagname')
To illustrate the answer from Karate-inventor Peter Thomas with an example:
Given a feature-file some.feature with multiple scenarios tagged by a tag-decorator:
#tagname
Scenario: A, base case that shares results to e.g. B
// given, when, then ... and share result in a map with keyword `uri`
* def uri = responseHeaders['Location'][0]
#anotherTag
Scenario: X, base case that shares results to e.g. B
// given, when, then ... and share result in a map with keyword `createdResourceId`
* def createdResourceId = $.id
In another feature we can call a specific scenario from that feature by its tag, e.g. tagname:
Scenario: B, another case reusing some results of A
* def result = call read('some.feature#tagname')
* print "given result of A is: $result"
Given path result.uri + '/update'
See also: demo of adding custom tags to scenarios
I noticed something weird happening when converting a Python environment into a TF environment using tf_agents.environments.TFPyEnvironment and I'd like to ask you what general changes occur.
To clarify the question please find below my code. I want the environment to simulate (in an oversimplied manner) interactions with a customers who want to buy fruits or vegetables. The agent should learn that when a customer asks for fruits, action 0 should be executed for example.
class CustomEnv(py_environment.PyEnvironment):
def __init__(self):
self._action_spec = array_spec.BoundedArraySpec(
shape=(), dtype=np.int32, minimum=0, maximum=1)
self._observation_spec = array_spec.BoundedArraySpec(
shape=(1,1), dtype=np.int32, minimum=0, maximum=1)
self._state = [0]
self._counter = 0
self._episode_ended = False
self.dictionary = {0: ["Fruits"],
1: ["Vegetables"]}
def action_spec(self):
return self._action_spec
def observation_spec(self):
return self._observation_spec
def _reset(self):
self._state = [0]
self._counter = 0
self._episode_ended = False
return ts.restart(np.array([self._state], dtype=np.int32))
def preferences(self):
return np.random.randint(2)
def pickedBasket(self, yes):
reward = -1.0
if yes:
reward = 0.0
return reward
def _step(self, action):
if self._episode_ended:
self._reset()
if self._counter<50:
self._counter += 1
basket = self.preferences()
condition = basket in self.dictionary[action]
reward = self.pickedBasket(condition)
self._state[0] = basket
if self._counter==50:
self._episode_ended=True
return ts.termination(np.array([self._state],
dtype=np.int32),
reward,
1)
else:
return ts.transition(np.array([self._state],
dtype=np.int32),
reward,
discount=1.0)
When I execute the following to code to check everything is working just fine:
py_env = ContextualMBA()
tf_env = tf_py_environment.TFPyEnvironment(py_env)
time_step = tf_env.reset()
action = 0
next_time_step = tf_env.step(action)
I get an unhashable type: 'numpy.ndarray' for the line condition = basket in self.dictionary[action] so I changed it into condition = basket in self.dictionary[int(action)] and it worked just fine. I'd also like to precise that it worked as a Python environment even without adding the int part. So I'd like to ask what changes the tf_agents.environments.TFPyEnvironment. I don't see how it can influence the type of action action since it isn't related to action_spec or anything (at least directly in the code).
Put basically, tf_agents.environments.TFPyEnvironment is a translator working between your Python environment and the TF-Agents API. The TF-Agents API does not know how many actions it is allowed to choose from, what data to observe and learn from or specially how the choice of actions will influence your custom environment.
Your custom environment is there to provide the rules of the environment and it follows some standards in order for the TFPyEnvironment to be able to translate it correctly so the TF-Agent can work with it. You need to define elements and methods in your custom environment, for example, such as:
__init__()
self._action_spec
self._observation_spec
_reset()
_step()
I'm not sure if your doubt came from the fact that you gave an action = 0 for the agent and, unrelated to the action_spec, the agent actually worked. The action_spec had no relation with your _step() function, and that is correct. Your step function takes some action and applies it to the environment. How this action is shaped is the real point.
The problem is you chose the value and gave it to the tf_env.step() function. If you had actually delegated the choice of action to the agent, by tf_env.step(agent.policy.action) (or tf_env.step(agent.policy.action.action), sometimes TF-Agents make me confuse), the agent would have to look to your action_spec definition to understand what the environment expects the action to look like.
If action_spec is not defined, the agent would not know what to choose between 0 for "Fruits", 1 for "Vegetables" - that you wanted, and defined - or unexpected results as 2 for "Meat", or [3, 2] for 2 bottles of water, since 3 could stand for "Bottle of Water". The TF-Agent needs these definitions so it knows the rules of your environment.
As for the actual changes and what they do with your custom environment code, I believe you would get a better idea by looking at the source code of the TF-Agents library.
I am working on processing a dataset that includes dense GPS data. My goal is to use parallel processing to test my dataset against all possible distributions and return the best one with the parameters generated for said distribution.
Currently, I have code that does this in serial thanks to this answer https://stackoverflow.com/a/37616966. Of course, it is going to take entirely too long to process my full dataset. I have been playing around with multiprocessing, but can't seem to get it to work right. I want it to test multiple distributions in parallel, keeping track of sum of square error. Then I want to select the distribution with the lowest SSE and return its name along with the parameters generated for it.
def fit_dist(distribution, data=data, bins=200, ax=None):
#Block of code that tests the distribution and generates params
return(distribution.name, best_params, sse)
if __name__ == '__main__':
p = Pool()
result = p.map(fit_dist, DISTRIBUTIONS)
p.close()
p.join()
I need some help with how to actually make use of the return values on each of the iterations in the multiprocessing to compare those values. I'm really new to python especially multiprocessing so please be patient with me and explain as much as possible.
The problem I'm having is it's giving me an "UnboundLocalError" on the variables that I'm trying to return from my fit_dist function. The DISTRIBUTIONS list is 89 objects. Could this be related to the parallel processing, or is it something to do with the definition of fit_dist?
With the help of Tomerikoo's comment and some further struggling, I got the code working the way I wanted it to. The UnboundLocalError was due to me not putting the return statement in the correct block of code within my fit_dist function. To answer the question I did the following.
from multiprocessing import Pool
def fit_dist:
#put this return under the right section of this method
return[distribution.name, params, sse]
if __name__ == '__main__':
p = Pool()
result = p.map(fit_dist, DISTRIBUTIONS)
p.close()
p.join()
'''filter out the None object results. Due to the nature of the distribution fitting,
some distributions are so far off that they result in None objects'''
res = list(filter(None, result))
#iterates over nested list storing the lowest sum of squared errors in best_sse
for dist in res:
if best_sse > dist[2] > 0:
best_sse = dis[2]
else:
continue
'''iterates over list pulling out sublist of distribution with best sse.
The sublists are made up of a string, tuple with parameters,
and float value for sse so that's why sse is always index 2.'''
for dist in res:
if dist[2]==best_sse:
best_dist_list = dist
else:
continue
The rest of the code simply consists of me using that list to construct charts and plots with that best distribution overtop of a histogram of my raw data.