How can I get the Variables.save to save all variables, and not just the one specified by the user. I want to make sure that future runs of scons use exactly the same setup.
Sadly there is no way to do this using core SCons functions. The decision on whether to save a variable value or not is hard-wired into Save, and only values that are not the defaults can be saved.
However, you can save all variables by abusing the not-well-documented "converter" function. It ain't pretty, but does the job.
When you call Variables.Save(), it calls each variable's converter function and checks the result to see if it is the same as the default (input) value. If you return something that is not the input value when called from Save then all values are stored in the cache file. Elsewhere you need the converter to return the input value as-is, or it would use incorrect variable values.
def save_all(value, env=None):
import traceback
# if called in Save method, return non-default
for line in traceback.format_stack():
lines = line.splitlines()
if lines[0].endswith('Save'):
return value + 'save'
# return correct value otherwise
return value
import SCons
var = Variables("config.py", ARGUMENTS)
# AddVariables accepts tuples of:
# Variable key, description, default-value, validator, converter
var.AddVariables(
('CC', 'The C Compiler', 'cc', None, save_all),
('CFLAGS', 'Flags to pass to compiler', '-Wall', None, save_all))
env = Environment(tools=[], variables=var)
var.Save('config.py', env)
print 'CC', env['CC']
print 'CFLAGS', env['CFLAGS']
Here the function save_all is the hack. It uses the stack trace to check if it is called from within Save. If it is, then it returns a tampered value, otherwise it returns the input value unchanged. You can check the result of this example in the cache file "config.py".
Related
I have a dataset and I want to make a function that does the .get_dummies() so I can use it in a pipeline for specific columns.
When I run dataset = pd.get_dummies(dataset, columns=['Embarked','Sex'], drop_first=True)
alone it works, as in, when I run df.head() I can still see the dummified columns but when I have a function like this,
def dummies(df):
df = pd.get_dummies(df, columns=['Embarked','Sex'], drop_first=True)
return df
Once I run dummies(dataset) it shows me the dummified columsn in that same cell but when I try to dataset.head() it isn't dummified anymore.
What am I doing wrong?
thanks.
You should assign the result of the function to df, call the function like:
dataset=dummies(dataset)
function inside them have their own independent namespace for variable defined there either in the signature or inside
for example
a = 0
def fun(a):
a=23
return a
fun(a)
print("a is",a) #a is 0
here you might think that a will have the value 23 at the end, but that is not the case because the a inside of fun is not the same a outside, when you call fun(a) what happens is that you pass into the function a reference to the real object that is somewhere in memory so the a inside will have the same reference and thus the same value.
With a=23 you're changing what this a points to, which in this example is 23.
And with fun(a) the function itself return a value, but without this being saved somewhere that result get lost.
To update the variable outside you need to reassigned to the result of the function
a = 0
def fun(a):
a=23
return a
a = fun(a)
print("a is",a) #a is 23
which in your case it would be dataset=dummies(dataset)
If you want that your function make changes in-place to the object it receive, you can't use =, you need to use something that the object itself provide to allow modifications in place, for example
this would not work
a = []
def fun2(a):
a=[23]
return a
fun2(a)
print("a is",a) #a is []
but this would
a = []
def fun2(a):
a.append(23)
return a
fun2(a)
print("a is",a) #a is [23]
because we are using a in-place modification method that the object provided, in this example that would be the append method form list
But such modification in place can result in unforeseen result, specially if the object being modify is shared between processes, so I rather recomend the previous approach
I am new to Python and am at a lost as to what I'm doing wrong. I am trying to use the fqdn variable that is being returned to the caller which is main() but I'm getting NameError: name 'fqdn' is not defined
I'm betting this is some type of global variable statement issue or something like that, but I've been researching this and can't figure it out.
If a function from a module returns a value, and the caller is main(), shouldn't main() be able to use that returned value???
Here's the layout:
asset.py
def import_asset_list():
# Open the file that contains FQDNs
openfile = open(r"FQDN-test.txt")
if openfile.mode == 'r':
# Remove CR from end of each item
fqdn = openfile.read().splitlines()
# Add https to the beginning of every item in list
fqdn = ["https://" + item for item in fqdn]
openfile.close()
return fqdn
tscan.py
def main():
import asset
asset.import_asset_list()
# Iterate through list
for i in fqdn:
if SCHEDULED_SCAN == 1:
create_scheduled_scan(fqdn)
launch_scan(sid)
check_status_scan(uuid)
else:
create_scan(fqdn)
launch_scan(sid)
check_status_scan(uuid)
Short Explanation
Yes, main() should be able to use the returned value, but it's only the value that is returned, not the variable name. You have to define a variable of your own name, to receive the value, and use that instead.
Long Explanation
The name of a variable inside any function is simply a "label" valid only within the scope of this function. A function is an abstraction which means "Give me some input(s), and I will give you some output(s)". Within the function, you need to reference the inputs somehow and, potentially, assign some additional variables to perform whatever it is you would like to. These variable names have no meaning whatsoever outside the function, other than to, at most, convey some information as to the intended use of the function.
When a function returns a value, it does not return the "name" of the variable. Only the value (or the reference in memory) of the variable. You can define your own variable at the point where you call the function, give it your own name and assign to it the returned result of the function, so you simply have to write:
def main():
import asset
my_asset_list = asset.import_asset_list()
# Iterate through list
for i in my_asset_list:
if SCHEDULED_SCAN == 1:
create_scheduled_scan(my_asset_list)
launch_scan(sid)
check_status_scan(uuid)
else:
create_scan(my_asset_list)
launch_scan(sid)
check_status_scan(uuid)
I don't know where the uuid and the sid variables are defined.
To make sure you have understood this properly, remember:
You can have multiple functions in the same file, and use identically-named variables within all those functions, this will be no problem because a variable (with its name) only exists within each specific function scope.
Variable names do not "cross" the boundaries of the scope, only variable values/references and to do this, a special construct is used, i.e. the return [something] statement.
I am writing a framework in Python. When a user declares a function, they do:
def foo(row, fetch=stuff, query=otherStuff)
def bar(row, query=stuff)
def bar2(row)
When the backend sees query= value, it executes the function with the query argument depending on value. This way the function has access to the result of something done by the backend in its scope.
Currently I build my arguments each time by checking whether query, fetch and the other items are None, and launching it with a set of args that exactly matches what the user asked for. Otherwise I got the "got an unexpected keyword argument" error. This is the code in the backend:
#fetch and query is something computed by the backend
if fetch= None and query==None:
userfunction(row)
elif fetch==None:
userunction (row, query=query)
elif query == None:
userfunction (row, fetch=fetch)
else:
userfunction (row,fetch=fetch,query=query)
This is not good; for each additional "service" the backend offers, I need to write all the combinations with the previous ones.
Instead of that I would like to primarily take the function and manually add a named parameter, before executing it, removing all the unnecessary code that does these checks. Then the user would just use the stuff it really wanted.
I don't want the user to have to modify the function by adding stuff it doesn't want (nor do I want them to specify a kwarg every time).
So I would like an example of this if this is doable, a function addNamedVar(name, function) that adds the variable name to the function function.
I want to do that that way because the users functions are called a lot of times, meaning that it would trigger me to, for example, create a dict of the named var of the function (with inspect) and then using **dict. I would really like to just modify the function once to avoid any kind of overhead.
This is indeed doable in AST and that's what I am gonna do because this solution will suit better for my use case . However you could do what I asked more simply by having a function cloning approach like the code snippet I show. Note that this code return the same functions with different defaults values. You can use this code as example to do whatever you want.
This works for python3
def copyTransform(f, name, **args):
signature=inspect.signature(f)
params= list(signature.parameters)
numberOfParam= len(params)
numberOfDefault= len(f.__defaults__)
listTuple= list(f.__defaults__)
for key,val in args.items():
toChangeIndex = params.index(key, numberOfDefault)
if toChangeIndex:
listTuple[toChangeIndex- numberOfDefault]=val
newTuple= tuple(listTuple)
oldCode=f.__code__
newCode= types.CodeType(
oldCode.co_argcount, # integer
oldCode.co_kwonlyargcount, # integer
oldCode.co_nlocals, # integer
oldCode.co_stacksize, # integer
oldCode.co_flags, # integer
oldCode.co_code, # bytes
oldCode.co_consts, # tuple
oldCode.co_names, # tuple
oldCode.co_varnames, # tuple
oldCode.co_filename, # string
name, # string
oldCode.co_firstlineno, # integer
oldCode.co_lnotab, # bytes
oldCode.co_freevars, # tuple
oldCode.co_cellvars # tuple
)
newFunction=types.FunctionType(newCode, f.__globals__, name, newTuple, f.__closure__)
newFunction.__qualname__=name #also needed for serialization
You need to do that weird stuff with the names if you want to Pickle your clone function.
I have created three functions. The first function is used in the other two functions but I am passing it a hardcoded filepath. I want to be able to pass this in as a parameter, but I seem to be getting an issue.
Essentially, given a file_path, my function will get the first item in the list and then the second item.
So far my code is as follows :
def sort_files(file_path):
"""Sort files in ascending order"""
files = os.listdir(file_path)
return sorted(files, reverse=True)
def current_day():
"""Get the current day file"""
return sort_files(file_path)[0]
def previous_day():
"""Get the previous day file"""
return sort_files(file_path)[1]
If you want a function to accept an argument, you need to define it as doing so by specifying the parameter name it will be known as in the function (as you did with sort_files).
How are you executing the call to the current_day and previous_day. You should make them as function that can take a parameter.
Also please post the code that you are using to execute the whole setup.
I have a tkinter GUI with an entry box I want to allow only numbers. Can someone explain to me what each command / line of code in validation does. I don't understand the vcmd variable and all the '%i' '%s' stuff. Thanks :)
UPDATE:
I have a different application to use this vcmd command with and dont understand it entirely. Here is my validation code:
def validate(self, action, index, value_if_allowed, prior_value, text, validation_type, trigger_type, widget_name):
if not int(action):
return True
elif text in '0123456789.-+':
try:
float(value_if_allowed)
return True
except ValueError:
return False
else:
return False
I dont get why in this code i need all of these:
action, index, value_if_allowed, prior_value, text, validation_type, trigger_type, widget_name
Why do i need all of these specific to my validation code for it to function correctly and what use are they?
The documentation you provided made sense but some of those '%s', '%i' stuff seemed unnecessary for my specific code yet it only works with all of them included like so:
vcmd = (self.master.register(self.validate), '%d', '%i', '%P', '%s', '%S', '%v', '%V', '%W')
I also want to know what self.master.register does please, i still cant figure that one out.
If you do not need any of the special arguments, you don't need to call register. For example, the following code will correctly call do_validation without any arguments:
import tkinter as tk
def do_validation():
value = entry.get()
if value == "" or value.isnumeric():
return True
return False
root = tk.Tk()
entry = tk.Entry(root, validate='key', validatecommand=do_validation)
entry.pack(fill="x", padx=20, pady=20)
root.mainloop()
However, in the above case the validation will always be one character behind. This is because validation happens before the character is inserted into the entry widget (ie: the first time it is called, entry.get() will return an empty string). The whole point of validation is to prevent illegal characters, so it makes sense to call the command before the character is inserted rather than after.
This is why the special arguments are useful -- they provide sufficient context in order to decide whether the character or characters are legal before they are inserted into the widget.
How register is useful
Tkinter is a wrapper around a Tcl interpreter, because tk (the core technology behind tkinter) is implemented in Tcl. Tcl is a programming language just like python.
In a Tcl program, you would use the validation features like this:
proc do_validation(new_value) {
return $new_value == "" || [string is integer $new_value]
}
entry .entry -validate key --validatecommand [list do_validation %P]
When Tcl detects a change, it performs substitution on the arguments, and then calls the defined procedure with the substituted arguments. For example, if you type "A", %P is converted to "A" and the function is called with "A" as the only argument.
In the case of Tkinter, there is no direct corollary for defining the function with arguments to be substituted at runtime.
Unfortunately, the validation feature wasn't implemented very well in the tkinter wrapper, so to properly use the validation we have to use a small workaround in order to get these special arguments passed to our function.
What register does is to create a new Tcl command that calls your python command. It also creates a new python command that is a reference to this new Tcl command. You can use this reference exactly like any other python command.
A simple example
In the simplest case, all you need is what the string would look like if the edit was to be allowed. You can then decide whether the edit is something that will result in valid input, or if it will result in invalid input. If the edit won't result in valid input, you can reject the edit before it actually happens.
The special argument that represents the value if the edit is allowed is %P 1. We can modify the function we are registering to accept this argument, and we can add this argument when we do the registering:
def do_validation(new_value):
return new_value == "" or new_value.isnumeric()
...
vcmd = (root.register(do_validation), '%P')
entry = tk.Entry(..., validatecommand=vcmd)
With that, when the underlying Tcl code detects a change it will call the new Tcl command which was created, passing in one argument corresponding to the special %P substitution.
1All of the mechanics of the validation are described thoroughly in the tcl documentation here: http://tcl.tk/man/tcl8.5/TkCmd/entry.htm#M7