when I try to create arrays in my script I get errors.
id[1]=string2; would generate error id[1]=string2: not found
I'm guessing it has something to do with the fact that im in a if statement or while loop since the use []? I'm running a VM so attached is a pic of the script thus far the array at top a[1]=string; generates no errors but the one in the logic id[1]=string2; does.
Posting as an answer so this question can be marked as resolved:
Your script is being executed by sh, not bash. Add a correct shebang line as the first line of the script file:
#!/bin/bash
Related
I'm fairly new to snakemake and inherited a kind of huge worflow that consists in a sequence of 17 rules that run in serial.
Each rule takes outputs from the previous rules and uses them to run a python script. Everything has worked great so far except that now I'm trying to improve the worflow since some of the rules can be run in parallel.
A rough example of what I'm trying to achieve, my understanding is that wildcards should allow me to solve this.
grid = [ 10 , 20 ]
rule all:
input:
expand("path/to/C/{grid}/file_C" ,grid = grid)
rule process_A:
input:
path_A = "path/to/A/file_A"
path_B = "path/to/B/{grid}/file_B" # A rule further in the worflow could need a file from a previous rule saved with this structure
params:
grid = lambda wc: wc.get(grid)
output:
path_C = "path/to/C/{grid}/file_C"
script:
"script_A.py"
And inside the script I retrieve the grid size parameter:
grid = snakemake.params.grid
In the end the whole rule process_A should be rerun with grid = 10 and with grid = 20 and save each result to a folder whose path depends on grid also.
I know there are several things wrong with this, but I can't seem to find were to start from to figure this out. The error I'm getting now is:
name 'params' is not defined
Any help as to where to start from?
It would be useful to post the error stack trace of name 'params' is not defined to know exactly what is causing it. For now...
And inside the script I retrieve the grid size parameter:
grid = snakemake.params.grid
I suspect you are mixing the script directive with the shell directive. Probably you want something like:
rule process_A:
input: ...
output: ...
params: ...
script:
"script_A.py"
inside script_A.py snakemake will replace snakemake.params.grid with the actual param value.
Alternatively, write a standalone python script that parses command line arguments and you execute like any other program using the shell directive. (I tend to prefer this solution as it makes things more explicit and easier to debug but it also means more boiler-plate code to write a standalone script).
I'm trying to Create a file and append all the content being calculated into that file, but when I run the script the very last iteration is written inside the file and nothing else.
My code is on pastebin, it's too long, and I feel like you would have to see exactly how the iteration is happening.
Try to summarize it, Go through an array of model numbers, if the model number matches call the function that calculates that MAC_ADDRESS, when done calculating store all the content inside a the file.
I have tried two possible routes and both have failed, giving the same result. There is no error in the code (it runs) but it just doesn't store the content into the file properly there should be 97 different APs and it's storing only 1.
The difference between the first and second attempt,
1 attempt) I open/create file in the beginning of the script and close at the very end.
2 attempt) I open/create file and close per-iteration.
First Attempt:
https://pastebin.com/jCpLGMCK
#Beginning of code
File = open("All_Possibilities.txt", "a+")
#End of code
File.close()
Second Attempt:
https://pastebin.com/cVrXQaAT
#Per function
File = open("All_Possibilities.txt", "a+")
#per function
File.close()
If I'm not suppose to reference other websites, please let me know and I'll just paste the code in his post.
Rather than close(), please use with:
with open('All_Possibilities.txt', 'a') as file_out:
file_out.write('some text\n')
The documentation explains that you don't need + to append writes to a file.
You may want to add some debugging console print() statements, or use a debugger like pdb, to verify that the write() statement actually ran, and that the variable you were writing actually contained the text you thought it did.
You have several loops that could be a one-liner using readlines().
Please do this:
$ pip install flake8
$ flake8 *.py
That is, please run the flake8 lint utility against your source code,
and follow the advice that it offers you.
In particular, it would be much better to name your identifier file than to name it File.
The initial capital letter means something to humans reading your code -- it is
used when naming classes, rather than local variables. Good luck!
I'm looking for help running a python script that takes some time to run.
It is a long running process that takes about 2hours per test observation. For example, these observations could be the 50 states of the usa.
I dont want to baby sit this process all day - I'd like to kick it off then drive home from work - or have it run while I'm sleeping.
Since this a loop - I would need to call one python script that loops through my code going over each of the 50 states - and a 2nd that runs my actual code that does things.
I've heard of NOHUP, but I have very limited knowledge. I saw nohup ipython mypython.py but then when I google I get alot of other people chiming in with other methods and so I don't know what is the ideal approach for someone like me. Additionally, I am essentially looking to run this as a loop - so don't know how that complicates things.
Please give me something simple and easier to understand. I don't know linux all that well or I wouldn't be asking as this seems like a common sort of command/activity...
Basic example of my code:
Two files: code_file.py and loop_file.py
Code_file.py does all the work. Loop file just passes in the list of things to run the stuff for.
code_file.py
output = each_state + ' needs some help!'
print output
loop_file.py
states = ['AL','CO','CA','NY','VA','TX']
for each_state in states:
code_file.py
Regarding the loop - I have also heard that I can't pass in parameters or something via nohup? I can fix this part within my python code....for example reading from a CSV in my code and deleting the current record from that CSV file and then re-writing it out...that way I can always select the top record in the CSV file for my loop (the list of states)
May be you could modify your loop_file.py like this:
import os
states = ['AL','CO','CA','NY','VA','TX']
for each_state in states:
os.system("python /dir_of_your_code/code_file.py")
Then in a shell, you could run the loop_file.py with:
nohup python loop_file.py & # & is not necessary, it just redirect all output of the file to a file named nohup.out instead of printing it on screen.
Is there a way to machine-generate some values, after the user has supplied some their values for the variables in cookiecutter.json?
The reason I ask is that:
one of the values I need to prompt for is rather hard for users to work out
but it's really easy for me to write some Python code to generate the right value
So I'd really like to be able to remove the user prompt, and calculate the value instead.
Things I've tried:
Searched online for an example pre_gen_project.py file to show how to do it
Read the cookiecutter Advanced Usage page
I'm using cookiecutter on the command line:
cookiecutter path_to_template
Am I missing any tricks?
I needed this exact capability just a few days ago. The solution I came up with was to write a wrapper script for cookiecutter, similar to what is mentioned in:
http://cookiecutter.readthedocs.io/en/latest/advanced_usage.html#calling-cookiecutter-functions-from-python
My script generates a random string for use in a Django project. I called my script cut-cut:
#! /usr/bin/env python
from cookiecutter.main import cookiecutter
import os
rstring = ''.join([c for c in os.urandom(1024)
if c.isalnum()])[:64]
cookiecutter(
'django-template', # path/url to cookiecutter template
extra_context={'secret': rstring},
)
So now I simply run cut-cut and step through the process as normal. The only difference is that the entry named secret in my cookiecutter.json file is prepopulated with the generated value in rstring from the script, provided via the extra_context passed.
You could modify the script to accept the template via the command line, but in my usage I always use the same template, thus I simply pass a hard coded value "django-template" as noted in the code above.
looking for something similar to .Net string format in a chef recipe ie.
string phone = String.format("phone: {0}",_phone);
I have a Chef recipe where I need to build up a command string with 30 of these params so hoping for a tidy way to build the string, in principle Im doing this
a=node['some_var'].to_s
ruby_block "run command" do
block do
cmd = shell_out!("node mycommand.js #{a}; exit 2;")
end
end
When I try this I get the error
Arguments to path.join must be strings any tips appreciated
Chef runs in two phases:
Compile and Execute (see https://www.chef.io/blog/2013/09/04/demystifying-common-idioms-in-chef-recipes/ for more details).
Your variable assignment to a happens at compile time, e.g. when chef loads all recipes. The ruby block will be execute in execution mode at converge time and cannot access the variable a.
So the easiest solution might be putting the attribute into the ruby block:
ruby_block "run command with argument #{node['some_var']}" do
block do
shell_out!("node mycommand.js #{node['some_var']}")
end
end
However:
If you don't need to execute Ruby code, consider using the execute or bash resource instead.
Keep in mind, that you must have a unique resource name, if you're building some kind of loop around it. An easy way is to put something unique into the name ruby_block "something unique per loop iteration" do ... end
What I really don't understand is your exit code 2. This is an error code. It will make chef throw an exception each time. (shell_out! throws an exception if exit code != 0, see https://github.com/chef/chef/blob/master/lib/chef/mixin/shell_out.rb#L24-L28)
The resource will be executed on each chef run. This is probably not in your interest. Consider adding a guard (test), to prevent unnecessary execution, see https://docs.chef.io/resource_common.html#guards