I recently created a Snakemake profile using the guide at Snakemake-Profiles/slurm. I was able to get the profile installed successfully, and it does work when calling the path directly. However, when using the profile name, such as
snakemake --profile slurm --dry-run
I get the error:
Error: profile given but no config.yaml found. Profile has to be given as
either absolute path, relative path or name of a directory available in either
/etc/xdg/snakemake or /home/GROUP/USERNAME/.config/snakemake.
I have indeed installed the profile under ~/.config/snakemake. Here is the tree of this directory:
/home/GROUP/USERNAME/.config/snakemake
.
└── slurm
├── cluster_config.yaml
├── config.yaml
├── CookieCutter.py
├── __pycache__
│ ├── CookieCutter.cpython-39.pyc
│ └── slurm_utils.cpython-39.pyc
├── settings.json
├── slurm-jobscript.sh
├── slurm-status.py
├── slurm-submit.py
└── slurm_utils.py
2 directories, 10 files
I can continue to specify the path to this profile when running Snakemake, but it would be useful to simply give it the name of the profile. Does anyone happen to know why Snakemake doesn't seem to be picking up that the profile slurm exists?
I've solved my issue by installing Snakemake in a Conda environment, and re-installing the profile. I'm not sure if it was the Conda environment or the profile re-install that fixed my issue.
Related
I've established a private github organization for common python repos - each repo is basically a unique python 3rd party package (like numpy for example) but that are homegrown. These are to be used across different projects.
At the moment, the repos are just source packages, not compiled with wheels or sdist for releases - so each has a setup.py, and directory structure for the modules/business logic of the library. Basically the repos look somewhat like this: https://packaging.python.org/tutorials/packaging-projects/
At the moment, I don't want to address compiling releases or a private PyPI server. What I need help/guidance on is what if its not just a library, but also has a CLI tool (that uses the library).
I expect the user to one of several things: clone it, set PYTHONPATH/PATH accordingly, and use it, or package and pip install it. but should the CLI tool be included inside that repo, or outside? how does one call it (ie. python -m ).
Whats strange to me is that packages seem more geared towards true libraries and not libraries+tools. Any help in my thought process on this and how to invoke?
Thanks to #phd for helping me walk the dog.
For my package project, I define a setup.py (surrogate makefile in python's parlance) which defines this entry point:
setuptools.setup(
name="pkg_name", # Replace with your package name
version="0.0.1", # see pep 440
...
scripts=['bin/simple_cli.py'], # callable script to register (updates PATH)
...
)
Now in the package project itself, basic structure is as follows and I'll highlight the bin/ directory:
$ tree -L 3
.
├── bin
│ └── simple_cli.py
├── contributing.md
├── LICENSE
├── makefile
├── pkg_name
│ ├── example_module.py
│ ├── __init__.py
├── README.md
├── requirements.txt
├── setup.py
└── tests
├── test_main.py
Once this is built (sdist, wheels, etc), we can use pip install . I test this in a virtual environment and here is where simple_cli.py exists:
The comments above have some references, but end-state is the file is installed in the venv bin/ directory (which is available on PATH with an activated venv).
I am getting this error when i am packaging my smart contract project in VS code IBM extendion
The Go smart contract is not a subdirectory of the path specified by the environment variable GOPATH. Please correct the environment variable GOPATH.
This is the image of my error and my path variables
https://imgur.com/DiNz7k6
First you should confirm that vscode is picking up your GOPATH. If you open the terminal view in vscode and type
echo $GOPATH
to confirm the gopath matches. Then you need to open VSCode to the location of your Go chaincode which must be in the src directory in your go workspace pointed to by your go path. For example here is my go chaincode project called testcc and the actual chaincode source is in mycc
└── testcc
├── bin
├── pkg
│ └── linux_amd64
└── src
├── github.com
├── golang.org
└── mycc
located at ~/mycode. Therefore I would have a GOPATH of ~/mycode/testcc and I would open vscode up at the mycc directory. For example I would launch vscode as follows
$ GOPATH=~/mycode/testcc code ~/mycode/testcc/src/mycc
My understanding is that Bazel expects projects to be under a monorepo with a WORKSPACE file at the top-level and BUILD files in every project:
Repo
├── ProjectA
│ └── BUILD
├── ProjectB
│ └── BUILD
└── WORKSPACE
However, going through the Bazel NodeJS rules documentation, it seems to suggest that every project should have it's own WORKSPACE file where it defines its dependencies. i.e. ...
Repo
├── ProjectA
│ ├── BUILD
│ └── WORKSPACE
└── ProjectB
├── BUILD
└── WORKSPACE
This looks similar to a multi-repo with every project referencing other projects as an external dependency, which seemed okay to me, until I realized that for external dependencies, Bazel requires all transitive dependencies to be specified in the WORKSPACE file for every package, which is definitely not ideal.
What's the easiest way to use Bazel with NodeJS projects, with some projects possibly written in other languages? Also, is there an example somewhere for Bazel being used in a multi-repo setting?
Thanks!
I think the 2 possible options are in fact
Repo
├── MyProject
│ └── BUILD
├── third_party
│ └── ProjectB
│ └─ BUILD
└── WORKSPACE
or
Repo
├── MyProject
│ └── BUILD
└── WORKSPACE
where in the second case WORKSPACE references ProjectB with npm_install rule as defined in https://github.com/bazelbuild/rules_nodejs#using-bazel-managed-dependencies
I'm still trying to figure this out myself, but what I've gathered so far is that there is only one WORKSPACE file at the root of the repo. You need to have a package.json file (probably at the root) containing all the dependencies used in the whole repo then call npm_install or yarn_install in the WORKSPACE file to download them all.
Then your package BUILD file can reference a dependency with #npm//some_package as in:
filegroup(
name = 'sources',
srcs = ['index.js'],
)
npm_package(
name = 'pkg',
srcs = [ 'package.json'],
deps = [
':sources'
'#npm//lodash'
],
)
There are a few different dependency edge cases I haven't figured out yet so this may not be perfectly correct. Good luck.
How to start the pyscaffold project?
I use this command to create the project putup sampleProject
but i don't know how to start this project?
You don't start a pyscaffold project per say -- Its goal is simply to create the files and folder that you will commonly need for your project. See my structure below from "putup MyTestProject". Look at all the nice stuff already created that you now don't have to do by hand.
To get started, you need to start adding packages/code to "..src/mytestproject" and run that code like you normally would.
Might I recommend for you the use of a good IDEA, such as pycharm? I think you will find it makes starting your journey much easier.
A second recommendation -- if you are just getting started, you might skip pyscaffold for now. While a great tool, it might add confusion that you don't need right now.
MyTestProject/
├── AUTHORS.rst
├── CHANGELOG.rst
├── docs
│ ├── authors.rst
│ ├── changelog.rst
│ ├── conf.py
│ ├── index.rst
│ ├── license.rst
│ ├── Makefile
│ └── _static
├── LICENSE.txt
├── README.rst
├── requirements.txt
├── setup.cfg
├── setup.py
├── src
│ └── mytestproject
│ ├── __init__.py
│ └── skeleton.py
└── tests
├── conftest.py
└── test_skeleton.py
[Edit]
With respect to why "python skeleton.py" gives an output, the library is simply providing an example to show the user where to start adding code, and how the code relates to the tests (test_skeleton.py). The intent is that skeleton.py will be erased and replaced with your code structure. This may be some python.py files or packages and sub packages with python.py files. Read it this way; "Your Code goes here ... and here is an arbitrary example to get you started."
But you have to ask yourself what you are trying to accomplish? If you are just creating a few scripts for yourself -- for nobody else in the world to see, do you need the additional stuff (docs, setup, licensing, etc?) If the answer is no - don't use pyscaffold, just create your scripts in a venv and be on your way. This scaffolding is meant to give you most of what you need to create a full, github worthy, project to potentially share with the world. Based on what I gather your python experience to be, I don't think you want to use pyscaffold.
But specific to your question. Were I starting with pyscaffold, I would erase skeleton.py, replace it with "mytester.py", use the begins library to parse my incoming command arguments, then write individual methods to respond to my command line calls.
Setup
I have the following tree structure in my project:
Cineaste/
├── cineaste/
│ ├── __init__.py
│ ├── metadata_errors.py
│ ├── metadata.py
│ └── tests/
│ └── __init__.py
├── docs/
├── LICENSE
├── README.md
└── setup.py
metadata.py imports metadata_errors.py with the expression:
from .metadata_errors.py import *
Thus setting a relative path to the module in the same directory (notice the dot prefix).
I can run metadata.py in the PyCharm 2016 editor just fine with the following configuration:
Problem
However, with this configuration I cannot debug metadata.py. PyCharm returns the following error message (partial stack trace):
from .metadata_errors import *
SystemError: Parent module '' not loaded, cannot perform relative import
PyCharm debugger is being called like so:
/home/myself/.pyenv/versions/cineaste/bin/python /home/myself/bin/pycharm-2016.1.3/helpers/pydev/pydevd.py --multiproc --module --qt-support --client 127.0.0.1 --port 52790 --file cineaste.metadata
Question
How can I setup this project so that PyCharm is able to run and debug a file that makes relative imports?
Today (PyCharm 2018.3) it is really easy but not obvious.
You can choose target to run: script name or module name by pressing the label "Script Path" in edit configuration window:
One of possible solutions could be to run your module through intermediate script which you'll run in debug mode.
E.g. test_runner.py:
import runpy
runpy.run_module('cineaste.metadata')
You might also try removing the last node (/cineaste) from the Working Directory. This configuration works (run and debug) for me (in Pycharm: 2017.2.2)
I would suggest not using * since that can cause many problems in the future, two classes or methods being named the same etc.