python3: can't open file - How to access directories with Docker - python-3.x

I have tried multiple options but I just can't figure out how to
access my starting foo.py file which is hiden in my directory structure.
The structure looks like this:
D:
│___ Dockerfile
│
└───Level1
└───Level2
foo.py
Now to start the project I need to access the foo.py file, which then works with different files spread
over Level1 and 2.
The problem is that I am not able to access the foo.py file with something like:
COPY /Level1 .
CMD ["python3", "/Level1/Level2/foo.py"]
The error is python3: can't open file
Please help

Your path is wrong. Change your CMD from CMD ["python3", "/Level1/Level2/foo.py"] to CMD ["python3", "/Level2/foo.py"]
You are copying from Level1/ to ., so you lose that Level1 folder.
As a side note, please include the entire error message in the post. the error message can't open file is very confusing, however the error message python3: can't open file '/Level1/Level2/foo.py': [Errno 2] No such file or directory is very clear.

Related

Running Detectron2 locally - windows - [Pytorch Config error]

I am trying to run this code locally:
https://gist.github.com/shashank524/74d8f46d5de633b84e2265fcc34774de#file-tabledetection-ipynb
After installing required packages, when I am trying to run this line:
import layoutparser as lp
# PubLayNet
model = layoutparser.Detectron2LayoutModel('lp://PubLayNet/faster_rcnn_R_50_FPN_3x/config',extra_config=["MODEL.ROI_HEADS.SCORE_THRESH_TEST", 0.81],label_map={0: "Text", 1: "Title", 2: "List", 3:"Table", 4:"Figure"})
I receive this error:
OSError: [Errno 22] Invalid argument: 'C:\\Users\\username/.torch/iopath_cache\\s/f3b12qc4hc0yh4m\\config.yml?dl=1.lock'
I looked into the directory and there was no config file available.
I tried to download the config file from here (https://layout-parser.readthedocs.io/en/latest/notes/modelzoo.html) and put it in the directory but it didn't solve the issue!
Even I got a similar error. I tried out manually some work around in Windows.
I am using your case as example: OSError: [Errno 22] Invalid argument: 'C:\Users\username/.torch/iopath_cache\s/f3b12qc4hc0yh4m\config.yml?dl=1.lock'
Please follow the following process.
Navigate to C:\Users\username/.torch/iopath_cache\s/f3b12qc4hc0yh4m\config.yml
Open that config.yaml file
Scroll down to WEIGHTS: https://www.dropbox.com/s/h7th27jfv19rxiy/model_final.pth?dl=1 should be around 265 line.
Copy that link and paste it in your browser, a 'model_final.pth' will be downloaded. Copy this file to your desired folder.
Now replace the path to WEIGHTS: your_desired_folder\model_final.pth
Save it and run the code it works!
But there is a small work around I think before you do this (if you have not done)
iopath work around
https://github.com/Layout-Parser/layout-parser/issues/15 (Github link to the issue)
Hope this helps!

Adding non-python files to colcon build

Building my workspace with colcon, some OSM files which are found in a directory "OSM" in a sub_package in the workspace are not found in built space. So when I go to the install space, the files are not there. I am not sure how to do this and if I should put it in the setup.py.
I tried putting this in the setup.py file in arguments of setup():
setup(
name=package_name,
version='0.0.0',
packages=[package_name, submodules, osm],
data_files=[
('share/ament_index/resource_index/packages',
['resource/' + package_name]),
('share/' + package_name, ['package.xml']),
(os.path.join('share', package_name), glob('launch/*.launch.py')),
('.package_name/sub_package', glob('OSM_folder/*.osm')),
],
.
.
.
) # close setup()
but it did not work.
I am using ROS2 Galactic.
Directory structure:
package_name
┃
┣━━━━setup.py
┣━━━━package.xml
┣━━━━resource/
┣━━━━launch/
┗━━━━package_name
┗━sub_package_name
┗━OSM
┣━__init__.py
┗━some_osm_files.osm
I have the OSM directory in the built workspace but it has only the init.py file
I solved it. I am not sure if this is the right way of doing this or is there like another better/proper way or not, but here we go.
In the setup.py file, I added the line
(os.path.join('lib/python3.8/site-packages/package_name/sub_package/OSM'),glob(package_name+'/sub_package_name/OSM/*.osm')), in the data_files variable.
The first part of the new line which is os.path.join('lib/python3.8/site-packages/package_name/sub_package_name/OSM') determines the new location of the files in the install folder after building the workspace.
The second part which is glob(package_name+'/sub_package_name/OSM/*.osm') determines the files original location in the project workspace.
So the result is that it takes the files from the location mentioned in the second part and puts them in the location mentioned in the first part.
The resulting block is:
setup(
name=package_name,
version='0.0.0',
packages=[package_name, submodules, osm],
data_files=[
('share/ament_index/resource_index/packages',
['resource/' + package_name]),
('share/' + package_name, ['package.xml']),
(os.path.join('share', package_name), glob('launch/*.launch.py')),
(os.path.join('lib/python3.8/site-packages/package_name/sub_package_name/OSM'), glob(package_name+'/sub_package_name/OSM/*.osm')),
],
.
.
.
)

Writing file denied

I am getting an error writing a file, that is driving me crazy.
I have an C# netcore 5 application running on RH Linux.
I mounted an shared folder (windows) using: sudo mount -t cifs -o username=MyDomainUsername,password=MyDomainUsernamePassword,domain=MyDomain,dir_mode=0777,file_mode=0777 //ipv4_from_destination/Reports /fileshare/Reports
Then I run the app, using just ./WebApi --urls=http://+:8060
The read/write test executes the following steps:
Create a text file.
Write the text file.
Delete de text file.
Creates a directory
Creates a text file inside that directory
Writes the text file
Deletes the text file
Deletes the directory.
Now the problem:
The text file is created
The write operation fails.
Where goes part of the log:
Creating file: /fileshare/Reports/test.616db7d1-07fb-4599-a0cf-749e6a8b34ec.tmp...Ok
Writing file: /fileshare/Reports/test.616db7d1-07fb-4599-a0cf-749e6a8b34ec.tmp...[16:22:20 ERR] ID:87988856-a765-4474-9ed9-2f04aef35771 PATH:/api/about ERROR:System.UnauthorizedAccessException:Access to the path '/fileshare/Reports/test.616db7d1-07fb-4599-a0cf-749e6a8b34ec.tmp' is denied. TRACE: at System.IO.FileStream.WriteNative(ReadOnlySpan`1 source)
at System.IO.FileStream.FlushWriteBuffer()
at System.IO.FileStream.FlushInternalBuffer()
at System.IO.FileStream.Flush(Boolean flushToDisk)
at System.IO.FileStream.Flush()
at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
at System.IO.StreamWriter.Flush()
at WebApi.Controllers.ApplicationController.TestFileSystem(String folder) in xxxxxxx\WebApi\Controllers\ApplicationController.cs:line 116
What I discovered so far:
I can create and delete the files and directories.
I cannot write to files.
Can someone give me an hint on this?
Solved using the cifs option nobrl

Referencing a YAML config file, from Python, when a softlink is defined

I have the following code;
#!/usr/bin/env python3
import yaml
with open('config.yml', 'r') as config_file:
config = yaml.load(config_file)
The file is called __init__.py which is in the directory ~/bin/myprogram/myprogram/ and in the same directory, I have a file called config.yml
My symlink is as follows;
user$ ls -la /usr/local/bin/
lrwxr-xr-x 1 user admin 55 27 Nov 13:25 myprogram -> /Users/user/bin/myprogram/myprogram/__init__.py
Every time I run myprogram, I get the error FileNotFoundError: [Errno 2] No such file or directory: 'config.yml'. I believe this is because the config.yml is not in /usr/local/bin/. What is the best way to work around this issue?
You can use __file__ to access the location of the __init__.py file when executing code in that file. It returns the full path, but care has to be taken as it may be the .pyc (or .pyo) version. Since you are using Python3 I would use the pathlib module:
import yaml
from pathlib import Path
my_path = Path(__file__).resolve() # resolve to get rid of any symlinks
config_path = my_path.parent / 'config.yaml'
with config_path.open() as config_file:
config = yaml.safe_load(config_file)
Please note:
If you have to use PyYAML, use safe_load(), even PyYAML's own documentation indicates .load() can be unsafe. It almost never necessary to use that. And in the unlikely event that safe_load() cannot load your config, e.g. if it has !!python/... tags, you should explicitly add register the classes that you actually need to the SafeLoader).
Since September 2006 the recommended extension for YAML files has been .yaml

python3, directory is not correct when import a module in sub-folder

I have a main folder called 'test', the inner structure is:
# folders and files in the main folder 'test'
Desktop\test\use_try.py
Desktop\test\cond\__init__.py # empty file.
Desktop\test\cond\tryme.py
Desktop\test\db\
Now in the file tryme.py. I want to generate a file in the folder of 'db'
# content in the file of tryme.py
import os
def main():
cwd = os.getcwd() # the directory of the folder 'Desktop\test\cond'
folder_test = cwd[:-4] # -4 since 'cond' has 4 letters
folder_db = folder_test + 'db/' # the directory of folder 'db'
with open(folder_db + 'db01.txt', 'w') as wfile:
wfile.writelines(['This is a test.'])
if __name__ == '__main__':
main()
If I directly run this file, no problem, the file 'db01.txt' is in the folder of 'db'.
But if I run the file of use_try.py, it will not work.
# content in the file of use_try.py
from cond import tryme
tryme.main()
The error I got refers to the tryme.py file. In the command of 'with open ...'
FileNotFoundError: [Error 2] No such file or directory: 'Desktop\db\db01.txt'
It seems like the code
'os.getcwd()'
only refers to the file that calls the tryme.py file, not the tryme.py file itself.
Do you know how to fix it, so that I can use the file use_try.py to generate the 'db01.txt' in the 'db' folder? I am using Python3
Thanks
Seems like what you need is not the working directory, but the directory of the tryme.py file.
This can be resolved using the __file__ magic:
curdir = os.path.dirname(__file__)
Use absolute filenames from an environment variable, or expect the db/ directory to be a subdirectory of the current working directory.
This behavior is as expected. The current working directory is where you invoke the code from, not where the code is stored.
folder_test = cwd # assume working directory will have the db/ subdir
or
folder_test = os.getEnv('TEST_DIR') # use ${TEST_DIR}/db/

Resources