How to change permissions or call a command without having to change permissions in an ONLINE Python environment? - python-3.x

I tried to make it so that my bot will create a directory inside a directory in the bot to store server data.
I originally just wanted to create the directory without worrying about permissions:
#client.event
async def on_guild_join(guild):
print(f'Recognized that Beatboxer has joined {guild.name}')
guild_path = rf'/guilds/{guild.id}'
if not os.path.exists(guild_path):
os.makedirs(rf'/guilds/{guild.id}')
An error message came up that looked like this:
Ignoring exception in on_guild_join
Traceback (most recent call last):
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/client.py", line 312, in _run_event
await coro(*args, **kwargs)
File "main.py", line 227, in on_guild_join
os.makedirs(rf'guilds/{guild.id}')
File "/usr/lib/python3.8/os.py", line 223, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: 'guilds/727168023101964298'
I then tried adding os.chmod into the code, but, for some reason, still had the same error message.
os.chmod("guilds", 777)
#client.event
async def on_guild_join(guild):
print(f'Recognized that Beatboxer has joined {guild.name}')
guild_path = rf'/guilds/{guild.id}'
if not os.path.exists(guild_path):
os.makedirs(rf'/guilds/{guild.id}')
Also, calling os.chdir and changing the directory into there did not work, and had a similar error message.
os.chmod("guilds", 777)
#client.event
async def on_guild_join(guild):
print(f'Recognized that Beatboxer has joined {guild.name}')
guild_path = rf'/guilds/{guild.id}'
if not os.path.exists(guild_path):
os.chdir('/guilds')
os.makedirs(rf'{guild.id}')
Finally, I attempted one last thing (which obviously still didn't work), which was os.popen, which opens a pipe for a command, allowing it to transfer the output to a file which is editable by other programs (which therefore should not regard what any permissions do):
#client.event
async def on_guild_join(guild):
print(f'Recognized that Beatboxer has joined {guild.name}')
guild_path = rf'/guilds/{guild.id}'
if not os.path.exists(guild_path):
os.popen(os.makedirs(rf'/guilds/{guild.id}'))
All of these attempted codes have very similar error messages, particularly Errno 13. Computer configuration will most likely not work. Please help? Thank you!

The problem is not that the functions you use are wrong.
You are actually giving a directory location you dont have permission to.
There are 2 kinds of paths to folders. Absolute paths and relative paths. In the examples you gave you used absolute path. When you use the absolute path you use the root directory as starting point. The thing with online IDE's is that you often dont have direct access to the root directory. Thus making new directories will give permission errors.
So how do we fix this? I suggest using relative paths instead. To fix this in your code is really easy, instead of doing this:
'/path/to/folder'
Do this:
'./path/to/folder'
By using ./ instead of /. You use the current folder as your starting point instead of the root directory. As you often have access to the current folder this wont give permission errors.

Related

FastText Error! ValueError: (file-name) cannot be opened for training

I Have installed fasttext module in Python and loaded the model [ 'cc.en.300.bin'].
I already made the data frame format according to the fasttext. and then generating the files
train.to_csv(" ecomm.train",columns=['Category_description'], index= False, header= False)
test.to_csv("ecom.test", columns=['Category_description'], index= False, header= False)
the files created successfully! then when I run this code
import fasttext
mod= fasttext.train_supervised(input='ecomm.train')
I get this error:
Traceback (most recent call last):
File "/Users/rosie/Documents/ProGraMinG/Python/pythonProject/FastText/FastText_overview.py", line 97, in <module>
mod= fasttext.train_supervised(input='ecomm.train')
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/fasttext/FastText.py", line 533, in train_supervised
fasttext.train(ft.f, a)
ValueError: ecomm.train cannot be opened for training!
{ UPDATE } !!!
Used both isfile() and exists() functions to check if the file exists:
path = 'Users/rosie/Documents/ProGraMinG/Python/pythonProject/FastText/ecomm.train'
check_file = os.path.isfile(path)
print("isfile method ",check_file)
check_file = os.path.exists(path)
print("exists method ",check_file)
Both methods returns ' False '.
I also checked if the file is readable or not
doc= open(' ecomm.train', 'r')
print('checking if the file is readable', doc.readable())
However, it returned 'True', now I'm confused. As for the size of the ' ecomm.train', it is 29.4 MB
Are you sure the file is readable, at the simple (local) path 'ecomm.train', from your Python process, given its current local orking directory?
For example, try specifying the file as its full absolute path – on MacOS probably something like /Users/yourusername/yourdirectory/etc/etc/ecomm.train. If that works, the problem was that your Python code's effective directory wasn't what you expected.
Alternatively, if the process that wrote the file was in some way a different user than the later process trying to read it, there might be permission errors.
Totally separate from fasttext, you could check, from the same code that's about to try fasttext operations, if the file is readable (via either the local path, or the absolute path) using a recipe lie that in this other answer: https://stackoverflow.com/a/44213239/130288
Even if it fails, it might give a more-explanatory error.

How do I resolve a permission error with django?

I am using Django to display my code in an html format using class based views. My code reads a log file and displays stats based on what is in there. When I try to access the log file from my computer this error message is displayed:
[Errno 13] Permission denied: 'C:/Users/bhattaar/Downloads/access.log'
I first tried going through the properties of the folder to make sure everything was set to read only (which it already was) and then I tried running the command prompt as an administrator and it still would not work.
The line the error appears on is this:
log = open('C:/Users/bhattaar/Downloads/access.log', 'r')
Does anyone know how I can resolve this issue?
Does this work?
Taken from here
log_file = r'C:/Users/bhattaar/Downloads/access.log'
with open(log_file) as f:
f = f.readlines()

FileNotFoundError: [WinError 2] The system cannot find the file specified while loading model from s3

I have recently saved a model into s3 using joblib
model_doc is the model object
import subprocess
import joblib
save_d2v_to_s3_current_doc2vec_model(model_doc,"doc2vec_model")
def save_d2v_to_s3_current_doc2vec_model(model,fname):
model_name = fname
joblib.dump(model,model_name)
s3_base_path = 's3://sd-flikku/datalake/current_doc2vec_model'
path = s3_base_path+'/'+model_name
command = "aws s3 cp {} {}".format(model_name,path).split()
print('saving...'+model_name)
subprocess.call(command)
It was successful, but after that when i try to load the model back from s3 it gives me an error
model = load_d2v("doc2vec_model")
def load_d2v(fname):
model_name = fname
s3_base_path='s3://sd-flikku/datalake/current_doc2vec_model'
path = s3_base_path+'/'+model_name
command = "aws s3 cp {} {}".format(path,model_name).split()
print('loading...'+model_name)
subprocess.call(command)
model=joblib.load(model_name)
return model
This is the error i get:
loading...doc2vec_model
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 7, in load_d2v
File "C:\Users\prane\AppData\Local\Programs\Python\Python37\lib\subprocess.py", line 339, in call
with Popen(*popenargs, **kwargs) as p:
File "C:\Users\prane\AppData\Local\Programs\Python\Python37\lib\subprocess.py", line 800, in __init__
restore_signals, start_new_session)
File "C:\Users\prane\AppData\Local\Programs\Python\Python37\lib\subprocess.py", line 1207, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
I don't even understand why it is saying File not found, this was the path i used to save the model but now i'm unable to get the model back from s3. Please help me!!
I suggest that rather than your generic print() lines, showing your intent, you should print the actual command you've composed, to verify that it makes sense upon observation.
If it does, then also try that exact same aws ... command directly, at the command prompt where you had been launching your python code, to make sure it runs that way. If it doesn't, you may get a more clear error.
Note that the error you're getting doesn't particularly look like it's coming from the aws command, of from the S3 service - which might talk about 'paths' or 'objects'. Rather, it's from the Python subprocess system & Popen' call. I think those are via your call tosubprocess.call(), but for some reason your line-of-code isn't shown. (How are you running the block of code with theload_d2v()`?)
That suggests the file that's no found might be the aws command itself. Are you sure it's installed & runnable from the exact working-directory/environment that your Python is running in, and invoking via subprocess.call()?
(BTW, if my previous answer got you over your sklearn.externals.joblib problem, it'd be good for you to mark the answer as accepted, to save other potential answerers from thinking that's still an unsolved question that's blocking you.)
try to add extension of your model file to your fname if you are confident the model file is there.
e.g. doc2vec_model.h3

shutil.rmtree() error when trying to remove NFS-mounted directory

Attempting to execute shutil.rmtree(path) on a directory managed by NFS consistently fails. Below you can see that os.rmdir(path) within shutil.rmtree(path) causes the exception. Is there a more robust way for me to achieve the expected result?
It appears that it removes all of the files, yet a hidden .nfs file remains in the directory for a short amount of time. I'm guessing that the process from which I'm calling rmtree has an open file handle to one of the files inside the directory, which, when deleted, apparently causes NFS to write a new hidden file. That would cause os.rmdir to fail on attempting to remove a non-empty directory.
Traceback (most recent call last):
File "/home/me/pre3/lib/python3.6/shutil.py", line 484, in rmtree
onerror(os.rmdir, path, sys.exc_info())
File "/home/me/pre3/lib/python3.6/shutil.py", line 482, in rmtree
os.rmdir(path)
OSError: [Errno 39] Directory not empty:
NFS details:
$ nfsstat -m
/home/me/nfs from XXX.YYY.ZZZ:/mnt/path/to/nfs
Flags: rw,relatime,vers=3,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,timeo=50,retrans=2,sec=sys,mountaddr=REDACTED,mountvers=3,mountport=832,mountproto=udp,local_lock=none,addr=REDACTED
I'm using Python 3.6.6 on Ubuntu 16.04.
If the python logging module is logging to the target output directory, it will maintain an open file. A workaround is to call logging.shutdown() first, then called shutil.rmtree(path). This is not a general answer to the broader question, however.
You could try defining an error handler function to be passed to the onerror arg for shutil.rmtree: https://docs.python.org/3/library/shutil.html#shutil.rmtree
def handle_rmtree_err(function, path, excinfo):
...
shutil.rmtree(my_path, onerror=handle_rmtree_err)
There are all sorts of reasons why a process may be holding onto a file, so I can't tell you what the error handler should do exactly.
If you haven't figured out what is holding onto the file, try $ lsof | grep .nfsXXXX.
If all else fails you could time.sleep(secs) and retry shutil.rmtree.

Why does draw() in pygraphviz/agraph not work on the server (but locally)?

I have a Python app using Pygraphviz that works fine locally, but on the server the draw function throws an error. It happens in make_svg. The following lines are the relevant part of the errors I get. (The full trail is here.)
File "/path/to/app/utils/make_svg.py", line 17, in make_svg
prog='dot'
File "/path/to/pygraphviz/agraph.py", line 1477, in draw
fh = self._get_fh(path, 'w+b')
File "/path/to/pygraphviz/agraph.py", line 1506, in _get_fh
fh = open(path, mode=mode)
FileNotFoundError: [Errno 2] No such file or directory: 'app/svg_files/nope.svg'
Logging type(g) gives <class 'pygraphviz.agraph.AGraph'> as expected.
I work in a virtualenv in a mod_wsgi 4.6.5/Python3.7 environment on a Webfaction server.
Locally I use a virtualenv with Python 3.5.
The version of Pygraphviz is 1.3.1.(First I had 1.5 on the server. The error was exactly the same, except for the line numbers.)
What can I do?
The same error is described in this bug report from last year.
I don't get which directory I am supposed to create. svg_files exists and has rights 777.
The draw function at the end of make_svg should create the SVG.(And at the end of extract_coordinates_from_svg the file is removed again.)The file name is a hash created in connected_dag (svg_name).
On the server app/svg_files seems not to describe the same place as locally.
I defined the path unambiguously, and now it works.
file_path = '{grandparent}/svg_files/{name}.svg'.format(
grandparent=os.path.dirname(os.path.dirname(__file__)),
name=name
)
g.draw(file_path, prog='dot')

Resources