Permission denied while downloading dataset from Kaggle API - permission-denied

I am using Google cloud services and I want to download kaggle dataset. I have followed the standard ways for authentication but still, I am getting "Permission denied" error
My approach:
I have tried the following method:
(1) Installing kaggle using pip
(2) placing the kaggle.json file in /.kaggle directory
(3) ''' chmod 600 /home/rushirajparmar23000/.kaggle/kaggle.json '''
(4) ''' echo '{"username":"USERNAME","key":"KEY"}' >/root/.kaggle/kaggle.json '''
error : -bash: /root/.kaggle/kaggle.json: Permission denied
(5) kaggle competitions download -c facebook-recruiting-iii-keyword-extraction
error: raise ValueError('Error: Missing %s in configuration.' % item)
ValueError: Error: Missing username in configuration.

Maybe this discussion in the kaggle forum might answer your question:
https://www.kaggle.com/general/51898
It has a script you can run to do what you want.

When I wanted to download a dataset, I got this error. (I think it's related to your question)
Traceback (most recent call last):
File "/home/matin/.local/bin/kaggle", line 5, in <module>
from kaggle.cli import main
File "/home/matin/.local/lib/python3.9/site-packages/kaggle/__init__.py", line 19, in <module>
from kaggle.api.kaggle_api_extended import KaggleApi
File "/home/matin/.local/lib/python3.9/site-packages/kaggle/api/__init__.py", line 22, in <module>
from kaggle.api.kaggle_api_extended import KaggleApi
File "/home/matin/.local/lib/python3.9/site-packages/kaggle/api/kaggle_api_extended.py", line 84, in <module>
class KaggleApi(KaggleApi):
File "/home/matin/.local/lib/python3.9/site-packages/kaggle/api/kaggle_api_extended.py", line 102, in KaggleApi
os.makedirs(config_dir)
File "/home/matin/miniconda3/lib/python3.9/os.py", line 225, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/kaggle.json'
Solution
sudo mkdir /.kaggle
sudo cp kaggle.json /.kaggle
sudo chmod 600 /.kaggle/kaggle.json
sudo chown `whoami`: /.kaggle/kaggle.json
export KAGGLE_CONFIG_DIR='/.kaggle/'
Explanation
In KaggleApi source code, it's trying to find a directory from the environment.
config_dir = os.environ.get('KAGGLE_CONFIG_DIR') or os.path.join(
expanduser('~'), '.kaggle')
if not os.path.exists(config_dir):
os.makedirs(config_dir)
So if you add KAGGLE_CONFIG_DIR to your environment, it doesn't try to make a directory!
If you got ValueError: Error: Missing username in configuration., you should just change the owner by using chown command.

This worked for me
removed the .kaggle directory that was created when installing the kaggle API:
rm -r .kaggle/
created a new .kaggle directory in the home directory:
mkdir .kaggle
downloaded the kaggle.json file from my kaggle account:
copied the kaggle.json to .kaggle/:
cp ~/Downloads/kaggle.json .kaggle/
chmod 600 .kaggle/kaggle.json

Related

FileNotFoundError while using crontab to run Python script

I'm totally lost here. I'm trying to create a scheduler to run python script on my Mac, but I'm getting the following error:
Traceback (most recent call last):
File "/Users/Root/Desktop/Project/Data/script.py", line 148, in <module>
run(
File "/Users/Root/Desktop/Project/Data/script.py", line 121, in run
config = get_config("config")
File "/Users/Root/Desktop/Project/Data/config/__init__.py", line 3, in get_config
with open(f"config/{config_type}.json", "r") as file:
FileNotFoundError: [Errno 2] No such file or directory: 'config/config.json'
So crontab convinces me that there is no such file or a directory, which is not true. I can run my script manually without errors.
My crontab is:
00 19 21 1-12 * /Library/Frameworks/Python.framework/Versions/3.9/bin/python3/ /Users/Root/Desktop/Project/Data/script.py >> /Users/Root/Desktop/Project/Data/cron.txt 2>&1
What am I doing wrong?
I'd be grateful for any help!
And is this possible without changing the relative path to an absolute path? I am aware of this solution
I assume crontab's cwd (Current Working Directory) is not same as where the script is stored.
this would solve your problem:
import os
script_dir = os.path.dirname(os.path.realpath(__file__))
os.chdir(script_dir)
You can get the directory where you script is by calling "os.path.dirname(os.path.realpath(file))"
if you change the current working directory "os.chdir(...dir...)" you can access you config/config.json by relative path,
Otherwise you will have to use a absolute path
Try running this and check your output file:
import os
script_dir = os.path.dirname(os.path.realpath(__file__))
print (os.getcwd())
print(script_dir)
os.chdir(script_dir)
print (os.getcwd())
Try to open with full path to the json file.

Python in Visual Studio Code getting winError[5] Access denied from idescripts

it is my first time posting here so I apologize if this question is not correctly formatted.
I installed VSC to use it in developing STM32 code. I found the damongranlabs idescripts in github that would help greatly. While running the update.py script, I get the winError[5] access denied. VSC is using powershell as a terminal and I am running windows 10 with Python 3.8 32bit.
I have tried running VSC as an admin with no luck.
The following is what I get in powershell after attempting to run the script:
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.
Try the new cross-platform PowerShell https://aka.ms/pscore6
PS D:\Development\STM32\cubemxprojects\476EncTest> & d:/Development/STM32/cubemxprojects/476EncTest/ideScripts/update.py
Update started.
Existing '.vscode' folder used.
One STM32CubeMX file found: 476EncTest.ioc
Existing 'Makefile' file found (restored from '.backup').
Copy of file (new name: Makefile): D:/Development/STM32/cubemxprojects/476EncTest/Makefile.backup
Makefile 'print-variable' function added.
Valid 'buildData.json' file found.
Valid 'toolsPaths.json' file found.
'toolsPaths.json' file updated!
Enter path(s) to OpenOCD configuration file(s):
Example: 'target/stm32f0x.cfg'. Absolute or relative to OpenOCD /scripts/ folder.
If more than one file is needed, separate with comma.
Paste here and press Enter: C:\Users\omis2\AppData\Roaming\GNUMCUEclipse\GNU MCU Eclipse\OpenOCD\0.10.0-12-20190422-2015\scripts\target\stm32l4x.cfg
Enter path or command for 'stm32SvdPath':
Paste here and press Enter: C:\Users\omis2\AppData\Roaming\GNUMCUEclipse\Keil.STM32L4xx_DFP.2.2.0\CMSIS\SVD\STM32L4x6.svd
ERROR (55 seconds).
Unexpected error occured during 'Update' procedure. Exception:
Traceback (most recent call last):
File "D:\Development\STM32\cubemxprojects\476EncTest\ideScripts\update.py", line 56, in <module>
makefileData = makefile.getMakefileData(makeExePath, gccExePath)
File "D:\Development\STM32\cubemxprojects\476EncTest\ideScripts\updateMakefile.py", line 93, in getMakefileData
projectName = self.getMakefileVariable(makeExePath, gccExePath, self.mkfStr.projectName)[0]
File " D:\Development\STM32\cubemxprojects\476EncTest\ideScripts\updateMakefile.py", line 366, in getMakefileVariable
proc = Popen(arguments, stdout=PIPE)
File "C:\Users\omis2\AppData\Local\Programs\Python\Python38- 32\lib\subprocess.py", line 854, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Users\omis2\AppData\Local\Programs\Python\Python38- 32\lib\subprocess.py", line 1307, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
PermissionError: [WinError 5] Access is denied
Thanks for any help

tensorflow.python.framework.errors_impl.NotFoundError adds $ sign

I'm trying to train a pre-trained object detection model to detect objects from my custom dataset. All is running on Google Colab. I prepared the images, created TFRecord files for train and test, installed Tensorflow Object detection API from source, and tested that it works.
First I suspected it is a PYTHONPATH problem, but even when adding the folder with the config to path, it does not work.
This is my command line (I invoke the script from research folder, as in documentation):
#From the tensorflow/models/research/ directory
PIPELINE_CONFIG_PATH='/content/gdrive/My\ Drive/AI/grape4/work/model/ssd_mobilenet_v2_oid_v4.config'
MODEL_DIR=os.path.join('/content/gdrive/My\ Drive/AI/grape4/work', 'model')
NUM_TRAIN_STEPS=5000
NUM_EVAL_STEPS=1000
!python object_detection/model_main.py \
--pipeline_config_path=${PIPELINE_CONFIG_PATH} \
--model_dir=${MODEL_DIR} \
--num_train_steps=${NUM_TRAIN_STEPS} \
--num_eval_steps=${NUM_EVAL_STEPS} \
--alsologtostderr
Below is the error I'm getting. I confirm the mentioned file exists in the folder. But what is strange to me is the added $ sign (dollar sign) in the trace:
Traceback (most recent call last):
File "object_detection/model_main.py", line 109, in <module>
tf.app.run()
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/platform/app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 300, in run
_run_main(main, args)
File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 251, in _run_main
sys.exit(main(argv))
File "object_detection/model_main.py", line 71, in main
FLAGS.sample_1_of_n_eval_on_train_examples))
File "/usr/local/lib/python3.6/dist-packages/object_detection-0.1-py3.6.egg/object_detection/model_lib.py", line 605, in create_estimator_and_inputs
pipeline_config_path, config_override=config_override)
File "/usr/local/lib/python3.6/dist-packages/object_detection-0.1-py3.6.egg/object_detection/utils/config_util.py", line 103, in get_configs_from_pipeline_file
proto_str = f.read()
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/lib/io/file_io.py", line 122, in read
self._preread_check()
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/lib/io/file_io.py", line 84, in _preread_check
compat.as_bytes(self.__name), 1024 * 512)
tensorflow.python.framework.errors_impl.NotFoundError: $/content/gdrive/My Drive/AI/grape4/work/model/ssd_mobilenet_v2_oid_v4.config; No such file or directory
Does anyone know where the problem might be?
I cannot exactly point what the problem is. But I think there might be 2 problems:
1. The path thats pointing to the .config file might be wrong.
2. The .config file might be corrupted.
Please take a look at this issues where problems #1 and #2 are being discussed clearly. I hope this helps!

Permission Error Permission Denied In simple basic Flask Application

I'm running this script on vagrant based Linux system, It's a simple basic code for running flask app, I'm getting this error:
raise child_exception_type(errno_num, err_msg)
PermissionError: [Errno 13] Permission denied
Here is the script:
#!/usr/bin python3
from flask import Flask
app = Flask(__name__)
#app.route('/')
#app.route('/hello')
def helloworld():
return 'Hello and Welcome to flask'
if __name__ == '__main__':
app.debug = True
app.run(host='0.0.0.0', port=5000)
And the error I got:
vagrant#vagrant:/vagrant$ python3 project.py
* Serving Flask app "project" (lazy loading)
* Environment: production
WARNING: Do not use the development server in a production environment.
Use a production WSGI server instead.
* Debug mode: on
* Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
* Restarting with stat
Traceback (most recent call last):
File "project.py", line 14, in <module>
app.run(host='0.0.0.0', port=5000)
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 943, in run run_simple(host, port, self, **options)
File "/usr/local/lib/python3.5/dist-packages/werkzeug/serving.py", line 988, in run_simple
run_with_reloader(inner, extra_files, reloader_interval, reloader_type)
File "/usr/local/lib/python3.5/dist-packages/werkzeug/_reloader.py", line 332, in run_with_reloader
sys.exit(reloader.restart_with_reloader())
File "/usr/local/lib/python3.5/dist-packages/werkzeug/_reloader.py", line 176, in restart_with_reloader
exit_code = subprocess.call(args, env=new_environ, close_fds=False)
File "/usr/lib/python3.5/subprocess.py", line 557, in call
with Popen(*popenargs, **kwargs) as p:
File "/usr/lib/python3.5/subprocess.py", line 947, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.5/subprocess.py", line 1551, in _execute_child
raise child_exception_type(errno_num, err_msg)
PermissionError: [Errno 13] Permission denied
I had the same problem, it turns out that the port needs to be greater than 5000:
from flask import Flask, request, abort, jsonify
app = Flask(__name__)
#app.route('/getSquare', methods=['POST'])
def get_square():
pass
return jsonify({'answer': num ** 2})
if __name__ == '__main__':
app.run(host='0.0.0.0', port=5001, debug=True) # make sure port is > 5000
Maybe it's a permission error since Vagrant is trying to run your flask application and doesn't have the necessary executable permission?
I say this because of this part of your stack trace:
PermissionError: [Errno 13] Permission denied
and appears to be failing on this line of code: app.run(host='0.0.0.0', port=5000)
Perhaps you could give executable permissions on your directory containing your flask project and see if that helps resolve your issue:
chmod -R 755 /path/to/directory
Hopefully that helps!
Since you're playing in a VM that you built using Vagrant, you must have done something like
vagrant#vagrant:/vagrant$ sudo pip3 install flask
first. Possibly you installed some other packages. Depending on what base OS you're using, using pip (or pip3) to install at the system level (even in a VM) can cause problems.
After wrestling with my base install getting mangled a few time, I now always use virtualenv inside of VMs so that all of the package installs are local.
I'll bet that
vagrant#vagrant:/vagrant$ sudo apt-get install -y python-virtualenv
vagrant#vagrant:/vagrant$ virtualenv --python=python3 venv
vagrant#vagrant:/vagrant$ venv/bin/pip install flask
vagrant#vagrant:/vagrant$ ven/bin/python project.py
will give you a better result.

PermissionError: [Errno 13] Permission denied while running a server-side CGI scripts coded in Python

I am trying to run a server-side cgi script coded in python but I am getting the following error while running it.
Traceback (most recent call last):
File "webserver.py", line 16, in <module>
srvrobj = HTTPServer(srvraddr,CGIHTTPRequestHandler)
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/socketserver.py", line 449, in __init__
self.server_bind()
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/server.py", line 137, in server_bind
socketserver.TCPServer.server_bind(self)
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/socketserver.py", line 463, in server_bind
self.socket.bind(self.server_address)
PermissionError: [Errno 13] Permission denied
Here is the script that I am trying to run.
"""
Implement an HTTP web server in Python that knows how to run server-side
CGI scripts coded in Python; serves files and scripts from current working
dir; Python scripts must be stored in webdir\cgi-bin or webdir\htbin;
"""
import os,sys
from http.server import HTTPServer, CGIHTTPRequestHandler
webdir = '.'
port = 80 #default http://localhost/, else use http://localhost:xxxx/
os.chdir(webdir)
srvraddr = ("",port)
srvrobj = HTTPServer(srvraddr,CGIHTTPRequestHandler)
srvrobj.serve_forever()
Any help will be appreciated. Thanks in advance.
Use this
chmod 777 script_name,
then execute the script if it fails again then use
sudo python script_name
or check how to run script using administrator privileges on your respective operating system.

Resources