I'm working with a custom data set in the format
folder
│
│
└--train
└--──class1
| │ file011
| │ file012
|
|
└───--class2
│ file021
│ file022
└--val
└--──class1
| │ file011
| │ file012
|
|
└───--class2
│ file021
│ file022
When trying to load the dataset
data_dir = r'PATH_TO_DATA/train'
dataset = datasets.ImageFolder(data_dir, ...)
FileNotFoundError: Found no valid file for the classes Cat, Deer, Dog, Human. Supported extensions are: .jpg, .jpeg, .png, .ppm, .bmp, .pgm, .tif, .tiff, .webp
The only issue I found similar to this was here, however in their case there seemed to be a .ipynb_checkpoints file which was causing the issue. It doesn't appear to be the case here.
I also checked for hidden files, and made sure the extensions are acceptable.
Edit: An important piece of information that I didn't realize was the issue seems to be the issue. I am hosting this data on a remote using Rclone, mounting my onedrive to access the data. When accessing the data directly, the dataset is read just fine. It seems to be an issues of ImageFolder accessing data via the remote access more than anything else.
One way to decompose the problem can be open a file with pillow.
import torch
import torchvision.transforms.funcional as TF
from PIL import Image
img = Image.open('PATH_TO_DATA/train/class1/file011')
img = TF.pil_to_tensor(img)
print(f'[DEBUG] img: {img.shape}, {img.min()}, {img.max()}')
Related
I have started using VSCode and was wondering what the / meant when I click on a file (see attached screenshot). Is it simply the full path of the file that I've clicked on? Thanks!
It means sub folder inside main folder you have created
I think you may be referring to the feature where a folder with a single subfolder are shown on the same line.
For example, let's consider the following structure:
X/
├─ a/
│ ├─ a1/
│ │ └- a1.txt
│ └- a2/
│ └- a2.txt
└ b/
└- b1/
└- b1.txt
Since b1 is the only folder under b the explorer window will show b and b1 on a single line:
eth-sig-util is child folder of metamask folder which is parent.
To access child we have to use '/' after parent folder.
Analyzing the path, the Node.js considering the root as the part of the directory:
/home/user/dir/file.txt
┌─────────────────────┬────────────┐
│ dir │ base │
├──────┬ ├──────┬─────┤
│ root │ │ name │ ext │
" / home/user/dir / file .txt "
└──────┴──────────────┴──────┴─────┘
C:\\path\\dir\\file.txt
┌─────────────────────┬────────────┐
│ dir │ base │
├──────┬ ├──────┬─────┤
│ root │ │ name │ ext │
" C:\ path\dir \ file .txt "
└──────┴──────────────┴──────┴─────┘
Is it actually so? Developing a new library, I am thinking must I consider the root as the part of directory, or no.
Well, actually the "directory" is a fuzzy term. According the definition,
In computing, a directory is a file system cataloging structure which
contains references to other computer files, and possibly other
directories.
Wikipedia
Nothing that answers on my question. What else we know?
When we are using the cd (the abbreviation of "change directory") command, we are specifying the path relative to current location. Nothing related with root.
The cd command works inside the specific drive (at least, on Windows). This indirectly could means that the root and directory could be combined but initially separated.
More exact terms are the "absolute path of the directory" and "relative path of the directory". But what is the "directory" itself?
For the Windows case, the data storage name could be different on separate computers but it does not affect to files structure inside the storage. Again, the root and directory are separate in this case.
I'm trying to get used to using the 'electronegativity' tool to look around inside opensource electron projects. Just to get familiar with it.
So I'm running it on a non-descript open source project, (this one if you would like to know https://github.com/hello-efficiency-inc/raven-reader)
──────────┤
│ CONTEXT_ISOLATION_JS_CHECK │ /home/ask/Git/raven-reader/src/main/pocket.js │ 11:23 │ Review the use of the contextIsolation option │
│ HIGH | FIRM │ │ │ https://git.io/Jeu1p │
This looks interesting, it tells me that there are some issues with the context isolation in the pcoket.js file at line 11.
This would in my mind indicate that I can find something like this:
contextIsolation: true
However, when I look in the code in the location that electronegativity indicates, I just find this:
const authWindow = new BrowserWindow({
width: 1024,
height: 720,
show: true
})
And I can't find any mention of context isolation anywhere in the file. Does this mean that context isolation is finding something that isn't there? or is there something smelly in the creation of the Browserwindow?
I'm writing a terraform script to create an EKS cluster with its worker nodes on AWS. First time doing it so I'm a bit confused.
Here is the folder organisation:
├─── Int AWS Account
│ ├─── variables.tf
│ ├─── eks-cluster.tf (refers the modules)
│ ├─── others
│
├─── Prod AWS Account
│ ├─── (will be the same than Int with different settings in variables)
│
├─── ReadMe.md
│
├─── data sources
│
├─── Modules
│ ├─── cluster.tf
│ ├─── worker-nodes.tf
│ ├─── worker-nodes-sg.tf
I am a bit confused regarding how to use and pass variables. Right now, what I'm doing is that I refer to ${var.name} in the module folder, in the eks-cluster.tf, I either put a direct value name = blabla (mostly avoiding it), or refer to the variable again and have a variable file in the account folder.
Is that correct?
I'm not sure if I get your question correctly but in general you would want to keep your module files with variables only, as modules are intended to be generic so you can easily include them in different environments.
When including the module in eks_cluster_int.tf or eks_cluster_prod.tf you would then pass the values for all variables defined in the module itself. This way you can use the environment specific values in the same module.
module "cluster" {
source = "..."
var1 = value1 # directly passing value
var2 = ${var.int_specific_var} # can be defined in variables.tf of environment
...
}
Does this answer your question?
I got a question that relates to (maybe is a duplicate of) this question here.
I try to write a pandas dataframe to an Excel file (non-existing before) in a given path. Since I have to do it quite a few times, I try to wrap it in a function. Here is what I do:
df = pd.DataFrame({'Data': [10, 20, 30, 20, 15, 30, 45]})
def excel_to_path(frame, path):
writer = pd.ExcelWriter(path , engine='xlsxwriter')
frame.to_excel(writer, sheet_name='Output')
writer.save()
excel_to_path(df, "../foo/bar/myfile.xlsx")
I get thrown the error [Errno 2] No such file or directory: '../foo/bar/myfile.xlsx'. How come and how can I fix it?
EDIT : It works as long the defined pathis inside the current working directory. But I'd like to specify any given pathinstead. Ideas?
I usually get bitten by forgetting to create the directories. Perhaps the path ../foo/bar/ doesn't exist yet? Pandas will create the file for you, but not the parent directories.
To elaborate, I'm guessing that your setup looks like this:
.
└── src
├── foo
│ └── bar
└── your_script.py
with src being your working directory, so that foo/bar exists relative to you, but ../foo/bar does not - yet!
So you should add the foo/bar directories one level up:
.
├── foo_should_go_here
│ └── bar_should_go_here
└── src
├── foo
│ └── bar
└── your_script.py