I have a shared drive where I have more than 2 million of wmv files of around 2TB total size. I am trying to access the drive by mounting it using smb protocol from my local MAC machine. When I run "
$ ls -a | wc -l
command to check the total count of files. I am getting different result every time. e.g if sometime I get the result as X then next time I am getting another result Y Here is the sample output sample output, which should not be as no other person is accessing this drive. When I investigate more I come to know that "ls" command output is different every time. This command should work as I have been using them since decade. Is it something I am doing wrong or in a large volume of data or network shared drive, this command fails ? I am sure there is no access or network issue while I am doing this activity. Any hint or work around will be much appreciated
I have faced the similar issue when I tried to access shared location which is having around 200K numbers of file.In my case the shared drive file system was NTFS file system. I believe there is a compatibility issue with SMB protocol and NTFS file system. Finally I tried to mount the shared drive using "NFS" instead of "SMB" and I could able to get the correct number of files in my mounted drive. This problem never happened in WINDOWS as I have mounted much higher number of files earlier using Windows many time. Hope this helps.
It is most likely because the file list is not immediately available to OSX from the network share. Apples implementation of SMB is still a bit buggy, unfortunately.
You may try: defaults write com.apple.desktopservices DSDontWriteNetworkStores -bool TRUE
And see if it helps.
Related
I am searching for a way to get all the metadata of the linux file system (ext2/3/4). The task is to find all the files (deleted/or not deleted) present on the linux partition. The metadata of the files should include creation time , modification time etc. (basically what you get from the command istat)
The problem i am facing is regarding the deleted files. I cannot find a way to get the inode of the deleted files currently present on the file system. Kindly suggest a way to solve this issue for the above mentioned file systems.
Thanks in advance.
You may find The Coroner's Toolkit to be quite useful. It includes tools to allow you to view any element of the metadata, directly view inodes, dump out all of the disk sectors that an inode references, dump disk sectors directly, etc. Since you are working with the inodes and sectors directly, it does not matter if they are deleted or not, they are all accessible.
I am working on a project for set top boxes and I need to mount a HDD partition formatted with xtvfs filesystem.
I have only received the following information regarding xtvfs:
The partition formatted with xtvfs is basically a type 1 (FAT12) to be used with xtvfs
I searched the internet and found a program called Copy+ that can be used to copy xtvfs images form one HDD to another.
The set top I am working on uses linux and so I need to mount and read the partition on it.
I have the following question:
I did not find any xtvfs named filesystem supported in the kernel. Can I be able to mount it using vfat?
If not, then how do I get to work such a filesystem in Linux.
Information on XTVFS is hard to find and support is not widespread - http://wiki.ph-mb.com/wiki/XTVFS is probably the best page I found. I could not find a linux driver for it. XTVFS is based on FAT32 and mounting that on Linux will get you part of the way - basically for everything but the video files. The video files have a separate Video FAT and video data section. I wrote some code to read files from an XTVFS image and will share it once it is tidied.
I have a Python script running under Linux that generates huge numbers of tiny files into a given directory. However, many Linux filesystems like ext4 have a fixed number of inodes set at creation time, so I want to make sure it's possible to save that many files into that directory before starting. From the command line, you can see this number using df -i /some/directory.
How do you find the number of free inodes on the filesystem that directory lives on, in Python?
This can be done using the statvfs system call. In Python (both 2 and 3), this can be accessed using os.statvfs. The call describes the filesystem containing the file/directory the path specifies.
So to get the number of free inodes, use
#import os
os.statvfs('/some/directory').f_favail
Also, it's possible that some percentage of the inodes are reserved for the root user. If the script is running as root and you want to allow it to use the reserved inodes, use f_ffree instead of f_favail.
I am new to the linux and working with centos system ,
By running command df -H it is showing 82% if full, that is only 15GB is free.
I want some more extra spaces, so using WINSCP i hav done shift deleted the 15G record.
and execured df -H once again, but still it is showing 15 GB free. but the free size of the deleted
file where it goes.
Plese help me out in finding solution to this
In most unix filesystems, if a file is open, the OS will delete the file right way, but will not release space until the file is closed. Why? Because the file is still visible for the user that opened it.
On the other side, Windows used to complain that it can't delete a file because it is in use, seems that in later incarnations explorer will pretend to delete the file.
Some applications are famous for bad behavior related to this fact. For example, I have to deal with some versions of MySQL that will not properly close some files, over the time I can find several GB of space wasted in /tmp.
You can use the lsof command to list open files (man lsof). If the problem is related to open files, and you can afford a reboot, most likely it is the easiest way to fix the problem.
My Linux-based system displays statistics for NFS-mounted filesystems, something like this:
Remote Path Mounted-on Stats
server1:/some/path/name /path1 100 GB free
server2:/other/path/name /path2 100 GB free
Total: 200 GB free
That works fine. The problem is when the same filesystem on the NFS server has been mounted twice on my client:
Remote Path Mounted-on Stats
server1:/some/path/name /path1 100 GB free
server1:/some/path/name2 /path2 100 GB free
Total: 200 GB free
server1's /some/path/name and /some/path/name2 are actually on the same filesystem, which has 100 GB free, but I erroneously add them up and report 200 GB free.
Is there any way to detect that they're on the same partition?
Approaches that won't work:
"Use statfs()": statfs() returns a struct statfs, which has a "file system ID" field, f_fsid. Unfortunately it's undefined and gets zeroed out over NFS.
"Don't mount the same partion multiple times." This is outside of my control.
"Use a heuristic based on available space." The method has to definitively work. Also, statfs() caches its output so it would be difficult to get this right in the face of large data movement.
If there's no solution I'll have to generate a config file in every potential mount point on the server side, but it would be a lot nicer if there was
some clean way to avoid that.
Thanks!
I guess if "stat -c %d /mountpoint" do what you want (I cannot test it right now)?
You probably want to read the remote system's shared file systems - using:
showmount -e server
That will give you the real paths that are being shared. When walking mounts from the remote system, prune them to the common root from the remote system and use that to determine if the mount points are from the same underlying file system.
This doesn't help you in the case that the file systems are separately shared from the same underlying file system.
You could add in a heuristic of checking for the overall file system size and space available, and assuming that if they're the same, and from the same remote server that it's on the same partition mapped to the shortest common path of the mount devices.
None of these help if you share from a loopback mounted file system that looks completely different in form from the others.
It doesn't help you in the case of a server that can be addressed in different names and addresses.