I have some files on a scratch drive of a HPC server. The server automatically deletes files which are 2 weeks old.
Using stat filename.txt I can see the follow information. Is there a way to somehow open/touch/manipulate files to update the Access date to prevent deletion without actually changing the file?
File: ‘name’
Size: 2583438768 Blocks: 4819945 IO Block: 524288 regular file
Device: xxh/xxxd Inode: 10354xxxx Links: 1
Access: (/-rw-r--r--) Uid: (/) Gid: (/)
Context: system_u:object_r:tmp_t:s0
Access: 2022-11-22 09:47:33.000000000 -0800
Modify: 2019-12-06 06:50:33.000000000 -0800
Change: 2022-11-22 16:54:55.000000000 -0800
Birth: -
Use the Linux touch command, eg:
$ touch filename.txt
Related
We test upgrade of clusters with Number of Bricks: 2 x 2 = 4 configuration from gluster 5 to gluster 9 (Centos 7 OS).
Often (in 90% of the cases) we see below problem.
After upgrade of the first node we trigger manual heal, waiting when everything will be healed.
But if after this we are trying to access files on upgraded node (ls -l), then Number of entries in heal pending goes up.
When we a looking at details of affected files on upgraded and not upgraded nodes, we see following:
On upgraded node afr have metadata bit set and trusted.glusterfs.mdata present:
fa05.sg05# getfattr -d -m . -e hex /data1/gluster/20200922-upg-004/many_files/562_4.log; stat /data1/gluster/20200922-upg-004/many_files/562_4.log
getfattr: Removing leading '/' from absolute path names
# file: data1/gluster/20200922-upg-004/many_files/562_4.log
trusted.afr.dirty=0x000000000000000000000000
trusted.afr.shared-client-1=0x000000000000000100000000
trusted.gfid=0x3bf30f76e2c94a67a0878bea1bd8db97
trusted.gfid2path.a9c4d7d45775ec60=0x36343866643935342d653534622d343762302d386631392d6339333636306561303838632f3536325f342e6c6f67
trusted.glusterfs.mdata=0x01000000000000000000000000632c3c3c000000000d6470f900000000632c3c3c000000000d552e6800000000632c3c3
metadata of file looks like this:
File: ‘/data1/gluster/20200922-upg-004/many_files/562_4.log’
Size: 4096 Blocks: 8 IO Block: 4096 regular file
Device: 820h/2080d Inode: 29306144911 Links: 2
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2022-09-22 10:43:08.221686087 +0000
Modify: 2022-09-22 10:43:08.223686248 +0000
Change: 2022-09-22 10:44:18.690378905 +0000
Birth: -
On not upgraded node everything is OK (no any changes of metadata)
We see that Change field differs on upgraded node, and change time on upgraded node corresponds to the time when ls -l operation has been performed:
fa05.sg05# date; ls -l /shared/20200922-upg-004/many_files/ | wc -l; date
Thu Sep 22 10:44:17 UTC 2022
10229
Thu Sep 22 10:44:35 UTC 2022
Not sure what can be the reason for such metadata update and how we can avoid this?
Because this lead to healing of huge amount of files and high I/O load on upgraded node.
*Details and logs are here: https://github.com/gluster/glusterfs/issues/3829
Thanks in advance!
A symbolic link created by zfs is returning as ../../zd0, tested through multiple ways to check. Not sure why the result is not /dev/zd0 as that is the actual link as far as I am aware.
admin#r2-compute1:~$ ls -l /dev/zvol/cpool/cinder-volumes
lrwxrwxrwx 1 root root 9 Jun 25 09:53 /dev/zvol/cpool/cinder-volumes -> ../../zd0
admin#r2-compute1:~$ stat /dev/zvol/cpool/cinder-volumes
File: /dev/zvol/cpool/cinder-volumes -> ../../zd0
Size: 9 Blocks: 0 IO Block: 4096 symbolic link
Device: 6h/6d Inode: 640 Links: 1
Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2020-06-25 10:07:28.610707769 -0700
Modify: 2020-06-25 09:53:52.174435310 -0700
Change: 2020-06-25 09:53:52.174435310 -0700
Birth: -
admin#r2-compute1:~$ file /dev/zvol/cpool/cinder-volumes
/dev/zvol/cpool/cinder-volumes: symbolic link to ../../zd0
I am working with openstack and zfs and am wondering why the symbolic link returns "../../zd0" instead of "dev/zd0". I am not sure what the ../../ means and understanding would assist in my openstack implementation understanding. First time posting and Google and SymbolHound did not produce any results in regards to the meaning behind ../../
Thank you!
Probably while creating the soft links ../../zd0 was adopted instead of the full path /dev/zd0 (or whatever the path was supposed to be given).
If absolutely necessary, you may try to unlink this first and recreate the link by providing FULL-PATH to the DEVICE/FILE.
NOTE: Try this with caution and ensure the paths are correct, this is just the symbolic code to help you correct the error:
unlink /dev/zvol/cpool/cinder-volumes
ln -s /dev/zvol/cpool/cinder-volumes /dev/zd0
Given a directory, I'd like to know whether the files in the directory have been modified or not. (Boolean) i.e. if the directory's state has changed from before.
I don't want to run a file watcher service for this as I don't need to know which file has been modified (or if many files change receive many events)
I've looked at atime, mtime, ctime from stat
eg: for a dir named taskmaster which already contains sample.txt
stat taskmaster
output
File: taskmaster
Size: 245760 Blocks: 480 IO Block: 4096 directory
Device: 802h/2050d Inode: 1309314 Links: 1
Access: (0777/drwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2020-05-22 21:25:06.226421200 +0530
Modify: 2020-05-22 21:25:06.222175900 +0530
Change: 2020-05-22 21:25:06.222175900 +0530
Birth: -
After I modify the folder contents
# modify an existing file
echo modify > taskmaster/sample.txt
stat taskmaster gives
File: taskmaster
Size: 245760 Blocks: 480 IO Block: 4096 directory
Device: 802h/2050d Inode: 1309314 Links: 1
Access: (0777/drwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2020-05-22 21:25:06.226421200 +0530
Modify: 2020-05-22 21:25:06.222175900 +0530
Change: 2020-05-22 21:25:06.222175900 +0530
Birth: -
The exact same output.
If no file is removed or deleted the access and modify times do not change. How can I achieve this?
I think you need to do stat on individual files, something like this :
previous="$(stat *)"
while sleep 60; do
current="$(stat *)"
if [[ $current != $previous ]]; then
echo "Some files changed."
fi
previous=$current
done
Earlier comment: stat -c %Y /path/to/directory also works, does have a ceveat.
There are several fields that the stat command reads and accesses and it depends on how the contents of the direcctory were modified.
stat command output uses st_mtime
printf("Last file modification: %s", ctime(&sb.st_mtime));
Source: https://www.pdl.cmu.edu/posix/docs/POSIX-stat-manpages.pdf
Stat documentation
The field st_mtime for a file is changed by file modifications, e.g. by mknod(2), truncate(2), utime(2) and write(2) (of more than zero bytes).
st_mtime of a directory is changed by the creation or deletion of files in that directory. The st_mtime field is not changed for changes in owner, group, hard link count, or mode.
Source: http://man.he.net/man2/stat
So if a file is created, written to or deleted within a directory, the directory modified time will be automatically updated, but not if an additional hardlink or permissions are changed on a file within that directory.
I cannot find information on casading the changes up towards / but the directories utime cchange 'should' propagate this upwards but possibly not immediately. I would cetainly test that though for your use case.
I am running Centos 7 kernel 3.10.0 on Oracle's VirtualBox 4.3.20 with an ext4 filesystem.
It seems that setting the "A" flag on a file doesn't keep the access time field from updating.
I created a file and used stat to check the access time -
touch ./foo
stat ./foo
which returns the following
File: ./foo
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
Device: fd01h/64769d Inode: 1444417 Links: 1
Access: (0777/-rwxrwxrwx) Uid: ( 1000/ user_1) Gid: ( 1000/ user_1)
Context: unconfined_u:object_r:user_home_t:s0
Access: 2015-08-02 11:52:23.451524456 -0700
Modify: 2015-08-02 11:52:23.451524456 -0700
Change: 2015-08-02 11:52:23.451524456 -0700
Birth: -
If I then change the attribute and rerun stat -
sudo chattr +A ./foo
stat ./foo
I get the following which (I think) shows the access time value as updated?
File: ./foo
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
Device: fd01h/64769d Inode: 1444417 Links: 1
Access: (0777/-rwxrwxrwx) Uid: ( 1000/ user_1) Gid: ( 1000/ user_1)
Context: unconfined_u:object_r:user_home_t:s0
Access: 2015-08-02 11:53:32.400974020 -0700
Modify: 2015-08-02 11:53:32.400974020 -0700
Change: 2015-08-02 11:53:32.400974020 -0700
Birth: -
Am I misunderstanding the use of the A flag? I thought it would keep the access time from changing?
Thanks for any insights you can offer. I am just trying to understand how things work.
Rog
+A will keep the same atime when you open the file with vi for example and not modify it. The result you've posted after modifying the attribute doesn't look right to me. Changing the attribute should only affect the 'Change' timestamp. In your case, all 3 timestamps are the same.
I have a node application that is having it's STDOUT being redirected to a log file:
node app.js > /var/log/app.out
From inside my node program I want to reset the log in order to keep it from growing too large. I've attempted fs.truncate and fs.open('/var/log/app.out', 'w') in order to reset it's length, but when I try to verify the file size using ls or stat, the file size seems to quickly reset to 0, then reset to it's previous size plus new messages (I currently am forcing a lot of output).
root#ubuntu-1304-mike:/var/log# stat app.out
File: ‘app.out’
Size: 5885622 Blocks: 16 IO Block: 4096 regular file
Device: fc00h/64512d Inode: 270560 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2013-09-26 17:20:06.784564421 -0600
Modify: 2013-09-26 17:20:49.088565625 -0600
Change: 2013-09-26 17:20:49.088565625 -0600
Birth: -
root#ubuntu-1304-mike:/var/log# stat app.out
File: ‘app.out’
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
Device: fc00h/64512d Inode: 270560 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2013-09-26 17:20:06.784564421 -0600
Modify: 2013-09-26 17:20:51.772565702 -0600
Change: 2013-09-26 17:20:51.772565702 -0600
Birth: -
root#ubuntu-1304-mike:/var/log# stat app.out
File: ‘app.out’
Size: 6038458 Blocks: 16 IO Block: 4096 regular file
Device: fc00h/64512d Inode: 270560 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2013-09-26 17:20:06.784564421 -0600
Modify: 2013-09-26 17:20:53.640565755 -0600
Change: 2013-09-26 17:20:53.640565755 -0600
Birth: -
It seems that STDOUT is re-writing it's buffer to the file (regardless of how large the buffer is).
How can I truncate a file that STDOUT is actively being redirected to, without breaking out of my node app?
It seems that STDOUT is re-writing it's buffer to the file (regardless of how large the buffer is).
Nobody is re-writing a buffer, it's already written on the disk. When application finishes writing something, it remembers position where it finished, and it starts with this position later. It's not a node issue, and there's nothing you can do about it, it's how unix pipes work (unless there's some ugly hack to rewind a pipe I don't know about).
How can I truncate a file that STDOUT is actively being redirected to, without breaking out of my node app?
Don't use stdout redirection and write to a file instead. If you can't modify your app, write another app that uses more clever approach and redirect stdout to it like that: node app.js | node write-stdin-to-app.out.js
I just ran into this issue, as well. Using the append operator is the best way to solve it:
node app.js >> /var/log/app.out
Now you can truncate the file with fs.truncate or even:
echo -n > /var/log/app.out