Truncating file that has STDOUT being redirected to it - node.js

I have a node application that is having it's STDOUT being redirected to a log file:
node app.js > /var/log/app.out
From inside my node program I want to reset the log in order to keep it from growing too large. I've attempted fs.truncate and fs.open('/var/log/app.out', 'w') in order to reset it's length, but when I try to verify the file size using ls or stat, the file size seems to quickly reset to 0, then reset to it's previous size plus new messages (I currently am forcing a lot of output).
root#ubuntu-1304-mike:/var/log# stat app.out
File: ‘app.out’
Size: 5885622 Blocks: 16 IO Block: 4096 regular file
Device: fc00h/64512d Inode: 270560 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2013-09-26 17:20:06.784564421 -0600
Modify: 2013-09-26 17:20:49.088565625 -0600
Change: 2013-09-26 17:20:49.088565625 -0600
Birth: -
root#ubuntu-1304-mike:/var/log# stat app.out
File: ‘app.out’
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
Device: fc00h/64512d Inode: 270560 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2013-09-26 17:20:06.784564421 -0600
Modify: 2013-09-26 17:20:51.772565702 -0600
Change: 2013-09-26 17:20:51.772565702 -0600
Birth: -
root#ubuntu-1304-mike:/var/log# stat app.out
File: ‘app.out’
Size: 6038458 Blocks: 16 IO Block: 4096 regular file
Device: fc00h/64512d Inode: 270560 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2013-09-26 17:20:06.784564421 -0600
Modify: 2013-09-26 17:20:53.640565755 -0600
Change: 2013-09-26 17:20:53.640565755 -0600
Birth: -
It seems that STDOUT is re-writing it's buffer to the file (regardless of how large the buffer is).
How can I truncate a file that STDOUT is actively being redirected to, without breaking out of my node app?

It seems that STDOUT is re-writing it's buffer to the file (regardless of how large the buffer is).
Nobody is re-writing a buffer, it's already written on the disk. When application finishes writing something, it remembers position where it finished, and it starts with this position later. It's not a node issue, and there's nothing you can do about it, it's how unix pipes work (unless there's some ugly hack to rewind a pipe I don't know about).
How can I truncate a file that STDOUT is actively being redirected to, without breaking out of my node app?
Don't use stdout redirection and write to a file instead. If you can't modify your app, write another app that uses more clever approach and redirect stdout to it like that: node app.js | node write-stdin-to-app.out.js

I just ran into this issue, as well. Using the append operator is the best way to solve it:
node app.js >> /var/log/app.out
Now you can truncate the file with fs.truncate or even:
echo -n > /var/log/app.out

Related

How to change a File's last Access/Modify/Change Date?

I have some files on a scratch drive of a HPC server. The server automatically deletes files which are 2 weeks old.
Using stat filename.txt I can see the follow information. Is there a way to somehow open/touch/manipulate files to update the Access date to prevent deletion without actually changing the file?
File: ‘name’
Size: 2583438768 Blocks: 4819945 IO Block: 524288 regular file
Device: xxh/xxxd Inode: 10354xxxx Links: 1
Access: (/-rw-r--r--) Uid: (/) Gid: (/)
Context: system_u:object_r:tmp_t:s0
Access: 2022-11-22 09:47:33.000000000 -0800
Modify: 2019-12-06 06:50:33.000000000 -0800
Change: 2022-11-22 16:54:55.000000000 -0800
Birth: -
Use the Linux touch command, eg:
$ touch filename.txt

how to check if any of the files in a directory changed

Given a directory, I'd like to know whether the files in the directory have been modified or not. (Boolean) i.e. if the directory's state has changed from before.
I don't want to run a file watcher service for this as I don't need to know which file has been modified (or if many files change receive many events)
I've looked at atime, mtime, ctime from stat
eg: for a dir named taskmaster which already contains sample.txt
stat taskmaster
output
File: taskmaster
Size: 245760 Blocks: 480 IO Block: 4096 directory
Device: 802h/2050d Inode: 1309314 Links: 1
Access: (0777/drwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2020-05-22 21:25:06.226421200 +0530
Modify: 2020-05-22 21:25:06.222175900 +0530
Change: 2020-05-22 21:25:06.222175900 +0530
Birth: -
After I modify the folder contents
# modify an existing file
echo modify > taskmaster/sample.txt
stat taskmaster gives
File: taskmaster
Size: 245760 Blocks: 480 IO Block: 4096 directory
Device: 802h/2050d Inode: 1309314 Links: 1
Access: (0777/drwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2020-05-22 21:25:06.226421200 +0530
Modify: 2020-05-22 21:25:06.222175900 +0530
Change: 2020-05-22 21:25:06.222175900 +0530
Birth: -
The exact same output.
If no file is removed or deleted the access and modify times do not change. How can I achieve this?
I think you need to do stat on individual files, something like this :
previous="$(stat *)"
while sleep 60; do
current="$(stat *)"
if [[ $current != $previous ]]; then
echo "Some files changed."
fi
previous=$current
done
Earlier comment: stat -c %Y /path/to/directory also works, does have a ceveat.
There are several fields that the stat command reads and accesses and it depends on how the contents of the direcctory were modified.
stat command output uses st_mtime
printf("Last file modification: %s", ctime(&sb.st_mtime));
Source: https://www.pdl.cmu.edu/posix/docs/POSIX-stat-manpages.pdf
Stat documentation
The field st_mtime for a file is changed by file modifications, e.g. by mknod(2), truncate(2), utime(2) and write(2) (of more than zero bytes).
st_mtime of a directory is changed by the creation or deletion of files in that directory. The st_mtime field is not changed for changes in owner, group, hard link count, or mode.
Source: http://man.he.net/man2/stat
So if a file is created, written to or deleted within a directory, the directory modified time will be automatically updated, but not if an additional hardlink or permissions are changed on a file within that directory.
I cannot find information on casading the changes up towards / but the directories utime cchange 'should' propagate this upwards but possibly not immediately. I would cetainly test that though for your use case.

I need to read a name file and work on it after that like finding out his inode-number and other things

I read a name file, just the name and I need to print inode-number, the number of hard-links and others. How do I do that?
I thought initially to add the absolute path to my name file but I do not know how to that.
Use the stat command. Here is an example output:
UbuntuVBox:~/stackOver$ stat main.cpp
File: ‘main.cpp’
Size: 234 Blocks: 8 IO Block: 4096 regular file
Device: 801h/2049d Inode: 18749058 Links: 1
Access: (0664/-rw-rw-r--) Uid: ( 1000/ root) Gid: ( 1000/ root)
Access: 2016-06-01 10:02:55.858362281 +0100
Modify: 2016-06-01 10:02:48.770820244 +0100
Change: 2016-06-01 10:02:48.782826244 +0100
Birth: -
As you can see you get the inode number and number of links amongst other things.

'chattr +A' doesn't appear to suppress access time update

I am running Centos 7 kernel 3.10.0 on Oracle's VirtualBox 4.3.20 with an ext4 filesystem.
It seems that setting the "A" flag on a file doesn't keep the access time field from updating.
I created a file and used stat to check the access time -
touch ./foo
stat ./foo
which returns the following
File: ./foo
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
Device: fd01h/64769d Inode: 1444417 Links: 1
Access: (0777/-rwxrwxrwx) Uid: ( 1000/ user_1) Gid: ( 1000/ user_1)
Context: unconfined_u:object_r:user_home_t:s0
Access: 2015-08-02 11:52:23.451524456 -0700
Modify: 2015-08-02 11:52:23.451524456 -0700
Change: 2015-08-02 11:52:23.451524456 -0700
Birth: -
If I then change the attribute and rerun stat -
sudo chattr +A ./foo
stat ./foo
I get the following which (I think) shows the access time value as updated?
File: ./foo
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
Device: fd01h/64769d Inode: 1444417 Links: 1
Access: (0777/-rwxrwxrwx) Uid: ( 1000/ user_1) Gid: ( 1000/ user_1)
Context: unconfined_u:object_r:user_home_t:s0
Access: 2015-08-02 11:53:32.400974020 -0700
Modify: 2015-08-02 11:53:32.400974020 -0700
Change: 2015-08-02 11:53:32.400974020 -0700
Birth: -
Am I misunderstanding the use of the A flag? I thought it would keep the access time from changing?
Thanks for any insights you can offer. I am just trying to understand how things work.
Rog
+A will keep the same atime when you open the file with vi for example and not modify it. The result you've posted after modifying the attribute doesn't look right to me. Changing the attribute should only affect the 'Change' timestamp. In your case, all 3 timestamps are the same.

how to get ubuntu file timestamp in millisecond

Linux friends, how do you guys get a file timestamp in the resolution of milliseconds in Ubuntu? When I tried the command "stat" on my system, I get:
File: `tmp.dat'
Size: 14 Blocks: 8 IO Block: 4096 regular file
Device: 811h/2065d Inode: 13895454 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 1000/ yqin) Gid: ( 1004/ dev)
Access: 2010-09-27 09:04:44.000000000 -0400
Modify: 2010-09-25 13:54:58.000000000 -0400
Change: 2010-09-25 13:54:58.000000000 -0400
It seems that the resolution is only up to second. Any ideas? Thanks a lot.
That depends on the filesystem. fat will give you 2 second resolution. ext2 and ext3 are based on seconds. ext4 has nanoseconds (http://www.softpanorama.org/Internals/Filesystems/linux_ext2_ext3.shtml, search for "minimum timestamp resolution").

Resources