node.js filesystem mangles Cygwin drive name - node.js

I recently installed Cygwin64 after using Cygwin32 for quite some time and I'm now having problems with one of our production scripts. Node.js's 'readFileSync' seems to be prepending the windows drive letter to the path and then failing to resolve - e.g. /cygdrive/c/foo becomes c:/cygdrive/c/foo.
I've found various mentions of similar issues online but so far I've been unable to resolve this problem. My co-worker has a seemingly identical setup and does not experience the problem.
Here it is in a nutshell -
$ cat filetest.js
var fs = require('fs');
function main(argv) {
console.log("fileName => ", argv[2]);
var data = fs.readFileSync(argv[2], 'utf8');
console.log("success");
}
main(process.argv);
$ s/node filetest.js filetest.js
fileName => filetest.js
success
$ ls -l /cygdrive/c/temp/Test.txt
----rwx---+ 1 bdodd Domain Users 14 Jun 3 14:12 /cygdrive/c/temp/Test.txt
$ s/node filetest.js /cygdrive/c/temp/Test.txt
fileName => /cygdrive/c/temp/Test.txt
fs.js:338
return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);
^
Error: ENOENT, no such file or directory 'C:\cygdrive\c\temp\Test.txt'
and just for completeness...
$ cat /etc/fstab
# /etc/fstab
#
# This file is read once by the first process in a Cygwin process tree.
# To pick up changes, restart all Cygwin processes. For a description
# see https://cygwin.com/cygwin-ug-net/using.html#mount-table
# This is default anyway:
none /cygdrive cygdrive binary,posix=0,user 0 0
$ cd /cygdrive/
$ ls -l
total 40
d---rwx---+ 1 NT SERVICE+TrustedInstaller NT SERVICE+TrustedInstaller 0 Jun 3 14:09 c
dr-xrwxr-x 1 Unknown+User Unix_Group+33 0 Apr 29 2014 u
Insight would be greatly appreciated. Thanks!

Related

how to recover files delted using "find /export/reports -mtime +112 -type f -delete"

I wanted to delete .pdf files, i used "find /export/reports -mtime +112 -type f -delete", however i did not applied any filter for .pdf file and end up deleting required files as well.
Can any one please help me with how to recover them?
you can try this:
Use debugfs to view a filesystems log
$ debugfs -w /dev/mapper/wks01-root
At the debugfs prompt
debugfs: lsdel
Sample output
Inode Owner Mode Size Blocks Time deleted
23601299 0 120777 3 1/ 1 Tue Mar 13 16:17:30 2012
7536655 0 120777 3 1/ 1 Tue May 1 06:21:22 2012
2 deleted inodes found.
Run the command in debugfs
debugfs: logdump -i <7536655>
5) Determine files inode
> ... ... .... output truncated
> Fast_link_dest: bin
> Blocks: (0+1): 7235938 FS block 7536642 logged at sequence 38402086, journal block 26711
> (inode block for inode 7536655):
> Inode: 7536655 Type: symlink Mode: 0777 Flags: 0x0 Generation: 3532221116
> User: 0 Group: 0 Size: 3
> File ACL: 0 Directory ACL: 0
> Links: 0 Blockcount: 0
> Fragment: Address: 0 Number: 0 Size: 0
> ctime: 0x4f9fc732 -- Tue May 1 06:21:22 2012
> atime: 0x4f9fc730 -- Tue May 1 06:21:20 2012
> mtime: 0x4f9fc72f -- Tue May 1 06:21:19 2012
> dtime: 0x4f9fc732 -- Tue May 1 06:21:22 2012
> Fast_link_dest: bin
> Blocks: (0+1): 7235938 No magic number at block 28053: end of journal.
With the above inode info run the following commands
dd if=/dev/mapper/wks01-root of=recovered.file.001 bs=4096 count=1
skip=7235938 file recovered.file.001 file: ASCII text, with very long
lines
Files been recovered to recovered.file.001.
there are also some tools which can be helpful:
Recovery Tools - Command Line :
testdisk,
photorec,
extundelete
Recovery Tools - Gui :
R-Linux,
R-Studio,
UFS Explorer,
Recovery Explorer

SNMP sysObjectID not translated when using Python3 easysnmp module

When I query SNMP sysObjectID using easysnmp module, then returned value is in numerical notation:
$ python3
Python 3.5.3 (default, Jan 19 2017, 14:11:04)
[GCC 6.3.0 20170118] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from easysnmp import Session
>>> session = Session(hostname="r1", community='public', version=2)
>>> session.get('sysObjectID.0')
<SNMPVariable value='.1.3.6.1.4.1.2636.1.1.1.2.21' (oid='sysObjectID', oid_index='0', snmp_type='OBJECTID')>
>>>
However, this does not seem to be because easysnmp is not able to find the correct MIB file. When I put the commands above into a file and execute it with strace, then correct MIB is accessed:
$ strace 2>&1 -f -e open python3 snmp_test.py | grep mib-jnx-chas-defines.txt
open("/usr/share/snmp/mibs/JuniperMibs_from_Juniper/mib-jnx-chas-defines.txt", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = -1 ENOTDIR (Not a directory)
open("/usr/share/snmp/mibs/JuniperMibs_from_Juniper/mib-jnx-chas-defines.txt", O_RDONLY) = 4
open("/usr/share/snmp/mibs/JuniperMibs_from_Juniper/mib-jnx-chas-defines.txt", O_RDONLY) = 3
$
I can double-check this using snmpget:
$ snmpget -v 2c -c public r1 sysObjectID.0
SNMPv2-MIB::sysObjectID.0 = OID: JUNIPER-CHASSIS-DEFINES-MIB::jnxProductNameMX960
$ strace 2>&1 -f -e open snmpget -v 2c -c public r1 sysObjectID.0 | grep mib-jnx-chas-defines.txt
open("/usr/share/snmp/mibs/JuniperMibs_from_Juniper/mib-jnx-chas-defines.txt", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = -1 ENOTDIR (Not a directory)
open("/usr/share/snmp/mibs/JuniperMibs_from_Juniper/mib-jnx-chas-defines.txt", O_RDONLY) = 4
open("/usr/share/snmp/mibs/JuniperMibs_from_Juniper/mib-jnx-chas-defines.txt", O_RDONLY) = 3
$
Still, just to be sure, I set the os.environ['MIBDIRS'], os.environ['MIBS'] and os.environ['PREFIX'] to same values as I see when I execute the snmpget -Dinit_mib -m ALL -v 2c -c public r1 sysObjectID.0 command, but this does not help either.
What might cause this?
This is because the value at the OID sysObjectID.0 is just being treated as a value. It looks like the use_sprint_value option enables the extra formatting on the return value:
>>> session = Session(hostname="abc", community='public', version=2, use_long_names=True, use_sprint_value=True)
>>> session.get('sysObjectID.0')
<SNMPVariable value='.iso.org.dod.internet.private.enterprises.2435.2.3.9.1' (oid='.iso.org.dod.internet.mgmt.mib-2.system.sysObjectID', oid_index='0', snmp_type='OBJECTID')>
Clearly the use_long_names option is also helpful to show the expanded name, though I don't have all the MIBs required to decode this example.

Bash script not producing desired result

I am running a cron-ed bash script to extract cache hits and bytes served per IP address. The script (ProxyUsage.bash) has two parts:
(uniqueIP.awk) find unique IPs and create a bash script do add up the hits and bytes
run the hits and bytes per IP
ProxyUsage.bash
#!/usr/bin/env bash
sudo gawk -f /home/maxg/scripts/uniqueIP.awk /var/log/squid3/access.log.1 > /home/maxg/scripts/pxyUsage.bash
source /home/maxg/scripts/pxyUsage.bash
uniqueIP.awk
{
arrIPs[$3]++;
}
END {
for (n in arrIPs) {
m++; # count arrIPs elements
#print "Array elements: " m;
arrAddr[i++] = n; # fill arrAddr with IPs
#print i " " n;
}
asort(arrAddr); # sort the array values
for (i = 1; i <= m; i++) { # write one command line per IP address
#printf("#!/usr/bin/env bash\n");
printf("sudo gawk -f /home/maxg/scripts/proxyUsage.awk -v v_Var=%s /var/log/squid3/access.log.1 >> /home/maxg/scripts/pxyUsage.txt\n", arrAddr[i])
}
}
pxyUsage.bash
sudo gawk -f /home/maxg/scripts/proxyUsage.awk -v v_Var=192.168.1.13 /var/log/squid3/access.log.1 >> /home/maxg/scripts/pxyUsage.txt
sudo gawk -f /home/maxg/scripts/proxyUsage.awk -v v_Var=192.168.1.14 /var/log/squid3/access.log.1 >> /home/maxg/scripts/pxyUsage.txt
sudo gawk -f /home/maxg/scripts/proxyUsage.awk -v v_Var=192.168.1.22 /var/log/squid3/access.log.1 >> /home/maxg/scripts/pxyUsage.txt
TheProxyUsage.bash script runs as scheduled and creates the pxyUsage.bash script.
However the pxyUsage.text file is not amended with the latest values when the script runs.
So far I run pxyUsage.bash every day myself, as I cannot figure out, why the result is not written to file.
Both bash scripts are set to execute. Actually the file permissions are below:
-rwxr-xr-x 1 maxg maxg 169 Mar 14 08:40 ProxySummary.bash
-rw-r--r-- 1 maxg maxg 910 Mar 15 17:15 proxyUsage.awk
-rwxrwxrwx 1 maxg maxg 399 Mar 17 06:10 pxyUsage.bash
-rw-rw-rw- 1 maxg maxg 2922 Mar 17 07:32 pxyUsage.txt
-rw-r--r-- 1 maxg maxg 781 Mar 16 07:35 uniqueIP.awk
Any hints appreciated. Thanks.
The sudo(8) command requires a pseudo-tty and you do not have one allocated under cron(8); you do have one allocated when logged in the usual way.
Instead of mucking about with sudo(8), just run the script as the correct user.
If you cannot do that, then in the root crontab, do something like this:
su - username /path/to/mycommand arg1 arg2...
This will work because root can use su(1) without neding a password.

running linux commands from R

I have a bunch of random files and I am going to run LINUX-file command on each file. Linux screen will be as follows
m7% file date-file.csv
date-file.csv: ASCII text, with CRLF line terminators
m7% file image-file.JPG
image-file.JPG: JPEG image data, EXIF standard
Only when Linux says that the file is a text file, I want to run a R script that goes through that file and finds all column names. In above screen, I want to run R script only on the first file. How could I achieve this conditional processing?
Is there any way I can run Linux commands from R? If i can do that then I can analyze the output given by Linux command to see if it contains text and then I can execute R script if required.
I am having difficulty achieving this and any help is appreciated
Try system()
08/27 7:08 [nodakai#kaidev01] ~/R$ R -q
> system("ls /")
bin boot dev etc home initrd.img initrd.img.old lib lib32 lib64 libGL.so libnss3.so lost+found media mnt opt proc root run sbin selinux sftp srv sys tmp usr var vmlinuz vmlinuz.old
> x = system("ls", TRUE)
> x
[1] "a.csv" "abi.pdf"
[3] "b.csv" "image.jpeg"
[5] "x86_64-pc-linux-gnu-library"
> for (f in system("ls", TRUE)) { if (length(grep("ASCII", system(paste("file", f), TRUE)))) { print(f) } }
[1] "a.csv"
[1] "b.csv"
http://stat.ethz.ch/R-manual/R-devel/library/base/html/system.html
Globbing
If you're sure all files ending ".csv" are really plain text CSV files, just use Sys.glob()
> list.files()
[1] "a.csv" "abi.pdf"
[3] "b.csv" "image.jpeg"
[5] "x86_64-pc-linux-gnu-library"
> Sys.glob("*.csv")
[1] "a.csv" "b.csv"
http://stat.ethz.ch/R-manual/R-devel/library/base/html/Sys.glob.html

How to get file creation date/time in Bash/Debian?

I'm using Bash on Debian GNU/Linux 6.0. Is it possible to get the file creation date/time? Not the modification date/time.
ls -lh a.txt and stat -c %y a.txt both only give the modification time.
Unfortunately your quest won't be possible in general, as there are only 3 distinct time values stored for each of your files as defined by the POSIX standard (see Base Definitions section 4.8 File Times Update)
Each file has three distinct associated timestamps: the time of last
data access, the time of last data modification, and the time the file
status last changed. These values are returned in the file
characteristics structure struct stat, as described in <sys/stat.h>.
EDIT: As mentioned in the comments below, depending on the filesystem used metadata may contain file creation date. Note however storage of information like that is non standard. Depending on it may lead to portability problems moving to another filesystem, in case the one actually used somehow stores it anyways.
ls -i file #output is for me 68551981
debugfs -R 'stat <68551981>' /dev/sda3 # /dev/sda3 is the disk on which the file exists
#results - crtime value
[root#loft9156 ~]# debugfs -R 'stat <68551981>' /dev/sda3
debugfs 1.41.12 (17-May-2010)
Inode: 68551981 Type: regular Mode: 0644 Flags: 0x80000
Generation: 769802755 Version: 0x00000000:00000001
User: 0 Group: 0 Size: 38973440
File ACL: 0 Directory ACL: 0
Links: 1 Blockcount: 76128
Fragment: Address: 0 Number: 0 Size: 0
ctime: 0x526931d7:1697cce0 -- Thu Oct 24 16:42:31 2013
atime: 0x52691f4d:7694eda4 -- Thu Oct 24 15:23:25 2013
mtime: 0x526931d7:1697cce0 -- Thu Oct 24 16:42:31 2013
**crtime: 0x52691f4d:7694eda4 -- Thu Oct 24 15:23:25 2013**
Size of extra inode fields: 28
EXTENTS:
(0-511): 352633728-352634239, (512-1023): 352634368-352634879, (1024-2047): 288392192-288393215, (2048-4095): 355803136-355805183, (4096-6143): 357941248-357943295, (6144
-9514): 357961728-357965098
mikyra's answer is good. The fact just like what he said.
[jason#rh5 test]$ stat test.txt
File: `test.txt'
Size: 0 Blocks: 8 IO Block: 4096 regular empty file
Device: 802h/2050d Inode: 588720 Links: 1
Access: (0664/-rw-rw-r--) Uid: ( 500/ jason) Gid: ( 500/ jason)
Access: 2013-03-14 01:58:12.000000000 -0700
Modify: 2013-03-14 01:58:12.000000000 -0700
Change: 2013-03-14 01:58:12.000000000 -0700
if you want to verify, which file was created first, you can structure your file name by appending system date when you create a series of files.
Note that if you've got your filesystem mounted with noatime for performance reasons, then the atime will likely show the creation time. Given that noatime results in a massive performance boost (by removing a disk write for every time a file is read), it may be a sensible configuration option that also gives you the results you want.
Creation date/time is normally not stored. So no, you can't.
You can find creation time - aka birth time - using stat and also match using find.
We have these files showing last modified time:
$ ls -l --time-style=long-iso | sort -k6
total 692
-rwxrwx---+ 1 XXXX XXXX 249159 2013-05-31 14:47 Getting Started.pdf
-rwxrwx---+ 1 XXXX XXXX 275799 2013-12-30 21:12 TheScienceofGettingRich.pdf
-rwxrwx---+ 1 XXXX XXXX 25600 2015-05-07 18:52 Thumbs.db
-rwxrwx---+ 1 XXXX XXXX 148051 2015-05-07 18:55 AsAManThinketh.pdf
To find files created within a certain time frame using find as below.
Clearly, the filesystem knows about the birth time of a file:
$ find -newerBt '2014-06-13' ! -newerBt '2014-06-13 12:16:10' -ls
20547673299906851 148 -rwxrwx--- 1 XXXX XXXX 148051 May 7 18:55 ./AsAManThinketh.pdf
1407374883582246 244 -rwxrwx--- 1 XXXX XXXX 249159 May 31 2013 ./Getting\ Started.pdf
We can confirm this using stat:
$ stat -c "%w %n" * | sort
2014-06-13 12:16:03.873778400 +0100 AsAManThinketh.pdf
2014-06-13 12:16:04.006872500 +0100 Getting Started.pdf
2014-06-13 12:16:29.607075500 +0100 TheScienceofGettingRich.pdf
2015-05-07 18:32:26.938446200 +0100 Thumbs.db
stat man pages explains %w:
%w time of file birth, human-readable; - if unknown
ls -i menus.xml
94490 menus.xml
Here the number 94490 represents inode
Then do a:
df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg-root 4.0G 3.4G 408M 90% /
tmpfs 1.9G 0 1.9G 0% /dev/shm
/dev/sda1 124M 27M 92M 23% /boot
/dev/mapper/vg-var 7.9G 1.1G 6.5G 15% /var
To find the mounting point of the root "/" filesystem, because the file menus.xml is on '/' that is '/dev/mapper/vg-root'
debugfs -R 'stat <94490>' /dev/mapper/vg-root
The output may be like the one below:
debugfs -R 'stat <94490>' /dev/mapper/vg-root
debugfs 1.41.12 (17-May-2010)
Inode: 94490 Type: regular Mode: 0644 Flags: 0x0
Generation: 2826123170 Version: 0x00000000
User: 0 Group: 0 Size: 4441
File ACL: 0 Directory ACL: 0
Links: 1 Blockcount: 16
Fragment: Address: 0 Number: 0 Size: 0
ctime: 0x5266e438 -- Wed Oct 23 09:46:48 2013
atime: 0x5266e47b -- Wed Oct 23 09:47:55 2013
mtime: 0x5266e438 -- Wed Oct 23 09:46:48 2013
Size of extra inode fields: 4
Extended attributes stored in inode body:
selinux = "unconfined_u:object_r:usr_t:s0\000" (31)
BLOCKS:
(0-1):375818-375819
TOTAL: 2
Where you can see the creation time:
ctime: 0x5266e438 -- Wed Oct 23 09:46:48 2013
stat -c %w a.txt
%w returns the file creation(birth) date if it is available, which is rare.
Here's the link
As #mikyra explained, creation date time is not stored anywhere.
All the methods above are nice, but if you want to quickly get only last modify date, you can type:
ls -lit /path
with -t option you list all file in /path odered by last modify date.
If you really want to achieve that you can use a file watcher like inotifywait.
You watch a directory and you save information about file creations in separate file outside that directory.
while true; do
change=$(inotifywait -e close_write,moved_to,create .)
change=${change#./ * }
if [ "$change" = ".*" ]; then ./scriptToStoreInfoAboutFile; fi
done
As no creation time is stored, you can build your own system based on inotify.
Cited from https://unix.stackexchange.com/questions/50177/birth-is-empty-on-ext4/131347#131347 , the following shellscript would work to get creation time:
get_crtime() {
for target in "${#}"; do
inode=$(stat -c %i "${target}")
fs=$(df "${target}" | tail -1 | awk '{print $1}')
crtime=$(sudo debugfs -R 'stat <'"${inode}"'>' "${fs}" 2>/dev/null | grep -oP 'crtime.*--\s*\K.*')
printf "%s\t%s\n" "${target}" "${crtime}"
done
}
even better:
lsct ()
{
debugfs -R 'stat <'`ls -i "$1" | (read a b;echo -n $a)`'>' `df "$1" | (read a; read a b; echo "$a")` 2> /dev/null | grep --color=auto crtime | ( read a b c d;
echo $d )
}
lsct /etc
Wed Jul 20 19:25:48 2016
Another trick to add to your arsenal is the following:
$ grep -r "Copyright" /<path-to-source-files>/src
Generally speaking, if one changes a file they should claim credit in the “Copyright”. Examine the results for dates, file names, contributors and contact email.
example grep result:
/<path>/src/someobject.h: * Copyright 2007-2012 <creator's name> <creator's email>(at)<some URL>>

Resources