Poking around I was unable to discover a way to detect hidden files in OS X with node (nodejs).
Of course, we can easily find the ".dot_hidden" files, but on the Mac, there are files/folders that are "protected" system files, that most users shouldn't fiddle with. In the Finder GUI, they are invisible or grey'd out when hidden files are forced to be shown via "AppleShowAllFiles".
I did discover a reference to UF_HIDDEN : 0x8000 here:
https://developer.apple.com/library/mac/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemDetails/FileSystemDetails.html
Using node's stat, we can return 2 additional bits of info that may provide a clue for the hidden status:
mode: 33188, // File protection.
ino: 48064969, // File inode number. An inode is a file
system data structure that stores
information about a file.
I'm not really a hex / binary guy, but it looks like grabbing the stat's "ino" property we can apply 0x8000 and determine if the file is being hinted as hidden or not.
I didn't have any success with the 0x8000 mask on the mode, but did have some with ino.
Here's what I've got, checking the "ino" returns 0 or 1726, when it's 1726 the file seems to match as a hidden file in OS X.
var fs = require("fs");
var dir = "/";
var list = fs.readdirSync(dir);
list.forEach(function(f){
// easy dot hidden files
var hidden = (f.substr(0, 1) == ".") ? true : false;
var ino = 0;
var syspath = dir + "/" + f;
if( ! hidden ){
var stats = fs.statSync(syspath);
ino = parseInt( stats.ino & 0x8000, 8);
// ino yeilds 0 when hidden and 1726 when not?
if(ino || dotted){
hidden = true;
}
}
console.log(syspath, hidden, ino);
});
So my question is if I'm applying the 0x8000 mask properly on the ino value to yeild a proper result?
And how would one go about parsing the ino property get at all the other flags contained within it?
The inode number (stats.ino) is a number which uniquely identifies a file; it has nothing to do with the hidden status of the file. (Indeed, it's possible to set or clear the hidden flag on a file at any time, and this won't change the inode number.)
The hidden flag is part of the st_flags field in the struct stat structure. Unfortunately, it doesn't look like the node.js fs module exposes this value, so you may need to shell out to the stat shell utility if you need to get this information on Mac OS X. (Short version: stat -f%f file will print a file's flags, represented in decimal.)
Related
I'm trying to understand how devicetrees work.
According to the kernel documentation, they are used, in arm architecture, in the following manner:
In the majority of cases, the machine identity is irrelevant, and the kernel will instead select setup code based on the machine’s core CPU or SoC. On ARM for example, setup_arch() in arch/arm/kernel/setup.c will call setup_machine_fdt() in arch/arm/kernel/devtree.c which searches through the machine_desc table and selects the machine_desc which best matches the device tree data. It determines the best match by looking at the ‘compatible’ property in the root device tree node, and comparing it with the dt_compat list in struct machine_desc (which is defined in arch/arm/include/asm/mach/arch.h if you’re curious).
The ‘compatible’ property contains a sorted list of strings starting with the exact name of the machine, followed by an optional list of boards it is compatible with sorted from most compatible to least.
I found the source code related to the comparison of machine_desc to the compatible parameter set in the dts file:
const struct machine_desc * __init setup_machine_fdt(void *dt_virt)
{
const struct machine_desc *mdesc, *mdesc_best = NULL;
#if defined(CONFIG_ARCH_MULTIPLATFORM) || defined(CONFIG_ARM_SINGLE_ARMV7M)
DT_MACHINE_START(GENERIC_DT, "Generic DT based system")
.l2c_aux_val = 0x0,
.l2c_aux_mask = ~0x0,
MACHINE_END
mdesc_best = &__mach_desc_GENERIC_DT;
#endif
if (!dt_virt || !early_init_dt_verify(dt_virt))
return NULL;
mdesc = of_flat_dt_match_machine(mdesc_best, arch_get_next_mach);
if (!mdesc) {
const char *prop;
int size;
unsigned long dt_root;
early_print("\nError: unrecognized/unsupported "
"device tree compatible list:\n[ ");
dt_root = of_get_flat_dt_root();
prop = of_get_flat_dt_prop(dt_root, "compatible", &size);
while (size > 0) {
early_print("'%s' ", prop);
size -= strlen(prop) + 1;
prop += strlen(prop) + 1;
}
early_print("]\n\n");
dump_machine_table(); /* does not return */
}
/* We really don't want to do this, but sometimes firmware provides buggy data */
if (mdesc->dt_fixup)
mdesc->dt_fixup();
early_init_dt_scan_nodes();
/* Change machine number to match the mdesc we're using */
__machine_arch_type = mdesc->nr;
return mdesc;
}
However, I didn't find machine_desc table definition.
If I'd like to read all machine_desc, Where can I find it?
TL;DR - The machine description is built by building and linking different source files into the kernel. So each machine source file adds an entry into the table.
The table is based in arch/arm/kernel/vmlinux.lds.S (or relevant architecture linker file). It is built with the macros MACHINE_START and MACHINE_END. This places a structure in the 'arch.info.init' sections of the object file. All of these objects get globbed together by the linker. This forms the table. So, it is constructed by linking different source files with the MACHINE_START and MACHINE_END macros. Therefore, it doesn't exist in one place.
However, you can use git grep -A10 MACHINE_START to get a fairly good list. This command works well, as typically, it is the last thing in the file so only five or six lines may print. Or you could write init code to dump the table by printing the machine_desc entries.
That said, the table is not too interesting as it is just function pointers to call at different times. The majority will be NULL as it used designated initializers.
Related: Control to 'dt_machine_start' on Android
I found a solution for my original question in another post Google Apps Script creates sheets version of excel file.
Testing with the code provided in the answer I ran into another issue. Every time I run the script it creates the Spreadsheets version of the .xlsx files again even if they already exist. I have tried modifying the code withing the last If with no results. Then went back to run your code as posted in case I have missed something but it keeps creating versions of the same files.
Any idea of what could I do to fix this will be really appreciated.
The code provided int he answer is the following.
// Convert the user's stored excel files to google spreadsheets based on the specified directories.
// There are quota limits on the maximum conversions per day: consumer #gmail = 250.
function convertExcelToGoogleSheets()
{
var user = Session.getActiveUser(); // Used for ownership testing.
var origin = DriveApp.getFolderById("origin folder id");
var dest = DriveApp.getFolderById("destination folder id");
// Index the filenames of owned Google Sheets files as object keys (which are hashed).
// This avoids needing to search and do multiple string comparisons.
// It takes around 100-200 ms per iteration to advance the iterator, check if the file
// should be cached, and insert the key-value pair. Depending on the magnitude of
// the task, this may need to be done separately, and loaded from a storage device instead.
// Note that there are quota limits on queries per second - 1000 per 100 sec:
// If the sequence is too large and the loop too fast, Utilities.sleep() usage will be needed.
var gsi = dest.getFilesByType(MimeType.GOOGLE_SHEETS), gsNames = {};
while (gsi.hasNext())
{
var file = gsi.next();
if(file.getOwner().getEmail() == user.getEmail())
gsNames[file.getName()] = true;
}
// Find and convert any unconverted .xls, .xlsx files in the given directories.
var exceltypes = [MimeType.MICROSOFT_EXCEL, MimeType.MICROSOFT_EXCEL_LEGACY];
for(var mt = 0; mt < exceltypes.length; ++mt)
{
var efi = origin.getFilesByType(exceltypes[mt]);
while (efi.hasNext())
{
var file = efi.next();
// Perform conversions only for owned files that don't have owned gs equivalents.
// If an excel file does not have gs file with the same name, gsNames[ ... ] will be undefined, and !undefined -> true
// If an excel file does have a gs file with the same name, gsNames[ ... ] will be true, and !true -> false
if(file.getOwner().getEmail() == user.getEmail() && !gsNames[file.getName()])
{
Drive.Files.insert(
{title: file.getName(), parents: [{"id": dest.getId()}]},
file.getBlob(),
{convert: true}
);
// Do not convert any more spreadsheets with this same name.
gsNames[file.getName()] = true;
}
}
}
}
You want to convert Excel files in origin folder to Google Spreadsheet and put the converted Spreadsheet to dest folder.
When the filename of converted file is existing in dest folder, you don't want to convert it.
If my understanding is correct, how about this modification?
From:
if(file.getOwner().getEmail() == user.getEmail() && !gsNames[file.getName()])
To:
if(file.getOwner().getEmail() == user.getEmail() && !gsNames[file.getName().split(".")[0]])
Note:
In this modification, when the filename of converted file is found in the dest folder, the file is not converted.
When the filename has the extension like ###.xlsx and it is converted to Google Spreadsheet, it seems that the extension is automatically removed. I think that this is the reason that the duplicated files are created. So I used split(".")[0] for this situation.
Reference:
split()
I have a QRCode image and it's content: P01 V0N S0000000
I tried to regenerate this image with different matchPattern using node-qrcode:
const $qr = require('qrcode')
const text = 'P01 V0N S0000000'
for (let i = 0; i < 8; i++) {
let path = `P01V0NS0000000-${i}.png`
const opt = {
maskPattern: i
}
$qr.toFile(path, text, opt, function (err) {
if (err) {
console.log(err)
} else {
console.log(opt)
}
})
}
but none of generated image has the same pattern as the target:
I also tried with different version paramter, but all version>1 will generate a much dense pattern.
Is this QRCode image generated with some kind of salt to make hard to replicate?
No need to try with another version as your first image clearly is a version 1 (21x21). Instead, it might be the format encoding (given the set of chars, the app might use alphanumeric and your original might well be byte for example)
As per the link you provided, I'd recommand reading this section about modes and try either with:
mode: 'alphanumeric'
or with:
mode: 'byte'
to see if any of those match your original code.
Another parameter that can change the aspect of the QR-code is the error correction level. I'd guess by default node-qrcode uses the highest level possible given version 1, and your original might use a lower one.
With those params, you have a set of 8(maskPattern)*2(mode)*4(errorCorrectionLevel) = 64 possible QR-codes with the same data inside, so maybe one of them will match!
Edit
As your data is 16 characters long, the scope is reduced as error correction level can only have the following values (source):
L,M,Q for alphanumeric mode
L for byte mode
Therefore, you have (3+1)*8 = 32 possible resulting images.
I am using Haxe and OpenFL and I got a program that generates a Xml file.
However, I can't figure out how to save that file. I can create Xml tree and check it's valid, but for my life I can't figure out how to write the file.
So, in simple, how to do I write(and create) a file in Haxe? I want to be able to save my newly created Xml File (they serve as sort of settings file and initialization files for my program) on computer, so that I can load it later?
Found the solution right after writing this question.
Solution is to first to use sys.io.File.write() to create the file, then File.saveContent() to save the data in. You can get string from Xml with toString function, the ultimate solution looks like this:
if (!FileSystem.exists("maps"))
{
FileSystem.createDirectory("maps");
}
var number:Int = 1;
if (FileSystem.exists("maps/" + filename + ".xml"))
{
while(FileSystem.exists("maps/" + filename + number +".xml"))
{
number++;
}
filename = filename + number;
}
File.write("maps/" + filename + ".xml", false);
File.saveContent("maps/" + filename + ".xml", root.toString());
This checks if the directory exist and if not, create it and if the file exist, create a new numbered file rather than override it (for the moment, still working on the save feature)
This solution only works on c++, haven't tested others much yet but Flash does not work
Suppose I want to invoke some command on all files in a directory and set a watch to invoke that command on all files that get created in that directory. If I do:
while( ( sdi = readdir( d )) != NULL ) { ... }
closedir( d );
/* Files created here will be missed */
inotify_add_watch( ... );
then some files will potentially be missed. If I call inotify_add_watch()
before the readdir(), files may be acted on twice (it would require
a fair bit of infrastructure to prevent acting twice, and it seems that
the edge cases would be difficult to handle). Is there a simple way to avoid
having to record the names of all files worked on during the readdir loop and
comparing those to the names returned in the inotify_event structure? I can
minimize the amount of necessary comparisons with:
while( ( sdi = readdir( d )) != NULL ) { ... }
inotify_add_watch( ... );
while( ( sdi = readdir( d )) != NULL ) { /* record name */ ... }
closedir( d );
And usually the second readdir() loop will do nothing, but this feels like a bad hack.
You simply can't. The more you hack, the more race conditions you'll get.
The simplest actually working solution is to set the watch before using opendir(), and keep a list (set) of already used names (or their hashes).
But this isn't perfect either. User can have the file open in a text editor; you fix it, user saves it and the directory contains unfixed file anyway, though it's on your list.
The best method would be to be able for the program to actually distinguish used files by their content. In other words, you set watch, call command on readdir() results, then call it on inotify results and let the command itself know whether the file is fine already or not.