Unexpected behavior when comparing two folders containing the same filenames on MacOS. On windows the comparison works. On MacOS .includes is never true.
Steps to reproduce:
Create 2 separate folders with filenames containing special characters ä, ö, ü.
for example:
'Aktivität.json',
'Anfängerin.json',
'Arbeitsgerät.json',
'Augenhöhle.json',
'Ausländer.json',
'Ängstlichkeit.json',
'Ärger.json',
'Ärztin.json',
'Bankdrücken.json',
'Bauchspeicheldrüse.json',
'Bäckerei.json'
Create and run node script:
import fs from "fs";
var dir = "../path/";
var path = `${dir}folder1/`;
var files = await fs.readdirSync(path);
var pathDone = `${dir}folder2/`;
var filesDone = await fs.readdirSync(pathDone);
console.log(files.length,filesDone.length)
files = files.filter((val) =>( !filesDone.includes(val)&&val.includes('.json')));
console.log(files)
console.log(filesDone)
I know it must be to do with how the filenames are encoded, but why would be comparing the two with the same filenames not work?
I have no idea what is causing this error! However it is not being caused by anything to do with coming from windows. Some of the files are being downloaded from my server using Cyberduck, there seems to be a problem with how the filenames are being saved (that is not visible to me in node, terminal or in finder!). I am now just going to use scp command instead of cyberduck for this tastk.
Related
I'm passing the string "C:\random-folder\ui-debug.log" file path to fs.readFile() on a virtualized Windows 11 installation under macos and a x64 bit electron installation and nodeJS 18 (idk if that matters).
Using the following code:
const filepath = path.resolve(path.normalize(pathStr))
const file = fs.readFileSync(filepath)
but continuously get this error:
TypeError [ERR_INVALID_ARG_VALUE]: The argument 'path' must be a string or Uint8Array without null bytes. Received 'C:\\random-folder\\ui-debug.log\x00'
The file path always seems to be interpreted incorrectly by readFileSync, the \x00 is always added to the end of the path it errors with. This same code works fine on macos.
What is the correct way to read arbitrary file paths with nodejs/electron under Windows? Thanks!
The issue was to do with how the string was being parsed in the first place. Before passing the filepath to readFileSync, it was being converted to ucs2
const pathStr = clipboard.readBuffer('FileNameW').toString('ucs2')
On windows some extra characters were added at the end of the string but was console logged as normal (some internal conversion going on?) so it looked like a normal path in console but internally it had some extra encoded characters on the end.
I solved it with some regex:
const pathStr = clipboard.readBuffer('FileNameW').toString('ucs2').replace(RegExp(String.fromCharCode(0), 'g'), '')
Now readFileSync reads the file just fine!
My python version is 3.5 through Anaconda on Windows 10 environment. I'm using Pyminizip because I need password protected for my zip files, and Zipfile doesn't support it yet.
I am able to zip single file through the function pyminizip.compress, and the encrypt function worked as expected. However, when trying to use pyminizip.compress_multiple I always encountered a Python crash (as pictures) and I believe it's due to the problem of my bad input format.
What I would like to know is: What's the acceptable format for input argument src file LIST path? From Pyminizip's documentation:
pyminizip.compress_multiple([u'pyminizip.so', 'file2.txt'], "file.zip", "1233", 4, progress)
Args:
1. src file LIST path (list)
2. dst file path (string)
3. password (string) or None (to create no-password zip)
4. compress_level(int) between 1 to 9, 1 (more fast) <---> 9 (more compress)
It seems the first argument src file LIST path should be a list containing all files required to be zipped. Accordingly, I tried to use compress_multiple to compress single file with command:
pyminizip.compress_multiple( ['Filename.txt'], 'output.zip', 'password', 4, optional)
and it lead to Python crash. So I try to add a full path into the args.
pyminizip.compress_multiple( [os.getcwd(), 'Filename.txt'], ... )
and still, it crashed again. So I think maybe I have to split the path like this
path = os.getcwd().split( os.sep )
pyminizip.compress_multiple( [path, 'Filename.txt'], ...)
still got a bad luck. Any ideas?
Pyminizip requires the path name (or relative path name from where the script is running from) in the files.
Your example:
pyminizip.compress_multiple( [os.getcwd(), 'Filename.txt'], ... )
gives a list of files of os.getcwd(), and then another file, 'Filename.txt'. You need to combine them into a single path using os.path.join()
in your filename example, you will need:
pyminizip.compress_multiple( [os.path.join(getcwd(), 'Filename.txt')],...)
conversly:
pyminizip.compress_multiple( [os.path.join(getcwd(), 'Filename1.txt'), os.path.join(getcwd(), 'Filename2.txt')],...)
From here - https://pypi.org/project/pyminizip/, the usage of compress_multiple is
pyminizip.compress_multiple([u'pyminizip.so', 'file2.txt'], [u'/path_for_file1', u'/path_for_file2'], "file.zip", "1233", 4, progress)
The second parameter is a bit confusing, but if used, it will create a zip file, which when uncompressed, will create a directory structure like:
I need to zip some files in amazon s3 without needing to write them to file locally first. Ideally my code worked in development but i don't have many write privileges in production.
folder = output_dir
files = fs.glob(folder)
f = BytesIO()
zip = zipfile.ZipFile(f, 'a', zipfile.ZIP_DEFLATED)
for file in files:
filename = os.path.basename(file)
image = fs.get(file, filename)
zip.write(filename)
zip.close()
the proplem is at this line in production
image = fs.get(file, filename)
Because i don't have write privileges.
My last resort is to write to /tmp/ directory which i have privileges to.
Is there a way to zip files from a url path or directly in the cloud?
I ended up using python tempfile which ended up being a perfect solution.
Using NamedTemporaryFile gave me the guarantee to create named and system visible temporary files that could be deleted automatically. No manual work.
Is there a way to make a call to the command line, (On Debian) from ActionScript in Adobe Flash? For example, execute files:
python update.py
Steps for this on Windows would also be appreciated!
If you are using Adobe Air version 2.0.x on Linux, you can use the NativeProcess() class (assuming your update.py is flagged as executable (a+x):
var nativeProcessStartupInfo:NativeProcessStartupInfo = new NativeProcessStartupInfo();
var file:File = File.applicationDirectory.resolvePath("update.py");
nativeProcessStartupInfo.executable = file;
var processArgs:Vector.<String> = new Vector.<String>();
processArgs.push("AnUpdateArgument");
nativeProcessStartupInfo.arguments = processArgs;
process = new NativeProcess();
process.addEventListener(ProgressEvent.STANDARD_OUTPUT_DATA, onOutputData);
process.start(nativeProcessStartupInfo);
public function onOutputData(event:ProgressEvent):void
{
var stdOut:ByteArray = process.standardOutput;
var data:String = stdOut.readUTFBytes(process.standardOutput.bytesAvailable);
trace("Got: ", data);
}
Communicating with native processes in AIR
Note: You can not do this via a browser based SWF
OK.
I see I can substitute the value of the file variable for what ever I need executed.That's fine if I want to execute that file. What about a simple command like cd MyDir or mkdir ThisAndThat Just plain old passes to the shell will be great. I am also using GNU Gnash with an SFW.
I have a file that is required in many other files, that are on different folders, inside the main directory.
Is there a way to just require the filename without having to write the relative path, or the absolute path? Like require('the_file'). And without having to go to npm and install it?
Create a folder inside your main directory , put the_file.js inside and set the NODE_PATH variable to this folder.
Example :
Let's say you create a ./libs folder within your main directory, you can just use :
export NODE_PATH = /.../main/lib
after that, you can require any module inside this directory using just :
var thefile = require('the_file')
To not have to do that every time, you'd have to add the variable to your .bashrc (assuming you're running a Unix system).
Or you can set a global variable inside your app.js file and store the path of your 'the_file' in it like so :
global.rootPath = __dirname;
Then you can require from any of your files using :
var thefile = require(rootPath+'/the_file')
These are the most convenient methods for me, short of creating a private npm, but there are a few other alternatives that I discovered when looking up an answer to your question, have a look here : https://gist.github.com/branneman/8048520