How to pass a buffer to a function instead of passing a file path?
This doesn't work (it's a bug) so I need another way to pass a buffer to the first argument which sends to stdout, but how?
page.render('/dev/stdout', {format : 'pdf'});
I can do something like this, but don't want to create a file
var pdf_file = 'test.pdf';
page.render(pdf_file, {format : 'pdf'});
var pdf = fs.read(pdf_file);
require('system').stdout.write(pdf);
fs.remove(pdf_file);
Related
I'm wondering if it's possible to somehow show the progress of reading a file with gridfs.GridFS() and pymongo. I could not file a callback I could pass to the .read() function.
my_db = MongoClient().test
fs = GridFSBucket(my_db)
# get _id of file to read.
file_id = fs.upload_from_stream("huge_test_file", "i carry lots of data!")
grid_out = fs.open_download_stream(file_id)
contents = grid_out.read()
Is there some way to actually retrieve the bytes that were already downloaded? Considering, the file may be being 5GB big, I want to give some download status feedback.
I know it's been 11 months.. but i think a way to do this is:
Make a new endpoint that returns the last chunks 'n' value, then compare this with the file 'chunkSize' value.
#app.route('/uploadProgress/<string:fileID>')
def uploadProgress(fileID):
doc = db.fs.files.find_one({"_id": fileID})
size = doc['chunkSize']
n = db.fs.chunks.find( { "files_id": fileID } ).sort([('n', -1)]).limit(1)[0]['n']
progress = str(n/size)
return progress
Now in javascript you can make a request to check the progress and update the progress bar
I am creating a personal assistant in Python. I am using Snowboy to record audio, and it works very well. Snowboy has a saveMessage() method that creates and writes a wav file to the disk. This wav file is later read and used as an AudioFile object by Speech_Recognition. I find it very inefficient that the program has to write and read wav files to the disk. I would much rather have the wav file be passed around as an object without EVER saving it to the disk.
Here is the snowboy saveMessage() module that I would like to reweite.
def saveMessage(self):
"""
Save the message stored in self.recordedData to a timestamped file.
"""
filename = 'output' + str(int(time.time())) + '.wav'
data = b''.join(self.recordedData)
#use wave to save data
wf = wave.open(filename, 'wb')
wf.setnchannels(1)
wf.setsampwidth(self.audio.get_sample_size(
self.audio.get_format_from_width(
self.detector.BitsPerSample() / 8)))
wf.setframerate(self.detector.SampleRate())
wf.writeframes(data)
wf.close()
logger.debug("finished saving: " + filename)
return filename #INSTEAD OF RETURNING filename I WANT THIS TO RETURN THE wav file object
Please note that the AudioFile class requires that the path for the wave file OR a "file-like" object must be passed into it. I am not sure what a "file-like" object is, so I will provide the AudioFile assert statement for the wav file argument:
assert isinstance(filename_or_fileobject, (type(""), type(u""))) or hasattr(filename_or_fileobject, "read"), "Given audio file must be a filename string or a file-like object"
I have tried to use an instance of BytesIO to save the wav data, BytesIO is apparently not a file-like object. Here was what I tried:
def saveMessage(self):
filename = 'output' + str(int(time.time())) + '.wav'
data = b''.join(self.recordedData)
#use wave to save data
with io.BytesIO() as wav_file:
wav_writer = wave.open(wav_file, "wb")
try:
wav_writer.setnchannels(1)
wav_writer.setsampwidth(self.audio.get_sample_size(
self.audio.get_format_from_width(
self.detector.BitsPerSample() / 8)))
wav_writer.setframerate(self.detector.SampleRate())
wav_writer.writeframes(data)
wav_data = wav_file.getvalue()
finally:
wav_writer.close()
logger.debug("finished saving: " + filename)
return wav_data
The error I got was: AssertionError: Given audio file must be a filename string or a file-like object
I am using python 3.7 on a Raspberry PI 3B+ running Raspbian Buster Lite kernel version 4.19.36.
If I can provide any additional information or clarify anything, please ask.
Thanks so much!
Something like this should work:
from speech_recognition import AudioData
def saveMessage(self):
filename = 'output' + str(int(time.time())) + '.wav'
data = b''.join(self.recordedData)
ad = AudioData(data, 16000, 2)
result = recognizer.recognize_google(ad)
Note that speech_recognition.listen can invoke snowboy internally, so you probably don't have to use external snowboy, you can just use listen with parameter snowboy_configuration.
In extension to this question.
Is it possible to read a file into a string without knowing the path to the file? - I only have the file as a 'def'/type-less parameter, which is why I can't just do a .getAbsolutePath()
To elaborate on this, this is how I import the file (which is from a temporary .jar file)
def getExportInfo(path) {
def zipFile = new java.util.zip.ZipFile(new File(path))
zipFile.entries().each { entry ->
def name = entry.name
if (!entry.directory && name == "ExportInfo") {
return entry
}
}
}
A ZipEntry is not a file, but a ZipEntry.
Those have almost nothing in common.
With def is = zipFile.getInputStream(entry) you get the input stream to the zip entry contents.
Then you can use is.text to get the contents as String in the default platform encoding or is.getText('<theFilesEncoding>') to get the contents as String in the specified encoding, exactly the same as you can do on a File object.
I'm writing an app using Flask and one of the things I want in it is the ability to upload a ZIP that conforms to a specific form; as such I have a small function that takes a FileStorage object from the form and first unzips it, checks the contents, and then tries to save. There's a problem, however - apparently unzipping it "breaks" the FileStorage object, as the following function:
def upload_modfile(modfile):
if not modfile.filename.endswith('.zip'):
raise ModError('Incorrect filename')
mod_path = join(get_mods_path(), secure_filename(modfile.filename))
if isfile(mod_path):
raise ModError('File ' + modfile.filename + ' already exists')
modzip = ZipFile(modfile)
base_filename = modfile.filename[:-4]
modzip_contents = modzip.namelist()
if join(base_filename, 'info.json') not in modzip_contents:
raise ModError('Could not validate file')
modfile.save(mod_path)
return True
saves modfile as a text file saying Archive: <filename>.zip. If I comment out the entire ZipFile bit (i.e. everything involving modzip), the file is saved just fine.
I'm pretty much brand new to Python and am a little confused as to what to do in this case, save for saving the file in /tmp. Should I somehow clone modfile by way of some stream? Is there a way of "rewinding" the stream pointer within FileStorage that I'm missing?
I'd like to pass a JSON object as a command line argument to node. Something like this:
node file.js --data { "name": "Dave" }
What's the best way to do this or is there another more advisable way to do accomplish the same thing?
if its a small amount of data, I'd use https://www.npmjs.com/package/minimist, which is a command line argument parser for nodejs. It's not json, but you can simply pass options like
--name=Foo
or
-n Foo
I think this is better suited for a command line tool than json.
If you have a large amount of data you want to use you're better of with creating a json file and only pass the file name as command line argument, so that your program can load and parse it then.
Big objects as command line argument, most likely, aren't a good idea.
this works for me:
$ node foo.js --json-array='["zoom"]'
then in my code I have:
import * as _ from 'lodash';
const parsed = JSON.parse(cliOpts.json_array || []);
_.flattenDeep([parsed]).forEach(item => console.log(item));
I use dashdash, which I think is the best choice when it comes to command line parsing.
To do the same thing with an object, just use:
$ node foo.js --json-object='{"bar": true}'
This might be a bit overkill and not appropriate for what you're doing because it renders the JSON unreadable, but I found a robust way (as in "works on any OS") to do this was to use base64 encoding.
I wanted to pass around lots of options via JSON between parts of my program (a master node routine calling a bunch of small slave node routines). My JSON was quite big, with annoying characters like quotes and backslashes so it sounded painful to sanitize that (particularly in a multi-OS context).
In the end, my code (TypeScript) looks like this:
in the calling program:
const buffer: Buffer = new Buffer(JSON.stringify(myJson));
const command: string = 'node slave.js --json "' + buffer.toString('base64') + '" --b64';
const slicing: child_process.ChildProcess = child_process.exec(command, ...)
in the receiving program:
let inputJson: string;
if (commander.json) {
inputJson = commander.json;
if (commander.b64) {
inputJson = new Buffer(inputJson, 'base64').toString('ascii');
}
}
(that --b64 flag allows me to still choose between manually entering a normal JSON, or use the base64 version, also I'm using commander just for convenience)