So I'm working on a Minecraft launcher (because why not, good experience), and I'm stuck when it comes to downloading the libraries.
I have a valid jar URL here. When you download it in the browser, it works fine. But, when you download it with Node.js, 7-zip gives this error when trying to open it:
An attempt was made to move the file pointer before the beginning of the file.
I'm using a module called snekfetch, but I've also tried it with request. Both items gave the same issue. Here's my current test code:
request.get('https://libraries.minecraft.net/com/google/code/gson/gson/2.8.0/gson-2.8.0.jar').then(r => {
fs.writeFileSync('./mything.jar', r.body);
});
Am I doing something wrong to download the jar file?
Okay, so now that I've seen this answer, I need to modify the question. I've gotten it to work using pipes, but I need inline-code because this is a for loop that's downloading (hence my usage of writeFileSync, and in my actual code I use await for the request). Is it even possible to download and write without piping?
It turns out this is an issue with the snekfetch library. Switching to snekfetch v3 fixed it.
You can check out the status of the issue here.
Related
Update: This question used to ask about Google Cloud Storage, but I have since realized the issue actually is reproducable merely trying to save the download to local disk. Thus, I am rephrasing the question to be entirely about file downloads in Typescript and to no longer mention Google Cloud Storage.
When attempting to download and save a file in Typescript with WebRequests (though I experienced the same issue with requests and request-promises), all the code seems to execute correctly, but the resultant file is corrupted and cannot be viewed. For example, if I download an image, the file is not viewable in any applications.
// Seems to work correctly
const download = await WebRequest.get(imageUrl);
// `Buffer.from()` also takes an `encoding` parameter, but it's unclear how to determine the encoding of a download
const imageBuffer = Buffer.from(download.content);
// I *think* this line is straightforward
const imageByteArray = new Uint8Array(imageBuffer);
// Saves a corrupted file
const file = fs.writeFileSync("/path/to/file.png", imageByteArray);
I suspect the issue lies within the Buffer.from call not correctly interpreting the downloaded content, but I'm not sure how to do it right. Any help would be greatly appreciated.
Thanks so much!
From what I saw in the examples for web-request, download.content is just a string. If you want to upload a string to Cloud Storage using the node SDK, you can use File.save, passing that string directly.
Alternatively, you could use one the solutions seen here.
I hope I'm saying this correctly. What I'm trying to do is write to a json file using fs.writeFile.
I can get it to work using the command line but what I want to do is call a function maybe a button click to update the json file.
I figure I would need some type of call to the node server which is local port 8080. I was researching and seen somebody mention using .post but still can't wrap my head around how to write the logic.
$(".button").on("click", function(event) {
fs.writeFile("./updateme.json", "{test: 1}", function(err) {
if(err) {
return console.log(err);
}
console.log("The file was saved!");
});
});
Using jQuery along with fs? Wow that could be great! Unfortunately that is not as simple as that!
Let me introduce you to server-side VS client-side JavaScript. Well actually there are a lot of resources on the net about that - just google it, or check the answers to this other StackOverflow question. Basically JavaScript can run either on a browser (Chrome, Mozilla...) or as a program (usually a server written in NodeJS), and while the language is (almost) the same, both platforms don't have the same features.
The script that you're showing should run in a browser, because it's using jQuery and interacting with buttons and stuff (aka the DOM). Can you imagine what a mess it would be if that script could interact with the file system? Any page you'll visit will be able to crawl around in your holiday pictures and other personal stuff you keep on your computer. Bad idea! That is why some libraries like fs are not available in the browser.
Similarly, some libraries like jQuery are not available (or simply useless) in the server, because there is no HTML and user interaction, only headless programs running.
So, what can I do to write a JSON file after a user clicks on a button?
You can set up:
A NodeJS server that will write a JSON file
Make jQuery call this server with the data to be written after the user clicks on a button
If you want further guidelines on this, tell me in the comments! I'll be ready to edit my question so as to include instructions on setting up such an environment.
This question is regarding the repo for react implicit grant for docusign using react.
https://github.com/docusign/eg-02-react-implicit-grant
When I tried to clone the repo using VSCode, I received an error
Git: fatal: cannot create directory at 'src/hoc/Aux': Invalid argument
When I looked into the repo, there is a file under src/hoc/aux/aux.js which has the text below.
const aux = (props) => props.children;
export default aux;
Is the file aux.js necessary? Because I was able to extract the zip files after skipping to extract aux.js.
I just tried to clone the https://github.com/docusign/eg-02-react-implicit-grant repo using the GitHub desktop and did not have any problems.
I suggest that you download the repo using git directly or use the download zip option from the repo's page.
Re: aux.js
TL;DR. -- Yes, I believe the aux.js is needed.
Details: Unfortunately I'm not a React expert. (Even though I wrote this example.)
The /hoc directory is used for React Higher Order Components. In this case, I'm using it for a simple component that checks to see if the user has a valid token. I believe that the js file is needed. But you can find out by not including it as seeing what happens.
Also, note that I wrote this React example almost a year ago. You'll want to update it to current React best practices.
Try renaming the aux folder and aux.js file inside it to any random name do it preferably in vscode it will let you know which name is allowed and which isn't, worked for me.
I'm trying to use Dropbox as a cloud-based file receptacle for an app/script. The script, written in Python, needs to take PDFs from the Dropbox and use the tika-python wrapper to convert to string.
I'm able to connect to the Dropbox API and use the files_download_to_file() method to download the PDFs to disk, and then use the tika from_file() method to pull that download file from the disk to process. Example:
# Download ex.pdf to local disk
dbx.files_download_to_file('/my_local_path/ex_on_disk.pdf', '/my_dropbox_path/ex.pdf')
from tika import parser
parsed = parser.from_file('ex_on_disk.pdf')
The problem is that I'm planning on running this app on something like Heroku. I don't think I'm able to save anything locally and then access it again. I'm not sure how to get something from the Dropbox API that can be directly referenced by the tika wrapper to run the same as above. I think the PHP SDK has a file_get_contents and a file_put_contents set of methods but it doesn't appear to have a companion in the Python SDK.
I've tried using the shareable links in place of a filename but that hasn't worked. Any ideas? I know there's also the files_download method which downloads the FileMetadata object but I have no idea what to do with this and am having trouble finding more about it.
TLDR; How can I reference a file on Dropbox with a filename string such as 'example.pdf' to be used in another function that is trying to read a file from disk, without saving that Dropbox file to disk?
I figured it out. I used the files_download method to get the byte string and then use the from_buffer method of tika instead:
md, response = dbx.files_download(path)
file_contents = response.content
parsed = parser.from_buffer(file_contents)
I'm trying out gun.js I have it installed as a node.js project, I have configured the amazon S3 bucket through the dotenv and I have tried adding a data.json file and still I cant get gun.js to save the file locally or to he S3 bucket.
I know its early days for gun, but I get the feeling I'm missing something obvious.
I'm expecting to find a .json file in he local file system and or in the S3 bucket but I get neither.
require('dotenv').config();
var Gun = require('gun');
var gun = Gun({
file: 'data.json', // local testing and development
s3: {
key: process.env.AWS_KEY, // AWS Access Key
secret: process.env.AWS_SECRET, // AWS Secret Token
bucket: process.env.AWS_BUCKET // The bucket you want to save into
}
});
gun.put({ hello: 'world' }).key('my/first/data');
#bill Just noticed this now, sorry for the late answer. Thanks to #paul-w for notifying me of this and his response earlier today.
This question and answer assumes you are running a version EARLIER than v0.4.x!
If you are in NodeJS and are getting the error “You have no persistence layer to save to”, it means the default storage drivers (S3, file.js) didn't get installed or were deactivated - which is unusual as this happens automatically.
Try installing gun (again?) via npm install gun in your local NodeJS project directory, not a git clone or a copy&paste.
I can only guess, given the context you explain, that you might have copied/moved gun (like the gun.js file) into your project. The browser will work with just the single file, but NodeJS needs more - it needs the S3/file.js modules, which will be included if installed with npm or properly git cloned.
Also unlikely (since your code doesn't show this), if you happen to (this is bad) Gun({wire: {put: null, get: null}}) (or something similar) it would intentionally break the persistence drivers.
If you are in the browser and getting the error (and assuming your not overwriting the persistence drivers like in the previous paragraph), it could be because of some weird situation like you are using an old version of IE or a browser that doesn't have JSON support. Again, all these things are unlikely but I'm just wanting to be comprehensive.
Note: The above applies to the question in your title. However your actual question doesn't ask about the error, it asks about not seeing data in data.json or in S3. Answering that below.
To which #paul-w is more on track. If you are using S3 then the file.js module (data.json) automatically deactivates itself. If you are using the file.js module (data.json) then S3 does not get activated. As #paul-w mentioned, v0.4.x will support easily having multiple storage engines simultaneously. However, you should see your data in at least one or the other - unless you are getting the "no persistence layer" error, in which case you won't see your data anywhere because there isn't any persistence! But again, default persistence layers are included with gun by default (unless installation was incorrect, or you explicitly overwrite them - both unusual things).
I hope this answers your question. Sorry I didn't see it till now. Please let me know if this works, and also join the conversation at https://gitter.im/amark/gun . Thank you for helping start the stackoverflow questions! We need more of these!
I think Mark is going to answer this more officially, but the quick answer is that in gun.js 0.3 (current) there is a single gun server peer or storage target, and when you run gun as a server (e.g. from node.js rather than a browser), S3 is preferred, if S3 credentials are specified. But gun is also saving your data changes in browser memory, or localStorage (up to the browser limit of 5MB), and S3 is there for a more permanent storage.
So in the example above, I think the problem is that the file entry will only be used if there is a problem saving changes to S3, and that's why you don't see the new data going there. Maybe try putting an error in the S3 credentials (e.g. add an 'x' for now) and see if it starts using the file path instead.
In gun.js 0.4 there are plans to make use of all peers specified in the constructor or dynamically, but that feature isn't here yet.
(And I probably butchered that answer, but hopefully Mark can correct any inaccuracies in this. I'm new to gun.js but had the same question.)