I am trying to PUT a file to an S3 pre-signed URL. Doing so with bash/cURL works fine but with Node I am getting the following error:
Error: write EPIPE
at WriteWrap.onWriteComplete [as oncomplete] (node:internal/stream_base_commons:94:16) {
errno: -32,
code: 'EPIPE',
syscall: 'write'
}
Here is the code
const fs = require('fs');
const request = require('request');
stream = fs.createReadStream('/tmp/file');
r = request.put('https://s3.eu-west-2.amazonaws.com/bucketname/path?X-Amz-Content-Sha256=....&...');
stream.pipe(r).on('error', function(err) {
console.log(err);
});
EPIPE means that the writing request failed because the other end closed the connection. Looks like there might be some additional settings required inorder to work with amazon s3. I know that curl has native support for multipart/form-data. You can use this library to create readable multipart/form-data streams.
https://nicedoc.io/form-data/form-data
Or you can use third party libraries to send data
https://www.npmjs.com/package/s3-upload-stream
https://www.npmjs.com/package/streaming-s3
Related
I would like download an file with Electron and save my file in path.
So, i've my request with net module, createWriteStream with my path :
const out = fs.createWriteStream(configuration.localFile);
const request = net.request(arg);
request.pipe(out); // Error: request.pipe is not a function
request
.on('response', ...
In the ClientRequest doc, i see ClientRequest implements the Writable Stream interface and is therefore an EventEmitter.
Anyone can help me ?
Thank you community !
I have a NodeJS and an API that handles get requests.
...
var apiRoutes = express.Router();
apiRoutes.get("/filter/:name",function(req, res){
// do something
res.json(result);
}
app.use('/api', apiRoutes);
Then in the client (not an important information but it is Angular2):
find(name:string): void{
name.trim();
this.http.get(encodeURI('http://server_address/api/filter/' + name))...
It works well for the parameters don't contain whitespaces, etc. In order to make it working with spaced inputs also, I used encodeURI function. However, when I give an input with whitespaces the server gives error:
undefined:0
^
SyntaxError: Unexpected end of input
at Object.parse (native)
at IncomingMessage.<anonymous> (/user/home/server/server.js:65:28)
at IncomingMessage.EventEmitter.emit (events.js:117:20)
at _stream_readable.js:920:16
at process._tickCallback (node.js:415:13)
Any idea what can I do to fix it?
I figured out the problem. I was doing something like:
apiRoutes.get("/filter/:name",function(req, res){
http.request(anotherURL + req.body.name)...
}
And thought that the name parameter is already encoded since it was encoded in the client. However I see that I have to encode it in the server again.
apiRoutes.get("/filter/:name",function(req, res){
http.request(anotherURL + encodeURI(req.body.name))...
}
I can bind to localAddress just fine when using HTTP, but as soon as I switch to HTTPS I get an error: bind EINVAL. Please consider this code:
var http = require('http');
var https = require('https');
var options = { host:'icanhazip.com',path:'/',localAddress:'2604:a880:1:20::27:a00f',family:6 };
callback = function(response) {
var data = '';
response.on('data',function(chunk) { data+= chunk; });
response.on('error',function(error) { console.log("error: "+error.message); });
response.on('end',function() {
console.log(data);
});
}
http.request(options,callback).end(); // Works. IP:2604:a880:1:20::27:a00f
https.request(options,callback).end(); // Fails. IP:2604:a880:1:20::27:a00f
https.request({host:'icanhazip.com',path:'/',family:6},callback).end(); // Works. IP:2604:a880:1:20::27:a00f
Here's the error while running node v5.0.0:
Error: bind EINVAL 2604:a880:1:20::27:a00f
at Object.exports._errnoException (util.js:860:11)
at exports._exceptionWithHostPort (util.js:883:20)
at connect (net.js:809:16)
at net.js:984:7
at GetAddrInfoReqWrap.asyncCallback [as callback] (dns.js:63:16)
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:82:10)
The only difference between the working and the failing code is setting localAddress and ironically, the last example binds to the correct IP address, but won't let you do it using localAddress.
The problem here is I have to make a request from a completely separate IPv6 address under my use case, and it works fine with HTTP but I need this to work for HTTPS requests. Currently I can only make this work while using cURL. Could you please provide some insight as to why this is happening or how I could make this work without additional libraries?
I had same issue as you. Figured it out. Update your node to the latest stable. They fixed it. Check it with node --version I'm on 6.6.0 and it works great.
The version I got from doing an apt-get was way too old.
The following code:
var output = '<p>Hello</p>';
require("jsreport").render({
template: {
content: output
}
}).then(function(out) {
out.result.pipe(res);
});
});
returns this error :
Error: { [Error: Error during rendering report: connect ECONNREFUSED]
code: 'ECONNREFUSED',
errno: 'ECONNREFUSED',
syscall: 'connect' }
I believe it's because it cannot connect to internal jsreport server, but don't know why. Any idea on how to debug it or why it happens ?
We changed the embedded jsreport to stand alone jsreport . So we use the remote client to connect to this stand-alone instance. Now it works ! I don't know why.
I'm trying to build a Twitter streaming app using node.js and the ntwitter module, here's my code :
var app = require('express').createServer(),
twitter=require('ntwitter');
app.listen(3000);
var feed = new twitter({
consumer_key: 'MY KEY',
consumer_secret:'MY SECRET KEY',
access_tocken_key:'MY ACCESS TOCKEN KEY',
access_tocken_secret:'MY ACCESS TOCKEN SECRET'
});
feed.stream('statuses/filter',{track: ['love', 'hate']}, function(stream){
stream.on('data',function(tweet){
console.log(tweet.text);
});
});
But here's what I get :
events.js:74
throw TypeError('Uncaught, unspecified "error" event.');
^
TypeError: Uncaught, unspecified "error" event.
at TypeError (<anonymous>)
at EventEmitter.emit (events.js:74:15)
at ClientRequest.<anonymous> (/Users/maximeheckel/Documents/My_Repositories/nodetwitter/node_modules/ntwitter/lib/twitter.js:251:14)
at ClientRequest.EventEmitter.emit (events.js:95:17)
at HTTPParser.parserOnIncomingClient [as onIncoming] (http.js:1628:21)
at HTTPParser.parserOnHeadersComplete [as onHeadersComplete] (http.js:119:23)
at CleartextStream.socketOnData [as ondata] (http.js:1523:20)
at CleartextStream.read [as _read] (tls.js:470:10)
at CleartextStream.Readable.read (_stream_readable.js:294:10)
at EncryptedStream.write [as _write] (tls.js:344:25)
I don't understand why I'm stuck with that as I'm following very carefully a lot of tutorials. Even when I clone the authors project I'm still getting this error.
Hope you can help.
UPDATE : I added
stream.on('error', function(error, code) {
console.log("My error: " + error + ": " + code);
});
To my stream function and I'm getting a HTTP 401 error
Any ideas ?
Everything with your code looks fine except for the spelling of "token" in the parameters passed to the constructor for twitter. You need to change access_tocken_key to access_token_key and access_tocken_secret to access_token_secret. Your error (401) is an authentication issue; that change should hopefully result in ntwitter passing the correct user authentication details to the Twitter API.
I had the same problem and it was because my servers system clock had floated by six minutes. Usually, API's give you about a five minute margin of error.