I have permissions, and can create Databases/Tables in the console just fine. but when i make code using the aws-sdk it fails with an InternalServerException: Internal Server Error. Is this only supported in the v3 api?
const params = {
DatabaseName,
Tags: getDefaultTags(customer,username)
};
console.log(params);
const promise = writeClient.createDatabase(params).promise();
to repeat, this is not a permissions issue, I have the region specified, I just get an internal error if I try to use the SDK:
InternalServerException: Internal Server Error
at Request.extractError (<<coderoot>>\node_modules\aws-sdk\lib\protocol\json.js:52:27)
at Request.callListeners (<<coderoot>>\node_modules\aws-sdk\lib\sequential_executor.js:106:20)
at Request.emit (<<coderoot>>\node_modules\aws-sdk\lib\sequential_executor.js:78:10)
at Request.emit (<<coderoot>>\node_modules\aws-sdk\lib\request.js:686:14)
at Request.transition (<<coderoot>>\node_modules\aws-sdk\lib\request.js:22:10)
at AcceptorStateMachine.runTo (<<coderoot>>\node_modules\aws-sdk\lib\state_machine.js:14:12)
at <<coderoot>>\node_modules\aws-sdk\lib\state_machine.js:26:10
at Request.<anonymous> (<<coderoot>>\node_modules\aws-sdk\lib\request.js:38:9)
at Request.<anonymous> (<<coderoot>>\node_modules\aws-sdk\lib\request.js:688:12)
at Request.callListeners (<<coderoot>>\node_modules\aws-sdk\lib\sequential_executor.js:116:18)
if I use the v3, I get the same error eventually, this example is creating a table, but it's the same idea:
InternalServerException: Internal Server Error
at deserializeAws_json1_0InternalServerExceptionResponse (<<coderoot>>\node_modules\#aws-sdk\client-timestream-write\dist-cjs\protocols\Aws_json1_0.js:899:23)
at deserializeAws_json1_0CreateTableCommandError (<<coderoot>>\node_modules\#aws-sdk\client-timestream-write\dist-cjs\protocols\Aws_json1_0.js:239:25)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async <<coderoot>>\node_modules\#aws-sdk\middleware-serde\dist-cjs\deserializerMiddleware.js:7:24
at async <<coderoot>>\node_modules\#aws-sdk\middleware-signing\dist-cjs\middleware.js:13:20
at async StandardRetryStrategy.retry (<<coderoot>>\node_modules\#aws-sdk\middleware-retry\dist-cjs\StandardRetryStrategy.js:51:46)
at async <<coderoot>>\node_modules\#aws-sdk\middleware-logger\dist-cjs\loggerMiddleware.js:6:22
at async Object.createTable (<<coderoot>>\timestream.db.js:228:22)
at async handleCustomerCreation (<<coderoot>>\router.js:60:21)
at async router (<<coderoot>>\router.js:266:13)
Related
I get the error message when I want to upload a file size > 1MB to S3 Bucket in my company's network:
(If file size < 1MB, upload file working fine)
500: null
at Request.extractError (...\node_modules\aws-sdk\lib\services\s3.js:711:35)
at Request.callListeners (...\node_modules\aws-sdk\lib\sequential_executor.js:106:20)
at Request.emit (...\node_modules\aws-sdk\lib\sequential_executor.js:78:10)
at Request.emit (...\node_modules\aws-sdk\lib\request.js:686:14)
at Request.transition (...\node_modules\aws-sdk\lib\request.js:22:10)
at AcceptorStateMachine.runTo (...\node_modules\aws-sdk\lib\state_machine.js:14:12)
at ...\node_modules\aws-sdk\lib\state_machine.js:26:10
at Request.<anonymous> (...\node_modules\aws-sdk\lib\request.js:38:9)
at Request.<anonymous> (...\node_modules\aws-sdk\lib\request.js:688:12)
at Request.callListeners (...\node_modules\aws-sdk\lib\sequential_executor.js:116:18) {
code: 500,
region: null,
time: 2022-11-22T09:07:55.279Z,
requestId: null,
extendedRequestId: undefined,
cfId: undefined,
statusCode: 500,
retryable: true
}
And I found this error is simlar to this issue (https://github.com/localstack/localstack/issues/1410),
but I still get the same error after I try to set s3ForcePathStyle to true while creating s3 object.
Here is my code:
const s3 = new AWS.S3({apiVersion: '2006-03-01', s3ForcePathStyle: true});
let result = await s3.upload( {Bucket: bucket_name, Key: file_path, Body: request.files.buffer}).promise();
// using upload and putObject api return same error
console.log(result);
But if I run the code in my home, there is no error, and file upload succeed.
In my code, I set the NODE_TLS_REJECT_UNAUTHORIZED diabled to avoid ssl certificate problem, but still not working
process.env['NODE_TLS_REJECT_UNAUTHORIZED'] = 0;
I also try to use AWS CLI CMD (aws s3api put-object ...) in my company, upload the file size > 1MB is working fine!!
I don't understand why I cannot using node.js code to upload the file size > 1MB in my company, doesn't AWS CLI and npm aws-sdk library using same protocol?
I guess this error maybe related to my company's environment, but AWS CLI is working fine. So, I think it probably could be fixed the problem in my code, hopping someone can help, thank you so much.
Finally, I try to set enpoint and remove s3ForcePathStyle while careting s3 object.
It's working now!
const s3 = new AWS.S3({apiVersion: '2006-03-01', endpoint: 's3.amazonaws.com'});
But I still don't know why?
I face really weird problem with Axios / nanoexpress. I started to write JEST test and copied working piece of code from Vuex that calls my backend. This code works reliably in Vue but here it fails with a network error. The Axios version is the same on both SPA and JEST test.
JEST:
let response = await axios.get(`${API}/users/1234`, getAuthHeader());
function getAuthHeader() {
const config = { headers: { } };
config.headers.Authorization = "bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiIxZTQwdjBiMWo1Iiwibmlja25hbWUiOiJsaXRlcmFrbCIsInB3ZFRpbWVzdGFtcCI6IjIwMjAtMDMtMjJUMTE6MTA6NDkuNDg2WiIsInJvbGVzIjpbImFkbWluOnBvbGwiXSwiaWF0IjoxNTg2NjkwNzc2LCJleHAiOjE1ODkzNjkxNzZ9.N5MfpZ9i9Sjv-izdYItR4gXCmzVkqkuVcVSEL_6Q89c";
return config;
}
When I remove the config with Authorization header, the code works well:
let response = await axios.get(`${API}/users/1234`, getAuthHeader());
This is the error:
Network Error
at createError (node_modules/axios/lib/core/createError.js:16:15)
at XMLHttpRequest.handleError (node_modules/axios/lib/adapters/xhr.js:83:14)
at XMLHttpRequest.<anonymous> (node_modules/jsdom/lib/jsdom/living/helpers/create-event-accessor.js:33:32)
at innerInvokeEventListeners (node_modules/jsdom/lib/jsdom/living/events/EventTarget-impl.js:316:27)
at invokeEventListeners (node_modules/jsdom/lib/jsdom/living/events/EventTarget-impl.js:267:3)
at XMLHttpRequestEventTargetImpl._dispatch (node_modules/jsdom/lib/jsdom/living/events/EventTarget-impl.js:214:9)
at fireAnEvent (node_modules/jsdom/lib/jsdom/living/helpers/events.js:17:36)
at requestErrorSteps (node_modules/jsdom/lib/jsdom/living/xhr-utils.js:121:3)
at Object.dispatchError (node_modules/jsdom/lib/jsdom/living/xhr-utils.js:51:3)
at EventEmitter.<anonymous> (node_modules/jsdom/lib/jsdom/living/xmlhttprequest.js:675:20)
at Request.<anonymous> (node_modules/jsdom/lib/jsdom/living/xhr-utils.js:384:47)
at Request.onRequestError (node_modules/request/request.js:877:8)
console.error node_modules/jsdom/lib/jsdom/virtual-console.js:29
Error: Error: Parse Error: Duplicate Content-Length
at Object.dispatchError (C:\Users\leos\WebstormProjects\nano-options\node_modules\jsdom\lib\jsdom\living\xhr-utils.js:54:19)
at EventEmitter.<anonymous> (C:\Users\leos\WebstormProjects\nano-options\node_modules\jsdom\lib\jsdom\living\xmlhttprequest.js:675:20)
at EventEmitter.emit (events.js:327:22)
at Request.<anonymous> (C:\Users\leos\WebstormProjects\nano-options\node_modules\jsdom\lib\jsdom\living\xhr-utils.js:384:47)
at Request.emit (events.js:315:20)
at Request.onRequestError (C:\Users\leos\WebstormProjects\nano-options\node_modules\request\request.js:877:8)
at ClientRequest.emit (events.js:315:20)
at Socket.socketOnData (_http_client.js:486:9)
at Socket.emit (events.js:315:20)
at addChunk (_stream_readable.js:297:12) undefined
The service is nothing fancy and it is never called when Authorization header is present and it is called when the Authorization header is missing.
app.get('/v1/users/:userId', async (req, res) => {
console.log("getUser handler starts");
const response = {};
res.send(response);
return res;
});
I have spent several hours trying to see the raw network communication but failed. It must be something with Axios because I can run successfuly exactly the same request with the Authorization header from Postman.
Update:
it seems to be related to NodeJS version. I submitted an issue: https://github.com/axios/axios/issues/2889
Update 2:
I switched from Axios to Got and the test is working properly. https://github.com/literakl/mezinamiridici/commit/e667c5661429fc8cba273716dfced2244e211abf
I am trying to get twit (https://www.npmjs.com/package/twit) to stream live tweets from specific twitter users.
The code I have is
var stream = T.stream('statuses/filter', { follow: ['nodejs'] })
stream.on('tweet', function (tweet) {
console.log(tweet.text)
})
but it gives the following error
Error: Bad Twitter streaming request: 406
at Object.exports.makeTwitError (/Users/simoncarr/dev/node/nodeletetweet/scripts/get_mp_data/node_modules/twit/lib/helpers.js:74:13)
at Request.<anonymous> (/Users/simoncarr/dev/node/nodeletetweet/scripts/get_mp_data/node_modules/twit/lib/streaming-api-connection.js:96:29)
at Request.emit (events.js:187:15)
at Gunzip.<anonymous> (/Users/simoncarr/dev/node/nodeletetweet/scripts/get_mp_data/node_modules/request/request.js:1083:12)
at Object.onceWrapper (events.js:273:13)
at Gunzip.emit (events.js:187:15)
at endReadableNT (_stream_readable.js:1094:12)
at process._tickCallback (internal/process/next_tick.js:63:19)
Emitted 'error' event at:
at Request.<anonymous> (/Users/simoncarr/dev/node/nodeletetweet/scripts/get_mp_data/node_modules/twit/lib/streaming-api-connection.js:99:14)
at Request.emit (events.js:187:15)
[... lines matching original stack trace ...]
at process._tickCallback (internal/process/next_tick.js:63:19)
simply tracking words in a tweet works just fine, for instance the following code works OK.
var stream = T.stream('statuses/filter', { track: ['apples'] })
stream.on('tweet', function (tweet) {
console.log(tweet.text)
})
The twitter documentation suggests that I should be able to simply replace track with follow.
https://developer.twitter.com/en/docs/tweets/filter-realtime/api-reference/post-statuses-filter.html
You need to use the numeric user ID, not the screen name, in the follow parameter. You can use the users/show endpoint to find the user ID for a given screen name.
We have an app build on nodejs (Server A), which is fetching data from another nodejs app(Server B) using REST API's.
Similarly for downloading files Server A make request on Server B and Server B fetches files from AWS S3 and pipes the data back to server A.
This system works perfectly and we are able to download files upto 1GB.
Issue is sometimes Server A crashes with below error:
buffer.js:488
throw new Error('toString failed');
^
Error: toString failed
at Buffer.toString (buffer.js:488:11)
at Request.<anonymous> (/var/www/sky/portal/node_modules/request/request.js:1145:39)
at emitOne (events.js:95:20)
at Request.emit (events.js:182:7)
at IncomingMessage.<anonymous> (/var/www/sky/portal/node_modules/request/request.js:1091:12)
at IncomingMessage.g (events.js:273:16)
at emitNone (events.js:85:20)
at IncomingMessage.emit (events.js:179:7)
at endReadableNT (_stream_readable.js:913:12)
at _combinedTickCallback (internal/process/next_tick.js:74:11)
at process._tickCallback (internal/process/next_tick.js:98:9)
Server A is calling server B using below code:
exports.downloadAsset = function(req, res, assetUrl, cb) {
res.cookie('fileDownload', 'true');
request(
config.api.baseUrl + assetUrl
,
function(error, response, body) {
}
).pipe(res);
}
We are not sure from where app is getting crashed.
Please suggest.
The Solution I used to sort above issue:
I had to modify our flow. We were streaming data from Server B to Server A, but now Server B is sending signed URL to Server A and file is being downloaded directly from S3 using that signed url.
I am trying to upload an image to my s3 bucket using the npm s3 module (https://www.npmjs.org/package/s3)
I used the following params
var params = {
localFile: 'image.png',
s3Params: {
Bucket: 'newstie.com',
Key: '/newsite/image'
}
};
And I am getting the following logs and I wasn't able to understand what I did wrong. Can you please help me? Many Thanks
progress 12181 0 12181
progress 12181 12181 12181
unable to upload Error
at Request.extractError (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/services/s3.js:257:35)
at Request.callListeners (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/sequential_executor.js:114:20)
at Request.callListeners (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/sequential_executor.js:115:16)
at Request.emit (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/sequential_executor.js:81:10)
at Request.emit (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/request.js:578:14)
at Request.transition (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/request.js:12:12)
at AcceptorStateMachine.runTo (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/state_machine.js:14:12)
at /Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/state_machine.js:28:10
at Request. (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/request.js:28:9)
at Request. (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/request.js:580:12)
progress 12181 0 12181
progress 12181 12181 12181
unable to upload Error
at Request.extractError (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/services/s3.js:257:35)
at Request.callListeners (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/sequential_executor.js:114:20)
at Request.callListeners (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/sequential_executor.js:115:16)
at Request.emit (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/sequential_executor.js:81:10)
at Request.emit (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/request.js:578:14)
at Request.transition (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/request.js:12:12)
at AcceptorStateMachine.runTo (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/state_machine.js:14:12)
at /Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/state_machine.js:28:10
at Request. (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/request.js:28:9)
at Request. (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/request.js:580:12)
I guess we need more code. I'm recommending using the aws-sdk. Works perfectly, easy to upload, sync, trigger jobs and so on.
But to answer your question and assuming the file you're trying to upload is located in the same directory as your script your code would look something like this:
var params = {
localFile: __dirname + '/image.png',
s3Params: {
Bucket: 'newstie.com',
Key: '/newsite/image'
}
};