nodejs abort fetch in lambda AWS environment - node.js

I managed to have a working abort controller in nodejs on a local environment, something as simple as this:
const signal = controller.signal;
setTimeout(() => controller.abort(), 5000);
fetch(url, { signal }).then(response => {
return response.text();
}).then(text => {
console.log(text);
});
So I am able to abort the request after n seconds.It works without issues in local environment, however when deployed to AWS in a Lambda I endup with and error
TypeError: Expected signal to be an instanceof AbortSignal
After digging in the fetch module source it seems that this line is the culprit:
!isAbortSignal(signal)
From this function::
function isAbortSignal(signal) {
const proto = signal && typeof signal === 'object' && Object.getPrototypeOf(signal);
return !!(proto && proto.constructor.name === 'AbortSignal');
}
The proto.constructor.name in local is correct as "AbortSignal" while in AWS lambda it ends up being just 'a'.
I searched around but I could not find anything specific about why this is not working. Any tips on that?

Are you using the node-fetch module? fetch and AbortController don't exist in native nodejs. You'll need to use the nodejs polyfill for each.
https://www.npmjs.com/package/node-abort-controller

Looks like it should be fixed in node-fetch v3 (currently in beta)
In my case, I'm using Webpack to bundle before uploading the code to AWS, after upgrading to 3.0.0-beta.4 it worked.
If you cant upgrade and using webpack try:
optimization: {
minimizer: [
new TerserPlugin({
terserOptions: {
mangle: false,
keep_classnames: true,
keep_fnames: true,
},
}),
],
},
and if it doesn't work try to config optimization.minimize to false

Related

Evaluate a Cypress configuration file

In my build scripts I need to evaluate a Cypress configuration file. I'm using the following script:
let appdata = process.env.LOCALAPPDATA;
let version = `11.0.1`;
let src = `${appdata}/Cypress/Cache/${version}/Cypress/resources/app/node_modules/#packages/data-context/src`;
const DataContext = require(`${src}/DataContext.js`).DataContext;
const ProjectConfigManager = require(`${src}/data/ProjectConfigManager.js`).ProjectConfigManager;
(async() => {
const ctx = new DataContext({
schema: null,
schemaCloud: null,
modeOptions: "run",
appApi: {},
localSettingsApi: {},
authApi: {
} ,
configApi: {
},
projectApi: {
} ,
electronApi: {
} ,
browserApi: {
},
})
let configManager = new ProjectConfigManager({
ctx,
configFile: 'C:\\work\\sample\\sample.config.ts',
projectRoot: 'C:\\work\\sample',
handlers: [],
hasCypressEnvFile: false,
eventRegistrar: null/*new EventRegistrar()*/,
onError: (error) => {},
onInitialConfigLoaded: () => {},
onFinalConfigLoaded: () => Promise.resolve(),
refreshLifecycle: () => Promise.resolve(),
})
configManager.configFilePath = "sample.config.ts"
configManager.setTestingType('e2e')
let cfg = await configManager.getConfigFileContents()
console.log(JSON.stringify(cfg));
})();
It works well for Cypress 10 version.
However, Cypress 11 has introduced some changes that break this script. Though I adjusted the paths, I'm still unable to make it work again.
It currently fails with this error:
Error: Cannot find module 'C:\Users\mbolotov\AppData\Local\Cypress\Cache\11.0.1\Cypress\resources\app\node_modules\graphql\index'. Please verify that the package.json has a valid "main" entry
How can I fix this problem (without making changes to the Cypress installation)?
OR
Is there any other way to evaluate a Cypress configuration file (say from the command line) and obtain its values?
The exact usage is unclear to me, but making some assumptions - a nodejs script in the /scripts folder of the project can compile and resolve the config using the Cypress module API.
It would need a test to run, a "null-test" can be generated from inside the script.
Note, the null-test must conform to the spec pattern of the project (below it's the std .cy.js)
const cypress = require('cypress')
const fs = require('fs')
fs.writeFileSync('../cypress/e2e/null-test.cy.js', 'it("", ()=>{})')
cypress.run({
project: '..',
spec: '../cypress/e2e/null-test.cy.js',
quiet: true
}).then(results => {
if (results.status === 'failed') {
console.log(results)
} else {
console.log(results.config.resolved) // output resolved config
}
})
I haven't attempted to duplicate everything you have in your code, as it's using Cypress internals and not publicly documented.
This may be because of Changelog 11.0.0
We have also massively improved our startup performance by shipping a snapshot of our binary instead of the source files.
Looking inside the cache folder for v10.11.0 (${process.env.CYPRESS_CACHE_FOLDER}/10.11.0/Cypress/resources/app), the /node_modules is fully populated and /node_modules/graphql/index.js exists.
But in v11.0.1 /node_modules/graphql is not fully populated.

Netlify functions not found on dev server

I have followed several blogs and questions related to the same problem I am having. It is exactly like this question. However, I am still having issues.
So, I am running netlify dev and trying to access my netlify functions. I have a function in /netlify/functions/ping. The function works as intended when I access the randomized port for the netlify functions (something like localhost:55832...).
However, using the localhost:8888/.netlify/functions/ping gives me a 404 error.
Here is my /netlify/functions/ping file:
import { Handler } from '#netlify/functions';
const handler: Handler = async (event, context) => {
return {
statusCode: 200,
body: JSON.stringify({ data: "pong" }),
};
};
export { handler };
here is where I am trying to call my function on a page
export default function HomePage() {
useEffect(() => {
async function pingpong() {
const res = await fetch(`/.netlify/functions/ping`);
console.log(res);
}
pingpong();
}, []);
return (
...
I have also tried to alter my netlify.toml with the following
[[redirects]]
from = "/api/*"
to = "/.netlify/functions/:splat"
status = 200
force = true
what's start script you use in package.json?
Keep in mind that, to enable Netlify Functions you have to use netlify-cli, i.e. ntl command to run on local dev server.
No more specific configuration, just follow docs, and use that simple netlify/functions/hello.js example.
Then run using ntl dev, you function will be avaiable on /.netlify/functions/hello. easy.

Cypress for Electron - Using FS

I am using Cypress to test my Electron application.
Since Cypress uses the browser mode, FS is not supported.
So I am getting this error:
Error in mounted hook: "TypeError: fs.existsSync is not a function"
And I found this on the documentation:
https://docs.cypress.io/api/commands/task.html#Event
So I added this on my test:
it('Sample test', () => {
cy.task('readSettingsJson', settingsFolder).then((content) => {
// This can print the JSON file contents correctly
console.log('content = ' + content)
})
})
And on my plugins/index.js:
on('task', {
readSettingsJson(foldername) {
if (!fs.existsSync(foldername)) {
fs.mkdirSync(foldername, { recursive: true })
// some command to copy the file
} else {
// This is what I am testing at this moment
return fs.readFileSync(path.join(filename, '/settings.json'), 'utf8')
}
return null
}
})
However, it doesnt seem to work. I still get the error:
Error in mounted hook: "TypeError: fs.existsSync is not a function"
And despite the test printing the json file correctly, my app still can't load the JSON file.
Am I missing anything? Help please!
Cypress support for Electron apps are in very early alpha release:
https://www.cypress.io/blog/2019/09/26/testing-electron-js-applications-using-cypress-alpha-release/
As an alternative, try using Spectron, which is the testing framework that is currently recommended by Electron team:
https://www.electronjs.org/spectron

Can't use Worker-Loader with Vuejs and Webpack

I am trying to get web workers up and running with Vue cli3 and I'm having trouble getting it to work.
I want to use the following package, worker-loader (and not vue-worker), as it looks well maintained and with more contributions.
Following their tutorial I attempted to modify webpack using the vue cli as follows:
module.exports = {
chainWebpack: config => {
config.module
.rule('worker-loader')
.test(/\.worker\.js$/)
.use('worker-loader')
.loader('worker-loader')
.end()
}
}
which I hope should match their
{
module: {
rules: [
{
test: /\.worker\.js$/,
use: { loader: 'worker-loader' }
}
]
}
}
which can be read here (https://github.com/webpack-contrib/worker-loader). I tried to follow the documentation for vue cli3 as best I could (found here: https://cli.vuejs.org/guide/webpack.html#simple-configuration).
My component is pretty simple:
import Worker from 'worker-loader!./../../sharedComponents/equations/recurringTimeComposer.js';
<...>
watch:{
recurringPaymentReturnObj: function(newVal, oldVal){
const myWorker = new Worker;
myWorker.postMessage({ hellothere: 'sailor' });
myWorker.onmessage = (e) => {
console.log('value of e from message return', e.data);
}
}
<...>
and in my ./../../sharedComponents/equations/recurringTimeComposer.js file I have:
onmessage = function(e) {
console.log('Message received from main script: ', e.data);
// var workerResult = 'Result: ' + e.data;
// console.log('Posting message back to main script');
postMessage('hello back handsome');
close();
}
I keep getting the error message:
ReferenceError: window is not defined a162426ab2892af040c5.worker.js:2:15
After some googling I came across this post: https://github.com/webpack/webpack/issues/6642, which suggests that the best way to fix this is to add the following to webpack:
output: {
path: path.join(__dirname, 'dist'),
filename: 'bundle.js'
publicPath: 'http://localhost:3000',
globalObject: 'this'
},
After modifying my vue.config.js file I have:
module.exports = {
chainWebpack: config => {
config.module
.rule('worker-loader')
.test(/\.worker\.js$/)
.use('worker-loader')
.loader('worker-loader')
.end()
config
.output
.path(path.join(__dirname, 'dist'))
.filename('bundle.js')
.publicPath('http://localhost:8080')
.globalObject('this')
}
}
...but still I am getting the window is not defined error.
Does anyone know what is going wrong? It seems to be a weird error in webpack.
Thanks!
EDIT: oh yeah, here is the MDN page for webworker as well: https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers.
Being new to Javascript I kept coming back to this issue when trying to use web workers with VueJS. I never managed to make it work with vue-worker or worker-loader.
It is now 2020 and Google has released worker-plugin.
To use it create a module my-worker with two files index.js and worker.js.
index.js creates the module:
const worker = new Worker('./worker.js', { type: 'module' });
const send = message => worker.postMessage({
message
})
export default {
worker,
send
}
worker.js contains the logic:
import _ from 'lodash'
addEventListener("message", async event => {
let arrayToReverse = event.data.message.array
let reversedArray = _.reverse(arrayToReverse)
// Send the reversed array
postMessage(reversedArray)
});
You will also need to update your vue.config.js to use the WorkerPlugin:
const WorkerPlugin = require('worker-plugin')
module.exports = {
configureWebpack: {
output: {
globalObject: "this"
},
plugins: [
new WorkerPlugin()
]
}
};
Now you can use you worker in your components:
Import it with import worker from '#/my-worker'.
Setup a listener in the mounted() lifecycle hook with worker.worker.onmessage = event => { // do something when receiving postMessage }
Start the worker with worker.send(payload).
I set up a starter code on github. I still haven't managed to make HMR work though...
This works for me (note the first line):
config.module.rule('js').exclude.add(/\.worker\.js$/)
config.module
.rule('worker-loader')
.test(/\.worker\.js$/)
.use('worker-loader')
.loader('worker-loader')
The first line excludes worker.js files, so two loaders wouldn't fight over the same js file
is this what you need ? Vue issue with worker-loader
Updating from the classic vue & webpack config, I found out that to make this one work, I needed to deactivate parallelization.
// vue.config.js
module.exports = {
parallel: false,
chainWebpack: (config) => {
config.module
.rule('worker')
.test(/\.worker\.js$/)
.use('worker-loader')
.loader('worker-loader')
.end();
}
};
I tried add web worker to a vue-cli4 project, and here is what I found:
using worker-loader and make configs in chainWebpack:
HMR works fine, but sourcemap broke, it show babel transformed code.
using worker-plugin as #braincoke mentioned:
HMR broke, but sourcemap works fined. and eslint broke while suggested disable all worker js file eslint instead.
Finally, My solution is tossing vue-cli away, and embrace vite.It support worker natively, and all just go fine now. (I think upgrade webpack to v5 can solve this, but i never tried.)

express body-parser utf-8 error in test

Super stumped by this. I have some server code that for some reason throws a UTF-8 error in my tests but works fine when running the server normally:
code:
export default ({ projectId = PROJECT_ID, esHost = ES_HOST } = {}) => {
let app = express();
app.use(cors());
app.use(bodyParser.json({ limit: '50mb' }));
let http = Server(app);
let io = socketIO(http);
let server = {
app,
io,
http,
status: 'off',
listen(
port = PORT,
cb = () => {
rainbow(`⚡️ Listening on port ${port} ⚡️`);
},
) {
this.http.listen(port, () => {
main({ io, app, projectId, esHost, port });
this.status = 'on';
cb();
});
},
close(cb = () => {}) {
if (this.http) {
this.http.close(() => {
this.status = 'off';
cb();
});
} else {
throw '❗️ cannot close server that has not been started ❗️';
}
},
};
return server;
};
usage (exactly the same, but in jest test body-parser isn't working properly):
import createServer from '../server'
let server = createServer()
server.listen(5050);
I'm using postman, post response outside of test:
{
"projects": [
{
"id": "test",
"active": true,
"timestamp": "2018-02-25T21:33:08.006Z"
},
{
"id": "TEST-PROJECT",
"active": true,
"timestamp": "2018-03-05T21:34:34.604Z"
},
{
"id": "asd",
"active": true,
"timestamp": "2018-03-06T23:29:55.348Z"
}
],
"total": 3
}
unexpected post response inside jest test server:
Error
UnsupportedMediaTypeError: unsupported charset "UTF-8" at /Users/awilmer/Projects/arranger/node_modules/body-parser/lib/read.js:83:18 at invokeCallback (/Users/awilmer/Projects/arranger/node_modules/raw-body/index.js:224:16) at _combinedTickCallback (internal/process/next_tick.js:131:7) at process._tickCallback (internal/process/next_tick.js:180:9)
So I was able to reproduce the issue and find the source of the issue and the workaround to make it work. The issue is caused by jest framework.
Before you jump on reading the rest of the thread, I would suggest you read another Jest thread I answer long back. This would help get some context internals about the require method in jest
Specify code to run before any Jest setup happens
Cause
The issue happens only in test and not in production. This is because of jest require method.
When you run your tests, it starts a express server, which calls the node_modules/raw-body/index.js as shown in below image
As you can see the encodings is null. This is because the iconv-lite module does a lazy loading of encodings. The encodings are only loaded when getCodec method gets executed.
Now when your test has fired the API, the server needs to read the body so the getCodec gets called
This then goes through the jest-runtime/build/index.js custom require method (which is overloaded if you read the previous link).
The execModule has a check for this._environment.global, which is blank in this case and hence a null value is returned and the module never gets executed
Now when you look at the exports of the encodings module, it just is a blank object
So the issue is purely a jest. A feature jest lacks or a bug mostly?
Related Issues
Related issues have already been discussed on below threads
https://github.com/facebook/jest/issues/2605
https://github.com/RubenVerborgh/N3.js/issues/120
https://github.com/sidorares/node-mysql2/issues/489#issuecomment-313374683
https://github.com/ashtuchkin/iconv-lite/issues/118
https://github.com/Jason-Rev/vscode-spell-checker/issues/159
Fix
The fix to the problem is that we load the module during our test itself and force a early loading instead of lazy loading. This can be done by adding a line to your index.test.js at the top
import encodings from '../../node_modules/iconv-lite/encodings';
import createServer from '#arranger/server';
After the change all the test pass, though you have a error in the url of the test so you get Cannot POST /
I'm adding a slightly different solution inspired from #Tarun Lalwani
Add the following lines at the top of your test file.
const encodings = require('./node_modules/iconv-lite/encodings');
const iconvLite = require('./node_modules/iconv-lite/lib');
iconvLite.getCodec('UTF-8');
I spent many hours trying to figure out why Jest would report a 415 error code when testing the Node.js server. Node.js is configured to use app.use(bodyParser.json(...)); on our system, too. That didn't solve the issue.
Solution
When using res.status(...), you MUST either chain on .json() or use res.json(), too. That means if you respond with a 500 error or otherwise and you don't return any JSON data, you still need to use res.json(). No idea why, as that defeats the whole purpose of app.use(bodyParser.json(...)); in the first place.
Example
const express = require('express');
const router = express.Router();
router.post("/register", (req, res) => {
// ...
res.status(500).json();
// ...
});

Resources