I'm trying to use Rx.js to handle the flow of Chrome extension webrequest API.
Each webrequest addListener() call takes a mandatory callback function as the first parameter. This sends request objects to the function. However, the callback can return a webRequest.BlockingResponse that determines the further life cycle of the request.
I'm struggling to handle the blocking response as part of the observable.
This code works well for examining all image requests, for example
onBeforeRequestHandler = function() {
var filteredURLs = ["http://*/*", "https://*/*"];
var resourceTypes = ["image"];
var filter = { urls: filteredURLs, types: resourceTypes };
var options = ["blocking"];
return Rx.Observable.create(observer => {
var listener = chrome.webRequest.onBeforeRequest.addListener(
function requestHandler(obj) {
observer.next(obj);
},
filter, options);
return unsubscribe => {
chrome.webRequest.onBeforeRequest.removeListener(listener);
};
});
};
I can then use all the Rx.js operators to manipulate the requests by doing this:
var source = onBeforeRequestHandler();
source.subscribe();
etc.
However, if during the course of working the images, I wish to cancel the request, I somehow need to return a blocking response object, like this {cancel:true} to the observable that is wrapping the chrome.webRequest.onBeforeRequest.addListener callback function.
At the moment I have no clue how to do this.
Any help much appreciated.
Related
Currently I'm on a legacy application using pug.js as view engine in a node.js express-app.
I want to implement a generic way to display feedback messages. I want to be able to display messages (successes, errors), even if the handler does reply with a redirect.
This is what I want:
handlePostRequest(req, res){
// do stuff with the post request
doStuff(req.body);
//This should be done of course somewhere else.
req.session.successes=req.session.successes|[];
//save some success-message for the user
req.session.successes.push("Your post has been saved. Thank you!");
//but reply with a 302
res.redirect(req.headers.referer);
}
//a get request. maybe the handler above redirected here
handleGetRequest(req,res){
// we do NOT get the successes here. Just the 'pure' data.
const renderData=getRenderData();
res.render('fancy-pug-template', renderData);
}
fancyMiddlewareForMessages(req, res, next){
//how to implement getRenderDataByBlackMagic()????
const renderData = getRenderDataByBlackMagic();
//set the messages
renderData.successes = req.session.successes;
//empty saved messages
req.session.successes = [];
next();
}
Obviously, I do not want to polute every handler which actually renders a template with some logic which retrieves the messages and adds them to the parameter object. I would like to move this cross-cutting concern in a middleware callback or something like that.
So, the question is: Can this be achieved? How? I'm fairly new to pug.js, maybe I'm overlooking something obvious.
Ok, I found a way. This is what I did:
const requestStorage = new AsyncLocalStorage<Request>();
function patchRenderFunction(req: Request, res: Response, next: NextFunction) {
const render = res.render;
res.render = function (view: string, options?: any, callback?: (err: Error, html: string) => void) {
const messages = new MessageManager(req);
//merge errorMessages
options.errorMessages = mergeMessageArrays(options.errorMessages, messages.errors);
//same for successMessages
options.successMessages = mergeMessageArrays(options.successMessages, messages.successes);
render.bind(this)(view, options, callback);
};
requestStorage.run(req, () => {
next();
});
}
export function applyAutomaticRenderAttributes(app: Express): void {
app.use(patchRenderFunction);
}
export function successMessage(message: string, req?: Request) {
if (!req) {
req = requestStorage.getStore();
}
if (!req) {
console.error('No request found in async storage. This should not happen. Please report this issue. (successMessage)');
return;
}
new MessageManager(req).addSuccessMessage(message);
}
//export function errorMessage(...) omitted
The MessageManager uses the requests session to store messages. It also filters them in some respect. I'm using the session because the application runs clustered (thank you, pm2). Since the session is stored in the db via express-mysql-session, I avoid problems with non-sticky sessions.
I am building a React/Typescript app, running completely client-side in browser. In componentDidMount(), I make a fetch request which I intercept successfully, to change the URL, and then make that request.
For reference, the API object is also from a third party library, loaded in via an HTML script tag, so I don't have access to the inner workings of the object. That's why I'm attempting to intercept the call instead to point the URL at a different endpoint.
async function makeRequest() {
let originalFetch = redefineFetch();
let data;
try {
data = await API.fetchData();
} catch (error) {
resetFetch(originalFetch);
return;
}
resetFetch(originalFetch);
return data;
}
const redefineFetch = () => {
const { fetch: originalFetch } = window;
let originalWindowFetch = window.fetch;
window.fetch = async (...args) => {
let [resource, config] = args;
resource = NEW_URL;
const response = await originalFetch(resource, config);
return response;
};
return originalWindowFetch;
};
const resetFetch = (
originalFetch: ((input: RequestInfo | URL, init?: RequestInit | undefined) => Promise<Response>) &
((input: RequestInfo | URL, init?: RequestInit | undefined) => Promise<Response>)
) => {
console.log("Resetting fetch");
window.fetch = originalFetch;
};
How I'm currently doing it:
I copied how it was done in this blog post. https://blog.logrocket.com/intercepting-javascript-fetch-api-requests-responses/.
As you can see, makeRequest() calls redefineFetch(), which redefines window.fetch to point to the NEW_URL instead.
redefineFetch() returns the original implementation of fetch as originalFetch.
After making the request, I call resetFetch() and pass originalFetch.
I then set window.fetch = originalFetch.
What I think is the issue
Every request including and after API.fetchData() now point to the NEW_URL.
These requests are out of my control in timing as they are made by 3rd party portions of my code.
I think I'm either not setting window.fetch back to its original value correctly, OR there's a race condition in which these mistakenly intercepted requests are being made before resetFetch() is called.
My Questions
How can I redefine fetch only for the API.fetchData() call without risking affecting any other calls made in my app?
Is there a better way to accomplish what I'm doing?
I implemented a recursive function in a requestHandler I made to serialize API requests and also to make sure the endpoint isn't currently being requested. To make sure that the endpoint isn't currently being requested, I add it to a Set and verify it with conditionals.
Problem is that this recursive approach consumes quite a lot of memory when a lot of requests are made to the same endpoint. Is there any way I could make it less memory intensive as well as performant at the same time? I would love to hear any alternative approach which I could use instead of recursion. Below you can find my code.
async request(endpoint, domain, method, headers, query, body, attachments) {
const requestURL = `${(domain === "discord") ? this.discordBaseURL :
(domain === "trello") ? this.trelloBaseURL : domain}/${endpoint}`;
if (this.queueCollection.has(endpoint) === false) { // queueCollection is the Set in which I store endpoints that are currently being requested by my requestHandler.
this.queueCollection.add(endpoint);
const response = await this.conditionalsHandler(endpoint, requestURL, method, headers, query, body, attachments);
this.queueCollection.delete(endpoint);
return response;
}
else {
const response = new Promise((resolve) => {
setTimeout(() => { // https://stackoverflow.com/a/20999077
resolve(this.request(endpoint, domain, method, headers, query, body, attachments)); // This is where I make the method recursive to call itself back until the endpoint is no longer in the queueCollection Set.
}, 0);
});
return response;
}
}
Yes, you can remove the recursion by making the queueCollection a Map<string, Promise> instead of a Set<string>, and instead of recursing asynchronously and polling the queue until it's empty, chain the request to the tail of the queue if it exists like this:
async request(endpoint, domain, method, headers, query, body, attachments) {
const requestURL = `${(domain === "discord") ? this.discordBaseURL :
(domain === "trello") ? this.trelloBaseURL : domain}/${endpoint}`;
// get existing queue or create a new one
const queue = this.queueCollection.get(endpoint) || Promise.resolve();
// schedule request on the tail of the queue
const request = queue.then(
() => this.conditionalsHandler(endpoint, requestURL, method, headers, query, body, attachments)
);
// prevent errors from propagating along the queue
const tail = request.catch(() => {});
// enqueue the request
this.queueCollection.set(endpoint, tail);
try {
// propagates error handling to consumer
// waits for request to settle before executing finally block
return await request;
} finally {
// only remove promise from Map if this settled request is at the tail of the queue
if (this.queueCollection.get(endpoint) === tail) this.queueCollection.delete(endpoint);
}
}
This approach allows request to throw without breaking the chain so the consumer can handle the error and all the requests will still happen in sequence without depending on previous requests being successful, and it will always clean up the queueCollection on the last pending request regardless of whether the request throws. The await is not redundant here for that reason.
I am trying to build a simple chatbot with DialogFlow.
My aim is to give information from user question, like : where can I slackline above water in croatia ? I have two parameters (croatia, waterline) and a list of slackline places.
So I need a data base to retrieve information from parameters. DialogFlow allows fulfillment with Firebase. I build a database with places (name, country, type of slack) and enable webhook call for my intent.
I use Inline Editor and index.js
const parameters = request.body.queryResult.parameters;
var country = parameters.country.toString();
function show(snap) {
console.log('snap');
agent.add(JSON.stringify(snap.val(),null,2));
}
function slkplc(agent) {
var testRef;
firebase.database().ref('slackplace').once('value',show);
}
// Run the proper function handler based on the matched Dialogflow intent name
let intentMap = new Map();
intentMap.set('slack place', slkplc);
agent.handleRequest(intentMap);
But I do not get the expected result while trying it on DialogFlow or Google Assistant. The function show is asynchronously called but too late and the response is not available for DialogFlow :
I see three way to deal with this problem :
use blocking call to database : another database ?
treat asynchronous message with DialogFlow ???
response to user that an error occured.
The third that I choose, but it is always on error.
After trying several things to wait data from database response, the only thing I managed is to freeze the response, therefore the timeout of DialogFlow - 5s -and Firebase - 60s - were reached.
A workaround
Another way to do it is to separate database acquisition and request/response from DialogFlow. The data of database is collected outside of the dialogflowFirebaseFulfillment
var data;
var inidata = firebase.database().ref().on('value',function(snap) {
console.log('snap');
data = snap.val();
});
exports.dialogflowFirebaseFulfillment = functions.https.onRequest((request, response) => {
const agent = new WebhookClient({ request, response });
...
function slkplc(agent) {
agent.add(JSON.stringify(data,null,2));
}
// Run the proper function handler based on the matched Dialogflow intent name
let intentMap = new Map();
intentMap.set('slack place', slkplc);
agent.handleRequest(intentMap);
}
Now I can do what I want with data, and I am able to find the place where I can practice waterline in croatia. But there is always something weird, the data of the database is duplicated ...
The "right" solution is option 2 that you suggest: since you're doing an asynchronous call, you need to handle this correctly when working with the dialogflow-fulfillment library.
Basically, if your handler makes an asynchronous call, it needs to be asynchronous as well. To indicate to the handleRequest() method that your handler is async, you need to return a Promise object.
Firebase's once() method returns a Promise if you don't pass it a callback function. You can take advantage of this, return that Promise, and also handle what you want it to do as part of a .then() clause. It might look something like this:
function slkplc(agent) {
var testRef;
return firebase.database().ref('slackplace').once('value')
.then( snap => {
var val = snap.val();
return agent.add( JSON.stringify( val, null, 2 ) );
});
}
The important part isn't just that you use a Promise, but also that you return that Promise.
Just playing around with my first Node/Express App.
What I am trying to do:
On submitting a form to /wordsearch (POST) the Wikipedia API should be called with the submitted keyword. After getting back the response from Wikipedia, I want to present it back to the user in a view. Basic stuff.
But I am missing some basic understanding of how to arrange that in NODE/JS. I read about callbacks and promises lately and understand the concepts theoretically, but seem to mix things up when trying to put it into code. If someone could shed light on where I am wrong, that would be highly appreciated.
Approach 1:
This is the controller function that is hit on submitting the form:
exports.searchSources = (req, res) => {
const term = req.body.searchTerm
const url = `https://en.wikipedia.org/w/api.php?action=opensearch&search=${term}&limit=10&namespace=0&format=json`
const client = new Client()
client.get(url, function (data, response) {
//this causes the error
res.json(data)
})
}
=> Error: Can't set headers after they are sent.
I know, that the error stems from trying to set response headers twice or when the response is already in a certain state, but I don't see where that happens here. How can I wait for the result of the Wiki request and have it available in the controller function so that I can render it?
Approach 2:
Again, the controller function:
exports.searchSources = (req, res) => {
const term = req.body.searchTerm
const url = `https://en.wikipedia.org/w/api.php?action=opensearch&search=${term}&limit=10&namespace=0&format=json`
const client = new Client()
const data = client.get(url, function (data, response) {
return data
})
res.json(data)
}
=> TypeError: Converting circular structure to JSON at JSON.stringify ()
This was just a try to make the response from Wiki available in the controller function.