How does the Chrome DevTools Protocol APIs work with Puppeteer? - node.js

Are the Chrome DevTools Protocol APIs open source? I'm trying to find out the source code to understand the working of some APIs in depth.
What exactly happens behind the scenes when we try to call those API using Puppeteer?
This is how we use them -
const tree = await page._client.send('Page.getResourceTree');
I want to understand the how does this interface (page._client.send) work. Does it also make a request using the internet bandwidth or it works completely offline?

Related

Register custom file protocols in Chrome/Firefox extension?

I want to register a custom file protocol in Chrome or Firefox, much like the way Electron does (please see their API for doing this: https://www.electronjs.org/docs/api/protocol). I am wondering if there is a way to implement this as a browser extension, or I have to modify the source code of the browsers.
I expect that the API would look like this (just a pseudocode to help explain what I mean):
registerHandler('myprotocol://', req => {
response('<body>You requested: ' + req.url);
});
Clarification: navigator.registerProtocolHandler is NOT what I need. What it does is to register a protocol that, when clicked, opens an external application to deal with that. But what I want is a protocol handler scheme that works in a request--response way, e.g. can be used in JS/CSS/HTML queries and responds with a content that can be rendered within the browser.
I would answer my own question because I found exactly what I need here: https://github.com/mozilla/libdweb
This is an experimental feature of Firefox Nightly that allows one to register a custom protocol and serve all requests to that protocol using firefox addon. This is not a WebExtension standard nor does it work on browser other than Nightly, but I'm glad to hear that someone is doing this.

Is there a way to access and control "Remote Devices" section of Chrome Devtools programmatically?

I understand that the Chrome Devtools protocol exposes the Google Chrome DevTools via APIs to be controlled programmatically, and puppeteer provides a Node.js implementation to do so. However, if I am not wrong, puppeteer doesn't allow us to do everything that the DevTools protocol can do.
If I want to remotely debug my android chrome browser, like I can do manually, by going to "Remote Devices" section of my DevTools, is there a way to do so bypassing puppeteer? Is there a package that lets us access more core functionalities of the DevTools protocol?
Yes, you can do that. puppeteer is just a wrapper around the Chrome DevTools Protocol. To understand how the protocol works, you might want to have a look at Protocol Fundamentals, where the basics are explained.
To communicate on this lower level, you can also use puppeteer. Check out the CDPSession documentation, which exposes the "low level" part of the API.
Minimal example:
const client = await page.target().createCDPSession(); // creates a "low level" session
await client.send('COMMAND'); // sends the command to the browser
client.on('MESSAGE', () => { /* message from the browser received */ });
Alternatives
If you don't want to use puppeteer, there are multiple other libraries. The most used one for Node.js is chrome-remote-interface, which is more low-level than puppeteer. According to the docs:
chrome-remote-interface [...] is a general purpose 1:1 Node.js binding for the Chrome Debugging Protocol. Use it if you need all the power of the raw protocol, e.g., to implement your own high-level API.
There are also multiple other libraries (also for other languages). You can find a full list here.

How can I communicate in a secure way with my firefox extension

I am working on a webpage that depends on browser extensions to perform certain tasks.
From the webpage I can communicate with the chrome browser using 'externally_connectable' and:
chrome.runtime.sendMessage(string extensionId, any message, object options, function responseCallback)
The good thing here is that from the point of view of my website I am sure I am communicating with my extension and not with a 'reverse engineered' version of my extension.
In firefox extensions however there is no support for externally_connectable and I have to use events or window.postmessage:
const event = new CustomEvent('msg-for-content-script');
document.querySelector('body').dispatchEvent(event);
This works ok, but the problem is that when somebody manages to reverse engineer my extension I am not able to check whether the extension I am communicating with is really my extension.
Is there anybody who can give advice on how to make sure that I am communicating with the right extension?

Native Client use outside of the Chrome App Store

I have successfully created a Native Client app that works using localhost and works once posted to the Chrome App Store.
I now need to find a way to embed this app in a web page outside of the App Store, which currently fails.
I've read that the usage of the nacl_io and specifically sockets is ONLY accessible when published via the Chrome App Store?
I also came across this https://developer.chrome.com/extensions/apps (I know it's essentially discontinued). Is there a replacement that could work? Or is it just the Chrome App Store?
The <embed> tag is mentioned here (https://developer.chrome.com/native-client/devguide/coding/application-structure). Am I right in saying even if we got this to load the .hmf file and communicate with the .pexe that it would still fail when using sockets due to the application not being passed through the Chrome App Store?
Any advice is welcome
Socket access is only allowed for applications in the Chrome Web Store. Similarly, Native Client applications (e.g. using a .nexe file) are also only allowed on the Chrome Web Store.
You can run a Portable Native Client (PNaCl) application on the open web, but it will not have access to the socket API.
nacl_io is still available to use. It's only the socket API that will fail if you try to use it.
You can still use the URLLoader and WebSocket APIs, though. Perhaps these will be enough for your application?

How to control web browser using some programming language?

I am looking for a way to control a web browser such as firefox or chrome. I need something like "selenium webdriver" but that will allow me to open many instances URL load, get http headers, response code, get response content, load time, etc.
Is there any library, framework, api that I could use to do it? I couldn't find one exactly that does all, selenium opens browser and go to url but I can't get http headers
Selenium and Jellyfish are strong options in general. Jellyfish is an option that uses Node.js - although I have no experience with it, I've heard good things from my colleagues.
If you just want to get headers and such, you could use the cURL library or wget. I've used cURL with NuSOAP to query XML web services in PHP, for example. The downside is that these are not functional browsers, and merely perform the HTTP requests and consume the response.
http://seleniumhq.org/
https://github.com/admc/jellyfish
http://curl.haxx.se/

Resources