I want to build a chrome extension for personal use.
The extension will scrape some webpages and it will render some information.
So I think puppeteer can help me with that.
I understand that I need to run node inside a chrome extension.
Is it possible?
I have found some answers but they are old.
I know this is 9 months late but I had the same use case at work on Window machines but you can make it work with Mac.
The trick is to use puppeteer-web
https://github.com/puppeteer/puppeteer/tree/master/utils/browser#bundling-for-web-browsers
Bundle the repository and place it in your chrome extension folder and then reference it in your popup.html with something like
<script src="./puppeteer/utils/browser/puppeteer-web.js"></script>
You'll then need to take advantage of Chrome's remote debugging functionality as puppeteer-web can't start its own instance via puppeteer.launch() and can only use puppeteer.connect() to connect to an already existing chrome instance.
On windows add --remote-debugging-port=9222 to the end of the target field of the chrome short cut as per How to make Chrome always launch with remote-debugging-port flag
Or on Mac /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --remote-debugging-port=9222 --no-first-run --no-default-browser-check --user-data-dir=$(mktemp -d -t 'chrome-remote_data_dir')
Once remote debugging is activated you'll be able to see the webSocketDebuggerUrl property by visiting http://127.0.0.1:9222/json/version on your browser. This is the browserWSEndpoint the connect method will invoke.
You will also need to add the port address to the permissions array in the manifest.json file otherwise ajax requests won't work in the chrome extension.
Eg:
"permissions": [ "tabs" , "identity", "http://127.0.0.1:9222/*"],
Example popup.html file
<!DOCTYPE html>
<html>
<head>
<title>Example popup</title>
<link rel="stylesheet" type="text/css" href="style.css">
</head>
<body>
<div>
<button id='puppeteer-button'>Do Puppeteer Things</button>
<script src="./puppeteer/utils/browser/puppeteer-web.js"></script>
<script type="module" src="popup.js"></script>
</div>
</body>
</html>
Example popup.js file
let browserWSEndpoint = '';
const puppeteer = require("puppeteer");
async function initiatePuppeteer() {
await fetch("http://127.0.0.1:9222/json/version")
.then(response => response.json())
.then(function(data) {
browserWSEndpoint = data.webSocketDebuggerUrl;
})
.catch(error => console.log(error));
}
initiatePuppeteer();
// Assign button to puppeteer function
document
.getElementById("puppeteer-button")
.addEventListener("click", doPuppeteerThings);
async function doPuppeteerThings() {
const browser = await puppeteer.connect({
browserWSEndpoint: browserWSEndpoint
});
const page = await browser.newPage();
// Your puppeteer code goes here
}
Hope that helps, I haven't had any issues by appending remote debugging to my target field on my work window machines, despite feeling a bit hacky. I wrote a short blog post on it with better syntax highlighting here.
Actually it is possible but with some limitations. Puppeteer use devtools-protocol (https://chromedevtools.github.io/devtools-protocol/) which is available inside chrome extension when you enable deubgger in your extension manifest https://developer.chrome.com/extensions/debugger. But inside extension is available only latest, stable version of protocol (for now is 1.3 https://chromedevtools.github.io/devtools-protocol/1-3).
But in my opinion you don't need devtools-protocol to handle your problem. Just use standard extension API https://developer.chrome.com/extensions/api_index to open any URL you need (chrome.tabs.update), parse page inside content.js and do with that data whatever you want.
Related
So I have here this:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Oh no!</title>
</head>
<body>
<label>Oh dear. A serious error occurred and the app needs to restart. Press the button below to restart.</label>
<br>
<button onclick="restart()">Restart</button>
<script>
const { app } = require("electron")
function restart() {
app.relaunch()
app.exit()
}
</script>
</body>
</html>
And now, when the app receives an unhandled error this will show... but when the user clicks the button, the app doesn't restart so how would I make the app restart?
You can't get the app object without using preload.js and neither is directly getting the app object safe. There is a method to do the above using preload.js and ipcRenderer which are pure Electon APIs
In electron (even in web development), there is server-side code and browser-side code. The code written in between the script tags in your snippet is server side code which will fail to execute in browser side.
Server-side code in your case is in NodeJS Backend and browser-side code is the one which is the HTML Page and its own javascript.
So to close the window (which only NodeJS can do, i.e., the backend) you need to use Electron's ipcRenderer which helps string based communication between the browser-side javascript and the server-side javascript.
While creating a browser window in electron using new BrowserWindow(options) where options is an object. Define the object as:
options = {
webPreferences: {
preload: preload.js, //You need to create a file named preload.js (or any name) in your code
nodeIntegration: true,
contextIsolation: false,
}
}
Now in a new file called preload.js:
window.ipcRenderer = require('electron').ipcRenderer;
In your snippet you added const { app } ... which should be done this way to inject the javascript using a preload property in the object.
Now in the main app.js file (whatever you named maybe index.js) where you created the browser window:
const ipc = require('electron').ipcMain; //Add to your pre-existing code
ipc.on("close-app", (event, message) => { //"close-app" can be anything but, you need to use the same key in the send message side (later in this answer)
browserWindow.close(); //If you named the browserwindow as browserWindow
});
Now in your HTML (i.e., send message side)
...
<script>
window.ipcRenderer("close-app", ""); //Second parameter is used if you want to send some extra message. The extra message can be viewed in the server side from the message parameter in the app.js code (just above this paragraph)
</script>
This is a bit difficult if you are doing it for the first time.
I've added more articles which will help you clear your confusions:
Highlight about server-side and browser-side code
Relation with socket.io communication in NodeJS
I'm creating an app using laravel + vue. Vue loads all the javascript codes in the app.js even in the login page in which the user is not yet authenticated. I don't want to expose all the js codes just yet until user is logged in. Is there any way that I can only show the necessary codes for the login page and not expose all the js code of the app? I'm doing this to also limit the size of app.js being downloaded for the login page to improve page loading time too.
If you're using Laravel mix, you can add a new entry point for your login script.
mix.js('resources/assets/app/js/app.js', 'public/app/js')
.js('resources/assets/app/js/login.js, 'public/app/js);
So, you can add a #yield statement in your base layout to place scripts declared in the page view.
base.blade.php
<html>
<head>
<title>App Name - #yield('title')</title>
</head>
<body>
#section('sidebar')
This is the master sidebar.
#show
<div class="container">
#yield('content')
</div>
#yield('scripts')
</body>
</html>
login.blade.php
#extends('layouts.base')
#section('title', 'Page Title')
#section('content')
<p>This is my body content.</p>
#stop
#section('scripts')
<script src="{{ mix('app/js/login.js') }}"></script>
#stop
Edited
For a SPA you can have another bundled entry point to just define the Login component:
Vue.component('login-form', {
... component code here
})
You can use vue-plugin-load-script to load the login.js script in the beforeEach method from Vue router. So the component will be available before render the login page.
router.beforeEach(async (to, from, next) => {
if (to === 'login'){
await Vue.loadScript('login entry point url here')
}
next();
});
Also, you can do it inside the page using the beforeMount hook. Here an example using a fake promise and entry point.
https://jsfiddle.net/7oj2sxun/3/
If you're using Vue CLI, it already supports lazy loading with dynamic imports.
Here a detailed article:
https://blog.logrocket.com/lazy-loading-in-vue-js/
I use node js without frameworks etc. and I have problem, I don't understand how set "start html page" for my first request to server.
I tried do it like this
var server = new http.Server();
server.listen(1137, '127.0.0.1');
server.on('request', function(req, res) {
fs.readFile('../public/index.html', function (err, html) {
if (err) {
throw err;
} else {
res.write(html);
res.end();
}
});
});
When I do request to 127.0.0.1:1137 - I got html in browser, but links to CSS/JS files isn't correct and how I can this to fix I don't know :(
I want get the html page ../public/index.html in browser when I will do first request to my server.
my server location
project/server/server.js
my html-page location
project/public/index.html
Your page includes references to images and stylesheets, which you said don't work.
Well, you are responding to every single HTTP request with the contents of the specified HTML page.
When the browser parses the HTML, it will see the image and stylesheet links and issue HTTP requests to those URL's. But those URL's don't respond with images or stylesheets. They respond with HTML.
GET /index.html
yields
<html>
<head>
<link rel="stylesheet" href="styles.css">
</head>
<body>
<img src="someimage.png">
</body>
</html>
The browser then requests
GET /styles.css
yields
<html>
...
</html>
The browser then requests
GET /someimage.png
yields
<html>
...
</html>
You need to make the response conditional based on the request. To prevent disclosure of information, like #minitech mentioned, you need to be careful not to blindly concatenate the paths. Then you have to worry about MIME types.
You're really best off using a framework like express.
You will need to return different files, depending on the path that was requested.
On the 'request' handler you are getting a ClientRequest object, which has a path property. Use this information to return the correct files (index.html, the CSS or JS files...). An example request callback could be:
function onRequest(req, res) {
var path = req.path;
fs.readFile('../public' + path, function(err, contents) {
if (err) {
res.writeHead(500);
res.end(err);
return;
}
res.end(contents);
});
}
This is still very basic, you will probably want to handle non-existent files with a 404 result code, return the correct Content-Type headers the different file kinds (text/html, text/css, etc).
Update: as minitech recommends, the path should be checked for ../, which would go up the filesystem and access sensitive files.
Hope this helps.
I'm trying to find out whether it's possible to invoke node.js functions from a web page. Is there any way to make node.js functions accessible from Google Chrome (so that they are run on the node.js server), as shown here?
(I'm aware that it's possible to do this using node-webkit (a non-standard Chromium implementation) without modifying the code, but I'd prefer to do this using an unmodified browser, which will require the code shown below to be modified in some way.)
<html>
<body>
<script type = "text/javascript">
var exec = require('child_process').exec; //node.js function
</script>
<p onclick = "exec('firefox')">
Click here to launch the Firefox web browser.
</p>
</body>
</html>
No, this is not possible, for clear security reasons.
You only have available to you what the browser gives you. node-webkit is the closest thing available, and does not meet your requirements.
NW has own method like node exec :
var gui = require('nw.gui');
gui.Shell.openItem('firefox', ,function(error, stdout, stderr) { });
Is there a way to check iOS to see if another app has been installed and then launched? If memory serves me this was not possible in early versions but has this been changed?
Doable, but tricky.
Launching installed apps, like the FB or Twitter apps, is done using the Custom URL Scheme. These can be used both in other apps as well as on web sites.
Here's an article about how to do this with your own app.
Seeing if the URL is there, though, can be tricky. A good example of an app that detects installed apps is Boxcar. The thing here is that Boxcar has advanced knowledge of the custom URL's. I'm fairly (99%) certain that there is a canOpenURL:, so knowing the custom scheme of the app you want to target ahead of time makes this simple to implement.
Here's a partial list of some of the more popular URL's you can check against.
There is a way to find out the custom app URL : https://www.amerhukic.com/finding-the-custom-url-scheme-of-an-ios-app
But if you want to scan for apps and deduce their URL's, it can't be done on a non-JB device.
Here's a blog post talking about how the folks at Bump handled the problem.
There is a script like the following.
<script type="text/javascript">
function startMyApp()
{
document.location = 'yourAppScheme://';
setTimeout( function()
{
if( confirm( 'You do not seem to have Your App installed, do you want to go download it now?'))
{
document.location = 'http://itunes.apple.com/us/app/yourAppId';
}
}, 300);
}
</script>
Calling this script from the web (Try to start MyApp), you can determine if your app with scheme "yourAppScheme" is installed on the device or not.
The App will launch if it is installed on the device and "yourAppScheme" is registered in it.
If the app is not installed you can suggest the user to install this app from iTunes.
To check if an app is installed (e.g. Clear):
BOOL installed = [[UIApplication sharedApplication] canOpenURL:[NSURL URLWithString:#"clearapp://"]];
To open that app:
BOOL success = [[UIApplication sharedApplication] openURL:[NSURL URLWithString:#"clearapp://"]];
Hides the error message if the app is not installed
At Branch we use a form of the code below--note that the iframe works on more browsers. Simply substitute in your app's URI and your App Store link.
<!DOCTYPE html>
<html>
<body>
<script type="text/javascript">
window.onload = function() {
// Deep link to your app goes here
document.getElementById("l").src = "my_app://";
setTimeout(function() {
// Link to the App Store should go here -- only fires if deep link fails
window.location = "https://itunes.apple.com/us/app/my.app/id123456789?ls=1&mt=8";
}, 500);
};
</script>
<iframe id="l" width="1" height="1" style="visibility:hidden"></iframe>
</body>
</html>
There's a second possibility that relies on cookies first and the javascript redirect only as a fallback. Here's the logic:
When a user without the app first taps on a link to your app, he or she is redirected straight to the App Store. This is accomplished by a link to your app actually being a dynamically-generated page on your servers with the redirect. You create a cookie and log a "digital fingerprint" of IP address, OS, OS version, etc. on your backend.
When the user installs the app and opens it, you collect and send another "digital fingerprint" to your backend. Now your backend knows the link is installed On any subsequent visits to links associated with your app, your servers make sure that the dynamically-generated redirect page leads to the app, not the App Store, based on the cookie sent up with the request.
This avoids the ugly redirect but involves a ton more work.
To my understanding, because of privacy issues, you can't see if an app is installed on the device. The way around this is to try and launch the app and if it doesn't launch to have the user hit the fall back url. To prevent the mobile safari error from occurring I found that placing it in an iframe helps resolve the issue.
Here's a snippet of code that I used.
<form name="mobileForm" action="mobile_landing.php" method="post">
<input type="hidden" name="url" value="<?=$web_client_url?>">
<input type="hidden" name="mobile_app" value="<?=$mobile_app?>">
<input type="hidden" name="device_os" value="<?=$device_os?>">
</form>
<script type="text/javascript">
var device_os = '<? echo $device_os; ?>';
if (device_os == 'ios'){
var now = new Date().valueOf();
setTimeout(function () {
if (new Date().valueOf() - now > 100)
return;
document.forms[0].submit(); }, 5);
var redirect = function (location) {
var iframe = document.createElement('iframe');
iframe.setAttribute('src', location);
iframe.setAttribute('width', '1px');
iframe.setAttribute('height', '1px');
iframe.setAttribute('position', 'absolute');
iframe.setAttribute('top', '0');
iframe.setAttribute('left', '0');
document.documentElement.appendChild(iframe);
iframe.parentNode.removeChild(iframe);
iframe = null;
};
setTimeout(function(){
window.close()
}, 150 );
redirect("AppScheme");
I struggled with this recently, and here is the solution I came up with. Notice that there is still no surefire way to detect whether the app launched or not.
I serve a page from my server which redirects to an iPhone-specific variant upon detecting the User-Agent. Links to that page can only be shared via email / SMS or Facebook.
The page renders a minimal version of the referenced document, but then automatically tries to open the app as soon as it loads, using a hidden <iframe> (AJAX always fails in this situation -- you can't use jQuery or XMLHttpRequest for this).
If the URL scheme is registered, the app will open and the user will be able to do everything they need. Either way, the page displays a message like this at the bottom: "Did the app launch? If not, you probably haven't installed it yet .... " with a link to the store.