I want to use Puppeteer to automate a particular task. However, for user-friendly purposes, I thought about implementing it within a chrome extension. Browsing the web, I encountered this article https://dev.to/tchan/web-automation-using-puppeteer-inside-a-chrome-extension-318o To be honest, I could not absorb some context from the article since I don't have enough experience about that kind of stuff. According to the article, I had to activate remote debugging for chrome. So I did, yet could not access localhost, thus webSocketDebuggerUrl property, either. Some connection refusal error hit on the browser while trying to access. But the thing about I care is that in the case that it even works, if I had published this extension, one would have to activate his/her remote debugging for chrome, too and that would not be much of user-friendly. In the end, I thought that if it is possible to run Puppeteer on nodejs with your own computer, it should be possible to run it on server-side(nodejs) of a website, too. Thus, you wouldn't need an extension, so the website itself would implement the task I have intended. Sorry for this long story-wise problem, but still I don't have much knowledge of server-side languages. So is it possible to run Puppeteer on server-side? So what would you recommend to do?
Thanks.
Related
I asked this question a little over a week ago on the Atom forums (link below), and didn't receive a response, so I am reposting it here in the hopes that someone may be able to provide insight on my problem.
Recently, I have taken up an open source project which uses Electron as it’s front-end. This project has two requirements: it must be cross-platform, and it must have an embedded web browser (which should be able to browse the web and render content like a typical browser). Considering the rather large footprint Electron has already netted my application, it seems like a bad idea to attempt to use another embedded web framework alongside it. So, in the interest of simplifying my project and retaining the UI built on top of Electron, I am looking into using Electron itself as the web browser. Here’s where I’ve come into a problem.
In a security page for Electron’s documentation, it is explicitly stated that,
it is important to understand … Electron is not a web browser
This quote comes within the context that Electron–or rather the code running on top of it–carries the unique ability to interact with the user’s operating system, unlike typical web applications. The page goes on to say,
displaying arbitrary content from untrusted sources poses a severe security risk that Electron is not intended to handle
At this point, I was tempted to give up on the idea of using Electron as an inbuilt browser, but further down on that same page, you can find another very interesting tidbit:
To display remote content, use the <webview> tag or BrowserView , [and] make sure to disable the nodeIntegration and enable contextIsolation
Link: https://electronjs.org/docs/tutorial/security#isolation-for-untrusted-content
First, in regard to using webviews, Electron’s own documentation recommends outright avoiding them:
Electron’s webview tag is based on Chromium’s webview , which is undergoing dramatic architectural changes. This impacts the stability of webviews , including rendering, navigation, and event routing. We currently recommend to not use the webview tag and to consider alternatives, like iframe , Electron’s BrowserView , or an architecture that avoids embedded content altogether.
Link: https://electronjs.org/docs/api/webview-tag
Seeing as though I cannot avoid embedded content, I opted to look into using a BrowserView, but what I found was not very motivating either. The advice, as it stands, is to do two things:
disable nodeIntegration
enable contextIsolation
After looking at the security and best-practices page, I will also append the following steps:
deny session permission requests from remote content (webcam, microphone, location, etc.)
catch webview elements in creation and strip default privileges
disable the creation of new windows
disable the remote module
That is a fair amount of steps to undergo in securing external content. Not to mention, there were several additional warnings scattered through the best practices page such as the following:
(On verifying webview options before creation)
Again, this list merely minimizes the risk, it does not remove it. If your goal is to display a website, a browser will be a more secure option.
Link: https://electronjs.org/docs/tutorial/security#11-verify-webview-options-before-creation
(On disabling the remote module)
However, if your app can run untrusted content and even if you sandbox your renderer processes accordingly, the remote module makes it easy for malicious code to escape the sandbox and have access to system resources via the higher privileges of the main process.
Link: https://electronjs.org/docs/tutorial/security#15-disable-the-remote-module
Not to mention, upon navigation to the BrowserView page, the whole class is listed as experimental.
This all isn’t even to mention the added attack surface created by Electron, such as a vulnerability in the webview component just last year: CVE-2018-1000136
Now, taking into account all of the above, numerous developers have still opted to create web browsers that routinely consume external and uncontrolled content using Electron.
Browser’s using Electron (linked directly from Electron’s website):
https://electronjs.org/apps/wexond
https://electronjs.org/apps/dot
https://electronjs.org/apps/beaker-browser
To me, it seems irresponsible to submit users to the above security implications as a trade-off for convenience.
That being said, my question is: can you safely, to the point at which you could ensure the integrity of your users, implement web browsing capabilities for uncontrolled content using Electron?
Thank you for your time.
Link to the original post:
https://discuss.atom.io/t/security-implications-in-electron-as-a-web-browser/70653
Some ideas that don't fit into a comment box:
[the project] must have an embedded web browser
So I presume then that this project isn't just a web browser. There's other content there that may have access to Node, but you just want the embedded-web-browser portion of it to be sandboxed appropriately, right?
Regarding the comments about <webview>, yes, it is considered unstable and Electron recommends using a BrowserView instead. I don't think that the fact that it's marked as "experimental" should necessarily deter you from using it (especially considering that the Electron team is recommending it [though maybe as the best of two evils]).
Experimental doesn't imply it's unstable. It can just mean that the Electron team is experimenting with this approach, but this approach may change in the future (at which point I would expect Electron to provide a transition path forward). Though this is one possible interpretation and ultimately Electron would have to comment on this.
The advice... is to do two things:
disable nodeIntegration
enable contextIsolation
I would also make use of the sandbox option inherited from BrowserWindows. BrowserView's docs on the constructor options say:
webPreferences Object (optional) - See BrowserWindow.
which tells me that BrowserView accepts the same options as BrowserWindow.
You would set it up like so:
new BrowserView({ webPreferences: {
sandbox: true,
nodeIntegration: false,
contextIsolation: true,
preload: "./pathToPreloadScript.js"
}});
See more information about this approach here. The preload script is what would expose some Node IPC APIs to the sandboxed content you're loading. Note the Status section at the bottom, which says:
Please use the sandbox option with care, as it is still an experimental feature. We are still not aware of the security implications of exposing some Electron renderer APIs to the preload script
If the content you're loading in the BrowserView never needs to communicate back to the application, then you don't need a preload script at all and can just sandbox the BrowserView.
After looking at the security and best-practices page, I will also append the following steps:
deny session permission requests from remote content (webcam, microphone, location, etc.)
catch webview elements in creation and strip default privileges
disable the creation of new windows
disable the remote module
Sure, those sound reasonable. Note that if your embedded browser needs to be able to open new windows (via window.open or <a target="_blank" />) then you'd have to allow popups.
That is a fair amount of steps to undergo in securing external content.
Sure, but is your main concern with the security of the app, or with how much work it takes to make it secure? Browser developers need to consider similar things to ensure webpages can't get access to the OS. It's just part of the game.
Again, this list merely minimizes the risk, it does not remove it. If your goal is to display a website, a browser will be a more secure option.
This is just saying that if all you're trying to do is display a website, then just use a browser since that's what they're there for.
If you need to do other things, well then you can't use a browser, so you'll have to make your own app, making sure it's reasonably secure.
I think that if you follow what's recommended in the Security document and keep up to date with new Electron releases, then you're doing the best you can do.
As for whether that's good enough, I can't say. It depends on what you're developing and what you're trying to protect against.
However, my thoughts can't substitute the more expert opinions of those on the Electron team. This question could certainly use some guidance from them.
I want to add a button to my visual C++ form that will open with a specific browser. So far for links I've been using:
System::Diagnostics::Process::Start("UrlHere")
Which, as standard, opens with whatever your default browser is.
I'm wondering what the process would be to force the URL to open with a specific browser and if it's possible without the use of ShellExecute?
Edit - You are correct, this is C++/CLI. Removed the C++ Tag.
Edit Edit - Apologies if it came across as misleading. Some slight elaboration;
The buttons will launch to application URL's, some of which can only be used in Internet Explorer, others that CAN (and should) be used in Chrome. This is why I need to avoid using the default browser and have different buttons using different browsers when launching URLs
Before answering the 'how', I'd like to ask the question "should you be implementing this?"
By not launching the user's default browser, you are subverting the user's decision.
Perhaps the user prefers a particular interface, and is willing to live with the incorrect renderings that come with it.
Perhaps the user has a browser addon that they really need, such as a screen reader for the blind.
You are requiring additional software installed that the user may or may not want.
Perhaps the user doesn't want Chrome. Perhaps the user prefers FireFox.
You are saying that you know which browser is best, now and forever.
What if the next version of IE makes it work with the sites that are currently Chrome-only? What if the next version of Chrome fixes the sites that are currently IE-only?
What if the site changes so that it works in more browsers?
Do you go back and release a new version of your software that changes the browser for particular sites?
You're trying to solve a problem that may already be fixed.
Both Chrome and Firefox support a addon that will render a tab using the IE engine. It can be set to automatically activate when certain URLs are seen.
Perhaps there is a browser that already works with all your sites, that you don't know about.
Therefore, my recommendation is no, do not do this. The user has decided which browser they want to use, respect that decision and use the default browser.
That said, here's how you would do it: You could use the CreateProcess method, but you're in managed-land, so you might as well use it. Use the Process class to launch the new process for you.
Process^ browserProcess = gcnew Process();
browserProcess->StartInfo->UseShellExecute = false;
browserProcess->StartInfo->FileName = "C:\\Program Files (x86)\\Internet Explorer\\iexplore.exe";
browserProcess->StartInfo->Arguments = "http://www.google.com";
browserProcess->Start();
I'm looking at AppJS as a possible candidate to build a cross-browser application. AppJS runs HTML5 content with a Chromium window with hooks to NodeJS.
Is there a NodeJS module that can give me an enumeration of displays currently active? I need to support dual-monitor setups and knowing the bounds of each display is required.
I've search the Node npm directory with no luck so far.
Update:
Based on sihorton's answer, I'm playing with a javascript approach. Browser security prohibits my first approach from working in normal Chrome, but I will try this on AppJS chromium when I get home. Code here and here.
The basic approach is running code like:
OpenWindow('http://jsbin.com/axiwad/3/','test',400,500);
and then the opened window runs:
window.moveTo(screen.width+2, 0);
setTimeout(function(){
$(".s2w").html(screen.width+2);
$(".s2h").html(screen.height);
}, 500);
The issue of course, is that browser do not allow a moveTo outside the active screen boundary. It remains to be seen if AppJS chromium fork allows this.
If you are running AppJS then you can do the screen detection using tricks in javascript running in the chromium window part of AppJS. I quick google search found the following link, I am sure there are others and if you experiment enough then hopefully you will get a cross platform solution:-
http://archive.cpradio.org/code/javascript/dual-monitors-and-windowopen/
An alternative approach if you want to stay node-native is to implement a common api and then implement separate code behind that for each platform. I don't personally know of any better solution at the moment, and obviously this is less preferable to the javascript solution above. However if you must go this route then there is node edge:
http://tjanczuk.github.io/edge
This lets you run .net code from nodejs so you should be able to cover windows machines with that api. For linux you could then look into connecting into dbus:
https://github.com/sidorares/node-dbus
https://npmjs.org/package/dbus-native
Hopefully you can find appropriate sample code on the internet to get display properties using dbus and .net and then just plug it together and go from there.
I'm tyring to get phonegap up and running on blackberry storm (9530 simulator). I had been testing my webapp from withing BB's built in browser, and it was looking ok, but then it totally bit once I tried to look at the some code from within phonegap, even though I was pointing phonegap to the same url (I hadn't yet gotten to the point of running code locally on the device).
I tried a test case on google and got similiar results. see below. I suspect that I'm missing something basic here. I would have expect both images to be nearly identical.
Browser
http://www.eleganttechnologies.com/outside/ImgDeviceBB9530WebGoogle.jpg
Phonegap
http://www.eleganttechnologies.com/outside/ImgDeviceBB9530PgGoogle.jpg
[Update]
To shed some light on what is happening, I ran the browser and the embedded browser (phonegap) against the W3 mobile web acid test: http://www.w3.org/2008/06/mobile-test/
I definitely notice differences between the two, but I don't yet know the 'why' and the 'how-to-address'.
Acid via built-in browser
(source: eleganttechnologies.com)
BTW - I ran this earlier today and got a couple more green square than just now.
Acid via browser embedded into phonegap
http://www.eleganttechnologies.com/outside/ImgDeviceBb9530PgAcid.jpg
Disclaimer: I don't know anything about phonegap, but have a pretty good theory. By default the embedded browser control on BlackBerry uses an older version of the rendering engine than the BlackBerry browser itself does.
At the BlackBerry developer conference last year, a talk was given about this, and there's an undocumented option to use the newer rendering engine. \
The option ID is 17000 (yes, a magic number, which could change, use at your own risk etc), and should be set to true. Not sure how you'd pass this option through phonegap (I'm not familiar with the toolkit) but using the BlackBerry APIs it's something like:
BrowserContent content;
...
content.getRenderingOptions().setProperty(RenderingOptions.CORE_OPTIONS_GUID, 17000, true);
I don't know the specifics of the browsers you are using, but I do know that most of the big sites will detect your OS + browser combination to decide what HTML to show you.
If Google is seeing a different user agent, you might get a generic mobile version of the HTML instead os the Blackberry specific HTML you get for the built in browser.
If you have access to a web server, try hitting it with both browser setups and see if there is any difference in the log file. That might tell you something interesting.
As we can see in your Acid tests...
One browser (the built-in one) is reporting correctly as a BlackBerry9530, and the other (phonegap) is not presenting the user-agent ["Testing with ."].
In this case, Google is providing you with the default view of their homepage, whereas when you are reporting yourself as a BlackBerry device, you will get the BlackBerry specific rendering.
By the sounds of things, using phonegap is removing the default user-agent (most probably because it's not recognising your device). As phonegap is open-source, the best bet is to get in there, and debug it and find out what happens with the user-agent when the http requests leave the device and track it back from there.
Maybe one browser has capabilities that another one does not?
Hm. By looking at the screenshot I would say that the second page is probably missing some resources. It may be missing some images, scripts and the CSS files, which would explain different l&f. Knowing how Blackberry Browser Field API works, I would guess that the implementation that uses the BrowserField was not done correctly. Just my guess. In addition to that, when the browser field is initialized the caller needs to configure it properly by enabling the appropriate browser features - scripts, styles etc. Again, the API is done in a very weird way, I have gotten myself into this trap once. When setting the options, you cannot just create one mask (like CSS | WML | SCRIPT) and make one call. Options are numeric and, I believe, non-overlapping - but you still need to call the API for setting each option independently.
Also the way asynchronous loading of the resources for BrowserField takes time to understand.
Just my $0.02.
I know this might be a no-brainer, but please read on.
I also know it's generally not considered a good idea, maybe the worst, to let a browser run and interact with local apps, even in an intranet context.
We use Citrix for home-office, and people really like it. Now, they would like the same kind of environment at work, a nice page where every important application/document/folder is nicely arranged and classified in an orderly fashion. These folks are not particularly tech savvy; I don't even consider thinking that they could understand the difference between remote delivered applications and local ones.
So, I've been asked if it's possible. Of course, it is, with IE's good ol' ActiveX controls. And I even made a working prototype (that's where it hurts).
But now, I doubt. Isn't it madness to allow such 'dangerous' ActiveX controls, even in the 'local intranet' zone? People will use the same browser to surf the web, can I fully trust IE? Isn't there a risk that Microsoft would just disable those controls in future updates/versions? What if a website, or any kind of malware, just put another site on the trust list? With that extent of control, you could as well uninstall every protection and just run amok 'till you got hanged by the IT dept.
I'm about to confront my superiors with the fact that, even if they saw it is doable, it would be a very bad thing. So I'm desperately in need of good and strong arguments, because "let's don't" won't do it.
Of course, if there is nothing to be scared of, that'll be nice too. But I strongly doubt that.
We use Citrix for home-office, and people really like it. Now, they would like the same kind of environment at work, a nice page where every important application/document/folder is nicely arranged and classified in an orderly fashion
I haven't used Citrix very many times, but what's it got to do with executing local applications? I don't see how "People like Citrix" and "browser executing local applications" relate at all?
If the people are accessing your Citrix server from home, and want the same experience in the office, then buy a cheap PC, and run the exact same Citrix software they run on their home computers. Put this computer in the corner and tell them to go use it. They'll be overjoyed.
Isn't it madness to allow such 'dangerous' ActiveX controls, even in the 'local intranet' zone ? People will use the same browser to surf the web, can I fully trust IE ?
Put it this way. IE has built-in support for AX controls. It uses it's security mechanisms to prevent them from running unless in a trusted site. By default, no sites are trusted at all.
If you use IE at all then you're putting yourself at the mercy of these security mechanisms. Whether or not you tell it to trust the local intranet is beside the point, and isn't going to affect the operation of any other zones.
The good old security holes that require you to reboot your computer every few weeks when MS issues a patch will continue to exist and cause problems, regardless of whether you allow ActiveX in your local intranet.
Isn't there a risk that Microsoft would just disable those controls in future updates / versions ?
Since XP-SP2, Microsoft has been making it increasingly difficult to use ActiveX controls. I don't know how many scary looking warning messages and "This might destroy your computer" dialogs you have to click through these days to get them to run, but it's quite a few. This will only get worse over time.
Microsoft is walking a fine line. On one hand, they regularly send ActiveX killbits with Windows Update to remove/disable applications that have been misbehaving. On the other hand, the latest version of Sharepoint 2007 (can't speak for earlier versions) allows for Office documents to be opened by clicking a link in the browser, and edited in the local application. When the edit is finished, the changes are transmitted back to the server and the webpage (generally) is refreshed. This is only an IE thing, as Firefox will throw up an error message.
I can see the logic behind it, though. Until Microsoft gets all of their apps 'in the cloud', there are cases that need to bridge the gap between the old client-side apps and a more web-centric business environment. While there is likely a non-web workaround, more and more information workers have come to expect that a large portion of their work will be done in a browser. Anything that makes the integration with the desktop easier is not going to be opposed by anyone except the sysadmins.
The standard citrix homepage (or how we use it) is a simple web page with program icons. Click on it, and the application get's delivered to you. People want the same thing, at work, with their applications/folders/documents. And because I'm a web developer, and they asked me, I do it with a web page... Perhaps I should pass the whole thing over to the VB guy..
Ahh... I know of 2 ways to accomplish this:
You can embed internet explorer into an application, and hook into it and intercept certain kinds of URL's and so on
I saw this done a few years ago - a telephony application embedded internet explorer in itself, and loaded some specially formatted webpages.
In the webpage there was this:
Call John Smith
Normally this would be a broken URL, but when the user clicked on this link, the application containing the embedded IE got notified, and proceeded to execute it's own custom code to dial the number from the URL.
You could get your VB guy to write an application which basically just wraps IE, and has handlers for executing applications. You could then code normal webpages with links to just open applications, and the VB app would launch them. This allows you to write your own security stuff (like, only launch applications in a preset list, or so on) into the VB app, and because VB is launching them, not IE, none of the IE security issues will be involved.
The second way is with browser plug-ins.
For example, skype comes with a Firefox plug-in, which looks for phone-numbers in web-pages, and attaches special links to them. When you click on these links it invokes skype - you could conceivably do something similar for launching your citrix apps.
You'd then be tied to firefox though. Writing plugins for IE is much harder than for FF, I wouldn't go down that path unless forced to.