I have built an extension that keeps track of all the pages which are loaded in a browser session. Recently, I started seeing extra pages in my logs, pages that were not getting actually rendered on the browser.
I tried to find out more about it and learned that it is the "page preload" feature of the browser that these websites are using. Basically, if you specify a link tag with preload option, the associated page will be loaded silently in the background (https://developer.mozilla.org/en-US/docs/Web/HTML/Preloading_content). For these pages, three of the four chrome.webNavigation events are getting fired - onBeforeNavigate, onCommitted and onDOMContentLoaded. I also looked into the transitionQualifier of the onCommitted event but does not give enough information to single out page preloads.
Any ideas around how do I detect that the page was not loaded in the conventional sense but was preloaded?
Also, if someone knows how can I disable the page preload feature in Chrome through commandline switches, I might just block all preloads and my extension would still work.
Related
I'm developing a chrome extension that renders and sidebar iframe in Gmail.
A while back, we ran into a Content Security Policy issue that broke our extension and ran into a well-known error that shows up in your console as
Refused to frame '[OUR DOMAIN]' because it violates the following Content Security Policy directive: ...
To fix it, we had to rewrite the CSP header as explained in this blog post.
However, recently, we've been getting the same error, even after having implemented the fix suggested in the blog post.
As you can see, even though we added the chrome.webRequest.onHeadersReceived listener to overwrite the CSP header, our domain doesn't seem to be working. As a result, our sidebar does not load.
Notes and attempts:
Note that upon refreshing the entire page, the sidebar will sometimes load successfully %50 of the times, which leads me to believe that we're not adding the above event listener fast enough. To solve this, I pulled the event listener out of our background class and placed it at the top of the background js script. However, this issue seems to still persist.
We've tried to remove and re-add the extension but that didn't yield any permanent results.
Throughout testing, I also noticed that adding another extension that implements a similar override on the CSP header will cause our extension to break. Removing and re-adding our extension brings us back to the 50-50 issue in the first note.
[Update]
It seems that being able to reproduce it %50 of the times has to do with the version of chrome. After upgrading it to 72, I'm now able to reproduce the issue. However, doing a hard refresh on the page will sometimes allow the page to load.
More details on our setup: What we do is basically have a background.html which contains html templates and we have one template that contains an iframe tag. To get the template with the iframe, we do a sendMessage, grab the response, which is the template in string, convert it to DOM element via jquery, append it to body, and change the iframe's src attribute. What happens is the iframe loads fine initially, but when you click on any link in the iframe that would redirect to another page, the same error above shows up.
I can provide snippets and more info if needed.
I'm working on a browser extension (compatible with Chrome, FF, Opera, and Edge) and I'm trying to figure out how to associate requests to domains outside of the current page. For example, when you go to google.com a lot of requests to domains other than google.com occur such as to gstatic.com.
An extension like NoScript shows all of the requested domains that a page made and lets you allow or deny. I'm trying to get a similar functionality.
Is this something that can be done in the content script or is there some way to keep state information in the background script that I can then display in the popup? Obviously it's possible but I'm just not seeing which callback I can use.
I recently published a Chrome extension (Source Code) and now discover some broken incoming links on the extension's website which must be related to that extension:
/track_install/search/ext/free/mebkekakcnabgndiakbbefcgpedlaidp/mixcloud_downloader
/webstore/detail/ext/free/mebkekakcnabgndiakbbefcgpedlaidp/mixcloud_downloader
On the chrome extension webstore I don't find such links. Do you have any idea where those links come from and what's their purpose? Would users exepect anything else than a 404 on that URLs?
The website is referenced in the extension's manifest homepage_url field and on the webstore item in the "Websites" field.
Update: I just noticed again one such request where the referer comes from this question.
Normally, those URLs are relative to the Webstore and are used for Analytics tracking and this stat page (only available to you). See this mention, for example.
/track_install/... is, quite obviously, used as a beacon to track installs.
/webstore/detail/ext/free/... tracks opening your extension's listing in Web Store.
Here's documentation on homepage_url, which I believe influenced this, including this quote:
If you distribute your extension using the Chrome Web Store, the homepage URL defaults to the extension's own page.
I believe that it's either a bug that those are sent out to your server instead, or a feature I haven't seen documented anywhere to let you track those instead. Note that those are just beacons sent from analytics code; you don't need to serve content on them.
In any case, it's worth reporting, either on the bugtracker or via the exceptionally well-hidden developer support form.
In Firefox or Chrome I'd like to prevent a private web page from making outgoing connections, i.e. if the URL starts with http://myprivatewebpage/ or https://myprivatewebpage/ in a browser tab, then that browser tab must be restricted so that it is allowed to load images, CSS, fonts, JavaScript, XmlHttpRequest, Java applets, flash animations and all other resources only from http://myprivatewebpage/ or https://myprivatewebpage/, i.e. an <img src="http://www.google.com/images/logos/ps_logo.png"> (or the corresponding <script>new Image(...) must not be able to load that image, because it's not on myprivatewebpage. I need a 100% and foolproof solution: not even a single resource outside myprivatewebpage can be accessible, not even at low probability. There must be no resource loading restrictions on Web pages other than myprivatewebpage, e.g. http://otherwebpage/ must be able to load images from google.com.
Please note that I assume that the users of myprivatewebpage are willing to cooperate to keep the web page private unless it's too much work for them. For example, they would be happy to install a Chrome or Firefox extension once, and they wouldn't be offended if they see an error message stating that access is denied to myprivatewebpage until they install the extension in a supported browser.
The reason why I need this restriction is to keep myprivatewebpage really private, without exposing any information about its use to webmasters of other web pages. If http://www.google.com/images/logos/ps_logo.png was allowed, then the use of myprivatewebpage would be logged in the access.log of Google's ps_logo.png, so Google's webmasters would have some information how myprivatewebpage is used, and I don't want that. (In this question I'm not interested in whether the restriction is reasonable, but I'm only interested in the technical solutions and its strengths and weaknesses.)
My ideas how to implement the restriction:
Don't impose any restrictions, just rely on the same origin policy. (This doesn't provide the necessary protection, the same origin policy lets all images pass through.)
Change the web application on the server so it generates HTML, JavaScript, Java applets, flash animations etc. which never attempt to load anything outside myprivatewebpage. (This is almost impossibly hard to foolproof everywhere on a complicated web application, especially with user-generated content.)
Over-sanitize the web page using a HTML output filter on the server, i.e. remove all <script>, <embed> and <object> tags, restrict the target of <img src=, <link rel=, <form action= etc. and also restrict the links in the CSS files. (This can prevent all unwanted resources if I can remember all HTML tags properly, e.g. I mustn't forget about <video>. But this is too restrictive: it removes all dyntamic web page functionality like JavaScript, Java applets and flash animations; without these most web applications are useless.)
Sanitize the web page, i.e. add an HTML output filter into the webserver which removes all offending URLs from the generated HTML. (This is not foolproof, because there can be a tricky JavaScript which generates a disallowed URL. It also doesn't protect against URLs loaded by Java applets and flash animations.)
Install a HTTP proxy which blocks requests based on the URL and the HTTP Referer, and force all browser traffic (including myprivatewebpage, otherwebpage, google.com) through that HTTP proxy. (This would slow down traffic to other than myprivatewebpage, and maybe it doesn't protect properly if XmlHttpRequest()s, Java applets or flash animations can forge the HTTP Referer.)
Find or write a Firefox or Chrome extension which intercepts all outgoing connections, and blocks them based on the URL of the tab and the target URL of the connection. I've found https://developer.mozilla.org/en/Setting_HTTP_request_headers and thinkahead.js in https://addons.mozilla.org/en-US/firefox/addon/thinkahead/ and http://thinkahead.mozdev.org/ . Am I correct that it's possible to write a Firefox extension using that? Is there such a Firefox extension already?
Some links I've found for the Chrome extension:
http://www.chromium.org/developers/design-documents/extensions/notifications-of-web-request-and-navigation
https://groups.google.com/a/chromium.org/group/chromium-extensions/browse_thread/thread/90645ce11e1b3d86?pli=1
http://code.google.com/chrome/extensions/trunk/experimental.webRequest.html
As far as I can see, only the Firefox or Chrome extension is feasible from the list above. Do you have any other suggestions? Do you have some pointers how to write or where to find such an extension?
I've found https://developer.mozilla.org/en/Setting_HTTP_request_headers and thinkahead.js in https://addons.mozilla.org/en-US/firefox/addon/thinkahead/ and http://thinkahead.mozdev.org/ . Am I correct that it's possible to write a Firefox extension using that? Is there such a Firefox extension already?
I am the author of the latter extension, though I have yet to update it to support newer versions of Firefox. My initial guess is that, yes, it will do what you want:
User visits your web page without plugin. Web page contains ThinkAhead block that would send a simple version header to the server, but this is ignored as plugin is not installed.
Since the server does not see that header, it redirects the client to a page to install the plugin.
User installs plugin.
User visits web page with plugin. Page sends version header to server, so server allows access.
The ThinkAhead block matches all pages that are not myprivatewebpage, and does something like set the HTTP status to 403 Forbidden. Thus:
When the user visits any webpage that is in myprivatewebpage, there is normal behaviour.
When the user visits any webpage outside of myprivatewebpage, access is denied.
If you want to catch bad requests earlier, instead of modifying incoming headers, you could modify outgoing headers, perhaps screwing up "If-Match" or "Accept" so that the request is never honoured.
This solution is extremely lightweight, but might not be strong enough for your concerns. This depends on what you want to protect: given the above, the client would not be able to see blocked content, but external "blocked" hosts might still notice that a request has been sent, and might be able to gather information from the request URL.
When I go to our web site through HTTPS mode, Chome is reporting an error saying that the page contains secure and not secure items. However, I used Firebug, Fiddler, and HttpDebuggerPro, all which are telling me that everything is going through HTTPS. Is this a bug in Chrome?
Sorry but I'm unable to give out the actual URL.
A bit late to the party here but I've been having issues recently and once I had found a http resource and changed it was still getting the red padlock symbol. When I closed the tab and opened a new one it changed to a green padlock so I guess Chrome caches this information for the lifetime of the tab
Current versions of Chrome will show the mixed content's URL in the error console. Hit CTRL+Shift+J and you'll see text like:
"The page at https://www.fiddler2.com/test/securepageinsecureimage.htm contains insecure content from http://www.fiddler2.com/Eric/images/me.jpg."
I was having the same issue: Chromium showing the non-secure static files, but when everything was http://.
Just closing the current tab and re-opening the page in another new tab worked, so I think this is a Chromium/Chrome bug.
Cheers,
Diogo
Using Chrome, if you open up the Developer Tools (View > Developer > Developer Tools) and bring up the Console and choose to filter to warnings, you'll see a list of offending URLs.
You'll see something like the following if you do have insecure content
The page at https://mysite/ displayed insecure content from http://insecureurl.
For the best experience in finding the culprit, you'll want to start your investigation in a new tab.
It is possible that a non-secure URL is referenced but not accessed (e.g. the codebase for a Flash <object>).
I ran into this problem when Jquery was being executing a a few seconds after page load which added a class containing a non-secure image background. Chrome must continually to check for any non-secure resources to be loaded.
See the code example below. If you had code like this, the green padlock is shown in Chrome for about 5 seconds until the deferred class is applied to the div.
setTimeout(function() {
$("#some-div").addClass("deferred")
}, 5000);
.deferred
{
background: url(http://not-secure.com/not-secure.jpg"
}
Check the source of the page for any external objects (scripts, stylesheets, images, objects) linked using http://... rather than https://... or a relative path. Change the links to use relative paths, or absolute paths without protocol, i.e. href="/path/to/file".
If all that if fine, it could be something included from Javascript. For example, the Google Analytics code uses document.write to add a new script to the page, but it has code to check for HTTPS in case the calling page is secure:
<script type="text/javascript">
var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
</script>
On the release of Chrome version 53 on Windows, Google has changed the trust indications to initiate the circle-i. Afterward, Google has announced a new warning message will be issued when a website is not using HTTPS.
From 2017 January Start, Popular web browser Chrome will begin
labeling HTTP sites as “Not Secure” [Which transmit passwords / ask
for credit card details]
If all your resources are indeed secure, then it is a bug. http://code.google.com/p/chromium/issues/detail?id=72015 . Luckily it was fixed.