Related
I found chrome.declarativeNetRequest only supports static rules, What I want is to call some custom methods before actions like redirect/request. I haven't found a solution so far. I'm not sure if I can still do this under the Manifest V3.
There are two usecases for my extension.
Before the redirect, I need to execute custom method.
chrome.webRequest.onBeforeRequest.addListener(
function(requestDetails) {
//
// I can get id from requestDetails.url,
// then do some custom business logic.
//
custom_function(requestDetails.url);
return {redirectUrl:"javascript:"};
},
{urls: [ "url_pattern?id=*" ]},
["blocking"]
);
Before some request, I want add/modify requestHeaders according to the user's browser.
chrome.webRequest.onBeforeSendHeaders.addListener(
function (details) {
details.requestHeaders.push({
"name": "User-Agent",
"value": navigator.userAgent + "version_1.0.0"
});
return {requestHeaders: details.requestHeaders};
},
{
urls: ["*://url_pattern"],
types: ["xmlhttprequest"]
},
["blocking", "requestHeaders"]
);
#wOxxOm Thank you very much for your patient answer !
I prefer to spinner.html. But I have another problem.
I can't set the regexSubstitution to the extension address,
I can use the extensionPath, but the corresponding capture groups doesn't work here.
"regexFilter": "google.com*"
The following are all incorrect:
can't use the corresponding capture groups.
"extensionPath": "/spinner.html?url=\\0"
can't use the extension's address.
"regexSubstitution": "spinner.html?url=\\0"
Is my configuration incorrect?
Adding/deleting headers can only accept static values and it's shown in the official example.
Conditionally adding/deleting/modifying headers based on response headers is tracked in https://crbug.com/1141166.
Nontrivial transformations that exceed the functionality of the actions listed in the documentation naturally cannot be re-implemented.
When https://crbug.com/1262147 is fixed we will be able to define a declarativeNetRequest rule to redirect to a page inside your extension via regexSubstitution or extensionPath and append the original URL as a parameter. This page will act as an interstitial, it will display some kind of UI or a simple progress spinner, process the URL parameters, and redirect the current tab to another URL.
In many cases this approach would introduce flicker and unnecessary visual fuss while the interstitial is displayed shortly, thus frustrating users who will likely abandon using such extensions altogether. Chromium team members who work on extensions seem to think this obscene workaround is acceptable so it's likely they'll roll with it, see also https://crbug.com/1013582.
Use the observational webRequest (without 'blocking' parameter) and chrome.tabs.update to redirect the tab. The downside is that the original request will be sent to the remote server. And this approach obviously won't work for iframes, to redirect those you'll have to inject/declare a content script, to which your webRequest listener would send a message with a frameId parameter.
Keep a visible tab with an html page from your extension, and use the blocking chrome.webRequest inside its scripts. It's a terrible UX, of course, even though endorsed by the Chromium's extensions team, with many extensions using this kludge the user's browsers will have to keep a lot of such tabs open.
P.S. The blocking webRequest will be still available for force-installed extensions via policies, but it's not something most users would be willing to use.
I'm working on a CLI with OCLIF. In one of the commands, I need to simulate a couple of clicks on a web page (using the WebdriverIO framework for that). Before you're able to reach the desired page, there is a redirect to a page with a login prompt. When I use WebdriverIO methods related to alerts such as browser.getAlertText(), browser.sendAlertText() or browser.acceptAlert, I always get the error no such alert.
As an alternative, I tried to get the URL when I am on the page that shows the login prompt. With the URL, I wanted to do something like browser.url(https://<username>:<password>#<url>) to circumvent the prompt. However, browser.url() returns chrome-error://chromewebdata/ as URL when I'm on that page. I guess because the focus is on the prompt and that doesn't have an URL. I also don't know the URL before I land on that page. When being redirected, a query string parameter containing a token is added to the URL that I need.
A screenshot of the prompt:
Is it possible to handle this scenario with WebdriverIO? And if so, how?
You are on the right track, probably there are some fine-tunings that you need to address to get it working.
First off, regarding the chrome-error://chromewebdata errors, quoting Chrome DOCs:
If you see errors with a location like chrome-error://chromewebdata/
in the error stack, these errors are not from the extension or from
your app - they are usually a sign that Chrome was not able to load
your app.
When you see these errors, first check whether Chrome was able to load
your app. Does Chrome say "This site can't be reached" or something
similar? You must start your own server to run your app. Double-check
that your server is running, and that the url and port are configured
correctly.
A lot of words that sum up to: Chrome couldn't load the URL you used inside the browser.url() command.
I tried myself on The Internet - Basic Auth page. It worked like a charm.
URL without basic auth credentials:
URL WITH basic auth credentials:
Code used:
it('Bypass HTTP basic auth', () => {
browser.url('https://admin:admin#the-internet.herokuapp.com/basic_auth');
browser.waitForReadyState('complete');
const banner = $('div.example p').getText().trim();
expect(banner).to.equal('Congratulations! You must have the proper credentials.');
});
What I'd do is manually go through each step, trying to emulate the same flow in the script you're using. From history I can tell you, I dealt with some HTTP web-apps that required a refresh after issuing the basic auth browser.url() call.
Another way to tackle this is to make use of some custom browser profiles (Firefox | Chrome) . I know I wrote a tutorial on it somewhere on SO, but I'm too lazy to find it. I reference a similar post here.
Short story, manually complete the basic auth flow (logging in with credentials) in an incognito window (as to isolate the configurations). Open chrome://version/ in another tab of that session and store the contents of the Profile Path. That folder in going to keep all your sessions & preserve cookies and other browser data.
Lastly, in your currentCapabilities, update the browser-specific options to start the sessions with a custom profile, via the '--user-data-dir=/path/to/your/custom/profile. It should look something like this:
'goog:chromeOptions': {
args: [
'--user-data-dir=/Users/iamdanchiv/Desktop/scoped_dir18256_17319',
],
}
Good luck!
Issue Context:
I am using meteor js for a mobile app.
I have hooked it up with google analytics calls and basically I am using two type of calls:
Screen views
Events
Screen views are just fine, but I'm facing an issue with the events.
When I go to Behavior -> Events -> Screens, in the google analytics dashboard, I can see the URL of every page that has triggered an event under the Screen Name column. My problem is that the page URLs for my login page look something like this:
meteor.local/login?username=*******&password=+++++++&rememberMe=on
Where ******* is an actual username and +++++++ is the corresponding password!
Reason:
Since I have to share this analytics account with multiple people, I do not want this information to be available over here.
Clues:
CLUE 1:
I used to do GET http calls, but I have changed them all to POST and it still has not fixed the issue as I expected it not to pass plain parameters through URL anymore.
CLUE 2:
I've noticed that the default google analytics js framework is working with http and not https. I was wondering if it is calling the analytics server with a GET as well. If so, is there anyway to change that?
CLUE 3:
Here is how I am initiating the GA instance:
(function (i, s, o, g, r, a, m) {
i['GoogleAnalyticsObject'] = r;
i[r] = i[r] || function () {
(i[r].q = i[r].q || []).push(arguments)
}, i[r].l = 1 * new Date();
a = s.createElement(o),
m = s.getElementsByTagName(o)[0];
a.async = 1;
a.src = g;
m.parentNode.insertBefore(a, m)
})(window, document, 'script', '//www.google-analytics.com/analytics.js', 'ga');
ga('create', googleKey, 'auto');
CLUE 4:
I have also noticed that these URLs are getting captured very occasionally. E.g. in the pas 12,500 unique events (about 30,000 total events) it has captured just 9 URLs with the username and password. The remaining 12,491 events have
meteor.local/login
OR
meteor.local/--
OR
localhost/--
as the Screen Name.
CLUE 5:
I have also put 4 "search and replace" global filters on the analytics account to search for this string
meteor.local/.*
and replace it with this one
meteor.local/concealedURI
This does not seem to be working either.
I have added this filter on 4 different fields (Since I still really don't know where the URLs are coming from):
Host Name
Page Title
Referral
Request URI
CLUE 6:
This is how I am calling the GA instance to send the event:
ga('send', 'event', 'button', 'click', eventName);
Okay. So, I had to run a lot of experiments and try out different things to solve this issue.
After trying all the things that I have described in the question, I finally found a way to address this problem.
The main cause of this problem was that I was using a google analytics account set to track an App, to capture the data from an app that was built with meteor js (which basically utilizes cordova).
Using meteor means that my app's screens are actually web pages rendered as a mobile app. It seems like meteor uses URLs to navigate through these screens.
On the other hand, google analytics looks at (and captures) the screen name of an app's page, when an event is triggered from that page. In native apps this screen name will be something similar to "About us", "Contact Us", "Home", etc.
Now since a meteor app is not the same, the screen name returned by meteor is actually the URL of the page that has triggered the event.
This does not have anything to do with the http calls (Whether or not they are GET or POST), because it is the local URL used by meteor for navigating that is being passed down to google analytics and not any http calls.
Solutions
1.
If I had the google analytics account set as a web page tracker, I could have access to "Exclude URL Query Parameters" field and I could potentially exclude username and password as was suggested by #Mike and #PhilipPryde in the comments.
However, I needed to use google analytics set as an app tracker. So, this did not work for me.
Failed
2.
I did put a filter on the whole view in the google analytics and searched for meteor.local/.* and replaced that with hiddenURL. The filters on
Host Name
Page Title
Referral
Request URI
did not work.
But when I put the same filter on
Screen Label
field, it worked.
However, this only looked at the screen names returned by screen view hits and not the event. Thus, this did not actually solve my issue either.
Failed
Finally, I had to do this:
There is a method call on GA instance that lets you set different options up. I ended up using this:
ga('set', 'screenName', 'hiddenURL');
This changed the screen name to "hiddenURL". So, I used this before every event and it worked for me.
My code for sending events to google analytics looked like this:
ga('set', 'screenName', 'hiddenURL');
ga('send', 'event', 'button', 'click', eventName);
PS:
This changes the screen name that was showing up in real-time reports of google analytics to "hiddenURL", whenever someone triggered an event. But, it changes back to a screen name as soon as they go to another page. So, it would not also mess with any of your screen view data either, since it is not being captured as a screen view.
Of course that is because, I pass the screen name to my GA instance every time I send a screen view. So it looks like this:
sendScreenViewToGA = function (screenName) {
ga('send', 'screenview', {
'appName': 'Something',
'screenName': screenName,
'appVersion': x.x
});
}
If I had used the screen name, that is being set on the environment tight now, I would have ended up with all my screen names in analytics set to "hiddenURL".
I really hope this post will help others with same issues and save them some time.
Content Script can be injected programatically or permanently by declaring in Extension manifest file. Programatic injection require host permission, which is generally grant by browser or page action.
In my use case, I want to inject gmail, outlook.com and yahoo mail web site without user action. I can do by declaring all of them manifest, but by doing so require all data access to those account. Some use may want to grant only outlook.com, but not gmail. Programatic injection does not work because I need to know when to inject. Using tabs permission is also require another permission.
Is there any good way to optionally inject web site?
You cannot run code on a site without the appropriate permissions. Fortunately, you can add the host permissions to optional_permissions in the manifest file to declare them optional and still allow the extension to use them.
In response to a user gesture, you can use chrome.permission.request to request additional permissions. This API can only be used in extension pages (background page, popup page, options page, ...). As of Chrome 36.0.1957.0, the required user gesture also carries over from content scripts, so if you want to, you could add a click event listener from a content script and use chrome.runtime.sendMessage to send the request to the background page, which in turn calls chrome.permissions.request.
Optional code execution in tabs
After obtaining the host permissions (optional or mandatory), you have to somehow inject the content script (or CSS style) in the matching pages. There are a few options, in order of my preference:
Use the chrome.declarativeContent.RequestContentScript action to insert a content script in the page. Read the documentation if you want to learn how to use this API.
Use the webNavigation API (e.g. chrome.webNavigation.onCommitted) to detect when the user has navigated to the page, then use chrome.tabs.executeScript to insert the content script in the tab (or chrome.tabs.insertCSS to insert styles).
Use the tabs API (chrome.tabs.onUpdated) to detect that a page might have changed, and insert a content script in the page using chrome.tabs.executeScript.
I strongly recommend option 1, because it was specifically designed for this use case. Note: This API was added in Chrome 38, but only worked with optional permissions since Chrome 39. Despite the "WARNING: This action is still experimental and is not supported on stable builds of Chrome." in the documentation, the API is actually supported on stable. Initially the idea was to wait for a review before publishing the API on stable, but that review never came and so now this API has been working fine for almost two years.
The second and third options are similar. The difference between the two is that using the webNavigation API adds an additional permission warning ("Read your browsing history"). For this warning, you get an API that can efficiently filter the navigations, so the number of chrome.tabs.executeScript calls can be minimized.
If you don't want to put this extra permission warning in your permission dialog, then you could blindly try to inject on every tab. If your extension has the permission, then the injection will succeed. Otherwise, it fails. This doesn't sound very efficient, and it is not... ...on the bright side, this method does not require any additional permissions.
By using either of the latter two methods, your content script must be designed in such a way that it can handle multiple insertions (e.g. with a guard). Inserting in frames is also supported (allFrames:true), but only if your extension is allowed to access the tab's URL (or the frame's URL if frameId is set).
I advise against using declarativeContent APIs because they're deprecated and buggy with CSS, as described by the last comment on https://bugs.chromium.org/p/chromium/issues/detail?id=708115.
Use the new content script registration APIs instead. Here's what you need, in two parts:
Programmatic script injection
There's a new contentScripts.register() API which can programmatically register content scripts and they'll be loaded exactly like content_scripts defined in the manifest:
browser.contentScripts.register({
matches: ['https://your-dynamic-domain.example.com/*'],
js: [{file: 'content.js'}]
});
This API is only available in Firefox but there's a Chrome polyfill you can use. If you're using Manifest v3, there's the native chrome.scripting.registerContentScript which does the same thing but slightly differently.
Acquiring new permissions
By using chrome.permissions.request you can add new domains on which you can inject content scripts. An example would be:
// In a content script or options page
document.querySelector('button').addEventListener('click', () => {
chrome.permissions.request({
origins: ['https://your-dynamic-domain.example.com/*']
}, granted => {
if (granted) {
/* Use contentScripts.register */
}
});
});
And you'll have to add optional_permissions in your manifest.json to allow new origins to be requested:
{
"optional_permissions": [
"*://*/*"
]
}
In Manifest v3 this property was renamed to optional_host_permissions.
I also wrote some tools to further simplify this for you and for the end user, such as
webext-domain-permission-toggle and webext-dynamic-content-scripts. They will automatically register your scripts in the next browser launches and allow the user the remove the new permissions and scripts.
Since the existing answer is now a few years old, optional injection is now much easier and is described here. It says that to inject a new file conditionally, you can use the following code:
// The lines I have commented are in the documentation, but the uncommented
// lines are the important part
//chrome.runtime.onMessage.addListener((message, callback) => {
// if (message == “runContentScript”){
chrome.tabs.executeScript({
file: 'contentScript.js'
});
// }
//});
You will need the Active Tab Permission to do this.
How do you track outbound links for your web site, since the request is logged on the destination server, not yours?
You can add a quick JQuery script to the page that will track external links and can either redirect them to a file on your server that will track the link and then forward to it, or add an ajax request that will submit on click for external links, and track them that way.
See:
http://www.prodevtips.com/2008/08/19/tracking-clicks-with-jquery-and-google-analytics/
https://web.archive.org/web/20090214024330/http://www.justskins.com/development/how-to-track-clicks-on-outgoing-links/132
Method #1: target="_blank", onclick and Google Analytics Events
Format your outgoing links with the following attributes:
outgoing
Define a javascript tracking function (requires google analytics to be loaded already):
function trackOutgoing(el) {
ga('send', 'event', {eventCategory: 'outbound',
eventAction: 'send',
eventLabel: el.getAttribute('href'),
eventValue: 1});
};
Pros:
Does NOT interfere with normal link behavior
Does NOT require redirecting to another url
Cons:
The onclick is not guaranteed to execute (user or browser could terminate the main window)
Method #2: Redirecting with Javascript and Google Analytics Callbacks
Format your outgoing links with the following attributes:
outgoing
Define a javascript tracking function (requires google analytics to be loaded already):
function trackOutgoingAndRedirect(el) {
var url = el.getAttribute('href');
ga('send', 'event', {eventCategory: 'outbound',
eventAction: 'send',
eventLabel: url,
eventValue: 1,
hitCallback: function() { document.location = url; }});
}
Pros:
Does not require target="_blank"
Higher chance of your event being registered with Google Analytics (compared to Method #1)
Cons:
Overrides the default behavior of links with return false;
Cannot open outgoing links in a new window
Method #3: Using a Redirect URL
Format your outgoing links with the following attributes:
outgoing
On your site you will need to implement a redirect script which is beyond the scope of this answer.
Your redirect script would most likely track the outgoing link and then redirect to the provided url.
Pros:
No Javascript required
Does NOT require Google Analytics
Does NOT interfere with the normal link behavior
Cons:
Harder to trigger Google Analytics Events
Links do not link to their original URL. Which may have negative SEO implications.
Add an onclick or onmousedown handler to the anchor tag. You can see many sites doing this, such as Google.
I don't like the redirect as described by Eric Tuttleman, as you unfortunately lose the 'search engine friendliness' of the link.
I handle this on a site I own by adding an onClick to my outgoing links, which fires a function which sends the link URL and a timestamp to my database. I then wrote a backend which retrieves the data, and lets me view it by such categories as 'Most clicked / 24h', 'Most clicked / 1w' etc.
I hope this helps.
On one system I've worked on, we ended up storing redirects in a database table and creating a redirect page that takes an id as an input. On our content pages, we link to the redirect page with an unique id from this table. Once the redirect page looks up the url via the id from the table, it then sends the client a redirect response, sending them to the ending page.
This does give us logging of external links, and as an added bonus, it makes mass changes to external urls a bit easier in some cases.
Some newer options that work without any hacks as explained in https://css-tricks.com/send-an-http-request-on-page-exit/ are Fetch with the keepalive-flag or navigator.sendBeacon.
keepalive is not yet (Aug. 2022) supported by Firefox (Can I Use), but navigator.sendBeacon works in all modern browsers (Can I Use).
// normal fetch, not guaranteed to work
someLink.addEventListener('click', function(event){
fetch('http://www.testing.local/?origin=classic-fetch');
});
// fetch + keep alive (not working in Firefox as of 103, Aug. 2022)
someLink.addEventListener('click', function(event){
fetch('http://www.testing.local/?origin=fetch-keep-alive', {
keepalive: true
});
});
// navigator.sendBeacon (all modern browsers)
someLink.addEventListener('click', function(event){
navigator.sendBeacon('http://www.testing.local/?origin=beacon');
});