Is there any possibility to cheat the Google PageSpeed Insights measurement?
I'm especially concerned by increased Cumulative Layout Shift (CLS) from Cookie Consent Banners. If I'd be able to detect that my page's speed is measured, I could deactivate the cookie consent banner for that purpose.
Second question is more of a legal point of view. If it would be possible to achieve the aforementioned, would Google impose a punishment on me, if they would detect my little cheat?
Yes, but...
You can detect Lighthouse by looking at the User-Agent, it will contain Chrome-Lighthouse, e.g. it might be Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4420.0 Safari/537.36 Chrome-Lighthouse.
Here is the but: it's not a good idea. You can trick the lab test by not using some features on your site when a test is being run, but Google isn't using the lab test for performance judgements- They are using the Chrome User Experience Report (CrUX), where they collect anonymous real world data from real people using Chrome.
If you were to disable your consent banner during tests, you'd hide the problem from yourself, but it would still affect your visitors, and its effect would be counted in Google's statistics. All you'd do is create a situation where you don't understand why Google judges your site differently in field data vs lab data.
Related
When running the pages of this website through Google Pagespeed Insights tool, I receive drastically different "Lab Data" (Time to Interactive, First Contentful Paint, Speed Index) speeds when comparing Mobile and Desktop. Desktop tends to receive values under 2 seconds, and as a result, the Pagespeed Insights score is generally in the 80s or 90s on each page. The Mobile score, however, suggests the page load speed is much slower, upwards to 10 seconds. As you may guess, I cannot reproduce anything close to these loading times on mobile. The mobile and desktop experience do not differ dramatically with the primary differences being styling using CSS media queries. Would love any help understanding why these values are so dramatically different!
Images for reference:
Desktop metrics
Mobile metrics
Page Speed Insights uses simulated CPU and Connection throttling to simulate mobile conditions people may experience when displaying your mobile score (no throttling exists on Desktop score).
Not everyone has a flagship phone (far from it) so they slow the CPU speed of their server by a factor of 4 to simulate the slower CPU speeds of mid and low end phones.
Similarly they also simulate a slow 4G connection to account for when people are out and about / have no WiFi connection. SO they add additional latency and slow the upload and download speeds to reflect this.
This is why you see such big differences on your site score between mobile and desktop.
If you want to simulate a similar speed yourself you can open developer tools in Google Chrome -> Network -> Look for the drop down that says "online" and change it to "Fast 3G".
Now reload your page and you can see the effects of additional latency and slower download speeds on your waterfall.
According to my analysis, this is due to the images on this page. However, Google PageSpeed Insights is very sensitive to mobile scores than desktop scores, so the stark difference between mobile and desktop scores is natural for this tool.
Try compressing the image first (you can use tinypng.com or other online tools), then use lazyload for image.
I've got different results running Google Page Speed Insights (mobile) from Chrome Dev Tools and Google Page Speed Insights page.
When I run Audits Performance (mobile, 3G) from Chrome Dev Tools I get higer score than "official page".
Running from Chrome dev tools it says that I've implemented some optimization but running the test from Google Page Speed Insights page it suggests that optimization.
I've tried test in different timing but the score on Google Page Speed Insights are always lower than Chrome dev tools.
I've implemented some optimization like "defer images not in view" with a lazy loading, I've deferred the css loading but only Page Speed in Chrome dev tools recognize this optimizations.
Can someone help me with this?
I've been having the same issue and I guess the reply here is in this guide at the following FAQ:
Why do the field data and lab data contradict each other? The Field data says the URL is slow, but the Lab data says the URL is fast!
The field data is a historical report about how a particular URL has performed, and represents anonymized performance data from users in the real-world on a variety of devices and network conditions. The lab data is based on a simulated load of a page on a single device and fixed set of network conditions. As a result, the values may differ.
I had this question for an interview.
Can you tell me how a server know which tab or browser I am using for a specific website?
There is a header called User-Agent that is sent to the server when a user makes a request from the browser. It looks like this:
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:12.0) Gecko/20100101 Firefox/12.0
It clearly tells you that the browser is mozilla, although there is some extra information for historical reasons that you souldn't worry about.
A header is a type of meta information that the users doesn't see but helps the server communicate with the client by sharing this kind of information. So you are able to serve different pages to different pages if the client is using an outdated browser.
You can't know what tab the client is using.
See this Wikipedia article.
In computing, a user agent is software (a software agent) that is
acting on behalf of a user. One common use of the term refers to a web
browser telling a web site information about the browser and operating
system. This allows the web site to customize content for the
capabilities of a particular device, but also raises privacy issues.
There are other uses of the term "user agent". For example, an email
reader is a mail user agent. In many cases, a user agent acts as a
client in a network protocol used in communications within a
client–server distributed computing system. In particular, the
Hypertext Transfer Protocol (HTTP) identifies the client software
originating the request, using a user-agent header, even when the
client is not operated by a user. The Session Initiation Protocol
(SIP) protocol (based on HTTP) followed this usage. In the SIP, the
term user agent refers to both end points of a communications session.
User agent identification When a software agent operates in a network
protocol, it often identifies itself, its application type, operating
system, software vendor, or software revision, by submitting a
characteristic identification string to its operating peer. In HTTP,
SIP, and NNTP protocols, this identification is transmitted in a
header field User-Agent. Bots, such as Web crawlers, often also
include a URL and/or e-mail address so that the Webmaster can contact
the operator of the bot.
Use in HTTP In HTTP, the User-Agent string is often used for content
negotiation, where the origin server selects suitable content or
operating parameters for the response. For example, the User-Agent
string might be used by a web server to choose variants based on the
known capabilities of a particular version of client software. The
concept of content tailoring is built into the HTTP standard in RFC
1945 "for the sake of tailoring responses to avoid particular user
agent limitations.”
The User-Agent string is one of the criteria by which Web crawlers may
be excluded from accessing certain parts of a Web site using the
Robots Exclusion Standard (robots.txt file).
As with many other HTTP request headers, the information in the
"User-Agent" string contributes to the information that the client
sends to the server, since the string can vary considerably from user
to user.
Format for human-operated web browsers The User-Agent string format is
currently specified by section 5.5.3 of HTTP/1.1 Semantics and
Content. The format of the User-Agent string in HTTP is a list of
product tokens (keywords) with optional comments. For example, if a
user's product were called WikiBrowser, their user agent string might
be WikiBrowser/1.0 Gecko/1.0. The "most important" product component
is listed first.
The parts of this string are as follows:
product name and version (WikiBrowser/1.0) layout engine and version
(Gecko/1.0) During the first browser war, many web servers were
configured to only send web pages that required advanced features,
including frames, to clients that were identified as some version of
Mozilla. Other browsers were considered to be older products such as
Mosaic, Cello or Samba and would be sent a bare bones HTML document.
For this reason, most Web browsers use a User-Agent string value as
follows:
Mozilla/[version] ([system and browser information]) [platform]
([platform details]) [extensions]. For example, Safari on the iPad has
used the following:
Mozilla/5.0 (iPad; U; CPU OS 3_2_1 like Mac OS X; en-us)
AppleWebKit/531.21.10 (KHTML, like Gecko) Mobile/7B405 The components
of this string are as follows:
Mozilla/5.0: Previously used to indicate compatibility with the
Mozilla rendering engine. (iPad; U; CPU OS 3_2_1 like Mac OS X;
en-us): Details of the system in which the browser is running.
AppleWebKit/531.21.10: The platform the browser uses. (KHTML, like
Gecko): Browser platform details. Mobile/7B405: This is used by the
browser to indicate specific enhancements that are available directly
in the browser or through third parties. An example of this is
Microsoft Live Meeting which registers an extension so that the Live
Meeting service knows if the software is already installed, which
means it can provide a streamlined experience to joining meetings.
Before migrating to the Chromium code base, Opera was the most widely
used web browser that did not have the User-Agent string with
"Mozilla" (instead beginning it with "Opera"). Since July 15, 2013,
Opera's User-Agent string begins with "Mozilla/5.0" and, to avoid
encountering legacy server rules, no longer includes the word "Opera"
(instead using the string "OPR" to denote the Opera version).
Format for automated agents (bots) Automated web crawling tools can
use a simplified form, where an important field is contact information
in case of problems. By convention the word "bot" is included in the
name of the agent[citation needed]. For example:
Googlebot/2.1 (+http://www.google.com/bot.html) Automated agents are
expected to follow rules in a special file called "robots.txt".
User agent spoofing The popularity of various Web browser products has
varied throughout the Web's history, and this has influenced the
design of Web sites in such a way that Web sites are sometimes
designed to work well only with particular browsers, rather than
according to uniform standards by the World Wide Web Consortium (W3C)
or the Internet Engineering Task Force (IETF). Web sites often include
code to detect browser version to adjust the page design sent
according to the user agent string received. This may mean that
less-popular browsers are not sent complex content (even though they
might be able to deal with it correctly) or, in extreme cases, refused
all content. Thus, various browsers have a feature to cloak or spoof
their identification to force certain server-side content. For
example, the Android browser identifies itself as Safari (among other
things) in order to aid compatibility.
Other HTTP client programs, like download managers and offline
browsers, often have the ability to change the user agent string.
Spam bots and Web scrapers often use fake user agents.
At times it has been popular among Web developers to initiate Viewable
With Any Browser campaigns, encouraging developers to design Web pages
that work equally well with any browser.
A result of user agent spoofing may be that collected statistics of
Web browser usage are inaccurate.
User agent sniffing Main article: Browser sniffing The term user agent
sniffing refers to the practice of Web sites showing different content
when viewed with a certain user agent. On the Internet, this will
result in a different site being shown when browsing the page with a
specific browser. One example of this is Microsoft Exchange Server
2003's Outlook Web Access feature. When viewed with Internet Explorer
6 or newer, more functionality is displayed compared to the same page
in any other browsers. User agent sniffing is now considered poor
practice, since it encourages browser-specific design and penalizes
new browsers with unrecognized user agent identifications. Instead,
the W3C recommends creating HTML markup that is standard, allowing
correct rendering in as many browsers as possible, and to test for
specific browser features rather than particular browser versions or
brands.
Web sites specifically targeted towards mobile phones, like NTT
DoCoMo's I-Mode or Vodafone's Vodafone Live! portals, often rely
heavily on user agent sniffing, since mobile browsers often differ
greatly from each other. Many developments in mobile browsing have
been made in the last few years,[when?] while many older phones that
do not possess these new technologies are still heavily used.
Therefore, mobile Web portals will often generate completely different
markup code depending on the mobile phone used to browse them. These
differences can be small, e.g., resizing of certain images to fit
smaller screens, or quite extensive, e.g., rendering of the page in
WML instead of XHTML.
Encryption strength notations Web browsers created in the United
States, such as Netscape Navigator and Internet Explorer, previously
used the letters U, I, and N to specify the encryption strength in the
user agent string. Until 1996, when the United States government
disallowed encryption with keys longer than 40 bits to be exported,
vendors shipped various browser versions with different encryption
strengths. "U" stands for "USA" (for the version with 128-bit
encryption), "I" stands for "International" – the browser has 40-bit
encryption and can be used anywhere in the world – and "N" stands (de
facto) for "None" (no encryption). Following the lifting of export
restrictions, most vendors supported 256-bit encryption.
Are there any good websites which feature how internet users use the internet. Which features stats like if they use cookies, browser type and version, the OS they use and a resolution? Or is the information used to design/build websites a standard, or do web designers/developers work to what they assume would be the most common and develop for that?
I suppose what I am asking is what standards do they work to? Industry standard, current and forthcoming use or is it just assumed?
http://gs.statcounter.com/ (not the statistic: drop down)
or
http://www.w3counter.com/globalstats.php
or google search for more
We are beginning to go down the path of mobile browser support for an enterprise e-commerce webapp (Java/Servlet based). Of course there are many decisions to be made, but it seems to me the cornerstone is to be able to reliably detect mobile browsers, and make decisions on the content to be returned accordingly. Is there a standard way to make this determination (quickly) based on the http request, and ideally glean more information about the given browser and device making the request (screen size, html capabilities, etc?).
I would also appreciate any supplemental information that would be of use from someone who has gone down this path of taking an existing large scale enterprise webapp and architect-ing out mobile browser support from the development side.
[edit] I certainly understand the request header and the information about a database of standard user agents is a great help. For those talking about 'other' request header properties, if you could include similar standardized name / resource of values that would be a big help.
[edit] Several users have proposed solutions that involve a call over the wire to some web service that will do the detection. While I'm sure this works, it is not a good solution for an enterprise e-commerce site for two reasons: 1) speed. A call over the wire for every page request to a third party would have huge performance implications. 2) dependency/legal. We'd tie our website response time and key functionality to their service, which is horrible for legal and risk reasons.
Wouldn't the standard way be to check the user agent? Here's a database of user agents you can use to detect mobile browsers.
#David's answer mentioned using WURFL -- which is probably your best option. Be forewarned, however, the success rate is usually around 60% (from mine and other's experience). With carriers changing UA's constantly and the amount of device profiles that exist (60,000+ ?), there's no bulletproof way to get all the right data you want.
Just a bit of warning before relying heavily on a device DB. I would try to keep the user's options open by allowing them to change session options in case i've guessed wrong.
You can use Modernizer to detect browser abilities
After days of searching for the right way of detecting a mobile device I've decided to keep it simple [ stupid ] and i shall put a 'Mobile device site' button on my index page.... it's only one click away!!
While you could detect a mobile browser through it's user agent the browser war on the PC platform has shown that sniffing user agents isn't really such a good thing to do.
What ideally should be done is that specific styles should be applied based on media type or that a different answer should be sent based on a header other than the user agent - such as the Accept-header which tells which kind of content that the browser prefers.
Right now it might be enough to code a site that works with the iPhone and with Opera through browser sniffing - but Googles Android is coming any minute now and there are many other mobile phones that will have browser functionality close to the iPhone's in the near future and it would be a waste to develop a mobile website that didn't support those devices as good as possibel from scratch.
This article (and its follow-up) seems nice.
Detect Mobile Browsers - snippets in various programming languages.
The following light weight Apache configuration does a pretty good job and remembers user preference if they prefer the PC version
<VirtualHost (your-address-binding)>
(your-virtual-host-configuration)
RewriteEngine On
RewriteCond %{QUERY_STRING} !ui=pc
RewriteCond %{HTTP_COOKIE} !ui=pc
RewriteCond %{HTTP_USER_AGENT} "^.*(iphone|ipod|ipad|android|symbian|nokia|blackberry| rim |opera mini|opera mobi|windows ce|windows phone|up\.browser|netfront|palm-|palm os|pre\/|palmsource|avantogo|webos|hiptop|iris|kddi|kindle|lg-|lge|mot-|motorola|nintendo ds|nitro|playstation portable|samsung|sanyo|sprint|sonyericsson|symbian).*$" [NC,OR]
RewriteCond %{HTTP_USER_AGENT} "^(alcatel|audiovox|bird|coral|cricket|docomo|edl|huawei|htc|gt-|lava|lct|lg|lynx|mobile|lenovo|maui|micromax|mot|myphone|nec|nexian|nook|pantech|pg|polaris|ppc|sch|sec|spice|tianyu|ustarcom|utstarcom|videocon|vodafone|winwap|zte).*$" [NC]
RewriteRule /(.*) http://bemoko.com/$1 [L]
RewriteCond %{QUERY_STRING} "ui=pc"
RewriteRule ^/ - [CO=ui:pc:(your-cookie-domain):86400:/]
RewriteCond %{QUERY_STRING} "ui=default"
RewriteRule ^/ - [CO=ui:default:(your-cookie-domain):86400:/]
</VirtualHost>
More background on this # http://bemoko.com/training.team/help/team/pc-to-mobile-redirect
I propose a free detection system which is based on uaprof and user agent:
http://www.mobilemultimedia.be
UAprof should be the primary key for detection when it's available as there are usually multiple user agents for the same uaprof. If you want to manage this on your own, you should then go for Wurfl because you can download the entire database and manage it locally by yourself.
When I had a similar need recently, I found this code that uses HTTP_X_WAP_PROFILE, HTTP_ACCEPT, and HTTP_USER_AGENT to identify a browser as mobile or non-mobile. It's PHP but could be converted fairly easily into whatever you need (I implemented it in VBScript for classic ASP).
Ironically, it turned out that I didn't end up using the code because we decided to provide specific URLs for mobile and non-mobile users, but it certainly worked when I was testing it ...
You will get most of the information like browser, device, accepted languages, accepted formats etc from the request header. The user agent mentioned above is part of the request header.
OK, here is a very simple answer - how about letting the user decide? on your login to your ap, provide a link to the mobile site. on the mobile site, provide a link "back to the main site" - try www.fazolis.com on your mobile device - they do a good job of this.
then, on the link to the mobile site from the browser site, register their "vote" and their user agent. You can build your own reliable list of YOUR clients who want the mobile site. Use this married to specs on screen size for these mobile devices, and you can build some pretty good logic for a satisfactory user experience. I would NEVER post out to a network source for something as elementary as this.
Oh and on your "mobile site" - if you write your ap semantically well, then you should be able to present a single site for both mobile and browser vs. having to write two separate page sets. Just something to think about - this is worth the extra thought and effort to save time later.
I can't see it posted on here, but another option I am looking into currently is www.detectmobilebrowser.com
The easiest way is to create an array with regular tags associated with mobile browsers. At least most mobile user agents must have the word mobile, mini, nokia, java ME, android, iphone, mobile OS, etc. If any is matched with the user agent, using php strpos, print a mobile button on top of the page. Leave the user to choose. I love full site cos my mobile browser gives me the same experience, except that I need to zoom or scroll most of the times.
You will have to check the user agent string with a previously defined list, like this one
you can use a webservice to detect mobile browsing like handsetdetection.com.
The fact is that just relying on the useragent is not good enough to detect mobile browsers.
Sure, years ago you could search it for certain strings and guess that it was a Nokia or something, but now there are so many phones out there, and so many that pretend to be things that they are not that something more sophisticated is needed.
I found a great site at link textwhich is based on the same solution that MTV use for all their mobile web sites. It is REALLY good as it has a device independent markup language but more importantly they offer a webservice call for isMobileDevice().
Just look in the manual then 'how it works'.
I've been using it for my customers sites and have yet to find a mobile browser that it doesn't detect accurately. Totally blinding!
Just ran across Device and feature detection on the mobile web with these contents:
Using device and feature detection to improve user experience on the mobile web
Introduction to device detection
Approaches to mobile site design
Do nothing
Providing a generic mobile site
Designing with mobile and adaptation in mind
Content adaptation and device grouping strategies
Device grouping
Content adaptation
Minimising the need for adaptation in the first place
Common approaches to device detection
Server side adaptation
Client-side adaptation
Server-side User Agent (UA) and header lookup
Server-side UA string combined with device database lookup
Server-side User Agent Profiles (UAProf) detection
Detection based on JavaScript technology
CSS media types
CSS media queries
Additional best practices
Redirection + manual link
Landing page + manual link
Downloadable sample page
you can use WURFL APIs to detect device type
http://wurfl.sourceforge.net/wurfl_schema.php
or Modernizer to detect browser abilities