Why does Google's Lighthouse bot (when run from Google's servers) not use gzip compression for loading from my site? - bots

I checked this behavior of Google's Lighthouse bot and it seems consistent across several sites I can check. I will use one page as example:
https://groupprops.subwiki.org/wiki/Groups_of_order_16
When I load this webpage on Chrome, the main page HTML loads as gzipped.
In desktop, when I look up the performance entry using window.performance.getEntriesByName I see a transferSize of 26866 and a decodedBodySize of 145861.
The corresponding numbers in mobile are transferSize of 24191 and decodedBodySize of 133739.
When I run Lighthouse from within Chrome Dev Tools, I similarly see a transferSize of 26866 and a decodedBodySize of 145861 for desktop , and transferSize of 24191 and decodedBodySize of 133739 for mobile emulation.
However, when I visit https://developers.google.com/speed/pagespeed/insights/?url=https%3A%2F%2Fgroupprops.subwiki.org%2Fwiki%2FGroups_of_order_16 to trigger a Lighthouse score calculation for the page, the Google bot that visits the page does not request them gzipped. In fact the transferSize for mobile emulation is 134337, a little higher than the decodedBodySize due to headers. (To capture this information I logged the data using JavaScript running on the page and sent it back to a server).
I confirmed across a diverse range of sites with different setups that Lighthouse bot loads of pages made through the PageSpeed Insights UI (https://developers.google.com/speed/pagespeed/insights/), or the corresponding API, do not use gzip compression even when it's enabled on the server and used by all modern clients.
Ideas?

Related

Is it possible to set cookies to Page Speed Insights tests?

When using WebPageTest it is possible to use custom scripts to set cookies. Is it possible to do the same on PageSpeed Insights or lighthouse?
For example in the European Union you may want to set cookies to disable the cookie consent popup which will not appear for repeat users who have already consented to the website's cookie policy.
Page Speed Insights
Page Speed Insights (PSI) is designed for simulating a first time user to the site, so your cookies popup would always show.
It does not cache cookies or anything else. There is no way to send cookies to PSI.
Lighthouse in the browser
If you are using Lighthouse in the browser simply untick the "Clear Storage" option in the top bar (you may have to click the gear icon top right to reveal this) and at that point run the page twice (once to set the cache), cookies will be stored and you will record a run with cached images, files etc.
If you wanted to create a run where cookies are stored but nothing else is cached you could go to the application tab in developer tools and remove everything other than the cookies.
However as this is not indicative of a real world scenario I do not see the benefit and is only included as it answers the original question.
Lighthouse CLI + Puppeteer
Your final option for complete control would be to run Lighthouse CLI on your own machine.
The source code for Lighthouse is available on GitHub, takes a short while to set up, installing chromium etc. but it is worth it.
At that point you have more control, if you partner it with Puppeteer then you have all the control to set individual cookies etc.
However it does appear that you should be able to pass cookies directly to Lighthouse via config according to this thread, but it might not work and ultimately ends up recommending puppeteer. Perhaps that is a feature that will get improved upon soon.

What could cause err_connection_reset one one page, and only on mobile devices?

We have a ColdFusion Server running on top of IIS. All of our pages can be viewed in desktop or mobile mode, which just changes some CSS around but shows essentially the same page. We've run into an odd situation where on one particular page of the site:
It works fine on a PC viewing the page in desktop mode
It works fine on a PC viewing the page in mobile mode
It returns a err_connection_reset on any mobile device viewing the page in mobile mode
What could cause one particular page on a website to show up fine in desktop browsers but return an err_connection_reset on mobile browsers.
This page is not large, and is made up mostly of common components that are used on other pages which are working fine. No error appears in the server logs (our own, ColdFusions, or IIS').
Google keeps returning me pages about problems with Chrome's data saver mode on pages with 302 redirections. However, this page does not redirect, and we've experienced this on Chrome, Safari, and the Android browser, on both Android and iOS devices.

Can we assign a responsive web site to act like a Progressive Web Application?

if we had a responsive web site with HTML, Css and Nodejs for it's back-end, can we make it to act like PWA application?
- I know the difference will the Service-worker file, but i wanna to find out is it the exact difference?
Here is the difference between PWA(Progressive Web Application) and RWD(Responsive Web Design):
PWA
Users access the website through any browser, and on a supported
browser (Chrome, Firefox, Safari, and even Edge!) it asks the user if
they want to install the web app to their home screen. This lets users
download and the store the website on their devices (which then gets
updated with Service Workers in the background).
RWD
Responsive web design (RWD) basically refers to the practice of
designing a website that can be accessed with any technology you might
use to build a site. RWD allows the website to be fully functional
(and look good) no matter what size screen the user has in front of
them. It has been around since 2001, when Audi launched the first
documented responsive site.
Therefore, PWAs and RWD are not two independent ways to create a website — in fact, PWAs will almost certainly utilize RWD.
Here's some tips:
The best solution is to choose the best solution for your particular site. If you’re looking to maximize those app-like features (like easy home screen app access, offline use, and high-quality, full screen functionality), you may want to choose a PWA.
If you’re looking to maximize site speed and accessibility for the most users, RWD might be the better choice.
If you're asking if you can make a PWA without a service worker then i believe you can't since it (service worker) and the manifest.json file are both required to save an application to the homescreen.
from the google documentation:
In order for a user to be able to install your Progressive Web App, it needs to meet the following criteria:
The web app is not already installed.
and prefer_related_applications is not true.
Meets a user engagement heuristic (currently, the user has interacted with the domain for at least 30 seconds)
Includes a web app manifest that includes:
short_name or name
icons must include a 192px and a 512px sized icons
start_url
display must be one of: fullscreen, standalone, or minimal-ui
Served over HTTPS (required for service workers)
Has registered a service worker with a fetch event handler

How to build web application that run in WeChat in app browsers?

I am trying to build a simple web application, which capture users photo and sent it my custom server there by connected to some other business use-case. My web page uses HTML's file input control to launch native camera or gallery pick up option.
var input = document.createElement('input');
input.setAttribute('accept', 'image/*');
input.setAttribute('capture', 'camera');
input.setAttribute('type', 'file');
input.click();
This web app, I placed in local webserver with a name "PhotoLocker" and testing with url like
https://localhost(ipaddress to access via mobile browser)/PhotoLocker/index.html
This link is working fine both on desktop and mobile chrome browsers and am able to debug any issues. Where as same link, I try to access from WeChat browser (just opening above link from chat window), it is not at all opening my application in WeChat in app browser.
After googling, I found that https URL scheme is not supported by WeChat. is it True? When I paste the same app url as weixin://ipaddress/PhotoLocker/index.html, I am able to see my web app home page but it is not working as expected.
My Question is - how to debug my webpage opened in WeChat browser? Do I need a official WeChat Dev account even to develop and test sample apps?
Additional Info :
I am able to debug webpage from WeChat web devloper tool as mentioned in below link. But, unable to debug mobile wechat page in this tool. It is always opening chrome dev tools.
https://mp.weixin.qq.com/wiki?action=doc&id=mp1455784140&t=0.06697335132505233#1
I am a frontend developer in China, Chinese. Forgive my English for any mistake, misunderstanding I could make. Some links (dev docs mostly) below contain sites complete in Chinese, because I can not find corresponding English ones for now.
how to debug my webpage opened in WeChat browser?
Tencent provide an IDE for developing regular web interface and WeChat-mini-program, with which developer can directly interact with:
JSSDK (basically a special weixin
Object lives only in in-WeChat-browser);
API provided in WeChat-mini-program.
If you download that IDE:
First it will ask you is to use you WeChat to scan the QRCode, then confirm login with your WeChat account;
Next it will show up two square button (image below), left one is for WeChat-mini-program, the right one is for you to testing regular web page.
Click the right blue one, then you can find your way out, it's just something built top on project Blink.
As you can see the part of debugging regular webpage in WeChat IDE is no more than a simulator (in the WeChat-mini-program part, developer can write code directly in it), and in my experience it does have bugs, you will still need to test code in real machine.
For that Tencent provide another tool called vConsole, tutorials here, with it you can do following things directly in in-WeChat-browsers:
View console logs;
View network requests;
View document elements;
View Cookies and localStorages;
Execute JS command manually
and so on
Do I need a official WeChat Dev account even to develop and test sample apps?
Depends.
You may know the Official Account inside WeChat, with webpages directly opened in any context inside in-WeChat-browser, it will have the ability to interact the weixin Object, or have some API like login with WeChat, pay with WeChat Pay:
API like close current in-WeChat-browser, hide-share-button will not required anything special, you don't need to register any Official Account;
But if you want yo do the Pay, Login thing, you need an Official Account and pay for the ability every year (not sure about this outside China).
The localhost problem you faced
I don't have my working machine with me now so I can not test. Regularly I can proxy localhost with Charles then debugging in WeChat, but never do the https, I will try it later.
All the information got regarding how to debug webpage opened in wechat browser redirects to how to see log or ajax/netwrok calls analysis.
Even with WeChat web devloper tool as mentioned in below link, I am unable to debug mobile wechat page in this tool. It is always opening chrome dev tools.
https://mp.weixin.qq.com/wiki?action=doc&id=mp1455784140&t=0.06697335132505233#1.
Hence further analyzed remote mobile webpage debugging and found that there is no way to put break points, watch, expressions and all just like in chrome dev tools is not possible.
As a work around - you are able to debug code, when you simulate page in dev tools but no way to debug webpage in mobile device.
Same webpage when tried to do remote debugging as per WeChat web devloper tool documentation. here we can only see console logs and network calls.

Developing a web site that can be accessed through mobile phone applications

I am developing a site that is tested only in Firefox and IE. Now I need to make the site accessible from mobile also.
So I need to know whether I need to calculate the time needed to shift the site. Is this created as a new application or the same application is modified?
When accessing stackoverflow.com from my mobile the design is entire changed. How is this done? Is it a separate application?
Thanks
Whether or not you need to create a new application for mobile depends on the site you have. The website at my workplace could not possibly fit on a mobile phone screen (too many frames), but other sites that have a more adjustment-friendly layout might just need a little tweak.
I would test your site on a mobile browser emulator, there are a bunch of them listed on this site.
Also, you might consider switching your firefox's user agent (here) so you can browse popular site's mobile versions, along with the source they used to lay it out.
Usually different CSS templates chosen using UA string matching. My phone has a fairly fully enabled web browser on it, so I get the whole of stackoverflow the same.
Some phone browser may also "mobile optimise" the layout, or in the case of opera mini, it does it on opera's proxy server and then sends modified data to the phone.
Javascript support is more of a problem, expect it to be minimal in most cases, although it is getting better.

Resources