Node.js express server with compression not work - node.js

From lighthouse chrome test page:
URL Transfer Size Potential Savings
/three.module.min.js 630.3 KiB 477.3 KiB
I minify but performance still need compression.
...
function shouldCompress (req, res) {
if (req.headers['x-no-compression']) {
// don't compress responses with this request header
return false
}
// fallback to standard filter function
return compression.filter(req, res)
}
hostingHTTP.use(compression({ filter: shouldCompress }))
...
My request :
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9,ru;q=0.8
Cache-Control: no-cache
Connection: keep-alive
Host: maximumroulette.com
Pragma: no-cache
Referer: https://maximumroulette.com/apps/magic/public/module.html
sec-ch-ua: "Not_A Brand";v="99", "Google Chrome";v="109", "Chromium";v="109"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
Sec-Fetch-Dest: script
Sec-Fetch-Mode: no-cors
Sec-Fetch-Site: same-origin
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36
My response:
Accept-Ranges: bytes
Cache-Control: public, max-age=0
Connection: keep-alive
Content-Length: 394590
Content-Type: application/javascript; charset=UTF-8
Date: Mon, 13 Feb 2023 20:39:00 GMT
ETag: W/"6055e-1864c7208a7"
Keep-Alive: timeout=5
Last-Modified: Mon, 13 Feb 2023 20:23:07 GMT
X-Powered-By: Express
Did i need some action onclietn part for decopression ?
If i setup on server res.set('Content-Encoding', 'deflate'); // gzip, deflate, br a get error on chrome:
net::ERR_CONTENT_DECODING_FAILED 200 (OK)
Any suggestion ?

Related

How to creat connection websocket qxbroker in python

how to bypass HTTP/1.1 403 Forbidden in connect to wss://ws2.qxbroker.com/socket.io/EIO=3&transport=websocket, i try change user-agent and try use proxy and add cookis but not work
class WebsocketClient(object):
def __init__(self, api):
websocket.enableTrace(True)
Origin = 'Origin: https://qxbroker.com'
Extensions = 'Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits'
Host = 'Host: ws2.qxbroker.com'
Agent = 'User-Agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36 OPR/94.0.0.0'
self.api = api
self.wss=websocket.WebSocketApp(('wss://ws2.qxbroker.com/socket.io/EIO=3&transport=websocket'), on_message=(self.on_message),
on_error=(self.on_error),
on_close=(self.on_close),
on_open=(self.on_open),
header=[Origin,Extensions,Agent])
request and response header this site protect with cloudflare
--- request header ---
GET /socket.io/?EIO=3&transport=websocket HTTP/1.1
Upgrade: websocket
Host: ws2.qxbroker.com
Sec-WebSocket-Key: 7DgEjWxUp8N8PVY7N7vyDw==
Sec-WebSocket-Version: 13
Connection: Upgrade
Origin: https://qxbroker.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36
-----------------------
--- response header ---
HTTP/1.1 403 Forbidden
Date: Sat, 11 Feb 2023 23:33:11 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: close
Permissions-Policy: accelerometer=(),autoplay=(),camera=(),clipboard-read=(),clipboard-write=(),fullscreen=(),geolocation=(),gyroscope=(),hid=(),interest-cohort=(),magnetometer=(),microphone=(),payment=(),publickey-credentials-get=(),screen-wake-lock=(),serial=(),sync-xhr=(),usb=()
Referrer-Policy: same-origin
X-Frame-Options: SAMEORIGIN
Cache-Control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Thu, 01 Jan 1970 00:00:01 GMT
Set-Cookie: __cf_bm=7TD4hk4.bntJRdP6w9K.AjXF5MsV9LERTJV00jL2Uww-1676158391-0-AZFOKw90ZYdyy4RxX1xJ4jZQMt74+3UkQDZpDrdXE8BxGJULfe8j0T8EZnpUNXr2W3YHd/FxRoO/bPhKA2Dc0E0=; path=/; expires=Sun, 12-Feb-23 00:03:11 GMT; domain=.qxbroker.com; HttpOnly; Secure; SameSite=None
Server-Timing: cf-q-config;dur=6.9999950937927e-06
Server: cloudflare
CF-RAY: 7980e3583b6a0785-MRS

Python POST request to retrieve base64 encode File

Im trying to POST request using Python to retreive a specific File. Since the URL is behind a server with authorized access theres no use posting it here
However the form data contains a field called base64 and lengthy which I cant figure out if its a form data value or base64 encoding of post request
Here are browser parameters
General:
Request URL: http://exampleapi.com/api/Document/Export
Request Method: POST
Status Code: 200 OK
Remote Address: XX.XXX.XXX.XX:XX
Referrer Policy: no-referrer-when-downgrade
Response Headers:
Access-Control-Allow-Origin: http://example.com
Cache-Control: no-cache
Content-Disposition: attachment; filename=location-downloads.xlsx
Content-Length: 7148
Content-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
Date: Tue, 23 Jul 2019 21:00:18 GMT
Expires: -1
Pragma: no-cache
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Request Headers :
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9
Cache-Control: max-age=0
Connection: keep-alive
Content-Length: 10162
Content-Type: application/x-www-form-urlencoded
Cookie: abcConnection=!UA7tkC3iZCmVNGRUyRpDWARVBWk/lY6SZvgxLlaygsQKk+vuwA1NxvhwE9ph4i+3NZlKeepIfuHhUvyQjl68fhhrT9ueqMx/3mBKUDcT
DNT: 1
Host: exampleapi.com
Origin: http://example.com
Referer: http://example.com/
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36
Form Data:
fileName: location-downloads.xlsx
contentType: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
base64: UEsDBAoAAAAAAAh4904AAAAAAAAAAAAAAAAJAAAAZG9jUHJvcHMvUEsDBAoAAAAIAAh490(shortened for simplicity)
Here is what I tried
url='http://example.com'
urllib3.disable_warnings()
headers = {
"Content-Type": "application/x-www-form-urlencoded",
"User-Agent": "Mozilla/5.0",
}
with requests.session() as s:
r=s.get(url,headers={"User-Agent":"Mozilla/5.0"},verify=False)
data=r.content
soup=BeautifulSoup(data,'html.parser')
form_data = {
"fileName":"location-downloads.xlsx",
"contentType":"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
}
r2=s.post('http://exampleapi.com/api/Document/Export',data=json.dumps(form_data,ensure_ascii=True).encode('utf-8'),headers=headers,verify=False)
print(r2.status_code)
Any idea how i should proceed. My status code also shows 500 for the post here

Expressjs Route contains weird characters

What could possibly be the reason for expressjs route to contain the following data? I am expecting it to return JSON data. I am making an ajax call to the server(expressjs) which gives me the below data with weird characters. Is this data gzipped? I have set the headers and contentType as follows:
headers: {"Access-Control-Allow-Origin":"*"}
contentType: 'application/json; charset=utf-8'
�=O�0�b��K�)�%7�܈9���G��%NOU���O'6��k�~6��S.���,��/�wأ%6�K�)��e�
The HTTP response is as follows:
General:
Request URL: http://localhost/expressRoute.js
Request Method: GET
Status Code: 200 OK
Remote Address: [::1]:80
Referrer Policy: no-referrer-when-downgrade
Response Headers:
Accept-Ranges: bytes
Connection: Keep-Alive
Content-Length: 29396
Content-Type: application/javascript
Date: Thu, 22 Nov 2018 00:50:36 GMT
ETag: "72d4-57b124e0c372e"
Keep-Alive: timeout=5, max=100
Last-Modified: Tue, 20 Nov 2018 05:57:12 GMT
Server: Apache/2.4.34 (Win32) OpenSSL/1.1.0i PHP/7.2.10
Request Headers:
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Cache-Control: no-cache
Connection: keep-alive
Host: localhost
Pragma: no-cache
Referer: http://localhost/index.html
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36

JSF-Login-Page in HTTP-Response despite valid JSESSIONID

I think this a HTTP-related problem.
I want to use my (JAX-RS) RESTeasy Service on a (JEE6) JBoss AS 7 Server from an Android Device. The RESTeasy Service is working fine. I am using on the Client-Side the Restlet-Client. This works too - without Security.
I want to use my JAAS-Formbased Security for the Pattern /rest/* in web.xml. So I have to send a HTTP-POST-Request with the Form-Data (j_username and j_password) to /foo/j_security_check.
I get the JSESSIONID from the first Response by the Server:
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Set-Cookie: JSESSIONID=uKUqlkUWdhX2l-FihiWyeSJr.undefined; Path=/foo
Pragma: No-cache
Cache-Control: no-cache
Expires: Thu, 01 Jan 1970 01:00:00 CET
X-Powered-By: JSF/2.0
Content-Type: text/html;charset=utf-8
Content-Length: 1028
Date: Wed, 15 Aug 2012 11:42:59 GMT
For this anonymous session I am authenticating ...
Header:
POST /foo/j_security_check HTTP/1.1
Date: Wed, 15 Aug 2012 11:42:58 GMT
Accept: text/html
Host: 172.24.47.5:8080
User-Agent: Restlet-Framework/2.0.14
Cookie: JSESSIONID=uKUqlkUWdhX2l-FihiWyeSJr.undefined
Content-Length: 62
Content-Language: *
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Content:
j_username=Bob&j_password=a
... and it works: JBoss-Security-TRACE:
2012-08-15 13:22:26,829 TRACE
[org.jboss.security.auth.spi.DatabaseServerLoginModule]
(http-0.0.0.0-0.0.0.0-8080-4) User 'Bob' authenticated, loginOk=true
Now the Problem: In the following request I want to GET the REST-URL (using the Cookie JSESSIONID):
GET /foo/rest/sync/products HTTP/1.1
Date: Wed, 15 Aug 2012 11:42:59 GMT
Accept: application/json
Host: 172.24.47.5:8080
User-Agent: Restlet-Framework/2.0.14
Cookie: JSESSIONID=uKUqlkUWdhX2l-FihiWyeSJr.undefined
Content-Length: 0
But instead of returning the Response with JSON Content, the server is returning the JSF-Login-Page, because it want's me to authenticate again(?):
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Pragma: No-cache
Cache-Control: no-cache
Expires: Thu, 01 Jan 1970 01:00:00 CET
X-Powered-By: JSF/2.0
Content-Type: text/html;charset=utf-8
Content-Length: 936
Date: Wed, 15 Aug 2012 11:42:59 GMT
<?xml version="1.0" encoding="utf-8"?> ... ... ... </html>
If I login with the Browser and then open the REST-URL it works fine. This is the GET-Request by the Browser:
GET http://localhost:8080/foo/rest/sync/products HTTP/1.1
Host: localhost:8080
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/536.5 (KHTML, like Gecko)
Chrome/19.0.1084.56 Safari/536.5
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip,deflate,sdch
Accept-Language: de-DE,de;q=0.8,en-US;q=0.6,en;q=0.4
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: JSESSIONID=royq26yLd7REOz2otiZdTl6j.undefined
Anyone has an idea? I think the problem lays in the last request (GET /foo/rest/sync/products), because in the Browser it works fine.
Thanks

Server headers - 302 temporary redirect

I have just checked the server headers for my website and this is what i got:
1 Server Response: http://www.pjnsports.co.uk
HTTP/1.1 302 Moved Temporarily
Content-Length: 0
Location: /?6690d3e0
I havent set up any 302 redirects - i assume this will be my host that is doing this. Is this normal practice?? will it have a negative affect on search results, site load speed etc? basically should i be going to them and telling them to do something about it?
Cheers
Paul
I'm not getting any 302 / 301 going to that site:
GET / HTTP/1.1
Host: www.pjnsports.co.uk
Connection: keep-alive
Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.597.107 Safari/534.13
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: PHPSESSID=emr10g0gs9srtjccadb4k7t846; language=en; currency=GBP; __utmz=239376578.1300041169.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utma=239376578.1365490247.1300041169.1300041169.1300041169.1; __utmc=239376578; __utmb=239376578.1.10.1300041169
HTTP/1.1 200 OK
Date: Sun, 13 Mar 2011 18:33:41 GMT
Server: Apache
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 6531
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=utf-8

Resources