Not receiving cookie that was previously set - node.js

I have a very mysterious puzzle here. I monitor user activity, and record their actions to user profiles, simple. I use a cookie, as the user's id. If I've never seen the user before, I issue a random value as a cookie. After that, if/when I see that guy again, I rewrite their cookie value on all subsequent visits. Pretty standard stuff so far. Mysteriously, after a few requests, very rarely, the inbound cookie is just not there as a part of the request. Here is a truncated view of my access logs for a given user:
2015-02-19 12:14:06 ip:108.45.172.178 cookie:tx7Xd8e8InQNyceDFPgWHS1424376343 /path1 "othercookie=nk1bwo:fxx-2gop9,nk1btk:fxx-2gop8; userid=tx7Xd8e8InQNyceDFPgWHS1424376343; HASESSION=8888; AWSELB=<LONGSTRING>"
2015-02-19 12:14:07 ip:108.45.172.178 cookie:tx7Xd8e8InQNyceDFPgWHS1424376343 /path2 "othercookie=nk1bwo:fxx-2gop9,nk1btk:fxx-2gop8; userid=tx7Xd8e8InQNyceDFPgWHS1424376343; HASESSION=8888; AWSELB=<LONGSTRING>"
2015-02-19 12:14:07 ip:108.45.172.178 cookie:tx7Xd8e8InQNyceDFPgWHS1424376343 /path3 "othercookie=nk1bwo:fxx-2gop9,nk1btk:fxx-2gop8; userid=tx7Xd8e8InQNyceDFPgWHS1424376343; HASESSION=8888; AWSELB=<LONGSTRING>"
2015-02-19 12:14:08 ip:108.45.172.178 cookie:ybJRVsoDVxzJJ6SsdaaZSS1424376845 /path4 "HASESSION=8888; AWSELB=<LONGSTRING>"
2015-02-19 12:14:09 ip:108.45.172.178 cookie:ybJRVsoDVxzJJ6SsdaaZSS1424376845 /path5 "HASESSION=8888; AWSELB=<LONGSTRING>; userid=ybJRVsoDVxzJJ6SsdaaZSS1424376845"
2015-02-19 12:14:10 ip:108.45.172.178 cookie:ybJRVsoDVxzJJ6SsdaaZSS1424376845 /path6 "HASESSION=8888; AWSELB=<LONGSTRING>; userid=ybJRVsoDVxzJJ6SsdaaZSS1424376845"
2015-02-19 12:14:11 ip:108.45.172.178 cookie:ybJRVsoDVxzJJ6SsdaaZSS1424376845 /path7 "HASESSION=8888; AWSELB=<LONGSTRING>; userid=ybJRVsoDVxzJJ6SsdaaZSS1424376845"
2015-02-19 12:14:12 ip:108.45.172.178 cookie:ybJRVsoDVxzJJ6SsdaaZSS1424376845 /path8 "HASESSION=8888; AWSELB=<LONGSTRING>; userid=ybJRVsoDVxzJJ6SsdaaZSS1424376845"
In order, you can see a timestamp, then ip, then the cookie value I'm using for that request, then the obfuscated path (lots of ugly irrelevant GET params), and then finally a dump of the inbound cookies on the request.
On the first request, this user has a userid cookie set, along with another irrelevant cookie, a cookie set by HAProxy for sticky sessions, and a cookie set by AWS ELB (138 characters long).
Then go to #4, I still have the haproxy cookie, and the awselb cookie, but the userid and the othercookie aren't part of that request? Where did they go? Consequently, I generate a new userid value, write it as part of the response, and that is subsequently received for all further requests. What's going on?
Here's another log showing what I'm writing to the header on outgoing responses for a different guy. I'm logging this immediately before my res.setHeader('Set-Cookie', cookie_list) line:
2015-02-18 17:47:47 ip:66.56.52.46 cookiessetting: ["userid=HxfmJNtK7rSXUB1vOnGDkC1424310323; Domain=.mydomain.net; Path=/; Expires=Sat, 18 Feb 2017 01:47:47 GMT"]
2015-02-18 17:47:48 ip:66.56.52.46 cookiessetting: ["userid=HxfmJNtK7rSXUB1vOnGDkC1424310323; Domain=.mydomain.net; Path=/; Expires=Sat, 18 Feb 2017 01:47:48 GMT"]
2015-02-18 17:47:49 ip:66.56.52.46 cookiessetting: ["userid=HxfmJNtK7rSXUB1vOnGDkC1424310323; Domain=.mydomain.net; Path=/; Expires=Sat, 18 Feb 2017 01:47:49 GMT"]
2015-02-18 17:47:50 ip:66.56.52.46 cookiessetting: ["userid=HxfmJNtK7rSXUB1vOnGDkC1424310323; Domain=.mydomain.net; Path=/; Expires=Sat, 18 Feb 2017 01:47:50 GMT"]
2015-02-18 17:47:51 ip:66.56.52.46 cookiessetting: ["userid=X0AF14Lbz6sJ8nh45C7Tws1424310463; Domain=.mydomain.net; Path=/; Expires=Sat, 18 Feb 2017 01:47:51 GMT"]
2015-02-18 17:47:53 ip:66.56.52.46 cookiessetting: ["userid=X0AF14Lbz6sJ8nh45C7Tws1424310463; Domain=.mydomain.net; Path=/; Expires=Sat, 18 Feb 2017 01:47:53 GMT"]
2015-02-18 17:47:54 ip:66.56.52.46 cookiessetting: ["userid=X0AF14Lbz6sJ8nh45C7Tws1424310463; Domain=.mydomain.net; Path=/; Expires=Sat, 18 Feb 2017 01:47:54 GMT"]
2015-02-18 17:47:56 ip:66.56.52.46 cookiessetting: ["userid=X0AF14Lbz6sJ8nh45C7Tws1424310463; Domain=.mydomain.net; Path=/; Expires=Sat, 18 Feb 2017 01:47:56 GMT"]
This shows the same thing, the request at 17:47:51 came in without that inbound cookie, so it generated a new one and that got used for all subsequent requests. How does the cookie setting look? I'm pretty sure that how it works ....
My set up is that I have AWS ELB directing requests to a fleet of ec2 instances, on each ec2 instance, I have HAProxy as a reverse proxy going to a bunch of different open ports. The ports/processes are node.js/express. Pretty standard technologies.
Do cookies like this sometimes just get lost? Do other players on the internet (isp's) do bad things like this? Why would they drop my userid cookie and not the AWSELB one? I'm really stumped on this one, and I would really appreciate some help here ....
EDIT
This is my code to do the cookie setting, I know there are newer ways to do it in express, but this should work right?:
var futuredate = new Date().getTime() + 2*365*24*60*60*1000;
var datestring = new Date(futuredate).toUTCString()
var cookie_list = []
cookie_list.push('userid ='+cookieval+'; Domain=.mydomain.net; Path=/; Expires='+datestring);
if(someconditional)
cookie_list.push('othercookie ='+val+'; Domain=.mydomain.net; Path=/; Expires='+datestring);
res.setHeader('Set-Cookie', cookie_list)

Related

{"message": "Cannot send an empty message", "code": 50006} gitlab

I want to integrate a bot via a weebhook between Gitlab and discord, so I've configured the bot first, copied his url and put it into the gitlab weebhook configuration input and set it for sending push updates to the Discord server.
With a real push test, I have (with the body)
Request headers:
Content-Type: application/json
X-Gitlab-Event: Push Hook
and as response
Response headers:
Date: Tue, 26 May 2020 18:46:48 GMT
Content-Type: application/json
Content-Length: 58
Connection: close
Set-Cookie: __cfduid=d374998c2f84e3e20b75bbdec88fb63d91590518808; expires=Thu, 25-Jun-20 18:46:48 GMT; path=/; domain=.discordapp.com; HttpOnly; SameSite=Lax, __cfruid=418f7199379a53d23012d37b15f2ac5a3aac36b6-1590518808; path=/; domain=.discordapp.com; HttpOnly; Secure; SameSite=None
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Ratelimit-Bucket: 3cd1f278bd0ecaf11e0d2391374c011d
X-Ratelimit-Limit: 5
X-Ratelimit-Remaining: 4
X-Ratelimit-Reset: 1590518811
X-Ratelimit-Reset-After: 2
X-Envoy-Upstream-Service-Time: 12
Via: 1.1 google
Cf-Cache-Status: DYNAMIC
Cf-Request-Id: 02f3e816a1000004823d920200000001
Expect-Ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: cloudflare
Cf-Ray: 5999a9376a790482-CDG
but got the error:
Response body:
{"message": "Cannot send an empty message", "code": 50006}
or also
Hook executed successfully but returned HTTP 400 {"message": "Cannot send an empty message", "code": 50006}
Thanks for help
You need to use the "Integrations" feature for "Discord notifications" instead of regular webhook.
See documentation here

Cookies conversion in Node.js

HI Guys Need a help in cookies
In one Login call i am getting this Cookie as response header but i need the format which is below for next requests:
Response header which i am getting [ 'ApplicationGatewayAffinityCORS=c26d807e1aaeccd5aadfffc649b97688; Path=/; SameSite=None; Secure',
'ApplicationGatewayAffinity=c26d807e1aaeccd5aadfffc649b97688; Path=/',
'.DWPLATFORMAUTH=; expires=Wed, 29-Apr-2020 13:39:20 GMT; path=/DocuWare/Platform; HttpOnly',
'.DWPLATFORMAUTH=BF0F76C29D2A57BBEC15244114BCE0ED2F81AF852C177C8C250C254F40454EAD1341AFAC394F9BF3261EC948C11D709CEBFE08F9F9683FCCB6BB9B7A56EE8C00BE3FDEEF5A9CA56BD9ABEAA651F7A5189698CEDFAAB3F29D1A46BD33E3FA1D692080B1DE68EA4C3A77BB7E9DE26B8F43ABB25575F6A40B223D55EACAA18DF365B98E88CC81D52098126E4FCC4DBF0C19768A1B100D50D0571EE65FF91BCD70CC23CAFEE3F484838AF70A7C995C89DC91CAC34F7A319F589DF00D534BDDF0D9E83791015969D532649981731E667D39A2D3F5D398CDF442E9FB65309282525CECC0BD01806B5CE4FA342A09F06D53E5D95A525470A323667463501EA105D5AD17B6D78EB27C5CEA047511BC55D55F2F2B34D5B196FAB5C69D07971435531CD786E324C2DA1EDEFE8509C863D7751D983A9EAE5BCDBB6556C0634E367BFDCA639FF2A6A56640D4E4D56FF92C4F5CB0CE083C113F05B537DC55C6C13206D865F9965448E4675D359981BD9E52B2E0A1570F831565B61635B4B5E26BB5E2F4EB890A4FC42E0EBEF4E7885DBA13AC8F0A44A4E91BFD7AB9846A906E449BBCE51AD532; path=/; HttpOnly',
'DWPLATFORMBROWSERID=; expires=Wed, 29-Apr-2020 13:39:20 GMT; path=/DocuWare/Platform; HttpOnly',
'DWPLATFORMBROWSERID=FF3A3868BB062A8E1C06EA73E419443B8DC6F073D8C3A8A3988AA9CC808C8FF369099345538486B3FA99DFEC097533BBBD621A12C12E0522521846D2D953F6D2C41DC91EAFFE3A442BC3DA738324F9A063DB88695243E19934325806DBF12D7579CA3BCCFF252A276F68334BD000358948860B23976C81326D8E571B0862045760C5B6FB9115C4F7715E951EC6AE0B779E017433676EAE7D7D202E9EE2ED83C1614317CE122828FF5D980BED4D57524F3BABF60AFFC70EA09D02E0001DD1603B952B03FE3436831E7E148EF1C76BB9A3B6F14940CFAA918A183664A586D049ED; expires=Fri, 30-Apr-2021 13:39:20 GMT; path=/; HttpOnly' ]
**but other request are accepting cookies in this format Which i need **:
'ApplicationGatewayAffinityCORS=71020739a3120956b31575e6a26cf347; ApplicationGatewayAffinity=71020739a3120956b31575e6a26cf347; .DWPLATFORMAUTH=22A566AC0C55B15C492B4CE3EEEFFB667BA0EA95DDA27084A8279F907AB2A475AF44E0D62C4733456628056F3F1E23AE8D60682A4305E8A43B95BC5F9A9A2B02B36B64892DAD484AE7C9C93430AE4B2276E591B2F9D0BCB33FBC14D3A5BF28951996029A55DD5C6B40630838A7A0770472FE8070CF546BF04AA7B31B6E4AD170C0A83FCDC253979084DAD5F23AE6B2DDEB9CF31968D67EBF1CA992B9B5FBFAA1BB3E59AF5F5E4A6D1F1CFFAFFC228CBA03561F60CAA0C47B4C01DA01909433ADF7132C114D98D36114976B9D31E669100EBF73F862ACA81781212819CB6D36A909D4BA9AB856F63B46EF95580A828AE9AB7E61C62F4C6DE925579B325330A6E77E786F0ACEF9C815B0559A7C5AAA12C733D1CE5A4B32ADF1919475C4C1451B304F4A778BD9DE4DD7B2BCCB30C159D503A634F8DA67308441C7A8B8B9B313CEED8C6EFACD20947670546C0EBCE4CC89498AD20CF566062128464077B6A7C12C77B0526F93AD7A6A2BD738C8A2CCAD8D864E23C11E2D0E036DF37E6A00E234B9B545B033929FB2ADB5880D4B8CF7D685ADB340083FFD671704A18259D6F9CD1C96; DWPLATFORMBROWSERID=D95CC701A52B6A15866C5A3966734BD6CF400DB18961F0E8FDA9B7D0F221544DC1D48799617FADC4CDA8F52CC5EBED9F1F8ECAB3969BA913E9DD3B2C346AB09E4083BABC307A4D41B0F3C436BCC5CBD2AC159480E5A48E7101B6BA0442F192B3AF38C00ECF9F3262FCFA911CF1EAF42BACAE869CCB176E49978860225D0FD614D4C2AA9B2BDE9CD376FBFFC16462622CB5235FF9D1F6DE2DDB2F4DADD2CA4497AE388C1745B12D7133FCCE0CF7A11292F49C33DDB25268C44C75C94BB381F2530D820A2FD886F5D36C1A853ECD84136B2541605C85C79C40DB73AF85DDA7701C'

Authenticate ColdFusion Application with Azure for creating contact groups

I am trying to publish contact group into Azure Active Directory.
Steps that I followed:
Register Application in Azure and get the Tenant ID and Client ID.
Send the CFHTTP Request for token (I need help here)
Send token and other parameters (I need help here) to publish the contact groups
cfhttp(method="POST", charset="utf-8", url="https://login.microsoftonline.com/common/oauth2/v2.0/response_type=code", result="result") {
cfhttpparam(name="Content-Type", type="header", value="application/x-www-form-urlencoded");
cfhttpparam(name="client_id", type="formfield", value="28d5e75-a762-45a2-9504-e9bbaa3294af");
cfhttpparam(name="grant_type", type="formfield", value="authorization_code");
cfhttpparam(name="mimetype", type="header", value="application/xml");
}
Response:
Header HTTP/1.1 404 Not Found Cache-Control: private Strict-Transport-Security: max-age=31536000; includeSubDomains X-Content-Type-Options: nosniff x-ms-request-id: caeef2f8-71a0-45f6-8820-63d880f02700 x-ms-ests-server: 2.1.10244.21 - WST ProdSlices P3P: CP="DSP CUR OTPi IND OTRi ONL FIN" Set-Cookie: x-ms-gateway-slice=prod; path=/; SameSite=None; secure; HttpOnly Date: Mon, 23 Mar 2020 18:20:59 GMT Connection: close Content-Length: 0
Mimetype Unable to determine MIME type of file.

Getting Error while opening hosted KaiOS app

I have a manifest.webapp hosted at my application root (https://localhost:5001/manifest.webapp), when I open it in KaiOS simulator, I get the following error:
Unable to access the app starting document https://localhost:5001/,
got HTTP code 405
Curl of its response is:
HTTP/1.1 200 OK
Date: Wed, 02 Oct 2019 21:18:41 GMT
Content-Type: application/x-web-app-manifest+json
Content-Length: 6097
Cache-Control: no-cache
Pragma: no-cache
Expires: -1
Last-Modified: Tue, 01 Oct 2019 07:35:10 GMT
Accept-Ranges: bytes
ETag: "1d5782ac10b5cd1"
Set-Cookie: ClientId=; expires=Thu, 01 Jan 1970 00:00:00 GMT; path=/; samesite=lax
Set-Cookie: ClientId=922A46E87C9646C18555E7E7DE84840F; expires=Mon, 02 Oct 2119 21:18:42 GMT; path=/; samesite=lax
Access-Control-Allow-Origin: *
x-frame-options: allow-from https://example.com/
x-web-server-version: 1.0.0.0
x-besku: UNKNOWN
{
"name": "abc",
"short_name": "abc",
"icons": [..],
"scope": "/",
"start_url": "/abc/?start_url=sss",
"display": "standalone",
"background_color": "#ffffff",
"theme_color": "#0078d7"
...
Any help would be appreciated.
Firefox os/kaios makes a HEAD call, before GET, so that should be implemented on your server, else this error will come.

How do I download an mp3 file with Python3

I am trying to download some playlists off soundcloud and found a site that does this for you. Of course if the playlist is long, then it's super tedious to click each link to download. So I saved the HTML of the page and have parsed out the links. The idea is to use urllib or requests to download the files.
Here's my code:
opener = urllib.request.build_opener()
opener.addheaders = [('User-agent', 'Mozilla/5.0')]
urllib.request.install_opener(opener)
track_url = 'https://scdownloader.io/download?track=zandex-hazerback-erox-stroke-bth-release&token=be1bc7997695495f756312886f566110'
track_name = 'BANG_THE_HOUSE___zandex-hazerback-erox-stroke-bth-release.mp3'
output_file = '/Users/ms/Desktop/playlist/{}'.format(track_name)
urllib.request.urlretrieve(track_url, output_file)
When I run the above code, it does save the file, but it arrives as a 1 byte file only.
I've tried other permutations using requests but basically either it doesn't work, downloads and saves a zero byte file, or does work to download and save a 1 byte file... just can't get the whole thing!
Also note, I have to send headers b/c otherwise I get a 403 error.
Any help is greatly appreciated!
Thank you!
EDIT:
Per the comments below, here's what the urlretrieve http response is:
Date: Fri, 15 Mar 2019 23:52:44 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: close
Set-Cookie: __cfduid=dcc5f95391fac83973cc77648c0e8c0391552693964; expires=Sat, 14-Mar-20 23:52:44 GMT; path=/; domain=.scdownloader.io; HttpOnly; Secure
X-Powered-By: PHP/5.6.36
Set-Cookie: PHPSESSID=fsnrrrtpnrav3vq5u2t9vfvrp7; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Vary: Accept-Encoding,User-Agent
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: cloudflare
CF-RAY: 4b82671d38067790-LAX

Resources