Packets don't have 'http' layer available - python-3.x

Question

**Hi all,
I am learning online about network packets. I came across 'Scapy' in python. I am supposed to have 'Http' section the packet results available in terminal. For some reason I don't see '###[ HTTP ]###' for some sites. In the video that I am learning from, the tutor is using the same code but he sees 'http' for every single site he browses on, but I can't duplicate his results.
I have python 2.7.18 and python 3.9.9 in my Kali. I tried using both 'python' and 'python3' header when calling the program in terminal(no change in finding 'http' layer in packers).
I am capturing some of the http packets but not all. I have been working on a python code on my Kali VM that would look for the packets transmission for Urls and login info and display those URL of in the Terminal. The Tutorial had pretty much my expected result but I don't have the same result. In Tutorial coach was doing the same as I did(Go to Bing, open a random image )
Am I doing something wrong...? I would appreciate help on this issue please.**
...
# CODE:
#!/usr/bin/env python
import scapy.all as scapy
from scapy.layers import http
def sniff(interface):
scapy.sniff(iface=interface, store=False, prn=process_sniffed_packet) #prn = call back function, udp= audio and
def get_url(packet):
return packet[http.HTTPRequest].Host + packet[http.HTTPRequest].Path
def get_login_info(packet):
if packet.haslayer(scapy.Raw): # When used, it will only show the packet with username and password.
load = packet[scapy.Raw].load
keywords = ["uname", "username", "user", "pass", "password", "login", "Email"]
for keyword in keywords:
if keyword in str(load):
return load
def process_sniffed_packet(packet):
#print(packet.show())
if packet.haslayer(http.HTTPRequest):
#print(packet.show())
URL = get_url(packet)
print("[+] HTTP >> " + str(URL))
login_info = get_login_info(packet)
if login_info:
print("\n\nPossible username and Password > " + str(login_info) + "\n\n")
sniff("eth0") # This is connected to the internet
...
RESULT IN TERMINAL: I was browsing to Bing.com and opening a random Image.
I have used print(packet.show()) for Final Image that I browsed. In tutorial there was a ###HTTP### Layer, but I didn't have that layer.Image of Packer info for a randowm Image
┌──(venv)─(root💀kali)-[~/PycharmProjects/hello]
└─# python packet_sniffer.py
[+] HTTP >> b'ocsp.digicert.com/'
[+] HTTP >> b'ocsp.pki.goog/gts1c3'
[+] HTTP >> b'ocsp.pki.goog/gts1c3'
[+] HTTP >> b'ocsp.pki.goog/gts1c3'
[+] HTTP >> b'ocsp.pki.goog/gts1c3'
[+] HTTP >> b'ocsp.pki.goog/gts1c3'
[+] HTTP >> b'ocsp.pki.goog/gts1c3'
[+] HTTP >> b'ocsp.digicert.com/'
^C
My Expectation: These are exactly the URLs That I visited for above result.
┌──(venv)─(root💀kali)-[~/PycharmProjects/hello]
└─# python packet_sniffer.py
[+] HTTP >> file:///usr/share/kali-defaults/web/homepage.html
[+] HTTP >> https://www.google.com/search?client=firefox-b-1-e&q=bing
[+] HTTP >> https://www.bing.com/
[+] HTTP >> https://www.bing.com/search?q=test&qs=HS&sc=8-0&cvid=75111DD366884A028FE0E0D9383A29CD&FORM=QBLH&sp=1
[+] HTTP >> https://www.bing.com/images/search?`view=detailV2&ccid=3QI4G5yZ&id=F8B496EB517D80EFD809FCD1EF576F85DDD3A8EE&thid=OIP.3QI4G5yZS31HKo6043_GlAHaEU&mediaurl=https%3a%2f%2fwww.hrt.org%2fwp-content%2fuploads%2f2018%2f01%2fGenetic-Testing-Test-DNA-for-Genetic-Mutations-Telomeres-Genes-and-Proteins-for-Risk-1.jpg&cdnurl=https%3a%2f%2fth.bing.com%2fth%2fid%2fR.dd02381b9c994b7d472a8eb4e37fc694%3frik%3d7qjT3YVvV%252b%252fR%252fA%26pid%3dImgRaw%26r%3d0&exph=3500&expw=6000&q=test&simid=608028087796855450&FORM=IRPRST&ck=326502E72BC539777664412003B5BAC2&selectedIndex=80&ajaxhist=0&ajaxserp=0`
^C
...

Answer 1

I was running into a similar issue, which turned out to be that the HTTP/1.0 packets I was attempting to analyze were not being sent over PORT 80. Instead, my packets were being sent over PORT 5000.
It appears that the scapy implementation by default only interprets packets as http when they are sent on PORT 80.
I found the following snippet in this response to a GitHub Issue (for a package which should not be installed, per Cukic0d in their answer to a similar question here).
scapy.packet.bind_layers(TCP, HTTP, dport=5000)
scapy.packet.bind_layers(TCP, HTTP, sport=5000)
Adding this snippet before my call to sniff() resolved my issue and allowed me to proceed.
Hope this helps.