I am working on idea to check port status of websites using NMAP library of python. so, I done some code, I used This Link to check 'open' word in dictionary I was getting in to print directly port number and status.
I need help to get result fast as I am not able to do it. I want to check port range of 80 to 443 ports.whenever I try to do that range It takes about 15 mins for one host (i.e. google.com). I have about 4-5 host names with range of 80 to 443 ports to check.
code image is for reference, what my code is looked like. but I used a list for host names. and basically two for loops to work all this. one for host name and other for range of port numbers.
any help is appreciated.
Thank you
Hmm, that’s really slow for a scan which would indicate to me there’s something wrong with the host you’re scanning (IDS?) or poor connection (you probably ruled this out). I believe IPs should be strings (as you correctly did in your script) but ports are numeric. Also maybe try scanning your local host or another local network machine which will be much faster and won’t block your IP.
Either way, some other ways to improve your speed include multi-threading (via threading, Queue and a host of other libraries), passing the T1-5 flags in Nmap (-T0 is slowest, -T3 is default, -T5 is insane mode). There’s also IDS evasive Nmap settings if you’re set on scanning actual remote hosts
Related
I am using my open source Serial-to-IP converter (Serial Network Interface, SNI) to communicate with headless Slackware server using ttyS0 console. SNI during connection sends back to Telnet command set ff fb 01 ff fb 03. This turns Telnet from Line mode to Char mode. But i notice that Telnet replaces (adds) my CR press (0x0d) to two chars 0x0d 0x00. The Slackware's TTY itself is immune to this, but some commands i run under it, at least cat and mcedit, are not. So i try to do the following:
Insert filter in my SNI so it just throw away NULLs at direction IP->Serial. It works, but it quite ugly, because no binary transfer will be allowed anymore.
Modify Telnet's source code to remove addind NULL to CR:
inetutils-1.9.4/telnet/telnet.c from line 2294:
case '\r':
if (!crlf)
{
NETADD ('\r'); <-- added
// NET2ADD ('\r', '\0'); <-- removed
}
else
{
NET2ADD ('\r', '\n');
}
bol = flushline = 1;
break;
It also work, so i ensure i am on the right way catching the problem.
But it is also ugly due to multiple reasons, include that it is quite not portable, and obligate me to have my own non-standard telnet binary.
So the question is: It is possible to command to Telnet not to modify my CRs? (looking at this source code piece, it is not possible at all with original code, but i am sure i miss something, and this should be possible without modify source code). Note: Replace 0x0d to 0x0d, 0x0a (this is present in code) will not work with Linux TTY: it interprets this as two CRs.
And sub-question is: Where is 0x0d+0x00 used at all? I do not know any hardware device, teletype machine, etc., and any TTY, where NULL after CR used, at all. Thanks.
So i take some investigation to see how this should be solved correctly, in terms of not to abuse the software, and to find the correct tools, or clearly show there are no such tools currently.
First of all, i show why my setup is exactly as it is, and why it uses (or not uses) standard tools.
My goal is very common and essential to Linux world (as i think before). Because Linux is network operating system, Linux box should be fully configurable via network (so can/should be fully useful when headless, i.e. without display monitor). But at time when Lilo/Grub starts, there is no network. Only serial ports are able at this time (and Lilo/Grub supports it). Why it is important to remote control at this time? Just because you can (remotely) compile new custom kernel for your Slackware box, and want to test it, adding as 2nd option in Lilo list, and want to return to original kernel even when remote machine stuck when booting, so no way to remotely edit/agjust Lilo options.
But serial console is really much more powerful tool for Linux machine.
It shows boot messages and shut down messages which impossible to see via ssh, due to network is not initialized at these moments. (And remember, we do not have display).
It lets you to (suddenly or intentionally) drop all network interfaces without a fear to lost your machine, when it is many miles from you.
Note: Serial console will not work out-of-box, but can be configured in quite well known standard ways, and described in many places. Example is http://docs.slackware.com/howtos:general_admin:serial_console
Note: There is problem that serial port should be BIOS-recognized, i.e., fully onboard. USB and PCIE devices will never work at stage when Lilo/Grub boots. Fortunately there is a good news. Hardware serial ports now (slowly) return to motherboards. I test new modern Asus J1900I-C for my server, it have two rs232 ports, and it all work fine.
To use serial console benefits remotely, some sort of SNI, Serial Network Interface, should be used (and it is the only way, AFAIK). SNI typically contains (simplest possible) TCP listening server, and raw IP-to-serial duplex translator. I start study and using SNIs when Lantronix X-Port was invented. It was in 2006, and was working fine with Slackware box. (I can't remember if there was \r\0 problems or not with XPort, but let's continue). XPort was too expensive, and was replaced with my own open source SNI immediately when Wiznet W5100 was issued. (Really, there is other, essential problem, why i need to replace Lantronix. It can't press Reset or Power buttons on motherboard, while my device can, and now server is absolutely under full remote control even after kernel panic; but it is offtopic here). Also nowadays there are many cheap no-named SNIs at online shops; i do not test these.
All these times i was using telnet to connect to SNI. The main reason was that XPort docs have examples of that. And, most times it working; i can't say that \r\0 problem stops my work. Most times it is not noticeable at all (say, mc commander is resistent to NULL 's). But mcedit fear the NULLs. And recently i start to catch the problem, so this question arrives as an result.
(now please re-read from begin of thread).
And now i try to answer to my own question. All these times i was misuse the software. Telnet was not written for human communication; instead, Telnet (suddenly) use Telnet protocol, not Raw protocol; and \r\0s are probably part of protocol. (probably, because RFC says nothing about requirement of NULLs anywhere).
While Telnet and Raw protocols may looks quite close, they are not match, so sometimes should work but sometimes not.
Using bruteforce and recompiling binary, i show that Telnet can be very easily turned to Raw mode; but it is non-standard software after that, and there is no chance that it can be pushed to worldwide repos.
So i search for Raw utilites.
netcat and ncat are not have Char mode, only Line mode, so only pure console possible, no ANSI colors, mc, passwords, etc. Putty is too complex and uses GUI. And... No more utilites i found! This was quite strange and annoying...
Then i try to use initially character-based terminals, minicom and gtkterm. They are both do not allow feed ip:port structure istead of ttyS* name. But there is data translators exist, i try socat. It connects to SNI server and creates virtual serial port; then character terminal software connects to that port.
And that, finally, work. Whoa.
But very many disadvantages in this chain; complex, hard to remember commands, can't be written in one line; when SNI drops TCP connection, it is impossible to see/catch that; a LOT of garbage issued by socat into virtual port at connection time (457 packets i count!). Here are the commands for brave people who may find my work useful.
sudo socat pty,link=/dev/ttyMYPORT,raw tcp:10.1.1.11:10001 &
then
sudo chmod a+rw /dev/ttyMYPORT; gtkterm -p /dev/ttyMYPORT
or
sudo chmod a+rw /dev/ttyMYPORT; minicom -o --color=on -D /dev/ttyMYPORT
And conclusion, i do not found any simple software nowadays, that can be directly used to communicate in human raw character mode with serial port via network bridge. (Please suggest something). Thanks.
I believe the solution you are looking for is a few lines higher:
if (!crlf)
If we set crlf = true, then we would get \r\n instead of \r\0. That's probably going to work for cat and friends.
Looking through the telnet source, it appears crlf is some sort of "toggle" option. Looking an man telnet and a bit of Googling, it appears you can run something like this:
$ telnet
telnet> toggle crlf
...and you'll get \r\n sent from then on.
I am new to Linux and I had to set DISPLAY variable for running a Java application. Somehow I managed to do that, and I understand that display can be set using
<host>:<display>[.<screen>]
but what I am doing is <host>:1001.
Now, this 1001 is 1001th display of this Linux? Are this many display possible in a machine or my understanding is wrong?
The DISPLAY variable is used by X11 to identify your display (and keyboard and mouse). Usually it'll be :0 on a desktop PC, referring to the primary monitor, etc.
If you're using SSH with X forwarding (ssh -X otherhost), then it'll be set to something like localhost:10.0. This tells X applications to send their output, and receive their input from the TCP port 127.0.0.1:6010, which SSH will forward back to your original host.
And, yes, back in the day, when "thin client" computing meant an X terminal, it was common to have several hundred displays connected to the same host.
The DISPLAY values are usually like :0, :0.0, etc. when running under the X Window server on the same host. Big numbers like in :1001 are typical for SSH passed X connection. The numbers are really summand to 6000 to get TCP port number; local ones start with 6000 and SSH passed ones could start with 7000. (This augment is different in different systems, e.g. 10 or 100 are also possible.)
As soon as these values are assigned dynamically, you should get the value for DISPLAY from an existing connection environment, provided that proper authorization token is also available (e.g. in ~/.Xauthority).
I have captured a TCP packet using libpcap, and I want to send this whole packet(without modifying it) to a specific port on another host(which has another sniffer listening to that port).
Is there any way I can do this?
Thanks a lot!
You didn't specify which programming language you're using and what you've tried so far.
Change the IP address field to the target IP and the TCP port field to the port you want. Don't forget to update both checksums.
If what you want is TCP forwarding, the Linux kernel already does this for you.
netcat may work in this case although I think you may have to reconstruct the header, have not tried.
How to escape hex values in netcat
The other option is to use iptables to tee the packet to the other sniffer while still catching it in you package analyzer
http://www.bjou.de/blog/2008/05/howto-copyteeclone-network-traffic-using-iptables/
Another option is using a port mirror, this goes by a few differnt names depending on the switch being used but it allows you to set a port on a a switch to be essentially a hub.
I think your best bet if you can't get netcat to work is to use iptables and you can add filters to that even.
I don't know whether you HAVE to use C or not, but even if you do, I would recommend building a prototype with Python/Scapy to begin with.
Using scapy, here are the steps:
Read the pcap file using rdpcap().
Grab the destination IP address and TCP destination port number (pkt.getlayer(IP).dst, pkt.getlayer(TCP).dport) and save it as a string in a format that you want (e.g. payload = "192.168.1.1:80").
Change the packet's destination IP address and the destination port number so that it can be received by the other host that is listening on the particular port.
Add the payload on top of the packet (pkt = pkt / payload)
Send the packet (sendp(pkt, iface='eth0'))
You will have to dissect the packet on the other host to grab the payload. Without knowing exactly what is on top of the TCP layer in the original packet, I can't give you an accurate code for this, but should be relatively straight forward.
This is all quite easy with Python/Scapy but I expect it to be much harder with C, having to manually calculate the correct offsets and checksums and things. Good luck, and I hope this helps.
After reading about serial ports, virtual serial ports and such, I need a little advice to see if this is even possible. I've set up two serial ports on a Linux machine (running Ubuntu). I'd like to route the two serial ports together. Is this even possible?
The two serial ports are automatically started through the /etc/init/ttyXXX.conf getty scripts. I'd like it so that when the first serial port receives a character, it outputs that character straight away to the second serial port, and also the vice versa.
Any easy way to do this through a program or bash scripts?
The idea is that both serial ports should be able to access the linux machine with commands. However, it would be nice to be able to see the outputs of the commands regardless of which port you are attached to. For example, if port 1, logged on as root, sends "echo testing", I'd like for port 2 to see the output, but also able to see that port 1 sent the command.
Thanks
A small Perl script like this might do what you're hoping, though I'm not quite sure what you're asking, so please comment if it's not working the way you'd hope. I've only got it going one way because I think they'd just keep sending the same character back and forth if it were two way. You might also need to change the port paths near the top to whatever yours are.
Just save it as serial.pl or similar, make it executable and run it.
#!/usr/bin/perl
use strict;
use warnings;
use Device::SerialPort;
my $port1_path = '/dev/tty1';
my $port2_path = '/dev/tty2';
my $port1 = Device::SerialPort->new($port1_path);
$port1->databits(8);
$port1->baudrate(19200);
$port1->parity("none");
$port1->stopbits(1);
my $port2 = Device::SerialPort->new($port2_path);
$port2->databits(8);
$port2->baudrate(19200);
$port2->parity("none");
$port2->stopbits(1);
while ($in = $port1->input) {
$port2->write($in);
}
It is possible to connect two serial ports between each other, with a crossover cable (so that the input of one port is connected to the output of the other port).
Assuming that you ports are correctly configured (drivers installed and loaded) and that your crossover cable is connected between your ports, you can type the following commands in two terminals:
Terminal 1: listen in the output port
$ tail -f /dev/ttyXXX
Terminal 2: write to the input port
$ echo "Hello!" > /dev/ttyYYY
If the two ports are correctly connected, the message "Hello!" will be displayed in terminal 1.
The hardest part is often to know which hardware port correspond to which device file.
If you just wanted to connect the two serial ports, you could use
socat /dev/ttyS0,raw,echo=0,crnl /dev/ttyS1,raw,echo=0,crnl
(see http://technostuff.blogspot.com/2008/10/some-useful-socat-commands.html)
But, since you want to interact with the command interpreter, I think you'll need to write a Perl script that
Opens both serial ports
Uses select to wait until one of the ports has some input for you
Pass that input to the shell
Write the output of the shell command back to both serial ports
I'm developing some code that is simulating network equipment. I need to run several thousand simulated "agents", and each needs to connect to a service. The problem is that after opening 1023 connections, the connects start to time out, and the whole thing comes crashing down.
The main code is in Go, but I've written a very trivial python script which reproduces the problem.
The one thing that is somewhat unusual is that we need to set the local address on the socket when we create it. This is because the equipment that the agents are connecting to expects the apparent IP to match what we say it should be. To achieve this, I have configured 10,000 virtual interfaces (eth0:1 to eth0:10000). These are assigned unique IP addresses in a private network.
The python script is just this (only runs to 2000 connnects):
import socket
i = 0
for b in range(10, 30):
for d in range(1, 100):
i += 1
ip = "1.%d.1.%d" % (b, d)
print("Conn %i %s" % (i, ip))
s = socket.create_connection(("1.6.1.1", 5060), 10, (ip, 5060))
If I remove the last argument to socket.create_connection (the source address), then I can get all 2000 connections.
The thing that is different with using a local address is that a bind must be made before the connection can be set up, so the output from this program running under strace looks like this:
Conn 1023 1.20.1.33
bind(3, {sa_family=AF_NETLINK, pid=0, groups=00000000}, 12) = 0
bind(3, {sa_family=AF_INET, sin_port=htons(5060), sin_addr=inet_addr("1.20.1.33")}, 16) = 0
connect(3, {sa_family=AF_INET, sin_port=htons(5060), sin_addr=inet_addr("1.6.1.1")}, 16) = -1 EINPROGRESS (Operation now in progress)
If I run without a local address, the AF_INET bind goes away, and it works.
So, it seems there must be some kind of limit on the number of binds that can be made. I've waded through all sorts of links about TCP tuning on Linux, and I've tried messing with tcp_tw_reuse/recycle and I've reduced the fin_timeout, and I've done other things that I can't remember.
This is running on Ubuntu Linux (11.04, kernel 2.6.38 (64 bit). It's a virtual machine on a VMWare ESX cluster.
Just before posting this, I tried running a second instances of the python script, with the additional starting at 1.30.1.1. The first script plowed through to 1023 connections, but the second one couldn't even get the first one done, indicating that the problem is related to the large number of virtual interfaces. Could some internal data structure be limited? Some max memory setting somewhere?
Can anyone think of some limit in Linux that would cause this?
Update:
This morning I decided to try an experiment. I modified the python script to use the "main" interface IP as the source IP, and ephemeral ports in the range 10000+. The script now looks like this:
import socket
i = 0
for i in range(1, 2000):
print("Conn %i" % i)
s = socket.create_connection(("1.6.1.1", 5060), 10, ("1.1.1.30", i + 10000))
This script works just fine, so this adds to my belief that the problem is related to the large number of aliased IP addresses.
What a DOH moment. I was watching the server, using netstat, and since I didn't see a large number of connects I didn't think there was a problem. But finally I wised up and checked the /var/log/kernel, in which I found this:
Mar 8 11:03:52 TestServer01 kernel: ipv4: Neighbour table overflow.
This lead me to this posting: http://www.serveradminblog.com/2011/02/neighbour-table-overflow-sysctl-conf-tunning/ which explains how to increase the limit. Bumping the thresh3 value immediately solved the problem.
You may want to look at sysctl settings related to net.ipv4 .
These settings include stuff like maxconntrack and other relevant settings you may wish to tweak.
Are you absolutely certain that the issue is not on the server side connection not closing the sockets? i.e. what does lsof -n -p of the server process show? What does plimit -p of the server process show? The server side could be tied up not being able to accept any more connections, while the client side is getting the EINPROGRESS result.
Check the ulimit for the number of open files on both sides of the connection - 1024 is too close to a ulimit level to be a coincidence.