ssh: connection refused on port 22 - linux

Question

I have installed Openssh-sever. Started the ssh-server using the following command :
:~$ sudo service ssh start
ssh start/running, process 20013
With the following command I can see port 22 is open :
:~$ sudo nmap localhost -p 1-65535
Starting Nmap 6.40 ( http://nmap.org ) at 2015-04-22 23:46 EDT
Nmap scan report for localhost (127.0.0.1)
Host is up (0.000013s latency).
Not shown: 65533 closed ports
PORT STATE SERVICE
22/tcp open ssh
631/tcp open ipp
Nmap done: 1 IP address (1 host up) scanned in 7.24 seconds
Furthermore, to check if the port is listening I used the following command with the results :
:~$ sudo netstat -plunt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.1.1:53 0.0.0.0:* LISTEN 1061/dnsmasq
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 20013/sshd
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 2009/cupsd
tcp6 0 0 :::22 :::* LISTEN 20013/sshd
tcp6 0 0 ::1:631 :::* LISTEN 2009/cupsd
udp 0 0 0.0.0.0:42198 0.0.0.0:*
But when I try to connect to the port from the same machine I get the following message :
:~$ ssh -vvv -p 22 address
OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to address [198.105.xxx.xxx] port 22.
debug1: connect to address 198.105.xxx.xxx port 22: Connection timed out
When I try to ssh from a different computer on a different network I get following message(xx.xxx.xxx.xxx is the external/public IP of the router) :
:~ssh bhuwan#xx.xxx.xxx.xxx
ssh: connect to host xx.xxx.xxx.xxx port 22: Connection refused
Whereas, when I try to ssh from a different computer on a same network I am able to connect. I do have port forwarding enabled on the router on port 22 to my local IP of the computer. I literally started using ubuntu 2 days ago and started working on ssh the same day. I'm not an expert on this as of now and any simplified explanation or guidance would be much appreciated. :)
below is my sshd_config file:
# Package generated configuration file
# See the sshd_config(5) manpage for details
# What ports, IPs and protocols we listen for
Port 22
# Use these options to restrict which interfaces/protocols sshd will bind to
#ListenAddress ::
#ListenAddress 0.0.0.0
Protocol 2
# HostKeys for protocol version 2
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_dsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
#Privilege Separation is turned on for security
UsePrivilegeSeparation yes
# Lifetime and size of ephemeral version 1 server key
KeyRegenerationInterval 3600
ServerKeyBits 1024
# Logging
SyslogFacility AUTH
LogLevel INFO
# Authentication:
LoginGraceTime 120
PermitRootLogin without-password
StrictModes yes
RSAAuthentication yes
PubkeyAuthentication yes
#AuthorizedKeysFile %h/.ssh/authorized_keys
# Don't read the user's ~/.rhosts and ~/.shosts files
IgnoreRhosts yes
# For this to work you will also need host keys in /etc/ssh_known_hosts
RhostsRSAAuthentication no
# similar for protocol version 2
HostbasedAuthentication no
# Uncomment if you don't trust ~/.ssh/known_hosts for RhostsRSAAuthentication
#IgnoreUserKnownHosts yes
# To enable empty passwords, change to yes (NOT RECOMMENDED)
PermitEmptyPasswords no
# Change to yes to enable challenge-response passwords (beware issues with
# some PAM modules and threads)
ChallengeResponseAuthentication no
# Change to no to disable tunnelled clear text passwords
#PasswordAuthentication yes
# Kerberos options
#KerberosAuthentication no
#KerberosGetAFSToken no
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes
# GSSAPI options
GSSAPIAuthentication yes
#GSSAPICleanupCredentials yes
AllowTcpForwarding yes
X11Forwarding yes
X11DisplayOffset 10
PrintMotd no
PrintLastLog yes
TCPKeepAlive yes
#UseLogin no
#MaxStartups 10:30:60
#Banner /etc/issue.net
# Allow client to pass locale environment variables
AcceptEnv LANG LC_*
Subsystem sftp /usr/lib/openssh/sftp-server
# Set this to 'yes' to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the ChallengeResponseAuthentication and
# PasswordAuthentication. Depending on your PAM configuration,
# PAM authentication via ChallengeResponseAuthentication may bypass
# the setting of "PermitRootLogin without-password".
# If you just want the PAM account and session checks to run without
# PAM authentication, then enable this but set PasswordAuthentication
# and ChallengeResponseAuthentication to 'no'.
UsePAM yes
And my ssh_config file as follows:
# This is the ssh client system-wide configuration file. See
# ssh_config(5) for more information. This file provides defaults for
# users, and the values can be changed in per-user configuration files
# or on the command line.
# Configuration data is parsed as follows:
# 1. command line options
# 2. user-specific file
# 3. system-wide file
# Any configuration value is only changed the first time it is set.
# Thus, host-specific definitions should be at the beginning of the
# configuration file, and defaults at the end.
# Site-wide defaults for some commonly used options. For a comprehensive
# list of available options, their meanings and defaults, please see the
# ssh_config(5) man page.
Host *
ServerAliveInterval 300
# ForwardAgent no
# ForwardX11 no
# ForwardX11Trusted yes
# RhostsRSAAuthentication no
# RSAAuthentication yes
# PasswordAuthentication yes
# HostbasedAuthentication no
# GSSAPIAuthentication no
# GSSAPIDelegateCredentials no
# GSSAPIKeyExchange no
# GSSAPITrustDNS no
# BatchMode no
# CheckHostIP yes
# AddressFamily any
# ConnectTimeout 0
# StrictHostKeyChecking ask
# IdentityFile ~/.ssh/identity
# IdentityFile ~/.ssh/id_rsa
# IdentityFile ~/.ssh/id_dsa
# Port 22
# Protocol 2,1
# Cipher 3des
# Ciphers aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc
# MACs hmac-md5,hmac-sha1,umac-64#openssh.com,hmac-ripemd160
# EscapeChar ~
# Tunnel no
# TunnelDevice any:any
# PermitLocalCommand no
# VisualHostKey no
# ProxyCommand ssh -q -W %h:%p gateway.example.com
# RekeyLimit 1G 1h
SendEnv LANG LC_*
HashKnownHosts yes
GSSAPIAuthentication yes
GSSAPIDelegateCredentials no

Answer 1

After a long haul found out that problem was the Modem that ISP provided. Above configuration should work fine if the modem/router has port forwarding not blocked by the ISP. With the modem/router ISP provided, even after I tried port forwarding, port 22 was not open which can be checked here: http://www.canyouseeme.org/.