tcp connect fails randomly under high load - linux

Our application uses a non-blocking socket usage with connect and select operations (c code). The pusedo code is as below:
unsigned int ConnectToServer(struct sockaddr_in *pSelfAddr,struct sockaddr_in *pDestAddr)
{
int sktConnect = -1;
sktConnect = socket(AF_INET,SOCK_STREAM,0);
if(sktConnect == INVALID_SOCKET)
return -1;
fcntl(sktConnect,F_SETFL,fcntl(sktConnect,F_GETFL) | O_NONBLOCK);
if(pSelfAddr != 0)
{
if(bind(sktConnect,(const struct sockaddr*)(void *)pSelfAddr,sizeof(*pSelfAddr)) != 0)
{
closesocket(sktConnect);
return -1;
}
}
errno = 0;
int nRc = connect(sktConnect,(const struct sockaddr*)(void *)pDestAddr, sizeof(*pDestAddr));
if(nrC != -1)
{
return sktConnect;
}
if(errno != EINPROGRESS)
{
int savedError = errno;
closesocket(sktConnect);
return -1;
}
fd_set scanSet;
FD_ZERO(&scanSet);
FD_SET(sktConnect,&scanSet);
struct timeval waitTime;
waitTime.tv_sec = 2;
waitTime.tv_usec = 0;
int tmp;
tmp = select(sktConnect +1, (fd_set*)0, &scanSet, (fd_set*)0,&waitTime);
if(tmp == -1 || !FD_ISSET(sktConnect,&scanSet))
{
int savedErrorNo = errno;
writeLog("Connect %s failed after select, cause %d, error %s",inet_ntoa(pDestAddr->sin_addr),savedErrorNo,strerror(savedErrorNo));
closesocket(sktConnect);
return -1;
}
. . . . .}
There are 80 such nodes and the application connects to all its peer in round-robin fashion.
In this phase, some of the nodes are not able to connect (api – connect + select) with error number 115.
In the below logs (of tcpdump output) for success scenario, we can
see (SYN, SYN+ACK, ACK) but no entry of even SYN is present for failed
node in tcpdump logs.
The tcpdump logs are:
387937 2012-07-05 07:45:30.646514 10.18.92.173 10.137.165.136 TCP 33728 > 8441 [SYN] Seq=0 Ack=0 Win=5792 Len=0 MSS=1460 TSV=1414450402 TSER=912308224 WS=8
387947 2012-07-05 07:45:30.780762 10.137.165.136 10.18.92.173 TCP 8441 > 33728 [SYN, ACK] Seq=0 Ack=1 Win=5792 Len=0 MSS=1460 TSV=912309754 TSER=1414450402 WS=8
387948 2012-07-05 07:45:30.780773 10.18.92.173 10.137.165.136 TCP 33728 > 8441 [ACK] Seq=1 Ack=1 Win=5888 Len=0 TSV=1414450435 TSER=912309754
All the above three events indicate the success information.
387949 2012-07-05 07:45:30.782652 10.18.92.173 10.137.165.136 TCP 33728 > 8441 [PSH, ACK] Seq=1 Ack=1 Win=5888 Len=320 TSV=1414450436 TSER=912309754
387967 2012-07-05 07:45:30.915615 10.137.165.136 10.18.92.173 TCP 8441 > 33728 [ACK] Seq=1 Ack=321 Win=6912 Len=0 TSV=912309788 TSER=1414450436
388011 2012-07-05 07:45:31.362712 10.18.92.173 10.137.165.136 TCP 33728 > 8441 [PSH, ACK] Seq=321 Ack=1 Win=5888 Len=320 TSV=1414450581 TSER=912309788
388055 2012-07-05 07:45:31.495558 10.137.165.136 10.18.92.173 TCP 8441 > 33728 [ACK] Seq=1 Ack=641 Win=7936 Len=0 TSV=912309933 TSER=1414450581
388080 2012-07-05 07:45:31.702336 10.137.165.136 10.18.92.173 TCP 8441 > 33728 [PSH, ACK] Seq=1 Ack=641 Win=7936 Len=712 TSV=912309985 TSER=1414450581
388081 2012-07-05 07:45:31.702350 10.18.92.173 10.137.165.136 TCP 33728 > 8441 [ACK] Seq=641 Ack=713 Win=7424 Len=0 TSV=1414450666 TSER=912309985
388142 2012-07-05 07:45:32.185612 10.137.165.136 10.18.92.173 TCP 8441 > 33728 [PSH, ACK] Seq=713 Ack=641 Win=7936 Len=320 TSV=912310106 TSER=1414450666
388143 2012-07-05 07:45:32.185629 10.18.92.173 10.137.165.136 TCP 33728 > 8441 [ACK] Seq=641 Ack=1033 Win=8704 Len=0 TSV=1414450786 TSER=912310106
388169 2012-07-05 07:45:32.362622 10.18.92.173 10.137.165.136 TCP 33728 > 8441 [PSH, ACK] Seq=641 Ack=1033 Win=8704 Len=320 TSV=1414450831 TSER=912310106
388212 2012-07-05 07:45:32.494833 10.137.165.136 10.18.92.173 TCP 8441 > 33728 [ACK] Seq=1033 Ack=961 Win=9216 Len=0 TSV=912310183 TSER=1414450831
388219 2012-07-05 07:45:32.501613 10.137.165.136 10.18.92.173 TCP 8441 > 33728 [PSH, ACK] Seq=1033 Ack=961 Win=9216 Len=356 TSV=912310185 TSER=1414450831
388220 2012-07-05 07:45:32.501624 10.18.92.173 10.137.165.136 TCP 33728 > 8441 [ACK] Seq=961 Ack=1389 Win=10240 Len=0 TSV=1414450865 TSER=912310185
Application Logs informing error on connect (i.e api - connect + select)
[5258: 2012-07-05 07:45:30]Connect [10.137.165.136 <- 10.18.92.173] success.
[5258: 2012-07-05 07:45:32]Connect 10.137.165.137 fail after select, cause:115, error Operation now in progress. Check whether remote machine exist and the network is normal or not.
[5258: 2012-07-05 07:45:32]Connect to server([10.137.165.137 <- 10.18.92.173], port=8441) Failed!
Success logs corresponding to first 3 entries of tcpdump. And failure log where there are no events in the tcpdump
My question is : When client initiates “connect” api for failed case,
i am not able to see any event in the tcpdump at client side (even
initial SYN). What can be the reason of this randomness.

You have hit EINPROGRESS. From the connect man page:
The socket is nonblocking and the connection cannot be completed immediately. It is possible to select(2) or poll(2) for completion by selecting the socket for writing. After select(2) indicates writability, use getsockopt(2) to read the SO_ERROR option at level SOL_SOCKET to determine whether connect() completed successfully (SO_ERROR is zero) or unsuccessfully (SO_ERROR is one of the usual error codes listed here, explaining the reason for the failure).
This is saying that EINPROGRESS is an indicator that the kernel is not able to complete the connection now, even though there are available local ports and routing cache entries. It seems this occurs when the socket state has not transitioned to "ESTABLISHED" yet. Just wait on the socket in select again, but call getsockopt afterwards to see if your connect had completed.
As to why, the socket transitions to SYN_SENT state during connect, but the packet may still be in the output queue and has not actually made it to the network device buffer yet.

After select() returns, you are not actually fetching the current status of the socket - you are seeing a stale value in errno (left over from the connect() call). Most likely your select() is simply returning after the timeout.
You need to call getsockopt(sktConnect, SOL_SOCKET, SO_ERROR, &err, ...) to get the actual status of the socket after select() returns.

Related

bind(): "Cannot assign request address"

I am aware of bind: cannot assign requested address and Cannot assign requested address - possible causes?, I have not been able to derive a solution from these.
I am trying to create a TCP stream (specifically here a std::net::TcpListener) directly with libc. I am encountering the error Cannot assign requested address and error code: 99 when running my code.
The exact output being:
error message: error code: : Cannot assign requested address
thread 'main' panicked at 'error code: 99', src/main.rs:43:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
My code (playground):
use libc; // 0.2.136
use std::mem::{size_of, transmute};
fn main() {
// Create socket
let socket_fd = unsafe {
let socket = libc::socket(libc::AF_INET6, libc::SOCK_STREAM | libc::SOCK_NONBLOCK, 0);
assert_ne!(socket, -1);
let optval = 1i32;
let res = libc::setsockopt(
socket,
libc::SOL_SOCKET,
libc::SO_REUSEPORT,
(&optval as *const i32).cast(),
4,
);
assert_eq!(res, 0);
socket
};
// Bind socket
// decimal 127.0.0.1 -> hexadecimal 007f.0000.0000.0001
let sin6_addr = unsafe { transmute::<_, libc::in6_addr>(*b"007f000000000001") };
let socket_address = libc::sockaddr_in6 {
sin6_family: libc::AF_INET6 as u16,
sin6_port: 8080,
sin6_flowinfo: u32::default(),
sin6_addr,
sin6_scope_id: u32::default(),
};
let socket_address_length = size_of::<libc::sockaddr_in6>() as u32;
unsafe {
let res = libc::bind(
socket_fd,
(&socket_address as *const libc::sockaddr_in6).cast(),
socket_address_length,
);
if res == -1 {
let err = errno();
print_error("error message: ");
panic!("error code: {err}");
}
}
assert_eq!(unsafe { libc::close(socket_fd) },0);
}
fn print_error(s: &str) {
unsafe {
libc::perror(s.as_ptr().cast());
}
}
fn errno() -> i32 {
unsafe { *libc::__errno_location() }
}
I set the option SO_REUSEPORT here as I need to create a stream that multiple processes can listen to.
https://lwn.net/Articles/542629/:
The basic concept of SO_REUSEPORT is simple enough. Multiple servers (processes or threads) can bind to the same port
The port does not appear to be in use:
jonathan#jonathan-pc:~$ sudo lsof -i -P -n
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
systemd-r 793 systemd-resolve 13u IPv4 19452 0t0 UDP 127.0.0.53:53
systemd-r 793 systemd-resolve 14u IPv4 19453 0t0 TCP 127.0.0.53:53 (LISTEN)
avahi-dae 855 avahi 12u IPv4 31377 0t0 UDP *:5353
avahi-dae 855 avahi 13u IPv6 31378 0t0 UDP *:5353
avahi-dae 855 avahi 14u IPv4 31379 0t0 UDP *:49594
avahi-dae 855 avahi 15u IPv6 31380 0t0 UDP *:58397
NetworkMa 859 root 36u IPv4 36212 0t0 UDP 192.168.0.22:68->192.168.0.1:67
NetworkMa 859 root 37u IPv4 110801 0t0 UDP 192.168.0.17:68->192.168.0.1:67
tor 1038 debian-tor 6u IPv4 31407 0t0 TCP 127.0.0.1:9050 (LISTEN)
rust-anal 4117 jonathan 46u IPv4 33176 0t0 UDP 127.0.0.1:35972->127.0.0.53:53
rust-anal 4189 jonathan 46u IPv4 33176 0t0 UDP 127.0.0.1:35972->127.0.0.53:53
firefox 4297 jonathan 3u IPv4 112786 0t0 TCP 192.168.0.22:46702->151.101.1.69:443 (ESTABLISHED)
firefox 4297 jonathan 29u IPv4 100032 0t0 TCP 192.168.0.22:55828->34.120.208.123:443 (ESTABLISHED)
firefox 4297 jonathan 56u IPv4 115182 0t0 TCP 192.168.0.22:50798->52.51.190.37:443 (ESTABLISHED)
firefox 4297 jonathan 75u IPv4 95741 0t0 TCP 192.168.0.22:48466->142.250.200.10:443 (ESTABLISHED)
firefox 4297 jonathan 81u IPv4 67879 0t0 TCP 192.168.0.22:55638->34.214.17.205:443 (ESTABLISHED)
firefox 4297 jonathan 116u IPv4 111739 0t0 TCP 192.168.0.22:37986->172.217.169.68:443 (ESTABLISHED)
firefox 4297 jonathan 121u IPv4 95751 0t0 TCP 192.168.0.22:46054->198.252.206.25:443 (ESTABLISHED)
firefox 4297 jonathan 129u IPv4 102073 0t0 TCP 192.168.0.22:51478->34.120.237.76:443 (ESTABLISHED)
firefox 4297 jonathan 136u IPv4 96742 0t0 TCP 192.168.0.22:36606->34.120.115.102:443 (ESTABLISHED)
firefox 4297 jonathan 139u IPv4 97943 0t0 TCP 192.168.0.22:36626->172.217.169.68:443 (ESTABLISHED)
firefox 4297 jonathan 144u IPv4 99520 0t0 TCP 192.168.0.22:49264->198.252.206.25:443 (ESTABLISHED)
firefox 4297 jonathan 157u IPv4 104859 0t0 TCP 192.168.0.22:58042->93.184.220.29:80 (ESTABLISHED)
jonathan#jonathan-pc:~$
You are transmuting b"007f000000000001" into a libc::in6_addr. But I don't thing that is the proper thing to do.
Looking at the man page, that struct is:
struct in6_addr {
uint8_t s6_addr[16];
};
That is, just 16 bytes, the proper thing for an IPv6. But your bytes are actually the ASCII values of that string: 30303766..., that while technically an IPv6 address, it is not a local one of yours so you cannot bind to it.
Moreover, in IPv6 the proper localhost address is ::1, not 127.0.0.1. That is 15 zeros followed by a single one.
If you want to bind to the IPv6 localhost by using a transmute that would be:
let sin6_addr = unsafe { transmute::<_, libc::in6_addr>([0_u8,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1]) };
Or if you insist on using a literal string (why?):
let sin6_addr = unsafe { transmute::<_, libc::in6_addr>(*b"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\x01") };

Why does shutdown write in the client cause the connection to be closed?

NOTE: No error appears if the shutdown in client is deleted
// server.rs
use std::io::BufReader;
use std::io::Read;
use std::io::Write;
use std::net::Shutdown;
use std::net::TcpListener;
use std::thread;
fn main() {
let listener = TcpListener::bind("127.0.0.1:4000").unwrap();
for stream in listener.incoming() {
let mut stream = stream.unwrap();
thread::spawn(move || {
let mut reader = BufReader::new(&stream);
let mut buffer = [0; 1024];
let len = reader.read(&mut buffer).unwrap();
// no sleep no error, just simulating a time-consuming operation
thread::sleep(std::time::Duration::from_secs(1));
stream.write_all(&buffer[0..len]).unwrap();
stream.shutdown(Shutdown::Write).unwrap();
});
}
}
// client.rs
use std::io::{Read, Write};
use std::net::TcpStream;
use std::thread;
fn main() {
let mut clients = Vec::new();
for _ in 0..1000 {
clients.push(thread::spawn(move || {
let mut client = TcpStream::connect("127.0.0.1:4000").unwrap();
client.write_all("hello".as_bytes()).unwrap();
// no error if remove the following line
client.shutdown(std::net::Shutdown::Write).unwrap();
let mut buffer = Vec::new();
client.read_to_end(&mut buffer).unwrap();
println!("{}", std::str::from_utf8(&buffer).unwrap());
}));
}
for client in clients.into_iter() {
client.join().unwrap();
}
}
As I understand, shutdown the write operation will append FIN after sending the previous data, and then the peer (server) can still continue to write data. But among these 1000 clients, some error appeared:
// server
<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 107, kind: NotConnected, message: "Transport endpoint is not connected" }', src/bin/server.rs:22:46
// client
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 104, kind: ConnectionReset, message: "Connection reset by peer" }', src/bin/client.rs:15:45
It seems that the connection is closed after the shutdown in client.
Update1:
I used Wireshark and this is one of the wrong connections:
No. Time Source Destination Protocol Length Info
1101 13.738139 127.0.0.1 127.0.0.1 TCP 56 10628 → 4000 [SYN] Seq=0 Win=65535 Len=0 MSS=65495 WS=256 SACK_PERM=1
1104 13.738157 127.0.0.1 127.0.0.1 TCP 44 4000 → 10628 [RST, ACK] Seq=409345761 Ack=1 Win=0 Len=0
1234 14.251615 127.0.0.1 127.0.0.1 TCP 56 [TCP Retransmission] [TCP Port numbers reused] 10628 → 4000 [SYN] Seq=0 Win=65535 Len=0 MSS=65495 WS=256 SACK_PERM=1
1250 14.251690 127.0.0.1 127.0.0.1 TCP 56 [TCP Port numbers reused] 4000 → 10628 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=65495 WS=256 SACK_PERM=1
1266 14.251726 127.0.0.1 127.0.0.1 TCP 44 10628 → 4000 [ACK] Seq=1 Ack=1 Win=2161152 Len=0
1376 14.251949 127.0.0.1 127.0.0.1 TCP 49 10628 → 4000 [PSH, ACK] Seq=1 Ack=1 Win=2161152 Len=5
1387 14.251970 127.0.0.1 127.0.0.1 TCP 44 4000 → 10628 [ACK] Seq=1 Ack=6 Win=2161152 Len=0
1402 14.251996 127.0.0.1 127.0.0.1 TCP 44 10628 → 4000 [FIN, ACK] Seq=6 Ack=1 Win=2161152 Len=0
1412 14.252013 127.0.0.1 127.0.0.1 TCP 44 4000 → 10628 [ACK] Seq=1 Ack=7 Win=2161152 Len=0
2092 15.261312 127.0.0.1 127.0.0.1 TCP 49 4000 → 10628 [PSH, ACK] Seq=1 Ack=7 Win=2161152 Len=5
2101 15.261384 127.0.0.1 127.0.0.1 TCP 44 10628 → 4000 [RST, ACK] Seq=7 Ack=6 Win=0 Len=0
Update2:
One of correct connections:
No. Time Source Destination Protocol Length Info
162 13.731960 127.0.0.1 127.0.0.1 TCP 56 10927 → 4000 [SYN] Seq=0 Win=65535 Len=0 MSS=65495 WS=256 SACK_PERM=1
166 13.731997 127.0.0.1 127.0.0.1 TCP 56 4000 → 10927 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=65495 WS=256 SACK_PERM=1
169 13.732013 127.0.0.1 127.0.0.1 TCP 44 10927 → 4000 [ACK] Seq=1 Ack=1 Win=2161152 Len=0
176 13.732035 127.0.0.1 127.0.0.1 TCP 49 10927 → 4000 [PSH, ACK] Seq=1 Ack=1 Win=2161152 Len=5
181 13.732046 127.0.0.1 127.0.0.1 TCP 44 4000 → 10927 [ACK] Seq=1 Ack=6 Win=2161152 Len=0
187 13.732059 127.0.0.1 127.0.0.1 TCP 44 10927 → 4000 [FIN, ACK] Seq=6 Ack=1 Win=2161152 Len=0
191 13.732074 127.0.0.1 127.0.0.1 TCP 44 4000 → 10927 [ACK] Seq=1 Ack=7 Win=2161152 Len=0
1495 14.746260 127.0.0.1 127.0.0.1 TCP 49 4000 → 10927 [PSH, ACK] Seq=1 Ack=7 Win=2161152 Len=5
1502 14.746369 127.0.0.1 127.0.0.1 TCP 44 10927 → 4000 [ACK] Seq=7 Ack=6 Win=2161152 Len=0
1505 14.746423 127.0.0.1 127.0.0.1 TCP 44 4000 → 10927 [FIN, ACK] Seq=6 Ack=7 Win=2161152 Len=0
1512 14.746529 127.0.0.1 127.0.0.1 TCP 44 10927 → 4000 [ACK] Seq=7 Ack=7 Win=2161152 Len=0
I strongly suspect it has to do with the backlog, the number of connections accepted by the OS TCP stack, but not yet handled by your application.
std doesn't allow you to control this number and defaults it to 128.
To control the number, I've re-implemented your server in tokio (well, it's actually just the main example on the tokio README), and set the backlog to 3000.
use std::time::Duration;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::{TcpListener, TcpSocket};
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let addr = "127.0.0.1:4000".parse().unwrap();
let socket = TcpSocket::new_v4()?;
socket.bind(addr)?;
let listener: TcpListener = socket.listen(3000)?;
loop {
let (mut socket, _) = listener.accept().await?;
tokio::spawn(async move {
let mut buf = [0; 1024];
loop {
let n = match socket.read(&mut buf).await {
Ok(n) if n == 0 => return,
Ok(n) => n,
Err(e) => {
eprintln!("failed to read from socket; err = {:?}", e);
return;
}
};
tokio::time::sleep(Duration::from_secs(1)).await;
if let Err(e) = socket.write_all(&buf[0..n]).await {
eprintln!("failed to write to socket; err = {:?}", e);
return;
}
}
});
}
}
This makes the problem disappear. Reducing it to socket.listen(128) makes it reappear. (Disclaimer: I do not suggest that 3000 is a sane number for the backlog.)
I said I strongly suspect that this is the cause because I can't fully explain how the sleep causes the problem. It may be that the many sleeping threads slow down the scheduler, and thus the speed at which your server can accept connections. But that is speculation.
(Side note: My default ulimit for open files was fairly low. I had to increase it with ulimit -nS 10000 to not get in the way when testing this.)

Node.js server only listening on ipv6

I am running a node.js server on port 5403. I can telent to the private ip on this port but cannot telnet to the public ip on the same port.
I assume the cause of this is because node.js is only listening on ipv6. This is the result of
netstat -tpln
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
PID/Program name
tcp 0 0 127.0.0.1:6379 0.0.0.0:* LISTEN
-
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
-
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN
-
tcp 0 0 127.0.0.1:5432 0.0.0.0:* LISTEN
-
tcp6 0 0 :::5611 :::* LISTEN
25715/node
tcp6 0 0 :::22 :::* LISTEN
-
tcp6 0 0 ::1:631 :::* LISTEN
-
tcp6 0 0 :::5403 :::* LISTEN
25709/node
How do I make the node server listen on ipv4
You need to specify an IPV4 address when you call the listen(), I had the same issue with the http module. If I use this:
var http = require('http');
var server = http.createServer(function(request, response) {
...
});
server.listen(13882, function() { });
It only listen on IPV6, as you can see from netstat output:
$ netstat -lntp
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp6 0 0 :::13882 :::* LISTEN
However, if I specify an IPV4 address like this:
var http = require('http');
var server = http.createServer(function(request, response) {
...
});
server.listen(13882, "0.0.0.0", function() { });
netstat will report the server as listening on IPV4:
$ netstat -lntp
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0 0.0.0.0:13882 0 0.0.0.0:13882 LISTEN
I'm using Ubuntu 16.04 and npm 5.3.0.
HTH

Can't connect to MongoDB from NodeJS on Ubuntu. Same code on OSX works fine

I have mongodb installed locally on my osx laptop and on a remote ubuntu server. Both have mongodb running and I can verify this using the http diagnostics on port 28017. I'm running the same code on both computers. On osx everything works fine, but on Ubuntu I can't make a connection to the database through NodeJS. I keep getting this error:
Error: failed to connect to [localhost:27017]]
message: listen EADDRNOTAVAIL
stack: Error: listen EADDRNOTAVAIL
at errnoException (net.js:769:11)
at Server._listen2 (net.js:892:19)
at listen (net.js:936:10)
at Server.listen (net.js:993:9)
at asyncCallback (dns.js:67:16)
at Object.onanswer [as oncomplete] (dns.js:120:9)
What I don't understand is that I can connect on Ubuntu locally via the mongo commandline interface. I can also connect to the database on Ubuntu via the mongo command on my OSX computer. So nothing seems to be wrong with the installation of MongoDB itself.
Can anyone think of a reason why I can't connect via NodeJS? I have tried using the mongodb and mongoose packages. Both give me the same error.
Here are the 2 ways I tried:
var mongo = require("mongodb");
var host = "localhost";
var port = mongo.Connection.DEFAULT_PORT;
var db = new mongo.Db('node-mongo-examples', new mongo.Server(host, port, {}), {});
db.open(function(err, db){
if(err){
log.error('MongoDB connection error:', err);
}else{
log.info("OPEN MONGO CONNECTION");
}
});
And the with mongoose:
var mongoose = require('mongoose');
var db = mongoose.createConnection('localhost', 'node-mongo-examples');
db.on('error', function(err){
log.error('MongoDB connection error:', err);
});
db.once('open', function () {
log.debug("OPEN MONGO CONNECTION");
});
In the logs I see nothing special, and nothing happens either
***** SERVER RESTARTED *****
Wed Sep 26 18:00:18 [initandlisten] MongoDB starting : pid=13377 port=27017 dbpath=/var/lib/mongodb 64-bit host=octo-dev
Wed Sep 26 18:00:18 [initandlisten] db version v2.2.0, pdfile version 4.5
Wed Sep 26 18:00:18 [initandlisten] git version: f5e83eae9cfbec7fb7a071321928f00d1b0c5207
Wed Sep 26 18:00:18 [initandlisten] build info: Linux ip-10-2-29-40 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_49
Wed Sep 26 18:00:18 [initandlisten] options: { config: "/etc/mongodb.conf", dbpath: "/var/lib/mongodb", logappend: "true", logpath: "/var/log/mongodb/mongodb.log" }
Wed Sep 26 18:00:18 [initandlisten] journal dir=/var/lib/mongodb/journal
Wed Sep 26 18:00:18 [initandlisten] recover : no journal files present, no recovery needed
Wed Sep 26 18:00:18 [websvr] admin web console waiting for connections on port 28017
Wed Sep 26 18:00:18 [initandlisten] waiting for connections on port 27017
..... except when I connect through the mongo commandline interface:
Wed Sep 26 18:30:40 [initandlisten] connection accepted from 127.0.0.1:38229 #3 (1 connection now open)
I ran out of things to try. Any suggestions for troubleshooting this?
Some extra info
sudo netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:27017 0.0.0.0:* LISTEN 13377/mongod
tcp 0 0 127.0.0.1:3306 0.0.0.0:* LISTEN 885/mysqld
tcp 0 0 0.0.0.0:1935 0.0.0.0:* LISTEN 1102/java
tcp 0 0 0.0.0.0:9999 0.0.0.0:* LISTEN 1102/java
tcp 0 0 192.87.219.76:10000 0.0.0.0:* LISTEN 31171/webserver
tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 1387/java
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 1076/apache2
tcp 0 0 0.0.0.0:28017 0.0.0.0:* LISTEN 13377/mongod
tcp 0 0 0.0.0.0:48466 0.0.0.0:* LISTEN 12418/java
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 3507/sshd
tcp 0 0 127.0.0.1:9016 0.0.0.0:* LISTEN 12418/java
tcp 0 0 0.0.0.0:5080 0.0.0.0:* LISTEN 1102/java
tcp 0 0 0.0.0.0:25 0.0.0.0:* LISTEN 1216/master
tcp 0 0 0.0.0.0:41018 0.0.0.0:* LISTEN 12418/java
tcp 0 0 0.0.0.0:8443 0.0.0.0:* LISTEN 1102/java
tcp 0 0 0.0.0.0:9090 0.0.0.0:* LISTEN 12418/java
tcp 0 0 0.0.0.0:29090 0.0.0.0:* LISTEN 12418/java
tcp 0 0 127.0.0.1:8100 0.0.0.0:* LISTEN 8535/soffice.bin
tcp 0 0 127.0.0.1:8005 0.0.0.0:* LISTEN 1387/java
tcp 0 0 0.0.0.0:389 0.0.0.0:* LISTEN 887/slapd
tcp 0 0 0.0.0.0:33736 0.0.0.0:* LISTEN 1102/java
tcp6 0 0 :::22 :::* LISTEN 3507/sshd
tcp6 0 0 :::389 :::* LISTEN 887/slapd
udp 0 0 192.87.219.76:123 0.0.0.0:* 721/ntpd
udp 0 0 127.0.0.1:123 0.0.0.0:* 721/ntpd
udp 0 0 0.0.0.0:123 0.0.0.0:* 721/ntpd
udp 0 0 0.0.0.0:5353 0.0.0.0:* 797/avahi-daemon: r
udp 0 0 0.0.0.0:55248 0.0.0.0:* 797/avahi-daemon: r
udp6 0 0 :::123 :::* 721/ntpd
udp6 0 0 :::35920 :::* 797/avahi-daemon: r
udp6 0 0 :::5353 :::* 797/avahi-daemon: r
Make sure port 27017 is opened to the web server in Ubuntu.
I found the problem!! The system had a messed up /etc/hosts file. Something with localhost configuration that was unusual. Correcting this file solved everything :D
Try firing the mongo command from your terminal.It will show an error if theres some problem with mongodb.

Dovecot not working pop3 with postfix

telnet localhost pop3
Trying ::1...
Trying 127.0.0.1...
telnet: Unable to connect to remote host: Connection refused
netstat -l
tcp 0 0 *:www : LISTEN
tcp 0 0 localhost.localdoma:ipp : LISTEN
tcp 0 0 *:smtp : LISTEN
tcp 0 0 localhost.localdo:mysql : LISTEN
when i run this service dovecot start i got
start: Rejected send message, 1 matched rules; type="method_call", sender=":1.553" (uid=1000 pid=26250 comm="start) interface="com.ubuntu.Upstart0_6.Job" member="Start" error name="(unset)" requested_reply=0 destination="com.ubuntu.Upstart" (uid=0 pid=1 comm="/sbin/init"))
on Dovecot.conf
protocols = imap imaps pop3 pop3s
disable_plaintext_auth = no
log_timestamp = "%Y-%m-%d %H:%M:%S "
mail_location = maildir:/var/spool/mail/%d/%n
mail_access_groups = mail
first_valid_uid = 106
first_valid_gid = 106
protocol imap {
}
protocol pop3 {
listen=*:110
pop3_uidl_format = %08Xu%08Xv
}
protocol lda {
postmaster_address = samer#aiu.com
mail_plugins = quota
log_path = /var/log/dovecot-deliver.log
info_log_path = /var/log/dovecot-deliver.log
}
auth default {
mechanisms = digest-md5 plain
passdb sql {
args = /etc/dovecot/dovecot-mysql.conf
}
userdb sql {
args = /etc/dovecot/dovecot-mysql.conf
}
user = root
}
If you get this message on the command line:
start: Rejected send message, 1 matched rules; type="method_call", sender=":1.553" (uid=1000 pid=26250 comm="start) interface="com.ubuntu.Upstart0_6.Job" member="Start" error name="(unset)" requested_reply=0 destination="com.ubuntu.Upstart" (uid=0 pid=1 comm="/sbin/init"))
It indicates that you're not doing service dovecot restart as root. So make sure you do:
sudo service dovecot restart
The directive protocols = imap imaps pop3 pop3s should be sufficient to activate pop3 with dovecot. You can add
listen = *
to be sure dovecot will listen on all available interfaces. You can verify this by netstat -apn | grep 110. Are there any failures while starting dovecot? Can you post the dovecot related logs?
By default, dovecot logs to syslog, you can explicitly specify the log files:
# Log file to use for error messages, instead of sending them to syslog.
# /dev/stderr can be used to log into stderr.
log_path = /var/log/dovecot.log
# Log file to use for informational and debug messages.
# Default is the same as log_path.
info_log_path = /var/log/dovecot.info.log

Resources