As we know the messages in buffer shall be still sent out even after socket closed. this is usefull, but there's a use case that we want to wait the messages is sent and acked by peer,
e.g. in embeded device, a firmware upgrade function in web ui. after upgrade done a message is sent to web ui then reboot the device. we want the message is sent and acked, or reboot may drop messages. then web ui lost reply.
so, is there a way in linux wait a socket (closed or not closed) sent out and acked all messages ?
Related
I am trying to implement SCTP support for an existing delphi application in Linux platform. I am using lksctp to support SCTP protocol. Also I am trying to use epoll(level triggered mode with EPOLLONESHOT flag) for event notification on the sockets. I am facing issue with the event handling and I am not getting where I am doing wrong or what I missed. Below is my scenario, It would be great if someone could help here to understand what is going wrong.
Currently from the server side I could see the server is reading(using sctp_recvmsg function) and processing same data repeatedly. Unfortunalely I could not post the code as the application code is very big.
Belows is the process flow of the application from the server side.
When the application starts epoll instance is created and the main thread creates a new thread which waits on the epoll handle for any network events(like EPOLLIN, EPOLLOUT etc..). The thread function waits on the epoll instance using epoll_wait() and once any event is notified by epoll_wait(), then the event type (like EPOLLIN, EPOLLOUT etc..) is recorded in a structure along with the socket descriptor and is pushed into a queue. Then the main application loop retrieves the structure one by one from the queue and performs the processing(sctp_send() or sctp_recvmsg() etc..) accordingly.
Down the line, the application creates a listening socket and adds(with EPOLLONESHOT flag enabled) this socket descriptor to the epoll handle created in step#1. Also the application starts listening on this listening socket.
Once the server receives a connection request from the client, the epoll handle signals the EPOLLIN event and the thread function pushes a structure with the event type and socket FD to the queue.
The main application loop retrieves the structure (with event details pushed to the queue in step#3) and processes the above connection request and it accepts the connection. At this point(ie after accepting connection) the listening socket is again re-armed for event notification using epol_ctl() and 'EPOLL_CTL_MOD'(This was done because of the EPOLLONESHOT used).
The newly accepted socket also now added(with EPOLLONESHOT flag) to the epoll handle created in step#1 for event monitoring.
Now the server receives a message ('UP' message) from the client on the newly accepted socket and epoll signals the EPOLLIN event.And the thread function pushes a structure with the event type and socket FD to the queue.
The main application loop retrieves the structure from the queue and does the processing, that means the Server now reads the 'UP' message from the newly accepted socket using sctp_recvmsg(). Immediately after this point the socket is re-armed for event notification using epol_ctl() and 'EPOLL_CTL_MOD'(This was done because of the EPOLLONESHOT used).
In response to the 'UP' message received in step#7 server sends 'UP_ACK' and 'NTFY' message back to client.
Client receives the 'UP_ACK' and 'NTFY' message from server and send back 'ACTIVE' message to server.
Now in the server side the epoll handle signals an EPOLLIN event and the server again tries to read the message using sctp_recvmsg(), now when the servers reads the message it is getting the same initial 'UP' message received in step#6 rather than the 'ACTIVE' message in step#9. Now the server agains sends 'UP_ACK' and 'NTFY' back to client and this goes on.
I checked from client side and i see it send the 'UP' message only one time.
Can someone give a hint on why the server is receiving the 'UP' message multiple times?
Once the server calls the first 'sctp_recvmsg()' for the 'UP' message shouldn't it remove the message from socket?.
Also can someone suggest the positions where I do the re-arming of sockets for event notification is correct? Or should I be doing it in the thread function where it is waiting for events using epoll_wait() ?
In the Bluetooth 4.0-4.2 Specifications Vol 3 Part F, I can find this text:
Commands and notifications that are received but cannot be processed, due to
buffer overflows or other reasons, shall be discarded. Therefore, those PDUs
must be considered to be unreliable.
I wonder, who receives? For Write Commands, is it the ATT server that receives from the ATT client, or might it also be the ATT layer of the sender host that receives a request to send a Write Command from the client app that drops the Write Command, i.e. it gets dropped before even being sent out over the air?
The air interface is often limited in the number of packets it can buffer for a Connection Event. Ditto Notification vs Indication in the other direction (server to client)
"Commands and notifications that are received..."
Commands, for example, Write request that is from Client to Server, so the Server is receiver. The process is that the client send out the write request to server, and when the server receive the write request(lower layer first received) but it find there is no buffer(or other reasons) for this write request and it will discard the write request(higher layer will not receive the write request).
Notifications, instead, from Server to Client, so the Client is receiver. When the client receive the notification but there is no buffer(or other reasons) it will discard the notification.
This is about flow control of communication protocols not just only for Bluetooth. So if you understand flow control you may not have such confusion.
I wonder, who receives? For Write Commands, is it the ATT server that receives from the ATT client,
Yes, it is the ATT server. Both ATT and GATT are not reliable. however the link layer is reliable. I assume the higher layer e.g. the application shall constitute reliability checking.
or might it also be the ATT layer of the sender host that receives a
request to send a Write Command from the client app that drops the
Write Command, i.e. it gets dropped before even being sent out over
the air?
This is out of spec,I think Bluetooth stack should return corresponding error e.g. "failer" due to no memory.
I have a TCP client sending data to a server continuously . After successful connection of client with the server , client sends data continuously with some intervals in terms of few seconds .
When the link between the client and server got disconnected after sending few data ,I came to know that TCP retransmits the data according to the value in TCP_retries2 , I configured this value to be 8 , such that I get write error after 100 secs .
But there will be some unacknowledged packets in send-q .
Is there way to read the content of this unacknowledged packets in send-q in my program before closing this socket or should i remember the send data and resend it after connecting again ? Is there any other way to implement this ?
You can get the size of sendq with an ioctl:
SIOCOUTQ
Returns the amount of unsent data in the socket send queue.
The socket must not be in LISTEN state, otherwise an error
(EINVAL) is returned. SIOCOUTQ is defined in
<linux/sockios.h>. Alternatively, you can use the synonymous
TIOCOUTQ, defined in <sys/ioctl.h>.
Note that sendq only tells you what the kernel of the remote system accepted, it does not guarantee that the application running on that host handled it. Most failures exist in the network between the communicating parties, but this metric can't be used for definite proof as successful transmission.
Once the application has given its data to TCP, it is the responsibility of TCP to keep track of the acknowledgement of the packets. If ACKs are not forthcoming, it tries its best to get the packet delivered based on RTO algorithm. Now until ACK is received, the data is kept in TCP_SEND_Q. I do not think there is any control from the application to determine current state of TCP_SEND_Q.
//should i remember the send data and resend it after connecting again//
How do you do this? The previous connection status is gone, isn't? Until the client and the server applications maintain some understanding as to what was received and sent offline, you have to start fresh with new connection.
No there isn't.
If you need to know that the peer application has received the data, you need to have the peer application acknowledge it back to your application via your application protocol, and treat any unacknowledged data as needing re-sending from your application somehow. This also brings in the question of transactional idempotence, so that you can resend with impunity.
It takes two to tango. You can close your end of the connection and it waits for the other end of the connection to drop, too. Think 3-way handshake in reverse.
How long do you wait between closing the connectiion and re-opening it? You must wait at least the TIME_WAIT before trying to reconnect using the same connection info.
I am writing a Client, Server-based chat. The Server is the central component and handles all the incoming messages and outgoing messages. The clients are that chat users. They see the chat in a frame and can also write chat messages. These messages are sent over to the server. The server in turn updates all clients.
My problem is synchronisation of the clients. Since the server is multi-threaded, both messages can be received from clients and updates (in form of messages) have to be sent out aswell. Since each client is getting updated in in its own thread, there is no guarantee that all clients will receive the same messages. We have a snychronisation problem.
How do I solve it?
I have messed with timestamps and a buffer. But this is not a good solution again because there is no guarantee that after assigning a timestamp the message will be put into the buffer immediately afterwards.
I shall add that I do not know the clients. That is, I only have one open connection in each thread on the server. I do not have an array of clients or something like that to keep track of all the clients.
I suggest that you implement a queue for each client proxy (that's the object that manages the communication with each client).
Each iteration of your server object's (on its own thread) work:
1. It reads messages from the queues of all client proxies first
2. Decides if it needs to send out any messages based on its internal logic and incoming messages
3. Prepares and puts any outgoing messages to the queues of all its client proxies.
The client proxy thread work schedule is this:
1. Read from the communication.
2. Write to the queue from client proxy to server (if received any messages).
3. Read from the queue from server to client proxy.
4. Write to communication channel to client (if needed).
You may have to have a mutex on each queue.
Hope that helps
I have minimum 3 TCP client, each has a Thread. I'm sending out messages and waiting for the answer, but sometimes I have to wait to receive the response from all client, this is depending what kind of message sent the server out. I already made to send messages to the clients and receiving, but when I have to wait for the other client response I couldn't do that until now.
As far as you didn't mention your environment/language, I assume C#/.NET 4
You need a mechanism for each client to signal the arrival of a response. This is usually done with AutoResetEvents: Each client sends his response back to the server. The server itself can extract from the reponse (or any other property, e.g. the connection) with client has sent it. Then he sets the apporpriate AutoResetEvent.
The thread that formerly initiated sending the message can afterwards wait for all AutoResetEvents to be set.