I've been trying to do a high performance server for 2 months now.
Using the high performance IO Completion Port functions.
My goal ? To have a server that handles many communications,
and clients that ask for some stuff to do, then send the server
a response of what they've done.
All the clients can launch many threads at the same time
to do many jobs. Those jobs take some time to be done,
that's why as soon as the client has a 'worker thread'
free to do something, it asks the server for a job to do.
The server then answers either a job to do,
or a 'wait' command (when there's nothing left to do).
What *really* annoys me is that sometimes after, say, 200 000
connexions, I don't why *at all*, it gets a
WSARecv failed with the error 'trying to recv on a thing that
is not a socket'.
The way I handle communication is very simple :
The client opens a communication with the server,
then send a packet = aks for something to do.
The server gets the packet, sends either the job or 'wait',
send waits for an answer. As soon as the client has got
the whole answer, it closes the communication. The server
sees the communication closed, everything went fine, okay.
Same for the results : the client opens a communication
with the server, then sends a packet = 'result of the job',
then waits for an answer.
The server answers either 'new jobs are here for you'
or 'wait', then waits for an answer (= waits for the
socket to be closed).
So far I've encountered 2 problems :
1) you're *always* forced to do something on a socket
to be able to be notified when it's closed : I first
thought : 'okay my server has sent an answer, it just
has to wait for the socket to be closed'. No way !
It seems you are forced to do something on the
socket to see when it's closed. Thus, just after sending
the answer, I have to do a WSARcv then I get, of
course, the close message (=> BytesReceveid=0) on
the IO completion thread. I even tried to close immediately
the socket just after sending, but sometimes, even if the
whole packet has not been sent, the socket is closed,
thus the client never got the whole message (this is
a normal behaviour afterthought), so I have no choice :
send the answer then wait for the socket to be closed
from the other side.
2) the problem i've been working on for weeks (and i'm
still not able to solve, that's why i do what i hate to
do : go back to the first echo server i made, and
trying to see if it has this problem too (it has it
actually)) : sometimes the connexion is closed
whereas I wouldn't expect it to be closed in any ways.
This leads to errors like sometimes WSARecv error :
WSARecv failed with the error 'trying to recv on a thing that
is not a socket' and so on.
The other thing i'm stuck with is very simple :
i want to simulate many many many many clients.
The way I do this is ugly, i don't like it and i don't
know if there are some other ways to do it. Here's
what I do :
Big loop for xx clients :
------
I create a non-blocking socket.
I link it with a home-made
completion key buffer (like all examples
do (including M$ samples))
Then I resolve the adress (here's something
I don't like : the resolv call is blocking,
do you know any example where i could call
a non-blocking resolv function ?)
See code sample CreateSocketClient below.
------
End of the big loop.
Once this big loop is finished,
I go into the main client loop, which is
infinite, where i call
WSAWaitForMultipleEvents : when
I want to leave, i just have to Set the
event, and in the loop i first check
if the thread is not terminated. If so, I quit.
If the thread is not terminated this means
a connect event has been raised.
Then I do something i really don't like :
a loop through all the sockets and, for each
socket, i call WSAEnumNetworkEvents() on this
socket to see if this socket has been connected
or if it has an error.
I'm sure there's something wrong with this.
Anyway, if the current socket has been connected,
i associate this socket with the completion port
through CreateIoCompletionPort(), and then if
everything went ok, i try to send the stuff
(= either ask for a job to do, or send the
result of the job done)
Once the loop through all the sockets is finished,
I loop to wait once again for one of the client
socket to be connected.
Another thing I don't like :
disconnection is handled by the completion threads,
but maybe, sometimes, the connect() fails so there's
an error in the infinite loop, which is a different
object. How could I handle both 'connection errors'
in the same function ?
WSAxxx functions are very hard to handle.
If you can help me or advice me in any way I would
be very happy.
Links to Delphi IO Completion stuff (I didn't find
any good web site actually).
Thanks in advance.
Olivier
function TThreadServerSocket.CreateSocketClient(
const ABufferCompletionKeySize:Integer):Boolean;
var
C : PCompletionKey;
begin
C := FGoBuffers.PopBufferCompletionKey(
ABufferCompletionKeySize, True);
{ Rappel : C = pointeur sur un TCompletionKey }
C^.Socket := WSASocket(AF_INET, SOCK_STREAM, 0, nil, 0,
WSA_FLAG_OVERLAPPED);
if (C^.Socket = INVALID_SOCKET) then
begin
Trace('WSASocket() : '+SysErrorMessage(WSAGetLastError()));
Result := False;
Exit;
end;
if WSAEventSelect(C^.Socket, FEventTerminate, FD_CONNECT)=SOCKET_ERROR then
begin
Trace('WSAEventSelect() : '+SysErrorMessage(WSAGetLastError()));
Result := False;
Exit;
end;
end;
