"York" <office@onlinebg.info> wrote in message
news:44d96a92$1@newsgroups.borland.com...
> The problem I experience is that if the file is let's say 1024KB and
> when I call idHTTP.Get it internally calls a procedure AdjustStreamSize
> which sets the size of MyStream to the size of the file.
As it should be. That is an optimization that speeds up the storage of the
downloaded file by preallocating the necessary storage space ahead of time.
> However the file is not downloaded and the stream contains 0 up to
> its end. So my parser thread reads zeros from the file.
>
> I know this sounds more like a non-indy issue but an algorithmic one
> so I thought you could give me some advice how to handle this situation.
Simply do not have your parser process the file until after Get() has
exited, indicating that the download is finished. You are starting your
parser too soon.
> Here are some more details:
The contents are irrelevant to this topic. You are simply trying to process
what is not yet available for processing. You need to delay your parser
until a later time.
The alternative is to write your own TStream class that overrides the
virtual Write() method. You can then have Write() push data to the parser
thread after each complete record has been downloaded.
> My parser thread starts from the beginning of the file in some time after
> the "get" is called and tries to read the first header. It usually
succeeds
> and after that tries to read the data portion followed by the first header
> (determined by the size recorded in the header)
You need to synchronize your threads better. At the very least, you could
have the parser thread wait on an event that the downloading code signals
after each complete record has been received. Each time the parser detects
a signal, have it process as many records as are actually available in the
file. That way, the parser is guaranteed to always have data available for
its processing.
> However for large data blocks the parser thread reads 0 because
> the entire portion of the content has not been downloaded.
The parser is not waiting for data to arrive. It needs to wait. This has
nothing to do with Indy at all.
> I have no chance to check the stream.size because it is already set
> to the size of the original file (AdjustStreamSize)
You should not be sharing a single stream across multiple threads that are
manipulating the stream's content at the same time. If the download stream
is writing the data to a file on the hard drive, then the parser thread
should be using its own separate stream that has the same file opened for
reading only. That way, the parser can maintain its own separate position
tracking in the file, without worrying about messing up the downloading
stream's own position tracking.
Gambit
