How Download Accelerator Works

Do you know how Download Accelerator Plus works. Let's go under the hood to find out.

Two weeks ago I posted the Understanding Protocols by using Sockspy to capture the various protocols (HTTP, IMAP4, POP3) in action. What we can do is to launch sockspy as a HTTP proxy and configure Download Accelerator to point to the localhost proxy.

I chose Tcl 8.4.15 source at http://jaist.dl.sourceforge.net/sourceforge/tcl/tcl8.4.15-src.tar.gz. This tarball is 3635163 bytes in size. If you were to do a HTTP HEAD on this URL, you will see the HTTP/1.1 Accept-Range values is bytes. What this mean is that web client may generate byte-range requests and that's what DAP does.

$ curl --head http://jaist.dl.sourceforge.net/sourceforge/tcl/tcl8.4.15-src.tar.gz
HTTP/1.1 200 OK
Date: Wed, 27 Jun 2007 15:04:33 GMT
Server: Apache/2.0.59 (Unix)
Last-Modified: Fri, 25 May 2007 16:32:11 GMT
ETag: "76ed8-3777db-4314df0eab8c0"
Accept-Ranges: bytes
Content-Length: 3635163
Connection: close
Content-Type: application/x-gzip

Do you know how DAP find out other mirror site before you start pushing the button to start the download. Below show the details of the HTTP requests from DAP and all the worker threads in requesting byte-range of the same file.

...
...
GET http://mirrorsearch.speedbit.com/cgi-bin/MirrorSearch.dll?ver10?Scheme=3&Site=jaist.dl.sourceforge.net&Path=/sourceforge/tcl/tcl8.4.15-src.tar.gz&Filename=tcl8.4.15-src.tar.gz&Size=4294967295&SizeHigh=4294967295&Pos=0&PosHigh=0&Country=&Region=&R=0&Advanced=1&Checksum=&BPS=1500000&V=8.0.4.8&DMD=1ee385283c6b5d14760142438cdd5044&D4=RVVeDg==&REF=&MID=e0Q5QTM2RTQ0LUVCRkItNGY5MS1BNTdFLTIwNjYyOTIxQURGOX17Qzg3MzZGNDUtMUU0RC00ZjcyLUI1REMtMEMyMUY2N0Y0REQwfQ==&SS=yEEuRJDfCkD1xEDDAQDJFIGAtgYAAAAAeH5BRJDfCkD1xUDDAQDJFIGAtgYAAAAADvyfRJDfCkD1xUDDAQDJFIGAtgYAAAAAknuCRpDfCkD1xUDDAQDJFIGAtgYBAAAA&SE=AA0AKgQAAAMABAQAA5l7gkboAAIABQQAABYAUgIAABcAIAQAACAALp+AACAARgQAACQARgMAACQARgIAACYALi8nACIAUgIAACMAUgMAACwALjQnACwAMwQAAC0AYwQAAC0AWwQAAC8ANwQAATAAN7sEATQANwEAATYAWwEAADYAZAIAADYAYwIAADYAWwIAABQALyonABQADgQAAR0ADgEAAB0ADgIAAB0AKgQAASMAKscEAS4AIAEAAC4AIAIAADAAKgIAADoALyonADoADgQAAQoADgEAAAoADgIAAAoAKgQA&SN=1&SV=3&TS_DInfo=tFgAAAAAAAAAAAAAAAAAAK58gkY=&TS_HAS=12.000211&TS_LAS=0.325744&TS_AAS=3.
407512 HTTP/1.1
Host: mirrorsearch.speedbit.com
Accept: */*
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1) (L0)
Connection: Close
Range: bytes=0-
...
...
GET http://jaist.dl.sourceforge.net/sourceforge/tcl/tcl8.4.15-src.tar.gz HTTP/1.1
Host: jaist.dl.sourceforge.net
Accept: */*
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)
Connection: Keep-Alive
Range: bytes=727031-1454064
Cookie: __utma=191645736.1777715428.1173594885.1173594885.1173594885.1; __utmz=191645736.1173594885.1.1.utmccn=(referral)|utmcsr=freemind.sourceforge.net|utmcct=/wiki/index.php/Main_Page|utmcmd=referral
...
...
GET http://jaist.dl.sourceforge.net/sourceforge/tcl/tcl8.4.15-src.tar.gz HTTP/1.1
Host: jaist.dl.sourceforge.net
Accept: */*
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)
Connection: Keep-Alive
Range: bytes=1454064-2181097
Cookie: __utma=191645736.1777715428.1173594885.1173594885.1173594885.1; __utmz=191645736.1173594885.1.1.utmccn=(referral)|utmcsr=freemind.sourceforge.net|utmcct=/wiki/index.php/Main_Page|utmcmd=referral
...
...
GET http://jaist.dl.sourceforge.net/sourceforge/tcl/tcl8.4.15-src.tar.gz HTTP/1.1
Host: jaist.dl.sourceforge.net
Accept: */*
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)
Connection: Keep-Alive
Range: bytes=2181097-2908130
Cookie: __utma=191645736.1777715428.1173594885.1173594885.1173594885.1; __utmz=191645736.1173594885.1.1.utmccn=(referral)|utmcsr=freemind.sourceforge.net|utmcct=/wiki/index.php/Main_Page|utmcmd=referral
...
...
GET http://jaist.dl.sourceforge.net/sourceforge/tcl/tcl8.4.15-src.tar.gz HTTP/1.1
Host: jaist.dl.sourceforge.net
Accept: */*
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)
Connection: Keep-Alive
Range: bytes=2908130-3635162
Cookie: __utma=191645736.1777715428.1173594885.1173594885.1173594885.1; __utmz=191645736.1173594885.1.1.utmccn=(referral)|utmcsr=freemind.sourceforge.net|utmcct=/wiki/index.php/Main_Page|utmcmd=referral

I am able to use Tcl and curl to simulate this. See these blogs that I posted two years ago - DAP using Curl, DAP using Tcl, with benchmark result.