This example shows an FTP server but the same works for HTTP server
/usr/bin/wget \ --verbose \ --mirror \ --wait=2 \ --random-wait \ --no-host-directories \ --cut-dirs=3 \ --directory-prefix=/home/ftp/doom/idgames_mirror \ --dot-style=binary \ ftp://ftp.fu-berlin.de/pc/games/idgames/
So ^that is how you can efficiently download all the things from a HTTP/FTP mirror (only if you have permission now!!). This is what I used to copy the idgames archive to my mirror at Michigan Tech, PSGNet.
I would look up what each of these options means on the wget man page if you aren’t sure.