Ahh, good choice in Varnish. :) Course, the problem with a big site is that while, yes there are lots of guests, there are also more members too - and it's hard to cache dynamic pages. It's possible though, but not without hacking vBulletin to pieces like I have done. :)
Keep up the good...