ExLibris wrote: I'd suggest sticking with a core group of playtesters and replacing them as and when they decide they need a break, rather than getting a whole new group for each major version.
There is definite value in maintaining core playtesters, which is why I don't rotate anyone out, although I will replace those that let me know they are no longer interested in playtesting for whatever reason (time, burnout, whatever.) There is also a great value in fresh eyes, but within reason. Every new playtester brings diminishing returns, and at some point the benefit of having fresh eyes goes negative.
I assume that many playtesters aren't really interested in playtesting, or even know how to playtest. They just want to see the new content early and I'm OK with that. They'll still report a broken link as they come across it, and on a game this complex, they may come across a scenario nobody else has found yet, or have an epiphany about a scene and share a different viewpoint on some comment threads, so there's still value there. I expect one in ten will be a gung-ho, hardcore playtester unless I actually employed some kind of screening process, which I don't plan to do. I was lucky enough to pick one up early in the testing (thanks ExLibris!)
Ultimately, I will limit BEW playtesting to 25 active testers, which should hopefully round out to 2-3 solid hardcore playtesters, 15 or so players who report occasional bugs, or continuity errors, and 5-10 players who find that someone else already reported the broken link they found.

I know that doesn't add up to 25, but I think it's clear by now that I can't count (or spell) anyway. Ultimately, few have the time available, or want to make the commitment to be a hardcore playtester, and I fully understand that. Especially at the wage I am paying. ($0.00/hr) Although, anyone who wanted to learn solid playtesting skills could probably have the opportunity on this project.
There will be three more groups of 5 to come on board for playtesting (well one group of three and two groups of 5...that whole counting thing again) over time. One group at the end of Day 2, another for the end of 3, and the last group at the end of 4. That is a good compromise for me to have fresh eyes on the game, without too much undue overhead. Bringing more on after that would probably be unproductive because of the reasons ExLibris mentioned regarding the time taken to come up to speed on all the content. So days 5-7 will have all the same playtesters. 25 might even be too many, but that really depends more on how many hardcore playtesters we pick up