We are about to rerelease a site that gets around 90,000 user registrations a day. On any given day we have around 250,000 unique visits.
Now, the new code for the site lays off the database a bit by storing more in the session. It equates to about 40k per logged in user, or 1-2k per anonymous user for up to a week of inactive time. This is obviously a significant amount of disk space to be using. In addition to the disk space, the session data is shared by 4 web servers via nfs.
My question is: how can I support this much session data and avoid the sessions being a bottleneck?
I have looked at sql based session storage (too slow, too much overhead), file based sessions (how well does this scale?) and msession (don't know enough about it). Any recommendations?