Lottery System

Get Started
1

Dear Java Gurus,

I have a program that does a lottery. It loads the entries from the database into array. it is working well with few entries upto about 1millions. I expect over 40Million entries. when I did a stress tests with 23Million, I am getting the Java Heap space.
Is there any best way to handle this? I thought from picking from the DB direct but it may be slow as it is suppose to show entries on the screen at random.

2

Here’s a creative idea.
Load 1 million entries and make a [single] choice.
Do that as many times as necessary to access the entire set of data (40 times?) plus one, final time to choose a winner among the “finalists”.
Statistically there is no difference how you make a choice of just one out of X.

3

Hm… I don’t know exact business rule is…but why would you load entire entries? You could simply get the count of the tables and randomly choose the row number. No need to load entire database in this case.

4

40 million record is quite big… I would use BIG Data technoloy like Hadoop.

5

There are couple of options what you can think: -

  1. Use some indexing technologies like Solr/ Lucene. They will help you in full text searching and pretty fast too.
  2. Use some NoSQL solution like Cassandra which provide 10x better performance than RDBMS for reads like Cassandra (We could managed to get around 20K reads per min for 50 concurrent users and that too on desktop like boxes).
  3. Use some Distributed caching solutions like memcache which can hold the GB’s of data very efficiently and retrieval is also pretty much optimized.

Whatever you choose but your decision should be based on the answers to the following Questions: -

  1. Size of Data and just not no of records (40 million is nothing, i have been using 2 TB of data and that too being refreshed/ updated on a daily basis.
  2. Volatility of Data - How frequently it will change?
  3. No of concurrent users, accessing this data