I have an audit table which has around 1.5 million rows and growing daily by around 100,000 entries. I found that when I query the table it’s a lot slower than it was yesterday, which was slower than the day before, and so on. I have a job which runs every 3 minutes and does a lot of queries against this table. The problem is now the job is taking longer than 3 minutes which causes a problem.
The job is basically doing this…
Check does an audit record exist within the last hour
If No > Insert an audit record
If Yes > Do nothing
Access to the audit data after this point is not speed critical, but the job of checking and inserting is.