Here is a simplified explanation of something I want to do in my app:
A user enters a sentence into a webform.
I split the string into separate words.
Create a query to see if any of the words match words in my members_keywords table (can be multiple instances of any one word).
Query returns an array of member id's.
I create a new insert query with one row for each member id returned in the previous select.
The thing is, this one process could potentially return thousands or member id's and so the insert query would equally need to insert thousands of rows.
I'm really concerned about the impact of this on the server as this process could potentially be running many times a day along with everything else in the web app.
Am I right to be concerned? Is there a way to make it less intensive? Do it in batches? Timeouts would be a nightmare. As it is, the query would be ok, but I need to make things future proof.
Any thoughts much appreciated.