Email to CRM Application Design Question

[bruin03]

So I am building an application to integrate email into a custom CRM with php/mysql and am wondering what approach to take. I have two options. The first does everything on one process. The second splits it in 2 seperate processes or services.

I am thinking of either
OPTION I
A single service that parses email contents directly into the live database.
1. reading the email.
2. Grabbing the Contents that I want.
3. Storing the contents that I want to the CRM database.
4. Deleting the Email.

OPTION II (two services)

SERVICE 1
Step 1. reads the email.
Step 2. storing the entire email in an intermediary database table.
Step 3. Deleting the email
.
SERVICE 2
Step 1. Loop through the the intermediary table 1 record at a time and parse the contents I want from a single field.
Step 2. Storing the contents that I want to the live CRM database.
Step 3. If the contents are stored successfully, update a flag in the intermediary table as processed for that record.

Which is a better way to do this and why?

[rguy84]

My first question is how many e-mails are you processing a day/hour? My second question, how soon after you putting into the live database, when will you be using it.

I'd say if it is under a few hundred, your first option is the way to go. If it is thousands, you may want to do it temp then do a cron job at 1am.

[Vali]

OPTION II will work better since you can scale your app easier.
Ex: if your parsing bounces off a few servers, but want to process them on a central server, then your "get email and put in db" script would be simple, small, and installed on all mail servers, and your main parsing script, on the main server.

[kyberfabrikken]

How do you receive mails? Do you poll a remote mailbox or do you configure a mail server to deliver it directly to your application?

[bruin03]

How do you receive mails? Do you poll a remote mailbox or do you configure a mail server to deliver it directly to your application?

I poll a mailserver and login via imap.

As for the question about the number of emails, I am looking at about 500+ a day.

[bruin03]

My first question is how many e-mails are you processing a day/hour? My second question, how soon after you putting into the live database, when will you be using it.

I'd say if it is under a few hundred, your first option is the way to go. If it is thousands, you may want to do it temp then do a cron job at 1am.

about 500+ emails a day. After getting into the live database, the customer service team may be using it immediately.

[bruin03]

For anyone trying to do something similar, I recommend first reading the following article at phpclasses.org.


http://www.phpclasses.org/blog/packa...using-PHP.html

[Czaries]

I poll a mailserver and login via imap.

As for the question about the number of emails, I am looking at about 500+ a day.

I would highly recommend email piping instead. Just pipe the incoming email to a php script that parses it and stores it in your database table. I used email polling with IMAP login for DisposeaMail, and recently had to switch to email piping due to the sheer volume of incoming email. The polling script always lagged behind, and eventually stopped working completely (timed out) as the unread email inbox grew larger and larger.

[bruin03]

Any links to good tutorials or articles explainin?

I hope it wouldn't require root access as the client has shared hosting with no plans of upgrading anytime soon.