The exec()'ed programs should have their own address spaces, so their use of memory doesn't affect PHP (unless what you're exec()'ing is a PHP script).
Without seeing code, it's hard to definitively help you determine why you're running out of memory. How big are the PDFs? How are you converting the PDFs? If you tried rolling your own PDF lib, you might want to check the PHP docs on PDFlib Lite to see if it meets your needs.
Also, take a look at the php.ini directive: memory_limit. You can increase the maximum size, if you really need to.