It would make no difference if it were feasible to encode a binary executable (?) but the important thing to note is that a hacker could easily 'wrap' that application and monitor all outgoing/incoming information whilst it's in use. Not only that, but any users logging in whilst the server is compromised are auto-magically screwed.
The best thing you can do if your server has been infiltrated? Ring up your host - tell them to remove the plug and send you the hard drives via FedEx. Put up a 'maintenance' page. Do this as fast as possible - more than a day of infiltration and you've pretty much lost all hope of being able to prevent them using the data they've already downloaded to their own servers.