I would go through /var/logs/auth.log if you have access to it. Not sure if all OS store their login logs there but Ubuntu and Debian sure do.
It stores logins not just for FTP but for anything - including SSH, Telnet (if you have it), etc.
If you don't have access to this kind of thing, then all this is really your host's responsibility I'd say...
When I recently was investigating a site that got hacked (not my site) this at least allowed me to find out which user account they had logged in as in order to gain access, and which IP address they came from.
If they logged in as you though, it's going to be virtually impossible to tell them and you apart unless you can figure it out from the dates and times.