It sounds rather like this is a VPS where you are the only client. If you don't know what you're doing you will definitely run into problems with this later, whereas cPanel manages much of this - you may be better going to a good shared cPanel host like asmallorange (can't recommend myself, but ASO are great).

I'm surprised though that they don't know enough to understand and offer the alternatives.

One option is to read the smoke signals you are getting and host elsewhere.

Another is to use SFTP to implement access over SSH very easily, if you chose. But once you have an SFTP password, the same password works for FTP and cPanel usually, unless FTP is firewalled off. Try asking whether you can use SFTP - or just try it if you've been given details, Filezilla provides an easy SFTP client and can be installed in seconds via (if you have Windows).

The problem is that without cPanel there are other issues that will become harder to manage - no auto software installation, logs etc are less available, etc etc.