You can do it in one of two ways (assuming your domain registrar provides free DNS services as many do): 1) Update your nameserver information at your registrar and use your VPS nameserver(s); 2) Use your registrar’s DNS (if it provides it) and create A Records for your non-www and www versions of your domain name with the value of the A Record being the IP Address of your VPS. You will also need a MX record as well if you plan to use email. The value of the MX can be your IP address (or you may be able to use your root domain name).
Your domain registrar will have some area where you can update your nameservers. You will have to get the names of your nameservers off your VPS and IP address (or welcome email when you signed up for it) and update your record at the registrar. The names of your nameservers will typically be something like ns1.yourdomain.com and ns2.yourdomain.com and so forth.
The downside to this is you have your nameserver on your web server, so if your web server goes down, you cannot point your DNS to any other place and will have to wait for propagation to occur.
If you use your registrar’s DNS, you will have more flexibility because if your web server ever goes down, you can change your A Records and point visitors to another IP address without concern for propagation time (which can take a few hours). If you have a high traffic web site, your registrar may not allow you to use their DNS if they provide it. So that is a consideration.
If you are using something like cPanel, it will manage your DNS server for you should you use your own. If you are using anything else, I have no idea what to do.