I’m wanting to build a UI which is updated which handles navigation, form submissions, etc without the need for a reload.
Are JavaScript Templating libraries such as Handlebars what I’ll need to use? Or, would it be something like AngularJS?
Yes, templating is definitely an important piece of rich client-side ui’s.
http://backbonejs.org/ is a great starting point (models, views, routing) but most people have migrated to the bigger frameworks Angular, React, Ember or Vue.
Any of those 5 options will help you to build what you’re after, which you choose is largely down to the style you prefer.