How do you embed footer links into every webpage? So that I can update one file and it fixes the footer on every webpage?

I know how to do it by using PHP files but I don't want to use that. I don't want every page to be xxxx.php