For the most part -- yes... either a numbered heading, or the appropriate heading-type tag for the element.
Numbered headings are NOT the only heading type elements in HTML. You have CAPTION which is basically a heading for a table, you have TH which is a heading for a row or column (typically for the column if in THEAD, or for the row if in TBODY -- assuming you use those structural/semantic elements PROPERLY, which of course few people do). You have LEGEND which is the title for a FIELDSET, a grouping of form INPUTS, you have LABEL which is a heading for a form input element like INPUT, TEXTAREA, SELECT, etc.
Oh, and about H1 -- since there should only one of these per page ANY other heading-type element you use on the page is basically saying it's a subsection of THAT... this is why IMHO it should be the first semantic element on the page since by definition EVERYTHING is a subsection of it... and so on and so forth down the list. You make a new heading, the content under that heading should be a subsection of the heading preceding it, with the headings fanning out like a tree.
Basically we're talking semantic markup -- saying what things ARE and NOT how they are going to appear; which also entails NOT using tags with meanings around the wrong elements... two or three words of text is not a GRAMMATICAL paragraph, and if you are thinking a typographical paragraph you are thinking APPEARANCE, which has NO BUSINESS in your HTML. Just because it's text doesn't mean it gets a P around it... and if it's not text, it most certainly should not have a P around it. You see people make that mistake with IMG all the time... only time it would be appropriate to be around IMG would be if said image is inline to the text like a smily