I’m reading about NS records and turns out this is kinda mandatory. But as per my experience setting up a github pages website with custom domain (from namecheap), I didn’t have to put the NS record. Why? I just put CNAME record and 4 A records.
The DNS can be complicated. If you search for other relevant articles then you will discover that they say that the Domain Name System (DNS) converts domain names to IP addresses. Yet how that happens can be complicated. In you situation, your A record(s) point to the website.
Instead of using an article about NS records to understand why you might not need NS records, look for beginner articles about how the DNS works, such as in how does dns find website - Google Search.