Managing releases of JS/CSS/image assets?

Working on a new site structure and one of the major changes is all JS, CSS and Image files will be moved to a CDN.

We’re going to have release cycles where some/most/all of those files will either change/be added/removed. We don’t want to keep around orphan files as it will just grow into a huge mess.

What’s the best way to keep “release assets” independent so that, for example, if even a single image is removed from one release to another - it is removed from the release and consequently the CDN?

To me the ideal situation is human intervention: “Oh, I removed this little widget which means images X Y and Z are no longer needed. I’ll go delete those” - but that simply won’t happen/won’t be consistent :frowning: