Building links - as opposed to earning links by having a high quality site - may well be one of the activities which causes your site to drop in rankings, if Google judges you to have violated their guidelines.
But search results do vary - that’s the nature of them. They can vary from day to day, from location to location, from one computer to another. There are many factors which influence your ranking, not all of thhem under your control. If one of your competitors makes improvements in their site, and rises above you in the rankings, this will, of course, cause your site to be pushed further down.
My advice would be to concentrate on the site, and do everything you can to ensure it’s a well-structured, quality site with unique content. At the end of the day, that’s what counts. Don’t try “quick fix” solutions to improve your ranking; they may work in the short term, but then you will see your site drop again.
The article @Twelve_Edin linked to in post #3 contains some good advice and is worth a read.