I am on a team of developers working on an API accessible to 3rd-party clients. An important fix may necessitate changing a variable name in the response body. A mischievious or unwitting team member may have the urge to update a model/data structure, to the detriment of unsuspecting consumers, who will end up with broken apps.
A friend suggested baking variable names into unit tests for those endpoints. The tests fail when conflicting information arrives. I wonder if it isn’t impossible for developer to equally update the test parameters so his updates can merrily leave for the CI server.
Is this the standard practice? Is there a way to enforce avoidance of such situations?
Your ‘unwitting’ developer realizes his blunder when the CI environment rejects his code because the unit tests fail, and reevaluates what he’s done.
“Mischievous” developer at that point is a malicious developer, who is intentionally bypassing sanity checks to release code changes he knows will fail.
Some, potentially. Mostly it’s just mitigation.
If your unit tests are kept by your testers, and your code is kept by the developers, you’d have to have a pair working together to bypass.
Note: Your API’s unit tests should already have ‘baked in’ variable names - otherwise how can they differentiate between pieces of data coming in?
If you tell me your API takes in 2 numbers, signal and mod, and I tell you my data is “6,3”, how do you know which one is which? How does your test know?
Like I asked earlier, what stops him from casually updating tests on his branch, and pushing to CI? But…
I agree with your point about baked in variable names. So, it’ll either take someone who doesn’t know what he’s doing, or a concerted effort to break 3rd party apps to equally update the tests. But you’re forgetting not every api/ organisation is being developed by separate testing teams and coding teams. In a typical situation with a handful of developers working autonomously, one individual is enough to carry out the deed
I am considering (at least, for such situations), a solution that encourages immutable responses. Compiling your code for production updates a hash (or API prefix) if it detects change in variable names. Then CI server is observing updates to this parameter, and can either deploy to a subdomain or whatever uri is convention
Your developer should not be writing the unit tests for his own code. When I worked DevOps, we had a simple structure to every story; There was a Dev, and a Test. The two worked on the story at the same time. Both agreed on the API’s behavior, and worked, while in communication, rarely did they actually look at each other’s code (until code review).
We also had gated checkins - code could only be commited to CI with a code review signoff (as well as code compliance enforced by the VS environment), and only made it to the second stage of deployment (which… i forget the acronym for) if it passed all of its unit tests.
So if it gets that far, your unwitting developer was following story, which means its gone through at least 3 (but probably more) sets of eyes - the scrummaster/manager/lead dev (who groomed the story), the dev, and the tester - and all 3 cleared the changes. If it gets that far without someone going ‘Hey this may have problems’, you need a better staff.
If you’re saying the situation is that your developers develop code changes and features without management’s approval and flow to release, you need a new situation. If this is the situation you are working in, then you have no control or ability to guarantee anything.