What methods do YOU use to debug/test your code?

[digitman]

I'm fairly new to coding in general. Only done about 2/3 serious projects. The problem I'm currently facing is that my code has too many bugs. After I have a working version of a script, making a change breaks something else in the code, and fixing one bug causes another to be produced somewhere else in the code. You get the idea.

I'm just wondering, what coding techniques do you all use to write less buggy and more manageable code? I'd like to hear how you make less mistakes in the first place instead of spending more time trying to find and correct them. Also, after you have an alpha version of the script ready, which methods do you use to test it, to make sure its working perfectly before submitting it to the clients for review / releasing it?

I'm looking forward to hearing your opinions on this

[otnemem]

Unit Testing and then Integration Testing (Check out SimpleTest/PHPUnit and Rephlux)

[Super Phil]

TDD, which leads to plenty of unit tests. Takes about 5 seconds to check if my script is working. I am planning to move on to automated builds/testing in the future.

[digitman]

Unit Testing and then Integration Testing (Check out SimpleTest/PHPUnit and Rephlux)

TDD, which leads to plenty of unit tests. Takes about 5 seconds to check if my script is working. I am planning to move on to automated builds/testing in the future.

Thank you for the responces. I've heard these terms a lot of times but can't figure out what they mean and how they work. Can someone explain it to me?

[Nigel McNie]

"Unit Testing" is a way to test a single class (or perhaps function) outside of the complete program where it normally resides. What you do is write a series of test cases that use just the part of the program that you are testing, then each time you make a change to your program you can re-run the test cases to demonstrate that what you have changes has not broken anything you have already written.

Instead of writing your code then testing it as normal, you would do something like this:

1) Write a test class (as in OO class) that will contain several methods that test out methods in the class you are testing. This class extends some class in a unit testing framework (typically called TestCase or similar), and the unit testing program will run each test method in your class when it is run. Of course, if you haven't written the class yet your tests will fail.
2) Write the class that you will test. Your first test will now pass.
3) Write a test for some non-existant piece of functionality that your class is supposed to have. Your tests will fail again.
4) Write the functionality into the class you are creating. Now your tests pass.
5) Repeat steps 3 and 4 until your class has all functionality that you need.

Of course, you can see the benefit of this: If you change the class in some way, whether by changing one of the methods already made or by adding new functionality, you can just run your test cases again to prove that it still works properly.

That's just a quick summary of unit testing.

[digitman]

thanks for that

I'm still interested in hearing what methods the rest of you guys use to debug and test your code!

[atu]

Validation of data before to use it. Try to programm in a defensive way. That means; never trust to anything. Under php4 you may use trigger_error() an under php5 exceptions for debuging. And of course unit tests. But it isnt always evident to write such tests for every code.

[NeverMind]

Unit testing all the way
I wrote an article about UnitTesting using SimpleTest:
http://www.devpapers.com/article/303
http://www.phpsimplicity.com/article...st_colored.htm (Mirror)

also PHP assertion could help and of course the most important thing is raising the error reporting level to include E_NOTICE and E_STRICT!

error_reporting(E_ALL | E_STRICT); 

[Dr Livingston]

You get the idea.

Yes, we do, and Unit Testing will certainly help but you may also want to consider that your scripts are not loosely coupled enough, if one script breaks elsewhere, when changes are made higher up?

That is a design issue I think

For some reason, E_STRICT with PHP5 will break Simple Test, and it's really annoying as I use E_STRICT by default, and then have to remove it. Would have thought by now, the problem would be remedied

[McGruff]

The problem I'm currently facing is that my code has too many bugs. After I have a working version of a script, making a change breaks something else in the code, and fixing one bug causes another to be produced somewhere else in the code.

Unit testing is the answer to these kinds of nightmares. I would find it impossible to go back to the old way I used to code now that I've tried it. I just wouldn't have the patience for those hours of banging-your-head-against-a-brick-wall debugging sessions.

Funnily enough, I had one of those this week. I'd just installed some software on a live site. All the tests were passing locally but when I ran them on the live server about a quarter would just mysteriously die with no output. Very strange.

Normally when tests fail they'll show you pretty accurately where the problem is. You might still have a bit of work to do to sort it out but it's not remotely as difficult as doing it without tests. For a start, the tests are automated. One click gives instant feedback on the changes you just made. You don't forget to check anything. No messing about with print_r etc no endless clicking around from page to page (with SimpleTest there's a webtester to do that for you). If it's hard to verify your changes you won't do it so often. With a longer period between tests you're not sure which of a number of edits might be producing the errors. It's more likely that multiple interacting bugs will infect the app. All this exponentially increases the difficulty of fixing problems when they crop up.

So, there I was with the tests passing locally but failing apparently at random on the live site. Because the failing tests weren't producing any output at all I didn't have a clue what was going wrong (that's a very unusual problem you probably won't encounter, I hope). It could have been anything from my own sloppy coding (likely), SimpleTest (unlikely), or even a different server set up. At times I was getting 502 gateway errors: was that something to do with the server's Zeus/fastcgi and remote responders? I was getting a bit out of my depth there.

The point is that tests can help enormously simply by telling you what's not failing thus narrowing down the scope of the problem and making it much easier to fix. With the tests themselves misbehaving, I'd lost my safety net. I hardly knew where to start looking: everything was a potentially a suspect and so had to be considered.

In essence unit-testing is all about breaking complex problems down into managable chunks. A test case focusses intensely on a single class. By the time they roll off the production line component parts of the app should have a high reliability. There's more craftsmanship to it - there has to be to pass all the tests. It's also just a lot more fun in general. Suddenly you are in control again and don't have any more of those nightmare WTF bugs where you don't even know the question to ask never mind possible answers. It's a very fluid way to work. You can add new tests for a particular class at any time. You have the confidence to make big changes, a step at a time, with the tests to back you up.

Testing has a whole range of benefits. The tests themselves can be read as documentation - docs which never get out of sync with the code. It's also a design tool if you test-first-code-later. Sometimes classes are hard to test and changing them purely to make it easier to write the tests often leads to a better design overall. That's a smell to watch for.

[lastcraft]

Hi.

Regarding failures on a different machine, it's likely that the memory limit setting is different and you are not reporting errors on the server. 8M is not enough for a full test run (1000+ tests) on a typical application. At least that's my theory .

Regarding TDD (the hardcore version of unit testing), I wrote a short introduction here:
http://www.developerspot.com/tutoria...ent/page1.html

It's object based, but you can use the same techniques for web tests or function tests. What is the first bug you want to tackle?

yours, Marcus

[Dr Livingston]

+1 to Marcus for the link

Btw, have you posted the link to the thread that Jason started recently...

[BerislavLopac]

Regarding TDD (the hardcore version of unit testing), I wrote a short introduction here:
http://www.developerspot.com/tutoria...ent/page1.html

Marcus, did you know that the pages get all mixed up when you try to print that?

[lastcraft]

Hi...

Marcus, did you know that the pages get all mixed up when you try to print that?

No I didn't. It's not my site I'm afraid.

yours, Marcus

[Ren]

For some reason, E_STRICT with PHP5 will break Simple Test, and it's really annoying as I use E_STRICT by default, and then have to remove it. Would have thought by now, the problem would be remedied

Yeah, I end up using SimpleTest from the CLI, so can php -d error_reporting=2047

[ssx-gun]

Sometimes the more primative ways are the best. Log onto your handy linux terminal (assuming you have one) and keep changing it until it works how you want while making sure to keep your version control up to date with any major changes. Gotta love the version control even though sometimes it seems useless.

[McGruff]

Sometimes the more primative ways are the best .... keep changing it until it works how you want while

If you'll excuse me for contradicting you, that is emphatically not recommended.

"Until it works" is far too vague. Tests for a single app might make thousands - even tens of thousands - of precise assertions. There's no way you can reliably verify these by hand.

Admittedly the bulk of those assertions are under-the-bonnet or white-box tests which you wouldn't need to check with "by hand" acceptance tests but it still leaves a lot to do. You could easily forget something. You won't check as often because it's much harder to do compared to clicking a button. In the meantime, bugs could be runnning rampant and they'll be much harder to deal with - exponentially harder - because you didn't catch them earlier.

Without tests, when you do find a bug, it probaby won't be immediately obvious what's causing it. With under-the-bonnet unit-tests, the tests will almost always tell you exactly where the problem lies. Testing seems like extra work but it makes you more productive overall.

Although we've mainly been talking about debugging, testing is much more than just a quality-control/continuous integration tool. It's also a design tool and a reliable source of documentation, as mentioned above.

I do agree though that version control is vital, testing or not.

[lastcraft]

Hi...

For some reason, E_STRICT with PHP5 will break Simple Test, and it's really annoying as I use E_STRICT by default, and then have to remove it. Would have thought by now, the problem would be remedied

I am open to suggestions. Because SimpleTest has to work with PHP4, and because it digs pretty deeply into the semantics of the language, E_STRICT compatibility is impossible right now. SimpleTest2 will be PHP5 only and at that point E_STRICT will be possible. You just have to wait awhile.

What errors are you catching with E_STRICT (please reply to this bit as I am genuinely interested)? Are they errors that could be caught with testing anyway, perhaps by adding a feature or two to the tester?

yours, Marcus

[lastcraft]

Hi...

I'm just wondering, what coding techniques do you all use to write less buggy and more manageable code? I'd like to hear how you make less mistakes in the first place instead of spending more time trying to find and correct them.

Just in case this thread turns into a "unit testing is the answer" thread, I'd just like to point out that it isn't the only trick you can use. After all, programs were written before Kent Beck and Ward Cunningham.

Now unit testing is very effective, finding 75% of errors for only about 10-20% extra work, so you should definitly do that first. It pays for itself many times over.

Code inspection finds 90%, but is more work to set up. Pair programming is the most "eXtreme" form of this, but you can get great value just from a friend looking at a knotty bit. If you code alone, then this forum will help. Spouses and girlfriends aren't really interested and are just being polite. The very act of preparing code for inspection will make you so nervous, that you will make every effort to make it better.

Clear and exlicit code are enablers for just about everything else. After all, if you cannot read the logic of your own code, what chance have you of debugging it? Avoid "clever" code that realise on assumptions and sneaky deductions. Keep things simple, holding a grudge against every unnecessary line. I don't have any clear figures for defect removal for clear code, but I find better programmers write less code.

The magic here is that all three of these three practices interact. Testing will force you to clarify your intent and will lead to more direct solutions with less unnecessary code. Other people will demand clearer code, just so that they can read it too, and clearer code will in turn get more responses on a newsgroup/forum.

Once you automatically write almost bug free code (a year away at most), you can start to look at design. This is the next higher plain of existence for developers. It means that you can take on much larger projects, and the whole area is a lot more fun.

yours, Marcus

[DarkAngelBGE]

Regarding TDD (the hardcore version of unit testing), I wrote a short introduction here:
http://www.developerspot.com/tutori...ment/page1.html

Great article, Marcus. Does not delve too deep into the topic. As a short introduction into the topic I bet there is hardly any article on the web which achieves the intended goal so well.

Talking about its content...one thing I have always wondered about is the phase where one things about all possible tests. Now that certainly is the human factor. Are there any techniques out there that help you realize all possible states of an application?

Or speaking of the example in the article...what if we forgot to include the text for blank lines in the configuration parser? Are there any "tools" that would have helped us realize that this would be a possible state of the input data?

[Dr Livingston]

Marcus, sorry if I came across as being slightly annoyed, I am but as you say, there is nothing you can do about it currently, that's life, ain't it?

I don't get any errors, bar the error reported that E_STRICT has a problem (with PHP5 only) about the use for example, of $var instead of public, private, et al. But that is because Simple Test was based solely on PHP4 originally.

I welcome a newer version for PHP5 and look forwards to this, thanks for asking btw.

[lastcraft]

Hi...

Marcus, sorry if I came across as being slightly annoyed,

You didn't, don't worry.

II don't get any errors, bar the error reported that E_STRICT has a problem

Sorry, that's not what I meant. I mean why do you feel the need to turn E_STRICT on. What errors in your code are you hoping to catch with this PHP setting. As SimpleTest forces this to be turned off in the tests (although not the web tests, because they are a separate process), is this causing a hole through which a coding error could slip.

yours, Marcus

[lastcraft]

Hi...

Now that certainly is the human factor. Are there any techniques out there that help you realize all possible states of an application?

I once saw a bad translation of a Russian phrase which captures this quite well: "train your fantasy". I don't know how to "train your fantasy", but I guess you have to think whimsically about all possible disasters. A broad experience, different programming languages and different coding environments, and even different problem domains, probably helps as well. Beyond that I have no answer other than getting other people to look at it.

I kinda' wrote about this too...
http://www.lastcraft.com/blog/index.php?p=12
...but I don't have an answer there either.

yours, Marcus

[Dr Livingston]

Sorry, that's not what I meant.

Oh!! I just put it at the top of each page, isn't it a requirement (best practice) now?

error_reporting( E_ALL | E_STRICT ); // is what I use 

[DarkAngelBGE]

Hi...



I once saw a bad translation of a Russian phrase which captures this quite well: "train your fantasy". I don't know how to "train your fantasy", but I guess you have to think whimsically about all possible disasters. A broad experience, different programming languages and different coding environments, and even different problem domains, probably helps as well. Beyond that I have no answer other than getting other people to look at it.

I kinda' wrote about this too...
http://www.lastcraft.com/blog/index.php?p=12
...but I don't have an answer there either.

yours, Marcus

Thanks for the clear-up, Marcus.