Data Access Layer - Is there a real seperation?

[Tony Marston]

This is the only place where my implementation diverges. Due to the passive nature of HTTP my controller tells a model to do something, and when it has done it the model gives its response back to the controller. The controller then takes this response and converts it into an XML document before giving it to the view (an XSL stylesheet) for processing. Thus my views have no communication with any model.

Looking at my code I think that defining my view component as nothing but the XSL transformation may be incorrect. I have a small number of generic controllers which are procedural, not OO, which may call other procedural functions as well as instantiating and communicating with objects in the business layer. The final step in every controller (when it needs to create XHTML output) is to call a procedural function buildXML() with a list of objects. This function pulls data out of the objects, constructs the XML document and finally performs the XSL transformation. If you regard this function to be part of the view then it (the view) does indeed pull data out of the object(s) before it pushes it to the XSL stylesheet.

[Tony Marston]

It is physically impossible for an XSL stylesheet to 'pull' data out of an application object - this has to be converted into an XML document so that it can be 'pushed' into an XSL transformation process.

That is, unless you use XSLT extensions and access your application components directly from your stylesheet.

http://www-106.ibm.com/developerwork...y/x-exslt.html

Very interesting, but the examples in that article only demonstrate that the EXSLT extension can be used to access a simple string (current date and time in the example) or another XML document. It does not indicate that you can access a database directly to retrieve multiple rows and/or columns.

[Tony Marston]

Having 100 or so functions in a class is a serious bad code smell. Doing these things and refactoring will drop the amount of code in your architecture by a ton.

Removing functions from a class so that the number does not exceed an arbitrary limit does not, in fact, reduce the amount of code. If those functions must exist then I am forced to put them into another class. This therefore increases the number of classes and so increases the amount of code as I now require additional code to instantiate and communicate with these new classes. If I have functions for an entity distributed accross several classes I also require extra code to determine which of those classes I need to communicate with at any moment in time. So instead of reducing complexity you are actually adding to it. My nose is beginning to twitch. Does it detect a bad smell?

[Tony Marston]

Good design is about finding the right balance. Design is navigating the tension between opposing principles. You can have classes which are too big and you can have classes which are too small. The trick is to make classes which are just right. Similarly, you can have too many objects and you can have too few objects. The trick is finding the balance.

The terms 'too many' and 'too few' are difficult to quantify, but I came across this description (I forget where) which gives some useful pointers:-

What is the definition of too many classes/variables/functions? More than you need.

What is the definition of too few classes/variables/functions? Less than you need.

I basically have one class per business entity. Anything more would be too much. Anything less would be too few.

Each class has enough variables to achieve its purpose. Anything more would be too much. Anything less would be too few.

Each class has enough functions to achieve its purpose. Anything more would be too much. Anything less would be too few.

Sounds reasonable to me.

[sweatje]

The terms 'too many' and 'too few' are difficult to quantify, but I came across this description (I forget where) which gives some useful pointers:-

I basically have one class per business entity. Anything more would be too much. Anything less would be too few.

Each class has enough variables to achieve its purpose. Anything more would be too much. Anything less would be too few.

Each class has enough functions to achieve its purpose. Anything more would be too much. Anything less would be too few.

Sounds reasonable too me.

It does sound reasonable, but it is addressing a different question than the point most posters in this thread are bringing up. It is good to have the "baby bear" number of methods (not too many, not too few ), but I think most of the objectors to your code feel that your choice of a "business entity" is too coarse grained, and thus attracts too much functionality to it.

Again this appears to be a definitional issue, where you are happy with your choices (related to the scope of the class) and others are free to disagree with them.

[dagfinn]

Removing functions from a class so that the number does not exceed an arbitrary limit does not, in fact, reduce the amount of code. If those functions must exist then I am forced to put them into another class. This therefore increases the number of classes and so increases the amount of code as I now require additional code to instantiate and communicate with these new classes. If I have functions for an entity distributed accross several classes I also require extra code to determine which of those classes I need to communicate with at any moment in time. So instead of reducing complexity you are actually adding to it. My nose is beginning to twitch. Does it detect a bad smell?

If you assume that extracting classes is done mechanically and mindlessly, you're right. It can only increase the amount of code.

But if you do it sensibly, you're likely to be able to extract some logical subunits of the large class. That could make the code more readable even if there is a little bit more of it.

More importantly in my experience, very large classes are practically a guarantee that there is duplication present, and the very size of the class makes it hard to find out what's duplicated. Breaking it down into smaller, more logical units will make it easier to spot duplication.

[Tony Marston]

And your point is .... ?

I should really have quoted some of McGruff's post, as mine was a responce to it - though now a responce to something deleted. He quoted a passage from your site saying that you did not care what others though, so I made an attempt to reiterate a point made by Brenden Vickery using your own words.

If I am guessing right you are wondering why I have classes with large numbers of functions (which makes them 'complex') whereas you quoted an article of mine:-

From Tony Marston's site: Large and Complex vs. Small and Simple

Consider the following business requirements:
The following functions are required for object 'A' (e.g. maintain customers or invoices)

• Enter selection criteria.
• List all occurrences that meet those criteria.
• Create a new occurrence.
• Display more details for a selected occurrence.
• Update the details of a selected occurrence.
• Delete the details for a selected occurrence.

Where does it say that all these functions MUST be provided in a single component? It does not. Users may expect to see all that functionality in a single component, but only because that is what they have been used to in the past, or because it is the only option proposed by inexperienced designers. Neither party may be aware that there is an alternative which is actually easier to implement and just as easy, if not easier, to use.

How can I account for this apparent contradiction? Simple. The six components in the above list are all presentation layer components, whereas the 'complex' classes exist in the business layer. In the 3-tier architecture you can have any number of presentation components access the same business component. This is so that the same business logic can be shared by any number of presentation components.

Each presentation component is 'small and simple' as it only performs a limited function. However, the business component has to cope with the demands of any and all presentation components which may access it, therefore it is more complex. 'More complex' is not necessarily the same as 'too complex'. Each business component contains just enough complexity to satisfy the demands of the various presentation components. If the business object contained more complexity than was required to fulfill its purposes then that would be 'too much'.

[Tony Marston]

I should really have quoted some of McGruff's post, as mine was a responce to it - though now a responce to something deleted.

I started to say something then I just thought: "don't feed the troll".

By my understanding a "troll" is defined as

A person who makes posts (on newsgroups or other forums) that are solely intended to incite controversy or conflict, or cause annoyance or offense.

My posts do not fall into that category as their intention is to defend my point of view which has been attacked most vociferously in this forum. It is people like you who take offense simply because I dare express an opinion which is different to your own, and I'm not afraid to defend that opinion.

[Tony Marston]

One thing that really annoys me about articles and tutorials on OOP that I have found on the web and in books - they all talk about creating a class called 'shape' with various subclasses for 'square', 'circle', 'triangle' etc. This is of absolutely no use when I want to build a system to deal with real-world objects such as 'customer', 'product' and 'invoice' which have corresponding database tables. This has often led me to believe that OOP is therefore unsuitable for building common-or-garden business systems as it appears to have been designed for nothing but graphical applications.

Could I ask you, Tony, to add a link to this discussion in your online "tutorials" (?). I'm scared that learners might be misled.

Just type in http://www.google.com/search?hl=en&l...ls&btnG=Search and follow the links.

[Version0-00e]

Simple.

Always got an answer for everything huh?

[Selkirk]

Removing functions from a class so that the number does not exceed an arbitrary limit does not, in fact, reduce the amount of code. If those functions must exist then I am forced to put them into another class. This therefore increases the number of classes and so increases the amount of code as I now require additional code to instantiate and communicate with these new classes.

But splitting a class up along an axis of change allows you to use different inheritance hierarchies for each axis of change. The smaller classes are more reusable. These factors can indeed help to reduce the total amount of code.

In the example that was posted here, you have a whole series of methods with the naming patterns _cm_* (23 methods) and _dml_* (10 methods) on your Default_Table class (86 methods!). Those naming patterns look to me like smaller classes screaming to get out of a big ball of mud.

[Tony Marston]

And then I wandered into this.

> Isaac Newton did not have to rediscover for himself a
> thousand years of Science. He stood on the shoulders of giants.

He had giants, I have pygmies.

I'm speechless, quite speechless.

For someone who is speechless you seem to be making a lot of noise.

The remark "He had giants, I have pygmies" is not meant to imply to everyone else in the programming universe is a pygmy, just those with whom I have come into personal contact over the last decade or so. Let me attempt to explain the situation from my point of view.....

While working for a software house in the mid-1980s I designed and built a development infrastructure in COBOL for a project for which I was the project leader. This infrastructure impressed by fellow workers and my bosses so much that they made it the company standard, and from then on it was used in every project that the company undertook.

That software house switched from COBOL (a 3rd generation language) to UNIFACE (a 4th generation language) in the 1990s, and so I redeveloped my infrastructure in that new language. With this new infrastructure I could build components at an average speed of one per hour, with the simplest taking no more than 20 minutes.

I was then headhunted to join a new project which (supposedly) had need of someone with proven expertise with development infrastructures. Despite my track record in this area the project architects ignored eveything I had to say and attempted to build an infrastructure using all the concepts and theories which are constantly occurring in this thread. To cut a long story short it was a total disaster. They spent a total of 3 man-years creating an infrastucture, and when they attempted to build the first simple 'live' components it took 2 (two) weeks. That is 70 (seventy) hours against my 1 (one)! The customer did not like the effect that this would have on his budget and timescales, so he (quite rightly, in my view) cancelled the project immediately in order to contain the damage.

I then joined another company who (supposedly) had need of my skills to assist them in upgrading their 2-tier system to 3-tier, but the project never materialised. As I had no desire to work as a maintenance programmer on their outdated infrastructure which took an average of 7 hours (1 day) to build each component I left.

I joined another company which had developed their own 3-tier infrastructure, but it was ful of so many deficiencies that their development times were 10 hours (1.5 days) per component.

I have worked briefly for a few other nondescript companies since then, but their development infrastructures have all been outdated, slow and cumbersome.

I have strived since 1985 to produce development infrastructures which allow developers to achieve a high rate of productivity. Anybody whose rate is lower than mine is a pygmy. Anybody whose rate is higher than mine is a giant. So far I have met only pygmies. Now do you understand where I am coming from?

I have created a development infrastructure for PHP that has enabled me to achieve a higher rate of productivity than I could with any of my previous infrastructures. I have documented it. I have made the source code available on my website. What do I get in response? Verbal abuse.

Your methods are all wrong!
You wouldn't catch a REAL oo pragrammer doing it that way!
Your classes are too big!
You are not supposed to write classes that way!
You are not using the right design patterns!
You don't know what 'separation of concerns' means!
et cetera, et cetera, et cetera

Words! Words! Words! Stop making wild accusations and spouting wild theories! Give me proof! Until someone can actually give me a sample of runnable code (like the one provided on my website) that outshines my own then I will have no option but to take all these words, add a pinch of salt, and file them away along with all the other "unproven theories". Stop boasting how clever you are and prove it. Either put up or shut up.

[Selkirk]

It seems to me that trying to apply the original smalltalk definition to Web MVC is one of the main sources of confusion on the subject.

Perhaps, but for me it was an aha moment that really clarified MVC for me. I feel that taking too broad a view of MVC is the main source of confusion about it, because then almost anything that attempts to separate domain and presentation can be called MVC (and often is). But, in that case, calling it MVC doesn't give you guidance for implementing it. The benefit of following a more strict smalltalk inspired definition of MVC is that you get then get a strict set of principles to help you implement the pattern and to answer questions like "where should X go?" or "what is a controller?"

The disadvantage to this is that it leaves other patterns for separating domain and presentation largely nameless (there is Presentation Abstraction Control) and outside the circle of wagons.

[Tony Marston]

Back on the good/bad oop page:

As far as I am concerned you bunch of tragicians [2] have taken a reasonable (note that I do not go so far as to say 'brilliant') concept called OOP and turned it into POOP (as in pooh, bowel movements, buffalo chips, compost, cow pats, crap, doo-doo, doggy-doo, droppings, dung, effluent, excrement, excreta, faeces, fertilizer, guano, jobbies, manure, muck, prairie oysters, road apples, sewage, turds [3]). Your petty rules are a hindrance to productivity rather than a help, so I suggest you take them away and flush them down the toilet where they belong. Instead of being OOPers you are POOPers [4].

It's not mentoring he needs: it's a therapist.

Colourful, isn't it?

I just love the way you broadcast my opinions to such a wide audience.

[Tony Marston]

That's why I just say that I respect your opinion even if it seems to differ from mine.

Then I shall strive to give your opinion the same respect.

[Tony Marston]

Simple.

Always got an answer for everything huh?

As I am supposed to be arrogant then isn't that supposed to be the case?

[Manor]

Designing according to the 'real' design practises, will give you less headache when discovering new problems and requirements in the future, plus giving good names and good roles makes your project documented by nature. Revisiting your code will be less painfull, and showing the code to others will not stress them out. These positive points will only be notionable after a longer period of using and changing the project. So, arguably, this method will only be better for a larger project.

Your way, trying extremely hard to accomplish things the simplest way possible, is not bad. At least, for some projects. Coding this way will hurt you on the longer period. That is, if your project needs to be maintained and changed more then once.

The usefulness of your development infrastructure will be rated by the fact which project they/you will be using it on.

Finally, your higher productivity rate may be better at first, but will get worser as time goes by in the projects lifetime.

The type of project defines the requirements. Tony's way has positive sides, too. As does the 'correct' way.

Just my opinion.

[Tony Marston]

I take a more precise and limited view point of MVC that is closer to the original smalltalk definition, with the controller focused on input, and translating that input into operations on the model.

It seems to me that trying to apply the original smalltalk definition to Web MVC is one of the main sources of confusion on the subject.

But your point about being able to change from one template engine to another is a good one. If there's logic in the Controller that depends on the nature of the template engine, then that's a limitation in the design. But if it's easy to extract that part of the controller, then it's less of a limitation. If it's in a separate class or classes, then you could consider it nicely separated but poorly labeled.

If you take a look at post #226 you will see that by relabelling part of my code I have 'moved' a single function out of the controller and into the view, thus enabling my view to pull data out of the model.

There, that was painless wasn't it?

I couldn't care less about "valid" or "invalid", "right" or "wrong". What counts is how difficult it is to make a certain kind of change.

I get the impression that we are both singing from the same hymn sheet!

[seratonin]

Very interesting, but the examples in that article only demonstrate that the EXSLT extension can be used to access a simple string (current date and time in the example) or another XML document. It does not indicate that you can access a database directly to retrieve multiple rows and/or columns.

It's not pretty, but it's possible:

http://code.box.sk/newsread.php?newsid=198

And here is a way to access a node set from a PHP function:

http://slides.bitflux.ch/phpug2004_1/slide_37.html

JT

[Tony Marston]

I basically have one class per business entity. Anything more would be too much. Anything less would be too few.

It does sound reasonable, but it is addressing a different question than the point most posters in this thread are bringing up. It is good to have the "baby bear" number of methods (not too many, not too few ), but I think most of the objectors to your code feel that your choice of a "business entity" is too coarse grained, and thus attracts too much functionality to it.

Then let me define precisely what I mean by "business entity". In my 16 years with COBOL we did not have 'classes', we had 'datasets' or 'database tables'. In my 10+ years with UNIFACE we worked with 'entities' within an application model (aka data dictionary) where each 'entity' equated directly with a 'database table'. It was possible to construct the database schema from the application model, and it was also possible to import an existing database schema and automatically create the application model. All data manipulation was done against 'entities' within the application model, and a database driver (delivered as a pre-compiled library file which had different versions for different database engines) would then handle all communication with the physical database.

In the original version of UNIFACE the components that we developed contained both presentation and business logic, with all data access controlled by the database driver, so it had a 2-tier architecture. In a later version which provided support for the 3-tier architecture we developed 'form' components which handled the presentation logic, and 'service' components which handled the business logic. Basically the 'form' components had screens while the 'service' components did not, and the 'form' components had to talk to 'service' components in order to carry out user requests.

Each of these 'service' components in the middle or business layer was initially created around a single 'entity' (database table), although it was possible to access more 'entities' (database tables) within the 'service' component. This allowed the physical implementation of an entity's structure to be hidden from the form component. It did not need to know that the data it was accessing was actually spread over several different tables.

I use the term 'business entity' instead of 'business object' as 'object' may be taken to mean 'something instantiated from a class' instead of just a 'thing'. If I describe each 'business entity' as a database table then I'm afraid that would resurrect this old argument.

If I describe 'customer', 'product' and 'invoice' as entities I actually mean database tables. If the 'customer' information is actually spread over several tables, such as 'address' or 'contact' then each of these would be a separate entity which would require their own OO class. I certainly would not have a single 'customer' class that would deal with every possible database table that was required to hold a customer's details. Even though each class in my business layer starts off as being concerned with a single entity (database table) I have the option to change the internals of any class so that it can access as many tables as is necessary to get the job done. Thus I can hide the physical structure of the database from the components in the presentation layer.

Does this clear it up? Or have I added to the confusion?

[Version0-00e]

As I am supposed to be arrogant then isn't that supposed to be the case?

Arrogant? Nah, you've just gone way past that in my view. Countless number of people on this forum have explained in any number of ways of how and why you could (and proberly should) implement a better design and development process, for example your class(es).

87 class methods does not make good OO Programming. Anybody who has ever developed with OOP to any extent will tell you that. But you just keep going and totally disregard this, with an unjustifyable comment of it being too complex.

You say you have had 20 years in the business? If so then you would already have the experience to know that a class wouldn't have that many methods. So what that you have developed any number of infastructures over the years? That doesn't mean that you've implemented best practices, nor took any notice of design and analysis at any level.

But you continue to bang your drum on how cool and great your infastructure is. I personally fear that your approaches give bad advice to younger less experienced developers further down the road.

You have some good methodologies sure but by no means is your infastructure, nor your attitude the only approach.

As I had no desire to work as a maintenance programmer on their outdated infrastructure which took an average of 7 hours (1 day) to build each component I left.

Did you ever actually give this job a chance to grow on you? In fact can you tell the board just how long you manage to last with this company before 'you' decided that you were 'just' too good for them?

This is the problem at the end of the day, it's your attitude and it sucks.

[Selkirk]

Regarding encapsulation versus visibility, i'll quote from Meilir Page-Jones' book "Fundamentals of object oriented design in UML," which i think is the best book on the subject.

Jones introduces a term called Connascence, which you can also interpret as interdependency:

Connascence, which is derived from Latin, means "having been born together." An undertone to this meaning is "having intertwined destinies in life." Two software elements that are connascent are born from some related need and share the same fate for at least one reason. Following is the definition of connascence as it applies to software:

Connascence between two software elements A and B means either

• That you can postulate some change to A that would require B to be changed (or at least carefully checked) in order to preserve overall correctness, or
• that you can postulate some change that would require both A and B to be changed together in order to preserve overall correctness.

The definition of interdependency is in terms of hypothetical change.

Jones goes on to discuss properties of connascence (explicit, implicit, directional, nondirectional, static, dynamic) and then enumerates several varieties:

Connascence of name
Connascence of type or class
Connascence of convention
Connascence of algorithm
Connascence of position
Connascence of execution
Connascence of timing
Connascence of value
Connascence of identity
Connascence of timing
Connascence of value
Connascence of identity

suffice it to say that there are many ways for two software elements to be interdependent. (here is another taxonomy of dependency along a different axis.)

Jones defines a particular type of Connascence, which he calls Contranascence, or "Connascence of difference."

Contranascence: Between software elements A and B, a form of connascence in which there is some property of A that must be held different that the corresponding property of B.

He gives this example:

Let's say that we have two declarations:
int i;
int j;
For correctness -- indeed, merely for the code to compile! -- the variable names, i and j, must differ from each other. There's a connascence at work here: If for some reason, we wanted to change the first variable name to j, then we'd also have to change the second name from j to something else. Thus, the two declarations are not independent.

Now, jones gives three rules based on these definitions for writing maintainable programs.

• Minimize overall connascence by breaking the system into encapsulated elements. This includes minimizing contranascence. Minimizing contranascence what limiting visibility does, through encapsulation and scoping.
• Minimize any remaining connascence that crosses encapsulation boundaries. (Coupling)
• Maximize the connascence within encapsulation boundaries. This is another way of saying that a class should do one thing or represent one abstration. (Cohesion)

One last quote:

So, a system that is not broken into encapsulated units has two problems: rampant connascence (chiefly through contranascence), and the confusion over what is true connascence and what is accidental similarity or adjacency.

So encapsulation and visibility help distinguish the accidental from the intentional.

Let's say that component A defines a global variable in the $_GLOBALS array. Now, that same variable is visible to component B, but unless component B *uses* that variable, I wouldn't say that it is dependent on it. Same goes for functions and classes. Just because a class can *see* a particular variable, function, or class does not mean that it is dependent on it. Visibility != Dependency

If a tree falls in the woods and no one is there, does it make a sound? If you declare something that no one uses, is there a dependency?

By the principle of contranascence, even being visible can introduce a dependency of difference. There are changes you can no longer freely make.

[Tony Marston]

If you assume that extracting classes is done mechanically and mindlessly, you're right. It can only increase the amount of code.

But if you do it sensibly, you're likely to be able to extract some logical subunits of the large class. That could make the code more readable even if there is a little bit more of it.

There is the problem. At the moment I can only see that breaking up my classes into smaller units will create a pile of poo. I am certain that it will actually generate more code, but I am also pretty certain that the result would be excrement instead of excellent. Only definitive working proof would be able to convince me otherwise, I'm afraid.

More importantly in my experience, very large classes are practically a guarantee that there is duplication present, and the very size of the class makes it hard to find out what's duplicated. Breaking it down into smaller, more logical units will make it easier to spot duplication.

AFAIAA there is no duplicate code in any of my classes. I have taken all reusable code and put it into separate functions that can be called from anywhere. That is one of the reasons why there are so any small functions within my classes.

[Tony Marston]

In the example that was posted here, you have a whole series of methods with the naming patterns _cm_* (23 methods) and _dml_* (10 methods) on your Default_Table class (86 methods!). Those naming patterns look to me like smaller classes screaming to get out of a big ball of mud.

For your information those classes beginning with '_cm_' are supposed to be private and abstract classes which may or may not be redefined within any subclass. Those beginning with '_dml_' are private classes which are used to communicate with a separate DML object (aka the DAO).

You keep telling me that I should split this class into smaller classes, but until you provide me with working samples to prove your hypothesis I shall continue to disbelieve you.

[Selkirk]

You keep telling me that I should split this class into smaller classes, but until you provide me with working samples to prove your hypothesis I shall continue to disbelieve you.

Don't make me refactor your architecture!