My next framework.

[iackay]

Until there are 10,000 concurrent users.

I've been burned in the past by taking the db for granted. Accessing the db is about the slowest thing the application can do.

This is a false position. You should be using a caching mechanism. Furthermore, since this is a PHP forum damned near all your datatypes that go past the int/string etc variants are going to be arrays.

Pick up a NoSQL solution for things that lives as keypair values. Use memcache to keep a highspeed buffer.

I just find it weird you are trying to code a framework to do what you want when you should be building a third party library for something like CodeIgniter then augmenting it with a custom Templating library.

If you are talking 10,000 users you have no business not using some sort of cache. Also, you should be looking at Nginx and fastcgi instead of Apache with mod_php

Anyways, I don't mean to sound so negative toward your project. I'd really like to see more code. You have a github or bitbucket repo?

[Michael Morris]

As far as I'm concerned the cache is just another model type alongside the db. I do plan to use caching on this thing extensively. But the structure of the framework isn't due to caching concerns but rather load concerns, particularly in trying to make sure the latency on js callbacks is as low as possible even on a machine that has no major burden.

[Jake Arkinstall]

Out of interest, how is your user-authentication for specific pages going to work?

[Michael Morris]

Out of interest, how is your user-authentication for specific pages going to work?

If the user is authenticated there will be db. No way around that. When I say 10,000 concurrent users I'm referring to guests, bots, crawlers and the like. Ideally the pages should not necessarily need the db to fulfill these requests. By taking that approach resources are freed for the authenticated users.

[TomB]

Why would authentication require the database after the initial log in? Can't you just use sessions for that?

[Michael Morris]

Yes, in the absence of a load balancer. I've not seen a good way to deal with sessions when a load balancer is used other than employ the db.

Anyway, at this point I'm commenting on things that I haven't fully framed out so sometimes the answer I should be giving is "I don't know."

[Immerse]

Memcache is a great option for sessions over a load balancer.

[mefisto]

While we are on the subject of user-auth, i would like to bounce off this idea:

Lets say that user has a cookie , which is made from

$key = $user_login_hash . '|' . $timestamp . '|' . $cookie_vertification_hash;

When user opens the site , system vertifies user like this :

• can i make the vertification_hash from rest of the cookie and the user's IP ?
• is the <timestamp> acceptable ( not too old, not too new )

If the timestamp is too old but the cookie is valid, system request new cookie ( involving DB queries and the whole show ).



So ... would this work, or am i missing something ?

[Michael Morris]

Memcache is a great option for sessions over a load balancer.

I'll look into that. In any event, Database and Cache(s) will be treated as separate model data services. I believe that is the best approach.

[iackay]

I'll look into that. In any event, Database and Cache(s) will be treated as separate model data services. I believe that is the best approach.

You should be looking into Class Interfaces at that point. You can framework out some common methods then implement them in classes specifically for what options is needed.

Moreover, since memcache and mongodb/couchdb or any key->val system should be able to make a very elegant system for each type of datastore. One for keypairs engines and one for relational database engines.

[Ren]

Just posting this here, http://yoshinorimatsunobu.blogspot.c...story-for.html

Very interesting, and can reduce complexity and still outperform using memcached as a simple query result cache.

[Michael Morris]

The whole framework feels built around this method more or less. This post answers several questions up thread now that I'm prepared to answer them.

public function __invoke( $url = '/' ) {
		$this->setInitialRequestEvent();
 
		// Let the heartbeat commence.
		do {
			// Get an event.
			$event = array_shift($this->queue);
 
			try {
				// If the event is a page, assign it and clear the queue and responses.
				if ( $event instanceof Page ) {
					$this->startPage($event);
				}
 
				// Validate the page (yes, pages validate themselves).
				$event = $this->page->validate( $event );
 
				// Parse the event.
				$result = $event( $this->page );
 
			} catch ( Exception $e ) {
				//Problem, start an error page.
				$this->startPage (
					new $this->services['settings']['core']['errorPage']( $e, $event, $this )
				);
			}
 
			if (is_array($result)) {
				foreach ( $result as $r ) {
					$this->route($r);
				}
			} else { // Handle just the one result.
				$this->route($result);
			}
		} while( count($this->queue) > 0);
 
		/**
		 * Now as our last step give the page the responses and compose the
		 * final response.
		 */
		$this->page->respond( $this->responses );
	}

The code that precedes this function's call exists just to get this thing running smoothly. To review what is upthread:

The framework is divided into events. Pages are a special type of event which provide a context for the others. GET events require a GET request from the browser to access, POST events require a post event. INIT events are called by the page anytime it's instantiated, and EXCEPTION events manage exceptions that must be reported to the outside world because they are uncaught or are the fault of the user (404 Not Found for example). Modules are areas of a page, they have events of their own.

Examples.

Homepage.

1. User arrives at site, URL is '/'.
2. Router determines "Page" is the class to handle this.
3. Page is started by dispatcher.
4. When page validates itself as an event it simply returns itself.
5. Page's own event fires. It returns an array with it's init event ("UserAuthOptionalEvent") and it's modules ("Login", "HeaderMenu", "HomeNews", "FooterMenu", "HeaderAd") and finally the requested event ("IndexEvent").
6. UserAuth returns a guest event.
7. The modules run, each returning their IndexEvent. Those are put on the queue.
8. The page indexEvent runs, returning a reponder. It is put on the responder queue.
9. Each module indexEvent runs, returning a responder. Again, those go to queue.
10. Having exhausted the queue, Event Dispatcher calls page->respond and gives it the responses.
11. Page->respond pulls the page's own index event from the response queue (it should be first because of LILO event resolution) and gives it the other responses and orders it to respond.
12. The responder composes the response and prints it.

The user logs in.

1. URL is /index.php?module=login&action=login - method is post/AJAX
2. Router determines "Page" is the class to handle the page.
3. Page is started by dispatcher.
4. When page validates itself as an event it simply returns itself.
5. Page's own event fires. This is an an AJAX request and a module is specified. Page does not do it's init, the module will. It's returns the login module, but it also returns the modules that are "listening" for the login action - the final list is "Login", "HeaderMenu", and "HeaderAd" (which is listening for * so refreshes on any action).
6. Login runs, is successful, return UserInfo IndexEvent,
7. The listening modules run, returning their indexEvent (A listener can be a unique event or a pointer to a refresh event.)
8. Each module indexEvent runs, returning an HTMLResponder for their new content and a JSresponder to execute their update.
9. Having exhausted the queue, Event Dispatcher calls page->respond and gives it the responses.
10. Page Responder diverts to AJAX response. This function creates an XMLResponder then harvests the array of responders into an XML tree (see example below) and returns it.
11. Clientside script updates relevant areas of the page.

That XML response looks like this

<r>
  <j> (the javascript callbacks of each module, in order.) </j>
  <UserInfo><![CDATA[ ( html) ]]></UserInfo>
  <HeaderMenu><![CDATA[ ( html ) ]]></HeaderMenu>
  <HeaderAd><![CDATA[ ( html ) ]]></HeaderAd>
</r>

And that's all. I hope it's clear now why I feel that this truely something different from the other frameworks out there - it is an Event Based MVC system rather than the traditional static MVC with AJAX tacked on as an afterthought, if at all.

Note that in the second example if the form is submitted without AJAX the page will be able rebuild itself and send a new copy of itself out.

[lampcms.com]

I agree that DB is the slowest part of CMS. MySql is great at first, but as you tables grow in size, mysql starts to use temporary tables during selects, especially when there are some table joins. I think if you want your cms to be able to handle very large data you should start looking at NoSQL database as a backend. Something like MongoDB or Cassandra or CouchDB. But of cause it's totally different way of thinking, not like mysql at all. In NoSQL there are always duplication of data, something that programmers try to avoid at all cost with traditional database design is a norm with NoSQL.

There is also a new emerging trend to use Node.js, a server-side javascript processing. This way you can write your templating language in javascript then you will be able to execute the same javascript on the server for server-side processing or in browser for ajax-based processing.

[Michael Morris]

As of this morning I have the first draft Control and Response system essentially completed. I still need to write some exception handling routines. Now comes the conversion of my old data access system over to using PDO. The framework will use that for the management of its own internals, but should be able to deal with a third party model such as Doctrine being used provided a wrapper is applied.

Hell, as much as I hate smarty, it can be used if a wrapper class is employed.

[PCSpectra]

And that's all. I hope it's clear now why I feel that this truely something different from the other frameworks out there - it is an Event Based MVC system rather than the traditional static MVC with AJAX tacked on as an afterthought, if at all.

Posting code would be best, but explaining how it's different from existing solutions, comparing to WordPress, Drupal, Joomla, CakePHP, CodeIgnitor, Zend and maybe Symphony.

MVC is a term loosely thrown around, yet most don't really enforce MVC rather they promote a controller architecture. Whether the developer implements business logic (or some of it) in the controller layer is entirely dependent on the implementor. Admittedly it is hard to enforce true model/controller/view seperation and whether all developers agree on what constitutes business logic and controller logic will vary as well.

It sounds as though you have a page assembly engine integrated with your interpretation of MVC.

I use Drupal a lot these days, and while it requires a lot of hacks to get things just as I like them. I am always curious how system X is better than Drupal? Worse than Drupal? Or Joomla, etc.

What is the key difference here? From what I gather from your post above, it is that events are triggered server side and AJAX on the server can query for view partials?

For example if someone logs into the site, the view partial responsible for showing the login FORM will automatically be replaced by the authenticated view, such as a "Welcome PCSpectra"???

Cheers,
Alex

[lampcms.com]

Right now what you have is an idea. When it's ready, let people download it and decide for themselves if it's different or better than other frameworks.

I am not an expert in ALL frameworks but I'm pretty sure Symfony 2 is event driven too.
take a look, it's right in their description, under "Flexible"
http://symfony-reloaded.org/

I know it's much more difficult to make a framework that handles ajax and non-ajax elegantly. It's easier said than done. Ajax is not just sending a small block of html via XHR response. You can't just have a router that looks at request and decides, hey ,this is ajax request, I will send ajax response, else I'll send normal HTML. In the real world it's often more complicated. Once you really start working on it, do a lot of hand-on development you will find it is not as simple as you may think now.

Anyway, good luck to you, post a link when your framework is ready.

[Michael Morris]

You only get one chance to make a first impression. And trust me, I am by no means underestimating the complexity of this endeavor. But it's not ready. It will be a few months yet before it's even close.

[PCSpectra]

Haha...I disagree (especially in software) you have unlimited attempts at making a first impression. How many times over the years I picked up CMS frameworks and dropped them a few weeks later. I started with a project called Midgard and later switched to one of the Nukes. Then Mambo, Joomla, Drupal, the latter two back and forth so many times I have lost count. Here I am today, convinced Drupal is the best there is at getting things done ASAP, with Joomla having a slight edge in terms of clean architecture. I've given CodeIgnitor and Zend so many 'firsts' I have all but given up on both for the most part.

Getting user feedback from the community will be impetus in improving your design. Otherwise, what I am willing to bet, is you end up with a system with flaws in the design, questionable API, etc. Once people start using it however, it's very difficult to change the design. Drupal I am sure, know they could do thngs better but changing that now would deprecate 1000's of existing modules and force everyone to re-learn, so they simply cannot change.

Cheers,
Alex

[Michael Morris]

Haha...I disagree (especially in software) you have unlimited attempts at making a first impression. How many times over the years I picked up CMS frameworks and dropped them a few weeks later. I started with a project called Midgard and later switched to one of the Nukes. Then Mambo, Joomla, Drupal, the latter two back and forth so many times I have lost count. Here I am today, convinced Drupal is the best there is at getting things done ASAP, with Joomla having a slight edge in terms of clean architecture. I've given CodeIgnitor and Zend so many 'firsts' I have all but given up on both for the most part.

Getting user feedback from the community will be impetus in improving your design. Otherwise, what I am willing to bet, is you end up with a system with flaws in the design, questionable API, etc. Once people start using it however, it's very difficult to change the design. Drupal I am sure, know they could do thngs better but changing that now would deprecate 1000's of existing modules and force everyone to re-learn, so they simply cannot change.

Cheers,
Alex

Which brings up a second reason to hold off until I'm completely happy with the API. If people start depending on it, you can't very well change it.

[Mittineague]

I tend to consider the release. i.e. if its Alpha I expect it to be rather crude, Beta a bit more polished but by no means pefected, RC ready to go with a few bugs but before the onslaught of feature bloat.

True, I think many are "scared away" or take the "I'll let others do take their chances first" approach, but I see nothing wrong with releasing a Beta.

In fact I think releasing a Beta is a crucial step in the process. There's no possible way I can test for all the various use scenarios.

For my own apps, they're Alpha until I'm happy with them. Then I release a Beta (or three ). Once I've gotten enough feedback (subjective I know) then the RC

I've yet to see any app that was "final" except where the developer stopped working on it. So don't suffer from "analysis paralysis" once you're happy with what you've got.

[Michael Morris]

I tend to consider the release. i.e. if its Alpha I expect it to be rather crude, Beta a bit more polished but by no means pefected, RC ready to go with a few bugs but before the onslaught of feature bloat.

True, I think many are "scared away" or take the "I'll let others do take their chances first" approach, but I see nothing wrong with releasing a Beta.

In fact I think releasing a Beta is a crucial step in the process. There's no possible way I can test for all the various use scenarios.

For my own apps, they're Alpha until I'm happy with them. Then I release a Beta (or three ). Once I've gotten enough feedback (subjective I know) then the RC

I've yet to see any app that was "final" except where the developer stopped working on it. So don't suffer from "analysis paralysis" once you're happy with what you've got.

Well, except for isolated snippets I'm not ready to have anyone look at it.

[gRoberts]

This point might be a little bit past your current conversation, but not ALL frameworks require a database connection at start.

CodeIgniter has the option to load the database class as soon as the request is received, but like my current setup, it will only ever load the database object, when the database is needed. Otherwise, a request doesn't even touch the database.

That said. I love CodeIgniter for it's ease of use and what not, but I'm finding the thought of writing my own framework daunting, yet I feel compelled to write one.

I haven't had the chance to read every single post, explaining your current framework and the reasons for setting it up, but I think at the very simplest form, an MVC framework is the way to go.

When I first looked into MVC, it was scary. But CodeIgniter made it seem VERY simple, simple enough that I could create the MVC structure very easy. The parts I struggled with was handling of data etc to make it scalable.

In the end, I simply gave up. Until now, when your post, threw my brain into, hell, lets try it again mode lol

Hope it all pans out mate, and as everyone else is saying, get some code up so we can have a fiddle with it.

[iackay]

This point might be a little bit past your current conversation, but not ALL frameworks require a database connection at start.

CodeIgniter has the option to load the database class as soon as the request is received, but like my current setup, it will only ever load the database object, when the database is needed. Otherwise, a request doesn't even touch the database.

That said. I love CodeIgniter for it's ease of use and what not, but I'm finding the thought of writing my own framework daunting, yet I feel compelled to write one.

I haven't had the chance to read every single post, explaining your current framework and the reasons for setting it up, but I think at the very simplest form, an MVC framework is the way to go.

When I first looked into MVC, it was scary. But CodeIgniter made it seem VERY simple, simple enough that I could create the MVC structure very easy. The parts I struggled with was handling of data etc to make it scalable.

In the end, I simply gave up. Until now, when your post, threw my brain into, hell, lets try it again mode lol

Hope it all pans out mate, and as everyone else is saying, get some code up so we can have a fiddle with it.

I went from Ruby on Rails to CodeIgniter back in ver 1.6 of CI and found it refreshingly simple. I didn't need migrations at the time and I felt more comfortable with pure SQL than some migration system so the CI db forge can make making your own simple migration system easy.

Also, CI makes it easy to find out if you have an AJAX request or not, to much of the chagrin of the poster on this page somewhere who said it could not.

Put that into the constants.php file in the app/config folder:

//
// Define Ajax Request
define('IS_AJAX', isset($_SERVER['HTTP_X_REQUESTED_WITH']) && strtolower($_SERVER['HTTP_X_REQUESTED_WITH']) == 'xmlhttprequest');
// 

Then you can use IS_AJAX in your controller to lock down if a call is ajax or not. That simple.

Your controller method would look like

public function get_data() {
if(IS_AJAX) {
//retrieve data from somewhere
} else {
//print error that it was not an ajax call.
}
} 

This sort of thing should probably be included in the base install but that is what makes CI so good, it can be tweaked to the needs of users. It might actually be something the OP wants to look into as it should be useful for any MVC.

[Michael Morris]

At this point I'm working through the unit testing system. Control and view seem solid but I want tests for them to prove it. This rebuild is taking longer than any previous version of the thing for better or worse.

[oddz]

@iackay

Using HTTP_X_REQUESTED_WITH for authentication is fundamentally flawed since it can be spoofed. Hopefully its not being used for that purpose otherwise anyone can spoof it and access actions without restrictions if no additional, server-side authentication exist. In that regards its pretty much worthless.