Help with sax parser and states

**Austin_Hastings** · Dec 14, 2004 21:36

Originally Posted by mwmitchell

I'm not sure when the replacements should occure. I'm thinking there should be a tokens replace filter somwhere along the chain. But where? I want to be able to use the tokens in attributes, and possibly in a tag like:

<set token="template" value="guest.tpl">
<if token="registered">
<set token="template" value="user.tpl"/>
</if>

<include file="{$template}"/>

Which means the replace filter would have to do it's thing first.

Sounds like you now have a SaxFilter ordering requirement, possibly the first.

I think your SET processor has to come late in the chain, so that the IF processor can prevent it from running. The INCLUDE processor will be difficult -- I think I tried to indicate that in the other thread -- since you'll need to process token replacement as you've indicated, then handle the include, which will create a whole new parser, etc.

I'm thinking something like this:

Parser->${token replacer} -> IF -> SET -> INCLUDE -> Writer

The INCLUDE filter should probably create a whole new filter-chain, with its successor (Writer) as the last node in the chain:

Parser->REPLACE->IF->SET->INCLUDE[Parser->REPLACE->IF->SET->INCLUDE->]->WRITER

Be careful about your substitution syntax. If you conflate it with PHPs syntax, things could get interesting. It is probably okay if you make sure you never do an eval, and make sure you quote all the things you pull from the user.

Also, the SAX parsers tend to give you "lines" of character data, instead of giving you the entire block. So if some user writes:

<set token = "reallylongname" value="1" />
<p>${reallylongname}</p>

the parser might break the second line in the middle, preventing you from recognizing the token.

You'll want to make your replace-o-mizer accumulate text until all tokens are closed (match ${ with } ) or until something other than a character data call occurs.

=Austin

**mwmitchell** · Dec 17, 2004 15:23

If I were to (while parsing) build a tree of object html elements that match their name type (a, td, tr etc.), how do I do this with tags that are suppossed to be closed but aren't. Do you know how this is dealt with? I guess the input would need to be converted to valid xhtml first? Example:

<html>

<head>


<meta XXXXXX>

</head>

<body>


<img src="">


<br/>

</body>

</html>

**Austin_Hastings** · Dec 17, 2004 16:27

I think you're going to have to handle that on your own. Look at the Tag Soup java parser for HTML for an example of how hard this can actually be.

=Austin

**mwmitchell** · Dec 20, 2004 22:24

OK yeah forget that! I'd rather start using DOM and PHP5.

I've now run into a problem of how to handle sub templates and set tags for them. I think I called it wrap before, but I'm going with "template". The problem that I'm having is that the template tag handler and the set tag handlers need to communicate, but I don't know the best way to do it.

When the set tag closes, it needs to set the content it's collected to it's parent template, and set it to it's "tag" value:

Code:

<set tag="title">the title to the current template</set>

When a template tag closes, it needs to either dump it's contents to the main/first template's buffer (in place)

Code:

My main document content here...
<template file="table.html"></template> // set to main template buffer

OR to the last sub-template, but *using the last "set" tag, "tag" value*:

Code:

My main document content here...
<template file="table.html">
    <set tag="body">
        <template file="blah.html"></template> // set to table.html->body
    </set>
</template>

I've thought about having all of this central (arrays of "set" and "template" tags) within the template class, which is basically a WACT DataSource and does the parser/handler set-up. And then pass around the template, but I don't really want to crud up the template class. I'd rather keep the stuff that's specific to each handler(s) inside of the handler itself, and if one type is dependant on another, it can handle getting the information it needs without the help of some other "global" object.

Any suggestions?

Matt

**Austin_Hastings** · Dec 21, 2004 08:13

I'm not sure I understand what you're doing.

The point of the Sax chain is that whatever you do in the middle, it all comes out the end.

If you're going to have a subtemplate, then it's either an include type statement (discussed earlier) or a subroutine (like xslt named/callable templates).

In the latter case, I'd just build a set of <define-template> / <use-template> tags and have the same object parse them. Other than behavior, it's basically just like <set>.

=Austin

**mwmitchell** · Dec 21, 2004 10:47

Everytime a new <template> tag is found, a new Template instance is created, which is the class that adds all of the handlers to the parser and is a datasource. I was thinking that this was the way to go because the template could be nested and each one would take care of itself. But maybe that's where my problem is? Because each new template instance has a new instance of the handlers/parser/datasource.

-matt

**Austin_Hastings** · Dec 21, 2004 12:39

I think that's a problem, yes.

The parserchain should not change unless something radical happens. (Note: Including another file is radical.)

When you see a template, you need to record the resulting stuff until the /template tag. What's your envisioned syntax for this tag? Is it like a macro?

=Austin

**mwmitchell** · Dec 21, 2004 20:51

Hey,

Well this is what I have so far... It's not working of course, but it's my first idea of how I think it should work. The problem with this is that I'm trying to access instances that aren't there. Note that all of my "template variable knowing" handlers (TemplateDataFilter) have access to the main template/datasource class: $this->template. The template class has a variable called output. It's a class that contains an array in which the content of each handler pushes onto (a buffer chain).

Matt

ps, is this what you mean? The php handler code or the template syntax?

PHP Code:


class TemplateTagHandler extends TemplateDataFilter{

    

    var $setHandler;

    var $subTemplate;

    

    function setSetTagHandler(&$h){

        $this->setHandler =& $h;

    }

    

    function open(&$parser, &$name, &$attrs, &$empty){

        if($name =='template'){

            $subTemplate =& new Template($attrs['file']);

            $subTemplate->_parent =& isset($this->subTemplate) ? $this->subTemplate : $this->template;

            $this->subTemplate =& $subTemplate;

        }

        if($this->subTemplate || $name=='template'){

            unset($name, $attrs, $empty);

        }

        parent::open($parser, $name, $attrs, $empty);

    }

    

    function data(&$parser, & $data){

        if($this->subTemplate){

            $data = NULL;

        }

        parent::data($parser, $data);

    }

    

    function close(&$parser, & $name, & $empty){

        if($name=='template'){

            $template =& $this->subTemplate;

            if(is_object($template->_parent)){

                $handler =& $template->_parent->_baseHandler;

                while($next =& $handler->getNextHandler()){

                    if(strtolower(get_class($next)) == 'settaghandler'){

                        $tag = $next->current_tag;

                        $template->_parent->set($tag, $template->toString());

                        break;

                    }

                    $handler =& $next;

                    unset($next);

                }

            }else{

                $this->template->output->write($template->toString());

            }

            unset($this->subTemplate);

        }

        if($name=='template' || $this->subTemplate){

            unset($name, $empty);

        }

        parent::close($parser, $name, $empty);

    }

    

}

**Austin_Hastings** · Dec 22, 2004 08:32

Originally Posted by mwmitchell

ps, is this what you mean? The php handler code or the template syntax?

I was thinking of the template syntax. What does the "<template>" tag actually mean?

If the parser is passing html tokens, and suddenly a template tag appears, what is the significance? Is a template supposed to be an html subroutine? or is it supposed to be a #include directive telling the parser to open another file, or what?

=Austin

**mwmitchell** · Dec 22, 2004 18:11

OK the template syntax would be:

Code:

<html>
<head><title>PAGE</title></head>
<body>

Page content/buffer

<template file="myTable.html">
    <set tag="left_col">The content for left_col</set>
    <set tag="right_col">Content for right_col</set>
</template>

</body>
</html>

And that's it. What I want that to do is get the myTable.html file as a new template, set the values that the "set" tags set and display the result in place (body/buffer content).

EDIT:

The myTable.html file:

Code:

<table><tr>

<td>{$left_col}</td>
<td>{$right_col}</td>

</tr></table>

**Dr Livingston** · Dec 22, 2004 18:28

Trying to follow this thread, out of interest have you managed to template multiple rows for your

Code:

myTable.html

template file?

When it comes to XML I use the DOM and have used SAX only about 3 times if that, and like I said, trying to follow your progress. Looks like you have an interesting approach btw

**mwmitchell** · Dec 22, 2004 18:57

Hey Dr.Livingston. Haven't seen you around in a while! Welcome back.

Do you mean creating a list in the template? I do have a loop tag handler. You assign it an array and it prints the list a lot like wact. I'm in the process of converting it to read an iterator instead though.

-matt

**mwmitchell** · Dec 22, 2004 23:15

Ugh. Got it! The problem I've been having is when template tag closes, it needs to append the previous template tags set tag. I was trying to set the previous template tags set tag value by calling set() on the sub-template. The template class has a var called sets and a var called children. I'm going to post a link to what I have in a day or so, so this all makes sense. It's finally all working!

-matt

PHP Code:


class TemplateTagHandler extends TemplateDataFilter{

    

    function & getCurrent(){

        $total_children = count($this->template->children);

        return $this->template->children[$total_children-1];

    }

    

    function open(&$parser, &$name, &$attrs, &$empty){

        $this->current_name = $name;

        if($name =='template'){

            $template =& new Template($attrs['file']);

                        $parent =& $this->getCurrent();

            $template->_parent =& $parent;

            $this->template->children[] =& $template;

            $name = NULL;

        }else{

            parent::open($parser, $name, $attrs, $empty);

        }

    }

    

    function data(&$parser, & $data){

        if($this->current_name=='template'){ $data = NULL; }

        parent::data($parser, $data);

    }

    

    function close(&$parser, & $name, & $empty){

        if($name=='template'){

            $current =& array_pop($this->template->children);

            $parent =& $this->getCurrent();            

            if( ! is_object($parent->_parent) ){

                $parent->output->write($current->toString());

            }else{

                $parent->sets[count($parent->sets)-1]['content'] .= $current->toString();

            }

            $name = NULL;

        }else{

            parent::close($parser, $name, $empty);

        }

    }

    

}

**mwmitchell** · Dec 26, 2004 16:19

OK, here is what I have (minus the loop handler). Please, have a look and let me know where I can improve and where problems could occur. This should be considered alpha!

Matt

http://www.telegraphtomcat.com/SAXTemplate.zip

**mwmitchell** · Jan 13, 2005 07:58

bump! I only yesterday figured out what bump means! Anyone want to have a look and check out the files in the link above?

Any comments or improvements?

-matt

**sike** · Jan 13, 2005 10:24

hi,

i just skimmed through the files in the zip without testing anything.
a couple of questions or comments:

- why did you base the Template class on Datasource? personally i would aggregate the datasource, not extending it. what about multiple datasources?

- the "plugin" system needs a bit of refactoring. as far as i can see you loop through the tags directory and include the files. in the template class you register the tags hardcoded. i would refactor this to register the class with its filename and do the instanciation in a loop

- createOpenTag and createCloseTag could be refactored into a CodeWriter which might take care of formatting or compressing and stuff. ahhh... missed the Output class. maybe its worth combining them.

thats all for now. will test the whole thing over the weekend

Cheers
Sike

**mwmitchell** · Jan 13, 2005 12:47

Hey. Much appreciated. I await more of your or anyone else's comments!

-matt

**mwmitchell** · Jan 16, 2005 14:44

Originally Posted by sike

hi,
- why did you base the Template class on Datasource? personally i would aggregate the datasource, not extending it. what about multiple datasources?

You mean prepare the DataSource and pass it to the Template class? How do you imagine using more than one DataSource? Merging? Show me what you mean.

Originally Posted by sike

- the "plugin" system needs a bit of refactoring. as far as i can see you loop through the tags directory and include the files. in the template class you register the tags hardcoded. i would refactor this to register the class with its filename and do the instanciation in a loop

Yeah this is really just to get the thing going. I really haven't thought of how to add the tag handlers. Although one idea is to load one if a tag name is encountered. If it hasn't been loaded then load it up, else ignore. But not sure about tag handler order...

Originally Posted by sike

- createOpenTag and createCloseTag could be refactored into a CodeWriter which might take care of formatting or compressing and stuff. ahhh... missed the Output class. maybe its worth combining them.

Yes you are right. Actually, I think compiling to PHP is the way to go at this point and not even worry about doing the logic in the tag handlers. Just write PHP. It seems much easier to comprehend also. I just figure out how to translate. So, that's my next move. I know this is a huge task!

-matt

**Dr Livingston** · Jan 16, 2005 16:36

Merging?

Have a look at the Visitor pattern, might give you some ideas

User Tag List

Thread: Help with sax parser and states

Thread Tools

Display

Bookmarks

Bookmarks

Posting Permissions