Key Takeaways
- PHP developers can add syntax sugar from other languages to PHP with the help of a pre-processor library called Yay, which allows for the creation of macros.
- Yay breaks down a code string into tokens, constructs an Abstract Syntax Tree, and then reassembles the PHP code with the macro elements replaced with real PHP code. This allows for the creation of more elegant and concise code.
- While there are limitations with variable scoping and the parser, Yay allows for the creation of cleaner and more efficient PHP code. For example, the macro allows developers to slice an array with $many[4..8].
I was originally going to call this “Micro Macros with Márcio’s Perfectly Pragmatic Pre-Processor library”, but I didn’t think Bruno would approve…
I get really excited when developers feel empowered to create new tools, and even new languages with which to solve their problems.
You see, many developers come to PHP from other languages. And many PHP developers can code in more than one language. Often there are things in those languages — small syntax sugars — that we appreciate and even miss when we’re building PHP things.
Adding these to a language, at a compiler level, is hard (or is it?). That is unless you built the compiler and/or know how they work. We’re not going to do anything that technical, but we’re still going to be empowered.
When I used to write Ruby (or languages similar in this regard), I used to use list index ranges. They look something like this:
few = many[1..3]
We can expect code like that to take the second, third and fourth item of the many list, and assign them to the few variable. We can do something similar, using array_slice:
$few = array_slice($many, 1, 3);
I think the Ruby equivalent is far more elegant. We can also use variables and negative values for the lower and upper bounds of the range. It’s brilliant!
Getting Started with Macros
Inspired by a recent SitePoint post, I decided to try and add this syntax to my applications. The trouble is I don’t understand the PHP interpreter well enough to be able to add it. Then I remembered Márcio Almada’s Yay library. It adds macros through pre-processing. Here’s an example of something simple you can do with it:
macro {
unless (···condition) { ···body }
} >> {
if (!(···condition)) { ···body }
}
$condition = false;
unless ($condition) {
print "look ma! macros...";
}
That example is straight out the readme (with a few style differences).
In order to run this, we need to download the pre-processor:
$ composer require yay/yay:dev-master
Once that’s downloaded, and you’ve saved the PHP-like code above to a file (let’s call it unless.yphp), we can compile it to regular PHP:
$ vendor/bin/yay unless.yphp >> unless.php
The result will look something like:
$condition = false;
if (!($condition)) { print "look ma! macros..."; }
How This All Works
Yay implements a parser. It breaks a code string up into tokens, constructs an Abstract Syntax Tree and puts the PHP code back together with all the macro stuff replaced with real PHP code.
These concepts can be difficult to get, at first. I suggest you read this series, which explains some of them using tools you know well.
When Yay matches the unless pattern we defined, it essentially does a string replace around the condition and body. We can use this to great effect, for the language syntax we want to add…
Adding Our Syntax
The first thing we need to do is identify the expression which will match a range index:
macro {
T_VARIABLE·A[
T_VARIABLE·B..T_VARIABLE·C
]
} >> {
array_slice(
T_VARIABLE·A,
T_VARIABLE·B,
T_VARIABLE·C - T_VARIABLE·B
)
}
This macro expression looks for a variable, like $many and labels it A. It will only match if it sees the equivalent of $many[$lower..$upper], which it will replace with:
array_slice(
$many,
$lower,
$upper - $lower
);
The array_slice function parameters are source array, starting offset and number of items to get. This is slightly different to the range syntax, because the range syntax defines a starting offset and an ending offset. We translate between the two by subtracting the upper bound from the lower bound.
T_VARIABLE is an AST token type, which identifies variables defined in a code string. There are quite a few different token types, but luckily we don’t have to remember them all…
We can try this, with some Yay PHP:
macro {
T_VARIABLE·A[
T_VARIABLE·B..T_VARIABLE·C
]
} >> {
array_slice(
T_VARIABLE·A,
T_VARIABLE·B,
T_VARIABLE·C - T_VARIABLE·B
)
}
$many = [
"She walks in beauty",
"like the night",
"of cloudless climes",
"and starry skies",
"And all that's best",
"of dark and bright",
"meet in her aspect",
"and her eyes",
"...",
];
$lower = 4;
$upper = 8;
$few = $many[$lower..$upper];
When we run this through the conversion process, we get something like:
$many = [
"She walks in beauty",
"like the night",
"of cloudless climes",
"and starry skies",
"And all that's best",
"of dark and bright",
"meet in her aspect",
"and her eyes",
"...",
];
$lower = 4;
$upper = 8;
$few = array_slice(
$many,
$lower,
$upper - $lower
);
This is a good start. We still want to allow plain integer bounds, which requires slightly more complex macro code:
macro {
→(···expression)
} >> {
··stringify(···expression)
}
macro {
T_VARIABLE·A[
···range
]
} >> {
eval(
'$list = ' . →(T_VARIABLE·A) . ';' .
'$lower = ' . explode('..', →(···range))[0] . ';' .
'$upper = ' . explode('..', →(···range))[1] . ';' .
'return array_slice($list, $lower, $upper - $lower);'
)
}
Let’s take this one step at a time. We begin by defining a macro to convert pattern matches to strings. ··stringify(...) is an expansion (which means it transforms other matches). It takes a match and converts it directly to a string. We can see this happening in the following example:
macro {
convert(···expression)
} >> {
··stringify(···expression)
}
convert($a, $b, $c);
If we run this code through Yay, we’ll see the following:
'$a, $b, $c';
I’ve chosen to create a kind of shorthand macro, converting →(...) to ··stringify(...), to make further macros more concise.
I ran into a couple of problems trying to capture numbers in the index range syntax. By the time Yay gets hold of the AST, 4..8 is tokenized as 4. and .8; instead of 4, .., and 8. This generates a parser error before Yay can apply the macros. A way around this is to convert the index range to a string and start parsing it from there.
An elegant way to do this would be to pass the index range string to a function, along with the source array. Unfortunately, PHP scope rules mean variable range bounds won’t be available inside the function, making it a bit useless.
The easiest way around this issue is to reference the variables inside a string, and eval them. The way this works is:
$manyis matched withT_VARIABLE·A4..8is matched with···expression- Inside
eval,$loweris defined as the first array index after exploding the string4..8by.. $upperis defined as the second array index of this explosion- The return value, of this
evalstring, is the originalarray_slicecode we used before
Any of the following lines will be matched by this macro:
$lower = 4;
$upper = 8;
$few = $many[$lower..$upper];
$few = $many[4..8];
$few = $many[4..$upper];
$few = $many[$lower..8];
…and they will generate code resembling the following:
$few = eval(
'$list = ' . '$many' . ';'.
'$lower = ' . explode('..', '$lower..$upper')[0] . ';' .
'$upper = ' . explode('..', '$lower..$upper')[1] . ';' .
'return array_slice($list, $lower, $upper - $lower);'
);
$few = eval(
'$list = ' . '$many' . ';'.
'$lower = ' . explode('..', '4..8')[0] . ';' .
'$upper = ' . explode('..', '4..8')[1] . ';' .
'return array_slice($list, $lower, $upper - $lower);'
);
$few = eval(
'$list = ' . '$many' . ';'.
'$lower = ' . explode('..', '4..$upper')[0] . ';' .
'$upper = ' . explode('..', '4..$upper')[1] . ';' .
'return array_slice($list, $lower, $upper - $lower);'
);
$few = eval(
'$list = ' . '$many' . ';'.
'$lower = ' . explode('..', '$lower..8')[0] . ';' .
'$upper = ' . explode('..', '$lower..8')[1] . ';' .
'return array_slice($list, $lower, $upper - $lower);'
);
That’s probably as clean as we can make this, given the limitations of variable scoping and the parser. It’s not the nicest PHP to look at, but it does the job. In the meantime, we can slice an array with $many[4..8]…
If you found this interesting, or have interesting ideas for how this library can be used, consider leaving a comment below.
Christopher PittChristopher is a writer and coder, working at Over. He usually works on application architecture, though sometimes you'll find him building compilers or robots.
