Hi everyone.
I’ve inherited an e-commerce system at work that I’ve been dinking with for about a year now. After hearing repeatedly on and on ad nauseum about the glorious heart-warming performance benefits one should get if scripts are loaded after CSS and the HTML, I thought I’d play around with trying to get such a thing to work on our system, as a test. Yes I’m also looking at painting/rendering but that’s a lot harder to tease out…
Of course a little site with a script or two is easy enough to put as the last child of the body, but our setup ensures it’s a complicated mess. What I’d like is some opinions on whether it’s dumb of me to try to hack around our current setup to get reasonable scripts loading later, or whether what I’m doing is good enough and I should consider pushing these changes.
The setup now
We’re using Flask to run e-commerce websites, and so this means we’re using Jinja2 templates. They’re nice, I like Jinja.
Our HTML templates work a bit like this:
There’s a main base “site” and every actual client site either uses those templates, or uses instead a local copy with anything specific to the client.
For example:
base.html -> has doctype, head, opening and closing body tags
structure.html -> basic structure of content inside the body
An example page might include templates/shoppingcart.html. Let’s say I make a new client site and they want special content on their shopping cart page. I’d make another shoppingcart.html and put it in their own template, meaning when the request comes in, their shoppingcart.html will get called rather than the shoppingcart.html in the base shop. Sometimes a client template will have some inline script that the base doesn’t have.
I’m not sure how typical this is of inherited templates so if this is what everyone else does, ignore the above.
Jinja allows “blocks” to be placed on a page. This means if there are 3 templates making up an HTML page, and they each call external scripts via a {% block javascript %}, when the page renders, the call to those scripts will all be together… near the top of the page.
(<script src=“/to/views/scripts.js”></script> etc etc)
Additionally, there are also many many many little inline scripts who only affect the little chunk of HTML loaded. I’ve been able to get rid of some of these and move them to our main script, but others require some template variables be checked first and since the HTML loads before scripts, I haven’t figured a way to do something like
{% if somVar %}
runnascript();
{% endif %}
where runnascript is sitting in someMainScript.js, since… someMainScript.js hasn’t been loaded yet, cause I’m loading everything at the bottom of the page.
Also, all of these little scripts expect jQuery, which the previous dev has chosen. So without jQuery, not a single little script will run.
In the original setup, about a gazillion scripts were called in the head: jquery, jquery-ui, and a bunch of our own proprietary scripts with caching names at the end. At the end of the body tag was about a gazillion jquery plugins.
So because jQuery got called in the head before anything else, any template chunk getting called after the <body> tag could run, as there was jQuery.
What I did when I showed up at this job is concatenated all the static stuff. I wrote a painful and fugly little bash script that cats all the scripts and I put this large superUberHugeJavascripts.js in the head, since it contains jQuery and needs to load first. At the bottom of the page I moved the cache-aware scripts, and concatinating them doesn’t seem like a good idea since these also get loaded within template conditions (and they can be pretty big, so let’s only call them if needed).
So that’s what’s running clients’ stuff right now, this halfassed semi-performant thing. The whole page waits for uberSuperGorillaJS.js to load and while on a fast connection the blocking isn’t noticeable, it will become so as we grow and as more people get the silly idea that using these sites with a tablet on some sh*tty network would be a great idea or something (god I don’t know why anyone would want to but it seems to be the hip thing these days. And no, the site’s not responsive and never will be, so everyone gets to pinch, zoom, and mis-click everything, because the bosses say this is fine).
The attempt to fix
So, wanting to move scripts to the bottom… would be difficult, if all the little scripts in all the templates need jQuery …and some of them also need jQuery-ui, and some of the ui parts used require a bunch of other ui plugins… so I think I don’t want to separate jQuery and ui from my concatenated JS.
Or do I? That’s question 1: is it better to leave the absolutely-required stuff, even though it’s a huge steaming pile of jQ, in the head and everything else concat’d at the bottom of the page? This would seem to defeat the whole purpose, but maybe all my checks and weirdness aren’t worth it.
What I’ve got so far:
I wrote a vanilla JS in the head. Its job is to read some data-foo attributes of any script passed to it (to get the src’s because they’re template-generated too), and add it to the bottom of the page. Since these scripts will need jQuery loaded, and since the current superUberLargeScripts.js thing takes a while to load, I need something to prevent these scripts from trying to run until jQuery is present. So also in the head, in my vanilla JS, is a timer that checks for jQuery. This timer is running a lot and in Chrome I see the browsers seems to enjoy repainting themselves every time the timer goes off. I am horrible disappoint.
Everyone is calling this timer, which I don’t like but seems safe (safe as in, nothing runs without jQ available first). What I’d rather have is my vanilla script use the timer just once (or something else) to wait until jQ has loaded, and then set some state variable or something that the other scripts can use to check.
So example of an old/current template:
<divs!> blah blah HTML....
{% if client_chose_this_functionality %}
<script>
$(function() {
some JS for the above HTML, needs to work down to IE7 :o
});
</script>
{% endif %}
Example of what I’m trying now
<divs!> HTML blah blah...
{% if client_chose_this_functionality %}
<script>
function IMakeUpASillyName() {
some JS for the above HTML, needs to work down to IE7 :o
}
checkJQ(IMakeUpASillyName);
</script>
{% endif %}
checkJQ is the vanilla function in the head with a timer.
function checkJQ(wannarun) {
if (window.jQuery && window.jQuery.ui) {
wannarun();
}
else {
window.setTimeout(function () {
checkJQ(wannarun);
}, 500);
}
}
I’m still playing around with the time since I know in my local env jQ doesn’t load all that quickly. Again, once anything gets loaded anywhere on a browser, we say Cache to browsers, and we also have caching in the back. It’s still a buttload of scripts though.
Similarly, the scripts I call on every page with the cache-names, and the views-level scripts, they also started with the self-invoking funcs:
$(function() {
run the moment the script is parsed
});
Only thing I could think of here was to give each of these somewhat more important and all-page scripts an _init() named func, similar to above
function _init() {
the original code otherwise
}
checkJQ(_init);
Everyone using _init is a script that’s actually external. I had looked at async and defer: defer is for external scripts with a src attr, not inline scripts sitting in HTML. Second, Mozilla has decided everyone should just be async unless you manually set them all to “false” and will generally try to run anyone who’s been parsed the moment they exist. Even when using these attrs, they won’t give me any guarantee of loading-order: since utils.js being added to the </body> is done in the same call as superUberGinormousJavascripts.js, and utils (one of the cache-named scripts) is smaller, it gets read/parsed before the one with jQuery, so even with a correct added-to-body order, utils could (and does) run first in some browsers.
So. This is the single script I have in the head of the pages:
<script id="initial_script"
data-src1="{{ template to a script }}"
{% if something's true %}
data-src2="{{ template to another script }}"
{% elif something else is true%}
data-src2="{{ template to another script }}"
{% endif %}
{% if some var %}
data-src3="{{ a cached script, True) }}"
{% endif %}
data-src4="{{ main utilities script, True) }}">
var fraggle=document.createDocumentFragment(),
initial_script=document.getElementById('initial_script');
appendScript(initial_script); //load this current set of scripts
function appendScript(javascript) { //also to be called by views/scripts as well
var attrs=javascript.attributes;
for (var i=0;i<attrs.length;i++) {
var attr=attrs[i].name.toString();
if (/data-src\\d/.test(attr)) {
var s=document.createElement('script');
s.src=javascript.getAttribute(attr);
fraggle.appendChild(s);
continue;
}
//yeah some clients want special css added if a certain script gets called for one-off pages
if (/data-css\\d/.test(attr)) {
var link=document.createElement('link');
link.rel='stylesheet';
link.href=javascript.getAttribute(attr);
document.head.appendChild(link);
}
}
// still dunno if this works well cross-browser, dunno if this works in Safari
var readyStateCheckInterval = setInterval(function() {
if (document.readyState==='complete') {
document.body.appendChild(fraggle);
clearInterval(readyStateCheckInterval);
}
}, 50);
}
function checkJQ(wannarun) {
if (window.jQuery && window.jQuery.ui) {
wannarun();
}
else {
window.setTimeout(function () {
checkJQ(wannarun);
}, 500);
}
}
</script>
And any template who knows it needs to call a views script does similar. Instead of
{% block javascript %}
<script src="{{ template link to a views script }}"></script>
{% if something true %}
<script src="{{ template link to another script }}"></script>
{% endif %}
etc
{% endblock %}
it’s now
{% block javascript %}
<script id="viewtype_scripts"
data-src1="{{ template link to script }}"
{% if something true %}
data-src2="{{ someother script }}"
{% endif %}>
var views_script=document.getElementById('viewtype_scripts');
appendScript(views_script);
</script>
{% endblock %}
and the views scripts are the ones who have the _init() wrapping them, and they also do a checkJQ(_init) check.
I came up with the above after scouring stackOverflow and remy’s pages looking for a small, easy way to check for page/body loaded, and jQ being loaded. I do not want the thing where you sniff for Webkit’s UA or silly super-complicated stuff that requires major browser monitoring. I also read a bit on what Require.js would need… well, it does best if you’re not dynamically adding random scripts all over the place as needed.
So you see, each little script makes this call. I think I’d like the state variable but have no idea how I could call those scripts to run if the variable is true.
2. Is using this weird init-check (checkJQ) setup okay, or stupid? Is it possible to use the set-a-state-variable thing instead? If so, how to call those scripts once that var is true?
I mean what I originally wanted was, my vanilla add-to-body-loader would then do a jquery-timer check, and once jQ was loaded, run the _init of all those whole-page scripts. I think I’m okay with how all the little in-template scripts are running, since they could do a
if (myStateVar) {
runme();
}
or something, but the scripts I’m loading at the same time as jQ on the first request, I think I’d like to say
if jQ
all scripts._init();
or something. Again, I’m probably not going to institute a large library/framework that might do this, unless I get like a month free and permission to completely rewrite ALL THE THINGS. Plus this means all existing clients will need a major update rather than a minor one, which means I’ll prolly get lucky and all their sites will die as I run into new and amazing bugs.
3rd question is, except for scripts who need access to variables coming from Python/the DB, while I’m trying to clean scripts out of the templates, there’s also a jQuery upgrade coming up for us soon: from 1.7 to 1.9 (we’ll be supporting old IEs even after M$ stops official support for XP in April this year, 'cause our smaller clients still need lots of time to upgrade). This means I have to crawl through all the old minified plugins and figure out what they do, who actually uses them and get updated versions. I’m not sure if there’s another opportunity somewhere regarding late-script loading here too?