Symphony.

A new extension, "Asset Compiler" is now available for download. Comments and feedback can be left here but if you discover any issues, please post it on the issue tracker.

Git Repo & Docs - also, please use the github issue tracker for any issues, cheers

Asset Compiler scans an XSLT Template (master.xsl by default) for lists of CSS and Javascript assets. These assets are then concatenated into a single file, minified and the new file tagged with a unique hash (to bust over-eager caches). The XSLT Template is then automatically updated with these new filenames. There is also a checkbox On/Off switch to control whether to serve these compiled assets. Javascript compilation is via an CURL API call to Google's Closure Compiler.

This is only my second Symphony extension so any advice/comments/etc are most welcome. This came out of the work I was doing on a command line tool and I thought it would make a good general purpose extension away from all the funky custom Symphony install stuff of that project.

Known issues: -

1. You can't comment out links between the two markers, the regexp doesn't discriminate between commented out and not, they'll all get compiled.

Attachments:

asset_compiler_results.jpg and asset_compiler.jpg

Nice!

Although I am wondering if there wasn't a more XML-like approach...

<asset:compile type="javascript">
  < script... />
</asset:compile>

or

< script... asset:compile="true" />

Although I guess the XML/XSL would need another processing step to remove them from the HTML output...

Thanks!

I really like your XML approach, I had a quick play around a lunch today and have got a way that gets rid of the <!-- START/END --> comments and uses a compile attribute instead which looks much cleaner...

< script compile="true" src="/workspace/scripts/main.js"/>

That's nice and I think I'll probably update the extension when I have time, but as you say we're still always going to have to have the <xsl:when> logic to switch the HTML output. I presume there's a post render delegate in the core somwhere that would allow removal of any compile='true' scripts/styles and append the compiled file on every page render but that seems like a lot of overhead and extra processing when a simple <xsl:when> switch suffices. A pity though because from a developer's point of view only having to add a simple compile='true' attribute to any assets would be sweet!

@phoque - Thanks again for the XML approach idea, you got me inspired!

Version 1.1 - What's New?

• No more xsl:when switching - simple add a data-compile='true' attribute to any < link> or < script> element in master.xsl and click the 'Compile' button in the Preferences page and you're done.
• Symphony's output is automatically updated (via the FrontendOutputPostGenerate delegate) with the compiled asset filename(s) (CSS file just before closing </head> and Javascripts just before closing </body>)
• Seperate controls for serving CSS and Javascript assets

I was initially concerned about the performance hit from re-writing Symphony's output, but in the end it's only a couple of calls to preg_replace which is going to be way more than offset by the performance gain in using single, minifed assets. Btw, the old version is still available as a git branch of xsl-when-version in case it's a better fit for anyone's projects.

EDIT - Clarify instructions

but in the end it's only a couple of calls to preg_replace which is going to be way more than offset by the performance gain in using single, minifed assets.

You can really speed up things if you do your minification during the release process, in stead of real-time. There is absolutely no reason why this is done server-side for each request. Pre-compiling assets is the only way forward.

@remie - It is pre-compiled - Sorry I should have been more clear, the minification/compilation process is only done once (when clicking the 'compile assets' button on the Preferences page). The preg_replace call on each request simply removes any data-compile='true' elements and adds a reference to the already pre-compiled asset. So it is already as you describe.

I've edited the instructions above to be clearer

Nice!

Oh this is very awesome and getting closer to viable for my projects.

I often store my scripts and stylesheets in a section with a file upload field so I can dynamically assign them to various templates (I'm loathe to put redundant js or css into the output if it's not going to do anything, although it could be argued that I'm going a bit too far considering I also use libraries like JQuery :D). I have used Page Attach fields to achieve this, although sometimes I manage them through a subsection in my entries depending on the structure of the back-end in a particular project, using url params to include certain assets is also sometimes a thing.

I imagine pre-compiling would probably be prohibitively complex using this method as you'd have to run through the Symphony engine/xslt preprocessor to get the actual file URIs, although perhaps a (secured via developer session check) query string trigger to set the compiler running once at runtime might be a good half way solution? Just thinking out loud really. Something like: http://www.example.com/compile/?url="/path/to/url"&js=true&css=true

To make that work with your ninja replacement method you'd probably need a path to the xsl stylesheet responsible for the assets.

Ideas for the future maybe?

Idealogically I see where you're coming from in not wanting to put redundent JS or CSS into your output but in practical terms I go completely the opposite direction. Unless you're creating some vast JS monster like Twitter, and serving it up to Google-level visitor numbers I personally think it's just so much more practical to bundle all your assets together in one and serve that up.

One thing to note about that though, I always use a CDN for JQuery (and other huge libraries) there's a very good chance that your visitors have already got JQuery in their cache from visiting other sites, in which case we can use it without a new download. And if they don't then it's a HTTP request to a different server so the download can happen in parallel. See Google's Hosted Libraries if you haven't already

The other thing you might want to try if you're wanting to not serve redundant code is very aggressive minification - Google's Closure Compiler (which Asset Compiler uses) has an ADVANCED_OPTIMIZATIONS setting that will do some pretty hefty rewriting and stripping of redundant code - you could try that. Though personally I find that level of minification often causes scripts to stop working altogether (especially when you factor in third-party libraries etc). For me the extra few Kb saved are not worth the effort, a better use of your time would be somehting like moving all your image assets Amazon S3/Cloudfront, it'll have more impact on loadtimes.

To make an extension do as you're describing I think you'd have to combine it with a page-caching extension of some kind that recorded each URL's output and cached the assets for that specific URL. But then you're talking about doing a lookup for each URL and substituting in the custom compiled assets. Then you have the problem of dynamic pages that can't be cached and the joy of cache-invalidation!

As the old saying goes, there's only two hard problems in Computer Science...

1. Naming things
2. Cache invalidation
3. Off by one errors

Very well explained and definitely something I'll take on board.

I definitely agree that ideology is getting a little in the way of practicality here, another hard problem in Computer Science! Although 'naming things' is another reason for assigning assets dynamically at the template, meaning you can manage assets graphically with 'one click' rather than uploading and then altering the template. It also allows automated cache-busting and the like and avoids those other clichés, the typo and reference error (or 404 in this case).

I experimented with CDN-ing JQuery and other libraries at one point, however I'm vary wary that the more external services you depend on, the risk of something breaking without your ability to influence the problem becomes higher eg. Twitter API (whhyyyyyy) / Github API (understandable) changes causing headaches all over the place. I guess the compromise on this would be to test the CDN and fallback to a local file if it fails, although this adds more overhead in other ways and perhaps defeats the point. Again this is perhaps ideology over pragmatism.

However, this is rapidly getting out of scope for the extension though so my apologies and thanks for adding an invaluable extension to the Symphony pool! I will definitely consider switching to this methodology in my next project. One day I might even catch up to Web 2.0.

I'm vary wary that the more external services you depend on, the risk of something breaking without your ability to influence the problem becomes higher

You can use a CDN version and fallback to a local version if necessary.

That's exactly the compromise I was considering, thanks David!

Yeah a local fallback is absolutely critical in my experience.

Here's the 3 lines of XSLT I use in my Symphony/HTML5Boilerplate mashup Symphony No.5 in case it helps anyone (the CDATA escaping can get a little scary!)

<xsl:variable name="jQueryVersion" select="'1.8.3'"/>
&lt;script src="//ajax.googleapis.com/ajax/libs/jquery/{$jQueryVersion}/jquery.min.js">&lt;/script>
&lt;script><![CDATA[window.jQuery || document.write('&lt;script src="/workspace/scripts/vendor/jquery-]]><xsl:value-of select="$jQueryVersion"/><![CDATA[.min.js">x3C/script>')]]>&lt;/script>

Oops, the script tags in that code snippet are being munched - here's a link to the lines on github

I'm probably confirming my self-taught web-luddite background here, but I prefer to use xsl:text (with or without disable-output-escaping as necessary) over CDATA, as I find it a lot easier to read (and write - how many square braces is that?). Any pitfalls to this approach?

I'm not an XSLT expert myself at all so you may well be right that xsl:text is a better way to go - I vaguely seem to remember reading something when I was first getting started with Symphony/XSLT that javascript should be put between CDATA blocks, and I know that the CDATA version has never given me any trouble, hence I've never bothered to explore further - but yeah, would be interesting to know from someone in the know... xsl:text certainly looks much easier

Asset Compiler updated to version 1.4 on 8th of February 2013

• Added checkbox option to the preferences pane that will update all manifest.*/config.php files with the newly compiled filenames upon compilation. In other words, if you're using Rowan Lewis' "Dual Manifest Folders" setup you no longer have to manually update the non-symlinked config.php files.

Well that didn't last long - found a show-stopping critical bug in my latest feature, updating multiple manifests. Have issued a new version with all that stuff stripped out again until such time as I can find a solution. v1.5 is therefore the same as v1.3

sorry folks!

Asset Compiler updated to version 1.6 on 16th of August 2013

• Finally got around to fixing this: Added option to update all manifest.*/config.php files with the newly compiled filenames upon compilation. In other words, if you're using Rowan Lewis' "Dual Manifest Folders" setup you no longer have to manually update the non-symlinked config.php files.