Symphony.

A new extension, “Cacheable Datasource” is now available for download. Comments and feedback can be left here but if you discover any issues, please post it on the issue tracker.

This is not a full-featured extension. It is packaged as an extension to make installation and version-tracking more easy, but it does require the developer to modify datasource PHP files.

I just finished implementing this on a site with some fairly large datasources (3000 queries) and some largely static datasources with much success!

Much thanks nick

Nick build killer extensions! ;-)

Nick build killer extensions!

He’s a show-off ;)

fairly large datasources (3000 queries)

Yikes! Yes, this will help ;-) Out of interest, does the section you are querying for this DS have many section links i.e. are there several section handles as attributes on each entry? Or is it that you’re querying many fields and many entries?

Yeah the datasource is pulling about 20 fields including a few bilink entries that had 3 or 4 children for one ‘entry’.

That datasource in particular was pulling 20~ entries in for a grid view…

The only thing I’ve noticed now is a slight delay in painting the page, almost like a white flash. Any idea how this could be less noticeable?

Some of the pages are using 6 or 7 Cacheable Datasources, so could there be a delay reading from the I/O?

Yeah the datasource is pulling about 20 fields including a few bilink entries that had 3 or 4 children for one ‘entry’.

Ah, that’ll be it. Bi-Link can be expensive.

The only thing I’ve noticed now is a slight delay in painting the page, almost like a white flash. Any idea how this could be less noticeable? Some of the pages are using 6 or 7 Cacheable Datasources, so could there be a delay reading from the I/O?

Hmm, I doubt it. If you ?profile the page and look at the DS rendering times, what do you see? The cached DSs will almost 100% definitely be faster. Could this be more of a CSS rendering issue?

Ah! Good to see you continued on our little discussion a few weeks ago!

Nick, I’m curious to your thoughts on a ‘problem’ that has arisen from using this extension.

The aforementioned large datasource sorta needs to be ‘real-time’ to reflect the current data. The datasource is used to pull in event information, such as pricing, sale times etc.

The issue is that if the change is made in the backend, it takes until the cache time expires for this to be reflected on the frontend. At present I have this time at 6 hours, which I know could be shortened, but it still doesn’t remove the problem.

Would it be possible to flush the datasource cache for a particular entry on the entry save through the backend? Would this be ideal as an automated approach, or would a frontend to manually flush datasources in the backend be useful?

I like this extension, and think it’s benefits are extreme (especially in this situation where it’s a fairly chunky DS), but it is sort of deal breaker at the moment (well for this DS anyway).

Thoughts are welcome :)

Would it be possible to flush the datasource cache for a particular entry on the entry save through the backend? Would this be ideal as an automated approach, or would a frontend to manually flush datasources in the backend be useful?

I added this exact functionality to the Cachelite extension. When an entire page is cached, the entries and sections used in the DSs that build the page are stored alongside the cache. When these entries or sections are updated in the backend, the cache of pages that references these are purged.

While I am tempted to add this to this extension, I would prefer to keep the Cacheable Datasource idea as simple as a small class without additional functionality. Your case seems like an edge-case that other users may not require.

As such, I’ll suggest here how you can solve it. But I don’t intend to build this into the extension officially ;-)

In the extension.driver.php add a function to make the extension subscribe to the delegates fired on entry creation and update:

public function getSubscribedDelegates() {
    return array(
        array(
            'page'      => '/publish/new/',
            'delegate'  => 'EntryPostCreate',
            'callback'  => 'flush_ds_cache'
        ),              
        array(
            'page'      => '/publish/edit/',
            'delegate'  => 'EntryPreEdit',
            'callback'  => 'flush_ds_cache'
        )
    );
}

Disable and Re-enable your DS to Symphony subscribes to the delegates.

Then write the callback function that these delegates are attached to:

public function flush_ds_cache($context) {
    $section = $context['section'];

    // section is Articles
    if ($section->get('handle') == 'articles') {

        $cache_directory = opendir(CACHE);

        while (false !== ($file = readdir($cache_directory))) {

            // find cache files starting with "datasource_articles"
            if (preg_match('/^datasourcearticles/', $extension)) {
                unlink(CACHE . '/' . $file);
            }
        }

    }
}

This should only execute when the entry is in an Articles section. It’ll loop through all files in /manifest/cache and delete those with filenames starting with the class name of your data source.

Haven’t tested this, but hope that’s enough to go on.

Thanks for that Nick, I won’t have to time to do that project today, but it’s something that will crop up this week.

I’ll be sure to post my results.

Cheers.

Hi, Nick
I made a small modification in your extension for myself. Now it can cache navigation DS’es.

It’s a quick and dirty solution but, maybe, could be interesting for you.

class.cacheabledatasource.php
In grab()

public function grab(&$param_pool=array()) {

    // Check that this DS has a cache time set
    if (isset($this->dsParamCACHE) && is_numeric($this->dsParamCACHE) && $this->dsParamCACHE > 0) {

and grab_xml() functions

public function grab_xml(&$param_pool){

    $result = new XMLElement($this->dsParamROOTELEMENT);

    try{
        if($this->getSource() == 'navigation'){
            include(TOOLKIT . '/data-sources/datasource.navigation.php');
        }else{
            include(TOOLKIT . '/data-sources/datasource.section.php');
        }
    }

Cacheable Datasource updated to version 0.2 on 15th of December 2009

Great idea andrrr, I’ve merged in your changes.

Cacheable Datasource updated to version 0.3 on 5th of February 2011

Cacheable Datasource updated to version 0.4 on 18th of February 2011

This update adds a page under the System menu to view the status of all cacheable data sources, and the option to clear their cache early (useful for debugging).

Is this still “relatively untested”, or considered ready for prime time now?

i love the readme.

This shit is relatively untested. Use with caution.

Symphony makes even exceptions somehow enjoyable by printing funny messages like “Symphony Exception Handler crapped out…” on your screen ;-)

[EDIT]: Removed information potentially exposing a security issue.

Anyone ever combined CacheableDatasource with Section Schemas?