Symphony.

I've been digging into the build-in caching manager of Symphony lately which made me wonder: Why isn't this used instead of creating seperate cache-files? It prevents an 'exploding bubble'-effect in your cache-folder.

At the time it was part of a wider caching project to prevent hits to the database. One less query meant one less read (of a potentially big blob) from a separate database server. The other reason is that Symphony's cache forces you to store a cached object with a key that is limited to 32 characters (MD5 hash). We wanted to prefix the hashed key with the name of the data source, so that the cache files can be deleted on a per-data source basis.

I've got used to seeing many thousands of files in my cache folder (given that JIT will create many cache files too) so I've never thought of it as being a problem. Are there good arguments for storing these in the database rather than the file system?

Well, I could think of some arguments:

• No exploding bubble, since you can set an expiracy time for each cached element
• No storing of physical files, which causes the cache-folder to get huge. (10.000+ files could have a impact on the time it takes for the server to lookup a file in the folder (at least on Windows it does ;-))).
• Make use of the built-in caching features of Symphony instead of re-inventing the wheel again (which could potentially lead to new bugs in the future).

You don't have to to use a MD5 Hash, you could also use a textual hash, but you are right about the 32 characters. But if you use various parameters to create a md5 hash you could do this again to retrieve the right object right?

I was looking into this extension today and have a few questions:

• What about @kanduvisla's remarks? :)
• I'm assuming this extension is only applicable to standard local DS'es, correct? For external Dynamic XML DS's we have the Update cached result every #n minutes cache option already, right?
• Is this considered stable(rer) with 2.2.4?

What about @kanduvisla's remarks? :)

Valid, but they won't help in certain situations. In one project I worked on the database server was separate to the server database and suffered some horrible latency. Using Cacheable Datasource eliminated hits to the database server producing a faster result.

For that project we actually extended the extension to include a EntryPostCreate hook that would purge the filesystem everytime an entry was created in that section. It worked because we had a strict naming convention between Datasources and Sections, not all users do this though.

In the future when the Cachable class is extensible for APC, file, Memcached etc. drivers, then moving to use the default Symphony class would probably be worthwhile, but I don't think it's that much of a worry at the moment.

I'm assuming this extension is only applicable to standard local DS'es, correct? For external Dynamic XML DS's we have the Update cached result every #n minutes cache option already, right?

Yep

Is this considered stable(rer) with 2.2.4?

I've been using fairly heavily across a multiple sites with no issues (ranging from 2.2.1 - 2.2.5).

Cool, thanks. Since most(!) of my DS'es (for this particular project) are actually external XML DS'es I just up the Cache in their option. I thought that, maybe, I've overlooked something that could be valuable for those but their 'native' option seems sufficient.

How can I find v0.6 of this extension? Need it for Symphony 2.2.x. Latest tag on git is 0.5 ...

Would love to get this to work for my project, I have one relly big datasource that is loaded everywhere. But after I install and set the timeout to 200 seconds, the cache isn't doing anything. I get no extra menu under system, and no speed improvement and no "cache-age" in the xml.

I do get the folder cache/datasources, but it stays empty

Sym 2.2.5, Cacheable datasource 1.1

thanks! J

Yes, the page in the backend has been removed because it was clutter and not required.

You're probably using the extension straight from Github, however this latest version is for Symphony 2.3 only (see note in the readme). For 2.2.5 you should get v0.5 here:

https://github.com/nickdunn/cacheabledatasource/tree/0.5

And then be sure to re-read "How do I use it?" which are different, and unfortunately more complex — you'll need to customise your data source PHP.

thanks! will try this!

Works great, although I suppose i have to choose between this and cachelite, since this one doesnt expire by a cachelite flush.

I guess I could delete all cachelite files and cachable-datasource in one go with a shell script?

since this one doesnt expire by a cachelite flush

It doesn't. But you wouldn't need to use them together, since CacheLite trumps Cacheable Datasource since it caches the entire page output, not individual datasource XML. Do you need event-based expiry of cached data sources? How do you need them to be flushed?

I wrote a really crude extension that will flush Cacheable Datasource files when any entry is saved or created in the backend, it might help.

I did say it was crude right?

I also had a modification of this (dbdatasourcecache) that flushes automatically per post. (difference is it saves to Database instead of file cache) - and there are some parameters that will only flush current updated post. I also keep track of the cache size in case it goes too big.

Using a version on production with a cache that goes up to 350m (compressed) without issues.

I'm working on a calendar based website where the same data (the calendar data) is displayed in a range of different ways. So the whole thing revolves mainly around one (big) datasource.

Uncached its 3-4 seconds (yikes!). Datasource caching that big culprit gets it down to an acceptable 1 second, but still too slow for regular browsing. Combining both caching mechanism i could make sure that the visitor gets the 3 second delay maximum once per visit, and a bunch of 1 seconds for first-hit pages.

If I'm using cachelite only, I'll get 3-4 seconds on every first hit on an updated page, after each global flush.

But when using both simultaniously then updating is a problem, unless both caches are purged simultaniously - But I'll try the abovementioned things. A global, manual, mega-purge would work. The client is OK with pressing "Flush" after updating

I guess the easiest would be to make my own flush.php that

1. deletes all .xml from /cache
2. then redirects to /?flush=site

You can make life easier - modify the cacheable datasource lib file. And in the Grab function make a check if user has ?flush attached. If it does include flush then just delete the Cache for the particular 'page' if that would suit you.

1. If it still takes more then 1 second after adding Cacheable Datasource then you have some inefficient XSL / too large XML. Try checking with ?profile and check what takes longest. If ds creation or XSLT.

Had a similar issue had to restructure most of the XML output for efficiency. Might mean more then 1ds on some pages but still less xml.

It's the DS (2000 queries!). I got this resource-heavy calendar included in the menu (=on every page). It includes hierarchies and subsections and a whole bunch of things, not much can be done about it performance-wise, I think.

Since the calendar is displayed on every page, a page-flush makes less sense than a global flush after an update. Then I also have a bunch of other datasources per-page. But for the big calendar datasource, Cachable datasource make absolute wonders. And then i can use cachelite as a catch-all for all other purposes. It works really great like this.

Quite happy how it works now. Only improvement would be to automatically call my flush.php after any update. Is there some central kind of 'hook' where this can be done?

J

yep there are some hooks - post save I cannot remember the name offhand.

Is there an easy way of getting the older version of Cachable Datasources working with the newer version of Subsection manager?

Subsection manager: 2.2.1 Cachable Datasources: 0.5

What happens, when a section with a subsection gets cached, none of the susections fields appear to have made it into the cached version. Has anybody else experienced this problem?

EDIT: I've added a post to the Subsection manager thread which contains more detail, see: http://getsymphony.com/discuss/thread/83270/2/#position-27