Symphony.

That seems to help.

P. S.: If it's possible it would be great if this setting could be more than just an interim solution. It is more intuitive than the double negation before.

Ah, by the way, is there any plan to have full page caching as in 1.7?

Hoi!

I've immense, at least I'd say immense, page rendering time problems on my production server (a shared, but really premium one). In sum I tested my site (Sym. rev5 – latest) on three different systems:

• local with MAMP = fast
• development server = fast, too (quite the same times as local) and
• on my productions server = snail plus very erratic times (from about 0.4 s to over 6.0 s, currently on average somewhere between 0.7 s to 1.8 s), (the backend feels slower, too)

i.e. times local:

General Details

Engine Initialisation: 0.0031 s
Page creation process started: 0.0043 s
XML Built: 0.0536 s
XML Generation: 0.0008 s
Page Built: 0.0570 s
XSLT Transformation: 0.0053 s
Page creation complete: 0.0667 s

Data Source Execution

content_main_page: 0.0342 s
tag_factor: 0.0166 s

Event Execution

login: 0.0019 s

Full Page Render Statistics

Total Database Queries: 83
Slow Queries (> 0.09s): 0
Total Time Spent on Queries: 0.0112 s
Time Triggering All Events: 0.0019 s
Time Running All Data Sources: 0.0508 s
XML Generation Function: 0.0008 s
XSLT Generation: 0.0053 s
Output Creation Time: 0.0667 s

example times on production server:

General Details

Engine Initialisation: 0.0580 s
Page creation process started: 0.1260 s
XML Built: 0.6366 s
XML Generation: 0.0008 s
Page Built: 0.6392 s
XSLT Transformation: 0.3720 s
Page creation complete: 1.1373 s

Data Source Execution

content_main_page: 0.4719 s
tag_factor: 0.0109 s

Event Execution

login: 0.0528 s

Full Page Render Statistics

Total Database Queries: 83
Slow Queries (> 0.09s): 0
Total Time Spent on Queries: 0.0045 s
Time Triggering All Events: 0.0528 s
Time Running All Data Sources: 0.4828 s
XML Generation Function: 0.0008 s
XSLT Generation: 0.3720 s
Output Creation Time: 1.1373 s

Above all XML and XSLT is very slow, but not only. Database queries are fast. Has someone an idea?

Could some PHP extension make the difference?

extensions production VS development server:

;;;;;;;;;;;;;;                 ;;;;;;;;;;;;;;;
; production ;                 ; development ;
;;;;;;;;;;;;;;                 ;;;;;;;;;;;;;;;

;extension=apc.so              ;extension=bcmath.so
;extension=bcmath.so           extension=bz2.so
;extension=bz2.so              ;extension=calendar.so
;extension=calendar.so         extension=curl.so
;extension=curl.so             extension=dba.so
;extension=dba.so              extension=dom.so
;extension=dbase.so            ;extension=exif.so
;extension=dbx.so              ;extension=ftp.so
extension=dom.so               extension=gd.so
;extension=eaccelerator.so     extension=gettext.so
;extension=exif.so             extension=gmp.so
;extension=ftp.so              extension=iconv.so
extension=gd.so                extension=imap.so
;extension=gettext.so          extension=ldap.so
;extension=gmp.so              ;extension=mbstring.so
extension=iconv.so             extension=mcrypt.so
;extension=imap.so             extension=mhash.so
;extension=json.so             ;extension=mssql.so
;extension=mhash.so            extension=mysql.so
extension=mbstring.so          extension=pdo_mysql.so
extension=mcrypt.so            ;extension=mysqli.so
extension=mysql.so             ;extension=odbc.so
extension=mysqli.so            ;extension=pdo_odbc.so
;extension=pcntl.so            extension=openssl.so
;extension=pdf.so              ;extension=pcntl.so
;extension=pdo.so              extension=pdo.so
;extension=pdo_mysql.so        extension=pgsql.so
;extension=pdo_pgsql.so        extension=pdo_pgsql.so
;extension=pdo_sqlite.so       ;extension=pspell.so
;extension=pgsql.so            ;extension=readline.so
;extension=posix.so            ;extension=shmop.so
;extension=pspell.so           ;extension=snmp.so
;extension=shmop.so            ;extension=soap.so
;extension=soap.so             ;extension=sockets.so
;extension=sockets.so          extension=sqlite.so
;extension=sysvsem.so          extension=pdo_sqlite.so
;extension=sysvshm.so          ;extension=sysvmsg.so
;extension=wddx.so              ;extension=sysvsem.so
;extension=xmlrpc.so           ;extension=sysvshm.so
extension=xsl.so               ;extension=wddx.so
;extension=zip.so              extension=xmlreader.so
extension=zlib.so              extension=xsl.so
                               extension=zlib.so
                               extension=simplexml.so
                               extension=session.so

Any help appreciated. Thanks in advance

What are your production server's load averages? You can find out by using the top command in the shell.

I am afraid that won't help you, because (quote from an administrator where it lives) :-/

Load average has absolutely no bearing on shared accelerators any more. And it shows load for the physical machine (as always with Solaris zones).

e.g. load average: 13.39, 16.48, 13.02

hmm, that is pretty high. I'm no sys admin, so not sure I can comment on what your admin is saying. However, read this article. I'm pretty certain your shared environment is getting bogged down with waiting processes.

I believe him, because currently I get something like

Total: 4 processes, 4 lwps, load averages: 1.55, 3.45, 3.93

and the site also needs over 1 s for Page creation complete. Besides that I get different results displayed with command prstat -Z on shared versus non shared, but I just asked if I can get somehow adequate ohter data on shared that has bearing...

A chief posted

well the one thing I can usually say about these is that they're usually because of something silly, like making it do hostname lookups in a database config file.

I guess it should not be something silly, because I run the same install on 3 different systems and only one creeps. In config.php database host is localhost and port 3306. I'd like to be a magus, damn, I'm not.

I believe him, because currently I get something like

Fair enough. Like I said, I am no sys admin.

So, lets look at it closer. We know that these are responsible for most of the render time:

Time Running All Data Sources: 0.4828 s
XSLT Transformation: 0.3720 s

XSLT Transformation is just that, the transformation of the XML. How big is your XSLT document? How much XML is there?

Of the DS's it looks like content_main_page is the main culprit. Since the time spent on queries is low, it suggests the problem lies elsewhere. Is there a large number of entries it needs to construct? What does this DS do?

How big is your XSLT document? How much XML is there?

I'd say very tiny and really little.

Is there a large number of entries it needs to construct? What does this DS do?

No. Only three and it does really nothing special.

The whole site is tiny in content and else – rather bridging of a gap. If you want (I guess you're short in time so perhaps no) I'll send you login data (for that I'd need your email).

Oh, and belonging load averages

They're 8+ CPU machines, so a load of 64-128 is where you're going to start seeing some issues assuming that you're not compensating by providing more of what's causing the it all (in our case it's almost always spindles), so yes in our testing we've even had loads of up to 1100 where the machine can still be logged into, is responsive and works well (why? because things are broken out into many dedicated spindles). Loads of <64 generally means that things are being used fine.

So the average seems to be quite well.

So the average seems to be quite well.

Makes sense.

If you want I'll send you login data

I think that would be the best option. I can poke around and see where the code is slowing down. I would need FTP details for that though. Is that possible? team [at] symphony21 [dot] com

Is that possible?

Yep. Message should be in your box.

I do not want to flood your box, so I say it here: Thank you very much for taking your time!

Got your e-mail. I'll get back to you soon.

Hello!

I just set up Symphony on my production server and whoa, I seem to have some problems with page rendering times - although the rendering on my development server isn't always the fastest to my newbie eyes, either (Page creation complete fluctuates from 0.5 s to 2.1 s).

First the development server:

General details

Engine Initialisation: 0.0061 s
Page creation process started: 0.0083 s
XML Built: 1.2848 s
XML Generation: 0.0222 s
Page Built: 1.3107 s
XSLT Transformation: 0.0259 s
Page creation complete: 1.3451 s

Datasource Execution

einleitungstexte: 0.0495 s
marken: 0.0419 s
navigation: 0.0044 s
produkte: 0.8476 s
farben: 0.2912 s

Full Page Render Statistics

Total Database Queries: 1315
Slow Queries (> 0.09s): 0
Total Time Spent on Queries: 0.0426 s
Time Triggering All Events: 0 s
Time Running All Data Sources: 1.2346 s
XML Generation Function: 0.0222 s
XSLT Generation: 0.0259 s
Output Creation Time: 1.3451 s

Now the production server:

General Details

Engine Initialisation: 0.0368 s
Page creation process started: 0.0449 s
XML Built: 4.4643 s
XML Generation: 0.0925 s
Page Built: 4.5771 s
XSLT Transformation: 0.2126 s
Page creation complete: 4.8351 s

Data Source Execution

einleitungstexte: 0.1568 s
marken: 0.0506 s
navigation: 0.0188 s
produkte: 2.6933 s
farben: 1.5200 s

Full Page Render Statistics

Total Database Queries: 1315
Slow Queries (> 0.09s): 0
Total Time Spent on Queries: 0.4215 s
Time Triggering All Events: 0 s
Time Running All Data Sources: 4.4395 s
XML Generation Function: 0.0925 s
XSLT Generation: 0.2126 s
Output Creation Time: 4.8351 s

Compared to my other pages, there is a lot of XML. The DS produkte needs to construct quite a big amount of entries (a few hundred).

This seems to be just another example of the problem already addressed in this thread – I must admit all these numbers are just starting to make sense to me, so I'd be thankful for any insight and ideas what I might do to speed things up. This is just too slow!

Total Database Queries: 1315

Total Time Spent on Queries: 0.4215 s

Do you have query caching enabled? If most of those queries are simple read's, then caching should help quite a bit.

produkte: 0.8476 s

farben: 0.2912 s

How much data are these 2 DS's pulling? I.E. how big is the resultant XML?

Query caching is enabled.

produkte is pulling a lot of data – it produces 1092 lines of XML.

farben shouldn't be doing anything right now, as its required URL parameter isn't set – it produces 3 lines of XML:

<farben>
    <error>No Records Found.</error>
</farben>

Is the site highly dynamic? Unfortunately there isn't a magic switch in rev5 to "make things faster". RC1 will address many of the speed issues with better DS/DB scaling and caching built into the core.

However, in the interim, we have some simple to implement page output caching code we used for a high profile site (1 million+ hits per day) that was, for the most part, totally static. Perhaps that might help you. I've actually been thinking of doing tutorial on how to set up this style of caching. The way it works is to allow your site to change when front-end input occurs ($_POST) and refresh every X minutes otherwise.

No, the site is nothing near highly dynamic – it contains a blog (1-2 entries per week - no comments), a nearly static product overview (this part is the one using produkte and farben), a few texts and a contact form. I don't know about the hits per day yet, but it's definately, definately no high profile site :)

The page output caching code sounds interesting – as I mentioned before, I'm just about to get a rough picture of what all this stuff actually does and means, but I do have some help on this side as well – a tutorial sounds great, though!

If you want to enable the kind of caching I mentioned, here is replacement code for index.php which resides in the root installation location:

<?php

    define('_CACHE_LIEFTIME_', 300);
    define('_COOKIE_PREFIX_', 'sym-');  

    Final class Cache{

        private $_cache_path;

        public function __construct($cache){
            $this->_cache_path = $cache;
        }

        public function write($key, $data){
            return @file_put_contents($this->_cache_path . '/' . $key, $data);
        }

        public function delete($key){
            if(file_exists($this->_cache_path . '/' . $key)) unlink($this->_cache_path . '/' . $key);
        }

        public function read($key, $expiry=NULL){

            if(!file_exists($this->_cache_path . '/' . $key)) return false;

            if(!is_null($expiry) && (time()-filemtime($this->_cache_path . '/' . $key)) > $expiry){
                return false;
            }

            return @file_get_contents($this->_cache_path . '/' . $key);
        }
    }

    $Cache = new Cache(realpath('./manifest/cache/'));
    $cache_id = md5($_SERVER['SCRIPT_URI'] . $_SERVER['QUERY_STRING']);
    if(empty($_POST) && !isset($_COOKIE[_COOKIE_PREFIX_])){
        if($output = $Cache->read($cache_id, _CACHE_LIEFTIME_)) die($output);
        define('_WRITE_TO_CACHE_', true);
    }

    else $Cache->delete($cache_id);

    ## Check if Symphony needs installing
    if(!file_exists('manifest/config.php')){

        if(file_exists('install.php')){
            header('Location: install.php');
            exit();
        }

        die('<h2>Error</h2><p>Could not locate Symphony configuration file. Please check <code>manifest/config.php</code> exists.</p>');
    }

    require_once('manifest/config.php');
    require_once(CORE . '/class.frontend.php');

    $Frontend = new Frontend();

    $output = $Frontend->display(getCurrentPage());

    ## Important not to write to the cache if there is POST data, or the user is logged in. May mean the
    ## cache data is incorrect. E.G. contains success/failure messages or author/admin specific information
    if(defined('_WRITE_TO_CACHE_') && _WRITE_TO_CACHE_ == true) $Cache->write($cache_id, $output);

    print $output;

    exit();

Tweak the 2 defines at the top of the page to suit. _CACHE_LIEFTIME_ is in seconds. _COOKIE_PREFIX_ shouldn't need to change unless you specifically changed the prefix in your configuration file.

As I mentioned, if you are logged in or there is data in the $_POST, the cache will not be used.

It's a tad rudimentary, however it seems to work well. We noticed sites going from only filling a few hundred requests per second to serving a few thousand. Of course the trade off is that the data may not be the absolute latest. But, mess with the cache lifetime until you're happy.

I've done something similar to the method Alistair has outlined using the CacheLite library. Speeds things up immensely.