Symphony.

Hello folks,

I have recently started playing around with Symphony and am really enjoying it. When I used to use Drupal (I've only been toying with Symphony for about ten days but am already happy to place that in the past tense), I had many gripes with its usability, performance, flexibility, etc. However, it was only when starting with Symphony that these really crystallised for me. I'm now converted and am looking forward to publishing my first project shortly, and thereafter starting to convert existing projects from Drupal. That's all by way of my background and also by way of thanks to the community and developers for your excellent work.

Now that that's out the way, down to business. With the usual apologies if I have missed any other relevant threads or if this question has already been asked elsewhere. I have looked but not found so please do point me in the right direction.

I have started to try to formalize a Symphony workflow. I am very familiar and comfortable with git and like the idea that I will manage projects between my local development machine, staging server, and production servers. I'm well on my way to getting myself up and running with this, and will be using Remie Bolte's Continuous Database Integration extension to manage structural database changes between environments. I'm confident that this won't cause any major problems and think it will be a much easier solution than I come across with other CMSs. So far, so good.

Despite this quite clear workflow, it strikes me that there is something substantial missing: tests. I'm a strong believer in integrating automated testing, and would really like to be able to run test suites on code as part of (say) the check-in process. This would have the usual sorts of advantages that unit, regression, and acceptance (perhaps alongside front-end usability, accessibility etc.) tests afford.

In particular, the git deployment workflow -- which seems to me by far the most robust way of managing Symphony deployments -- begs that both extensions and Symphony core are managed through git. Which is obviously sensible. However, without a decent coverage of tests, it is really difficult to know whether it's safe to (say) upgrade an extension because we have no automated way of validating that the upgrade is compatible with core and with other extensions. If these tests were in place, we would be able to quickly gain a high-confidence indication of whether or not upgrades were successful.

So, I guess the bottom line is:

• Is there a suite of tests which exercise Symphony core? (And if so, where?)
• How common is it that extensions are provided with test suites? (And, if common, where do they reside?)
• Does anybody have experience of integrating automated testing into a Symphony workflow that they could share?

Assuming that testing is rare, I would hope to start a dialogue which might persuade the inclusion of a decent test suite in future Symphony versions and establish extension-writing guidelines which strongly encourage the provision of tests. So if anyone would like to chip in, please do.

Thanks in advance for your thoughts!

First of all: welcome!

Despite this quite clear workflow, it strikes me that there is something substantial missing: tests

You are very right. Unit testing is something we are currently talking about. However, at this moment the core is very hard to test, because the elements are rather tightly coupled - it was not written with testability in mind.

So, while I think testing would be one of the best things we can do for a next big release, it is a waste of time to provide tests for the current core; it's just too much work. The same goes with extensions, the only way to test extensions is by instantiating them, and believe me, instantiating extensions is one of the most difficult things to do within Symphony at this moment.

instantiating extensions is one of the most difficult things to do within Symphony at this moment.

Can you clarify your meaning here?

Are features for 2.5 already planned? If not, I propose that we don't introduce any new features in 2.5 and instead use this release for a major core refactoring, so all upcoming changes after 2.4 (database, relationships, etc.) can be implemented and tested in a solid way.

Rowan wrote symphony_tests...

We will definitely have a rollover effect from 2.4 features as we seem to have a severe lack of developer time for things as they are.

When @brendo is on the core solidly from the new year, I expect things will pick up on core coding, although I do think it a little unfair to leave the main brunt of the coding to one person. If anyone wants to start planning things in relation to unit testing and refactoring, then all I suggest is to get on with it now ;o) Yes there will be a giant shift in the core code during 2.4, but things can start early as we know where the main shift will be. Perhaps starting on the other parts of the core will help identify issues early.

Can you clarify your meaning here?

Sure. What I am trying to say is that because the core is very tightly coupled, and most extensions will need to do something to the core, you'll have to pretty much rebuild symphony in a test in order to run it.

Let me illustrate this with an example; if your extension needs access to an entry, you'd call the entrymanager class. This class will in turn make calls to the Symphony object, which holds the logger, the database, etcetera.

So, in order to test this, you'd have to create the Symphony object, which in turn loads all extensions, sessions, sets up an error handler, reads from the config, uses apache variables and so on. This means you'll have to create either a mock for each of these, and hope you haven't forgotten about one static method that we introduced. What should have been a rather trivial thing to do: test some code that loads data from a section, has now become quite a nightmare where you don't really know what you're testing.

In fact, because you have so many (hidden) dependencies and tightly coupled classes, unit testing is pretty much impossible, because you can't isolate the code you want to test (the extension, in this case).

Does that explain it a bit better? If it doesn't, please watch the talk by Misko Hevery, he can explain this much, much better than I can.

Yeah, I think I understand what you mean now. Thanks for the explanation.

Thanks for the explanation.

No problem.

By the way, I am not too sure how badly we want to test the core, and if we're willing to rewrite big parts of it just to make it testable, but I could free one evening a week to work on this if that would help the progress?

Hi guys,

Thanks for your incredibly quick responses. The rumors about the community here seem true, which is great news.

I absolutely don't want to tread on any toes, but I would be keen to help in any way possible. As such, please feel free to ignore any of these comments or just point me to where I can read up on things!

I haven't watched the talk you mentioned, creativedutchmen, but I will take a look tomorrow. The coupling issues you highlighted seem serious and are what I had begun to expect from briefly looking through code. I gather that when we talk about refactoring for a major release, we are talking Symphony 2.4/5. However, might it make more sense to timetable a core re-write for 3.0 and apply future feature updates to a 'cleaner' core codebase? I guess this is essentially jens' idea, except I don't have much of a feel for the development roadmap.

Meanwhile, is this a reasonable/appropriate place to start a discussion about how testability could be factored into the next major releases? It strikes me that systematically writing for testability requires the creation or updating of a Symphony styleguide. Does such a document exist already? More generally, I guess I don't have much of a feel for how Symphony development proceeds. Are there Working Groups under whose remit these issues fall, or perhaps members of the team who would want to take responsibility?

As I say, I would really like to contribute if possible and practical, so please do point me in the right direction.

By the way, I am not too sure how badly we want to test the core

We should, same for extensions. Improving code quality should always be a priority over features.

I would rather spend some dedicated time to help with this instead of having to fix annoying issues and run into fatal errors on every client project.

but I could free one evening a week to work on this if that would help the progress?

@creativedutchmen

I'm eager to help if you're willing to take on a leading role and give me very specific instructions what to do and where to start.

I'm also currently discussing some database stuff with Alistair. Implementing this would require some refactoring too, so maybe now is a good time to do both in one effort?

brendo, I believe is working on some testing stuff but I don't think it's part of the core.

I'm also currently discussing some database stuff with Alistair. Implementing this would require some refactoring too, so maybe now is a good time to do both in one effort?

Yes! The database class is one of the most important dependencies, one that pretty much all the other code is built upon, so it would be a shame if we had to rewrite it to make the core testable:)

The good part is that it is one of the parts with least dependencies, so writing it in a testable manner shouldn't be too hard - although the way it is used will change when we make the rest of the core testable.

Shall we have a little chat about this on talker, google chat or anything else?