Symphony.

Hello everyone! :)

Symphony 2 beta 1 was released almost a year ago, and after beta 5 a long period of no updates followed while we worked on the elusive "RC1" version. I'm going to tell you about what happened over this time, and what we have just decided for Symphony's immediate future, but I'll do those in reverse order so that if you just want the juice, you only need to read the very next paragraph.

Versions

RC1's to-do list required a full rewrite of the codebase and is still growing constantly as we fine-tune its new concepts. In the meantime, the existing, stable and soon-to-be-deprecated beta was gathering a decent number of extensions and being used for many critical projects. So we've decided to take the responsible course of action and correctly call RC1 "Symphony 3" and postpone its development while we fix the remaining bugs in Symphony 2 beta and release the result as the full monty, non-beta Symphony 2.0 version.

There will no longer be a Symphony 2 Release Candidate. Instead, we're going to release Symphony 2.0 final as a bug-fix update to the current beta and save our progress on RC1 for the next major version, Symphony 3.

Some benefits of this are:

• Extension developers can breathe a sigh of relief since their hard work was not in vain.
• Sites running on Symphony 2 beta can get continued updates and bug fixes.
• Compatibility with the current beta is assured for all future 2.x updates.
• We plan to support Symphony 2 long into the future, even after Symphony 3 is released. (More about that below)
• Community development is welcomed and encouraged. Fork the git repository and send us a pull request if you have a patch you'd like to contribute.
• All future versions of Symphony are Free Software (under the MIT licence), so there's no obligation to license or contribute any changes you make if you work for Super-Secret Co.
• The Symphony Team is under a lot less pressure to finish Symphony 3 asap.

While there are a few improvements and features we plan to add to Symphony 2 in the meantime (in addition to as many bug fixes as possible), we want to focus most of our attention over the next 12 months on Symphony 3. But don't let that stop you from suggesting improvements/features on the bug tracker (and bugs too, of course) since if they are useful enough someone else may wish to implement them.

Continued... (Apparently Vanilla has size limits)

Continuing...

Delays

It's difficult to explain why Symphony development is so often plagued with delays, so I'll give you a concrete example:

1. The main goal for Symphony 2 beta 6 was to convert Symphony's XML processing over to use the DOMDocument interface. (Alistair explained this in depth.) Alone, this would have been quite a substantial update.

2. We realised that since there's a DOMDocument object in memory while a page gets rendered, it can be queried anytime with XPath, even while it's still being built. This is exactly what param output was designed to do, except XPath does it better - you don't need to "set up" the data source that has information you want, you can pull information from multiple fields of a single data source, you can use all of XPath's built-in functions to compare data, and the whole process winds up being more efficient.

   That was a killer feature. We couldn't not implement this. But it was a massive amount of work since it meant most of the data source editor had to change, the section link field became useless so we had to figure out some other way to convey linkedness in the interface, just to mention a couple of consequences. Talk about scope-creep!

3. We found more uses for XPath the more we used it, and it started to become annoying that Symphony's injected params ($root, $today, and friends) were outside XPath's reach. Simple solution: use XML for param data instead.

   With that tiny change, it was scary how much of the interface could be converted to use XPath. Events could have boolean conditions specified through an XPath query to indicate whether or not they should perform an action. Where we'd formerly used magical syntax to combine strings (e.g., "a,{$b},c"), the XPath concat() function could take over. Needless to say, this wasn't helping us reach our deadline.

4. Since nearly all of the relation of data in Symphony can be done through XPath and XSLT, the next logical step is to move from a relational database to an XML database. This is really the ideal fit for Symphony, but we've been blinded for so long by our old PHP + MySQL habits.

   It's still in the drawing board phase so we haven't progressed very far with it, but this massive redesign of the underlying data model will mean we should probably change the terminology of "fields" to "elements", at the very least. There are probably more much less trivial consequences, I just don't know what they will be yet!

So that's the basic direction that we are taking Symphony 3 in. There are a few other small improvements here and there, but the main changes are those above.

Divergence

The other reason we decided to clearly separate v2 and v3 is for compatibility reasons. Symphony 3 may (it hasn't been decided yet) have stricter system requirements than Symphony 2, and so if this was the case, it would make sense to keep both versions around with continued development and support. I don't want to make any promises, however; the latest version of MySQL has support for XML input and output, and by the time we release Symphony 3, most shared hosting providers might meet that requirement.

It's also about time that we started using sensible version numbers, rather than try to cram the kitchen sink into minor point releases.

Partnership

Some more good news: We've recently formed a partnership with a London company called Airlock to focus on getting more value out of Symphony. We're very happy with the arrangement since it lets us spend more time working on Symphony and in particular involve Symphony in a number of high profile projects.

Although it took some time for us to get settled in, this should mean that updates will flow more frequently from now on, and we'll also have a lot more time to work on improving the Symphony and Overture websites.

I think it's also Symphony's 3rd birthday today. (I'm pretty sure 1.0 was released on 1 Dec 2005.) If that's not cause for celebration, I don't know what is!

Scott,

Thanks for this very informative update. It sounds like S3 will be well worth the wait, and now we can get down to some more extension building for S2!

"Since nearly all of the relation of data in Symphony can be done through XPath and XSLT, the next logical step is to move from a relational database to an XML database"

All I can say is, bring on eXist! I was just learning about XRX this weekend. I think Symphony could be the first XRX cms, and it sounds like you're headed that direction!

Thanks again!

I think Symphony could be the first XRX cms, and it sounds like you're headed that direction!

XQuery is frighteningly interesting, and was what started us thinking along these lines. XRX sounds like a very powerful combination. I don't know how future-proof XForms are, but you never know!

Thanks, Scott. This is indeed good news and cause for celebration.

Happy Birthday, Symphony! Thanks for the update, Scott. I hope we don't have to wait 12 months before getting to play with beta versions of Symphony 3. Sounds very intriquing indeed.

I'm glad to hear that Symphony 2 will be coming out of limbo. My contributions had also gone into limbo, waiting for the next big thing. I didn't want to be working on ensembles, projects and tutorials that would be obsolete with the final release.

Can we look forward to the resolution of the front end entry form checkbox bug?

Scott, this is a great news!
Question: Will we get the calendar back with the release of Symphony 2?

this is good news. at work, I hear people say things like "oh, v1 isn't backwards compatible but it's this next release is v1.4" which doesn't make any sense whatsoever. but i'm glad you guys are taking the "responsible route" with changing the version number because of all the major changes going on.

i've always wondered why more projects are not approached in this manner so that people can really distinguish the differences.

can't (but going to have to) wait to see what's in stored for v3!

Great news indeed. Symphony's third incarnation sounds more than promising. Ninja Turtles!

@bauhouse:

I hope we don't have to wait 12 months before getting to play with beta versions of Symphony 3.

Apparently we got this question a lot. I'm sorry! I just picked that number to make the whole post symmetrical since it started with 12 months ago and I like symmetry.

In this global financial crisis, it's safest for us to spend more time on work that is directly and immediately profitable, so we may not get Symphony 3 done in the shortest time possible, but I'm very confident it will be much sooner than 12 months from now. (And even after we release Symphony 3, we're still going to continue working on improving it.)

Can we look forward to the resolution of the front end entry form checkbox bug?

Absolutely. [EDIT: We won't have time to fix this before 2.0, but we'll definitely get around to it soon after.]

@Nils:

Will we get the calendar back with the release of Symphony 2?

I don't see why not, however I won't have time for this before the 2.0 release, which will be soon.

I'd made the date field for Symphony 3 use a grouping of 4 select boxes by default which, despite its simplicity, I found to be very effective (easy to use with keyboard only, etc). Timepickr is another unique and interesting design for date input, so maybe we might see lots of different date picking extensions.

Thanks, Scott. Glad to hear that "12 months" may be more of an arbitrarily chosen phrase. Symmetry is nice, but getting to play with shiny new toys at Christmas time sounds even better. No pressure, though.

Regarding the crisis, I've heard that the Chinese combine two characters to create the character for crisis: danger and opportunity. It makes sense to sharpen focus during such a dangerous opportunity as the one in which we all find ourselves.

At the same time, many hands make light work. If there is anything we can do to help, I'd be happy to contribute. For instance, how does the decision to have two versions impact Mark Lewis's commitment to developing documentation? Will this double his work, or should the community pitch in to develop documentation and support for Symphony 2, while Lewis focuses on documentation for the final release of Symphony 3?

... we're going to release Symphony 2.0 final as a bug-fix update to the current beta ...

Another question: Will the bug tracker be "cleaned"? There are many entries that have been solved but neither the solution has been posted there nor the status has been changed.

I want to echo bauhouse in saying "please give us work if there's work we can be trusted with."

@bauhouse:

For instance, how does the decision to have two versions impact Mark Lewis's commitment to developing documentation?

I feel bad for this. We certainly don't want to double Mark's workload, but I think documentation (especially of the kind I hear he is planning on, which focuses on high-level explanations of how pages, fields, data sources, etc are related) will translate well between versions, so it might not even be necessary to have two versions of documentation.

Whatever Mark decides will be of great benefit regardless. It's important to us to provide API-level documentation for extensions, so this will be another focus of ours in the short term.

@Nils:

Will the bug tracker be "cleaned"?

I'd rather not start afresh, but instead go through them all and update as needed. We'll start on this today in fact.

@czheng:

Thanks! We really appreciate this. If all goes well, 2.0 should be released later this week, and after that, more bug-testing, patches and improvements would be very helpful.

This week! Cool!

Cool. I assume you guys will brief us on how best to submit patches and such... Cheers down the homestretch!

Thanks for all the explanations, it makes a lot of sense now. I'm looking forward to putting my hands on it and I'm willing to help with patches, bugs, etc.

Thank you guys!

@czheng

I assume you guys will brief us on how best to submit patches and such...

We intend on using Git exclusively now, and it has a fantastic system for allowing contributions via forking, and issuing pull requests. I will be writing up an article on the topic shortly after release. I really look forward to allowing everyone channels for contribution. Exciting times!

@bauhouse

I've heard that the Chinese combine two characters to create the character for crisis: danger and opportunity.

Whilst a fairly romantic notion, it's not entirely accurate. If you'd like to know a little more about it, have a read of http://www.pinyin.info/chinese/crisis.html. It's very informative.

Hmm. Very informative, indeed. I guess I heard wrong. Thanks for setting me straight.

This is all good news to me.