Symphony.

Not necessarily. Using XSLT you can combine the different data sources into one XML nodeset and iterate through them. The problem occurs when feeds have different date formats, making ordering difficult. But you can achieve a lifestream using either Dynamic XML or native Section-based data sources. Something like:

<xsl:template name="merge-data-sources">

    <xsl:variable name="entries">

        <xsl:for-each select="/data/twitter-feed/entry">
            <entry>
                <date><xsl:value-of select="..." /></date>
                <text><xsl:value-of select="..." /></text>
                <type>Twitter</type>
            </entry>
        </xsl:for-each>

        <xsl:for-each select="/data/flickr-photos/entry">
            <entry>
                <date><xsl:value-of select="..." /></date>
                <text><xsl:value-of select="..." /></text>
                <type>Flickr</type>
            </entry>
        </xsl:for-each>

        <xsl:for-each select="/data/blog-posts/entry">
            <entry>
                <date><xsl:value-of select="..." /></date>
                <text><xsl:value-of select="..." /></text>
                <type>Blog</type>
            </entry>
        </xsl:for-each>

    </xsl:variable>

    <xsl:for-each select="exsl:node-set($entries)/entry">
        <xsl:sort select="date" date-type="number" order="descending" />
        <xsl:copy-of select="." />
    </xsl:for-each> 

</xsl:template>

<xsl:template match="/">

    <xsl:variable name="entries">
        <xsl:call-template name="merge-data-sources" />
    </xsl:variable>

    <ul>
        <xsl:for-each select="exsl:node-set($entries)/entry">
            <li>...</li>
        </xsl:for-each> 
    </ul>

</xsl:template>

One named template does the work of creating a variable, which essentially becomes a nodeset. Inside this nodeset are Twitter, Flickr and Blog entries. The purpose of this is to get each "object" into a common format, so in the example above each new "entry" has a Date, Text and Type. Once the variable is created, these new entries are sorted by the Date element. In reality this would need to be a numerical timestamp.

In your main content template you can then call this template and cache the nodeset into a variable ($entries), and loop.

The Twitter, Flickr and Blog posts could well be Dynamic XML (external RSS feeds), or Symphony Sections, or a combination of the two. I prefer to use the pre-caching method into Sections — the main reason being that the date formats are standardised, and I can sort the nodes on the @timestamp attribute.

The support for the "mixer" application like Tubes or Yahoo! Pipes is to simplify this process and remove the dependency from Symphony to do crunching of the caching, merging and sorting.

I've made the small changes needed to save microblog entries from Jaiku. If anyone is interested, let me know and I'll post.

My PHP skills are a bit dusty. A typical Delicious rss feed item looks like this:

    <item>
      <title>News after Newspapers: Nuts and bolts: How newspapers can optimize use of social media</title>
      <pubDate>Mon, 29 Dec 2008 20:44:28 +0000</pubDate>
      <guid isPermaLink="false">http://delicious.com/url/2849290c2608a313c4f71b3cee980bdd#thoresson</guid>
      <link>http://newsafternewspapers.blogspot.com/2008/12/nuts-and-bolts-how-newspapers-can.html</link>
      <dc:creator>thoresson</dc:creator>
      <comments>http://delicious.com/url/2849290c2608a313c4f71b3cee980bdd</comments>
      <wfw:commentRss>http://feeds.delicious.com/v2/rss/url/2849290c2608a313c4f71b3cee980bdd</wfw:commentRss>
      <source url="http://feeds.delicious.com/v2/rss/thoresson">thoresson's bookmarks</source>
      <category domain="http://delicious.com/thoresson/">blog</category>
      <category domain="http://delicious.com/thoresson/">media</category>
      <category domain="http://delicious.com/thoresson/">internet</category>
      <category domain="http://delicious.com/thoresson/">social_media</category>
    </item>

How can I get the categories into the array in the try-part?

How are you planning on storing the categories on your section? A tag field?

Yes. A tag field. So I want to iterate over the unknown number of cateogy-nodes and add each to a comma separated list.

My problem is to come up with the logic. All I can think of is iterating over all the nodes of each <item>, locking for <category>. But I guess there is a more efficient way.

Inside the try-catch you want to iterate through each category node to concatenate the tags. so you're already iterating over the item nodes I presume. Inside this loop, add this:

$category_tags = "";
foreach ($tweet->getElementsByTagName("category") as $category) {
    $category_tags .= $category->nodeValue . ", ";
}

Then store it in the array with the other variables:

array_push($new_tweets, array($tweet_id, $tweet_text, $tweet_date, $category_tags));

Then where you build the post variables, add the category field (assuming your tag list field is called "Categories").

$post .= "&fields[$i][categories]=" . $new_tweets[$i][3];

Thanks. I'm not familiar with the DOM (as you are using here, right?) and had no knowledge of the nodeValue method. Will give this a go next week, but it looks promising.

Yes. It worked. Thanks, Nick.

Is there a maximal length that the POST request can take? When saving delicious bookmarks, not all are saved and often the last one is truncated.

Theoretically no. There's a MAX_FILE_SIZE in my PHP (which Symphony adds by default). Is it always the last one? Perhaps it's a case that the data isn't being URL encoded before sending, which can sometimes lead to truncation.

Yes. It was that I had forgot to escape the data that caused the problem.

I was just moving my site from dev to live server. And this caching method stopped to work. It took me some time to realize it was because I had set the site in maintenance mode. Because of that, DOMDocument::load-method couldn't load the tweets already in my database or save the new ones.

Just a note, if someone else would run into the same problem.

First off thanks Nick for the excellent article. Using your technique I’m trying to back up a blog that I want to convert over to symphony. I’m backing up locally and then eventually exporting an ensemble and database to the live site.

So I’ve been able to back up the entry text. So my last hurdle is to save the images in the content node. I’m really not sure how to go about this.

Below is the xml in the rss feed. Also the content node is being saved as a text field in the section.

Any Thoughts?

<entry>
    <title type="html">Wearing a hat!</title>
    <id>tag:the.com,2009-09-27:/id/145/</id>
    <published>2009-09-26T21:16:29-04:00</published>
    <content type="html">&lt;img src="http://somewhere.com/blog//includes/thumb.php?file=../uploads/1st-day-at-school---6.jpg&amp;amp;max_width=500&amp;amp;max_height=500&amp;amp;quality=100" alt="1st-day-at-school---6.jpg" /&gt;&lt;br /&gt;&lt;br /&gt;&lt;p&gt;Wearing big hat!&lt;/p&gt;
    </content>
</entry>

Bumpedy bump. Anyone with some insight?

You’ll need some custom PHP to do this. You could try regular expressions; or HTML decoding the string, parsing into a DomDocument (loadHTML()) and then apply an XPath expression to squeeze out each image. Not simple though.

Yeah kinda figured it was going to be a hard one to do… Thanks Nick. Time to do some searching and hope I can put it all together.