Symphony.

Hi guys,

I retrieve some data from an xml feed that gets populated at different times of the day every day. I wish to ensure to update such values on my website as soon as such values are updated in the xml feed. I think running a cron job every minute: 1) is exhaustive just to update some values 2) the update time does not exactly reflect when the value was updated in the feed.

The other data that is extracted from this XML feed should also be cached and only this value should be purged as soon as it is pushed to symphony.

Anyone implemented such functionality?

Thanks

Do you have control of the XML feed where the data is getting populated? If so, you could set it up so that when it updates it hits a Symphony URL.

This URL could do two things:

1. You push content to it via POST and then let a Symphony event handle it (this would require the POST content to match you Section though)
2. Tell Symphony, 'hey, data is ready, come get it', and then use custom logic/XML Importer to pull the data off the feed.

mmm...that sounds like two interesting proposals except for the fact that we do not have control of the XML feed where the data is being populated.

Where is it coming from? Do they offer callbacks or anything?

I'm not sure how you would fully integrate 'push' without having some sort of hook/listener for when the XML feed is updated.

The XML Importer would help you out with duplicate content, as it allows you to set a field as unique.

have got something similar. As well have no control over the feed which was also somewhat slow.

What I was thinking about is that I have a server used internally for file sharing; so its not a production server; and I would put a script/daemon thread running on this server whose role is basically to hit this XML feed every few seconds say 5-10 seconds since I would like my data to update immediately. Then when I notice that there is in fact some new data push this with a post to Symphony.

I am using a custom extension to populate this data at the moment since XML Feeds used before were pretty in-efficient. So basically was going to create a custom php script that connects to the Symphony Database and updates the field where this value should go. reducing as much as possible overhead on the database by doing a single update call rather then running a page of events etc.

However I am also planning a cron every few hours to force an update on all the data. So if anything in the feed changed after I detected an update/insert I also get to update this data eventually if not straight away. This also helps in making sure all data is synchronized if anything goes wrong with the push functionality.

I retrieve some data from an xml feed that gets populated at different times of the day every day. I wish to ensure to update such values on my website as soon as such values are updated in the xml feed. I think running a cron job every minute: 1) is exhaustive just to update some values 2) the update time does not exactly reflect when the value was updated in the feed.

What information does the feed contain? Does it need to be instant, or does an offset of a few minutes matter?

If you want push, the publishing process of the XML feed will need to fire off a callback to your service, that's how push works. Otherwise you'll need to schedule a pull.

You could try running a pubsubhubbub server, but that sounds like a lot of work if a simple pull every few minutes would suffice.

Thanks Nick and Brendo for both of your replies. Feed would be preferably be instant but since we have to control over it, a schedules pull would have to suffice.

Thanks guys for all the great work and support!