0 users online. Create an account or sign in to join them.Users

Search

Import- and export-functions are now handled by seperate drivers, providing more flexibility for different types of fields

Awesome! I like the direction this is going.

Importing is now done with AJAX-calls, solving memory- and loadtime issues when trying to import, say, 2000+ entries

You are kidding me. Only this morning I added AJAX-based exporting so that CSVs are created in batches, and a final request concatenates them into a single file. Admittedly my code is quite rough and ready, and likely won't merge with your latest updates... but would you like to implement? Or shall I?

I also had a few other ideas:

  • the option to add an "Export as CV" link to all sections (perhaps a config setting to choose which sections to append this to)
  • support for filtering. So if I filter the backend with Publish Filtering and then click Export as CSV, to pass this filter through to the exporter. This is actually very simple to implement, is a case of duplicating the code from content.publish.php that gets the $_GET, instantiates the field, builds the $where and $joins that you need to pass to EntryManager::fetchByPage

We have a site at work with 250,000 entries in a Competition Entries section and downloading theCSV maxes the memory limit after about 2500 entries. So I built the AJAX output, which is still remarkably slow (about 40s for a page of 1000 entries). I wonder if you could speed that up somehow. Of course, adding the filtering would mean the client doesn't have to download all 250,000 entries; they could filter by competition first, and only download the entries they need.

This might be only slightly related, but for the new Email Newsletters extension, Michael and I are working on a very reliable background process script.

Do you think the code could help with your extension? It isn't fully completed yet, but I think you might be able to get it to work with just a few modifications.

Import/export CSV updated to version 0.2.1 on 24th of June 2011

  • Speed optimised (up to 10 times faster with importing!)
  • Security fix (see note below)
  • Code cleanup

Please note:
The previous version 0.2 had a security issue where the uploaded CSV-file was not deleted after importing. You might have to delete this file manualy in your manifest/tmp-folder.
This update prevents something like this from ever happening again, since it stores the uploaded CSV data directly into the cache-table in the database, instead of putting a physical file on your domain.

Geil, I have just tried this extension since the last upgrade, and I get the following errors (see attached image) for every single batch and therefore every entry.

The extension tells me that all entries are imported, but none are.

Attachments:
Screen shot 2011-07-01 at 17.25.27.png

Also, just for reference, this field type is 'TextBox'. Does that have anything to do with it?

Just tried this again after changing the fieldtype to 'Text Input' and it's the same error.

As soon as I have time I'll look into this.

Unfortunately I'm very very busy at the moment, and since we use Symphony on a commercial base, it means that I mostly update/fix extensions when the client has new requirements or we stumble into some issues ourselves.

So to make a short story shorter: as soon as I have a project in which I am using the importer/exporter again, I'll look into this issue.

Perhaps someone in the community has some spare time to look into it?

Working with Reference Links field

It's basically the same as selectbox link. So just copy the driver file(ImportDriverselectboxlink.php), rename it to ImportDriver_referencelink.php. Then, find "selectbox_link" and replace by "referencelink" in the new PHP driver file.

@kanduvisla Thank you so much for this awesome extension. Symphony CMS and it's developers kick asses!

Import/export CSV updated to version 0.2.2 on 7th of July 2011

  • updated documentation
  • added driver for Reference Link Field (thanks @edubaldan)

If more people make drivers for this extension, please share them! It will make the extension harder, better, faster, stronger.

@designermonkey: is 'TextBox' the only field in the section in which you are trying to import? Or are there more fields in the section?

@kanduvisla, any reason why a CSV export from a section would output css and html as well as mixed bag of content into the CSV file?

If what is meant above by referencelink is "Select Box Link" then I'll give the updated Extension a go and see if that helps.. cos I have 3 select box links in the section.

@moonoo2: it outputs css and html? which field type? If a fields' value is CSS/HTML (like a textarea could have for example), I can imagine it puts this in the values. Or if a textarea has a rich text editor (like CKEditor or TinyMCE), it's exported value is HTML. Could you provide more details on when this happens? Could it be that it's a Symphony Error message that gets exported into the CSV file?

Reference Link is just another field type. It has no effect on Select Box Link whatsoever, since that already has it's own driver.

Just one, but it won't work with any field type though. So it's technically not CSV data as there is only one field...

@kanduvisla, it is looking like the symphony error is included in the file on closer inspection.

@moonoo2: Could you provide me with the error message?

@designermonkey: I tried to reproduce the error by creating a section with one field of type 'input'. Created some entries, exported it, everything went OK. Edited the CSV-file, imported it, everything went OK. Are you sure your CSV-file isn't corrupted or something?

The only way I seemed to reproduce the error is by selecting Don't use as field name when importing at step 2. Are you sure you have selected a target field to import to?

screenshot

@kanduvisla, I've pasted a private pastie for the moment to see if I can make sense of the output.. here's the first bit I can read: SELECT SQL_CACHE * FROMsymentriesdata117WHEREentryid= ;

@moonoo2: and how can I see this private pastie? :-P

Ummm mail me and I'll send link.

Or I can mail you the CSV that's output!

@moonoo2: could you replace drivers/ImportDriver_selectbox_link.php with the attached file and tell me if it works?

Attachments:
ImportDriver_selectbox_link.php

Create an account or sign in to comment.

Symphony • Open Source XSLT CMS

Server Requirements

  • PHP 5.3-5.6 or 7.0-7.1
  • PHP's LibXML module, with the XSLT extension enabled (--with-xsl)
  • MySQL 5.5 or above
  • An Apache or Litespeed webserver
  • Apache's mod_rewrite module or equivalent

Compatible Hosts

Sign in

Login details