Import/export CSV
This is an open discussion with 142 replies, filed under Extensions.
Search
This might be only slightly related, but for the new Email Newsletters extension, Michael and I are working on a very reliable background process script.
Do you think the code could help with your extension? It isn't fully completed yet, but I think you might be able to get it to work with just a few modifications.
Import/export CSV
updated to version 0.2.1
on 24th of June 2011
- Speed optimised (up to 10 times faster with importing!)
- Security fix (see note below)
- Code cleanup
Please note:
The previous version 0.2 had a security issue where the uploaded CSV-file was not deleted after importing. You might have to delete this file manualy in your manifest/tmp
-folder.
This update prevents something like this from ever happening again, since it stores the uploaded CSV data directly into the cache-table in the database, instead of putting a physical file on your domain.
Geil, I have just tried this extension since the last upgrade, and I get the following errors (see attached image) for every single batch and therefore every entry.
The extension tells me that all entries are imported, but none are.
Also, just for reference, this field type is 'TextBox'. Does that have anything to do with it?
Just tried this again after changing the fieldtype to 'Text Input' and it's the same error.
As soon as I have time I'll look into this.
Unfortunately I'm very very busy at the moment, and since we use Symphony on a commercial base, it means that I mostly update/fix extensions when the client has new requirements or we stumble into some issues ourselves.
So to make a short story shorter: as soon as I have a project in which I am using the importer/exporter again, I'll look into this issue.
Perhaps someone in the community has some spare time to look into it?
Working with Reference Links field
It's basically the same as selectbox link. So just copy the driver file(ImportDriverselectboxlink.php), rename it to ImportDriver_referencelink.php. Then, find "selectbox_link" and replace by "referencelink" in the new PHP driver file.
@kanduvisla Thank you so much for this awesome extension. Symphony CMS and it's developers kick asses!
Import/export CSV
updated to version 0.2.2
on 7th of July 2011
- updated documentation
- added driver for Reference Link Field (thanks @edubaldan)
If more people make drivers for this extension, please share them! It will make the extension harder, better, faster, stronger.
@designermonkey: is 'TextBox' the only field in the section in which you are trying to import? Or are there more fields in the section?
@kanduvisla, any reason why a CSV export from a section would output css and html as well as mixed bag of content into the CSV file?
If what is meant above by referencelink is "Select Box Link" then I'll give the updated Extension a go and see if that helps.. cos I have 3 select box links in the section.
@moonoo2: it outputs css and html? which field type? If a fields' value is CSS/HTML (like a textarea could have for example), I can imagine it puts this in the values. Or if a textarea has a rich text editor (like CKEditor or TinyMCE), it's exported value is HTML. Could you provide more details on when this happens? Could it be that it's a Symphony Error message that gets exported into the CSV file?
Reference Link is just another field type. It has no effect on Select Box Link whatsoever, since that already has it's own driver.
Just one, but it won't work with any field type though. So it's technically not CSV data as there is only one field...
@kanduvisla, it is looking like the symphony error is included in the file on closer inspection.
@moonoo2: Could you provide me with the error message?
@designermonkey: I tried to reproduce the error by creating a section with one field of type 'input'. Created some entries, exported it, everything went OK. Edited the CSV-file, imported it, everything went OK. Are you sure your CSV-file isn't corrupted or something?
The only way I seemed to reproduce the error is by selecting Don't use as field name when importing at step 2. Are you sure you have selected a target field to import to?
@kanduvisla, I've pasted a private pastie for the moment to see if I can make sense of the output.. here's the first bit I can read: SELECT SQL_CACHE * FROM
symentriesdata117WHERE
entryid= ;
@moonoo2: and how can I see this private pastie? :-P
Ummm mail me and I'll send link.
Or I can mail you the CSV that's output!
@moonoo2: could you replace drivers/ImportDriver_selectbox_link.php
with the attached file and tell me if it works?
Create an account or sign in to comment.
Awesome! I like the direction this is going.
You are kidding me. Only this morning I added AJAX-based exporting so that CSVs are created in batches, and a final request concatenates them into a single file. Admittedly my code is quite rough and ready, and likely won't merge with your latest updates... but would you like to implement? Or shall I?
I also had a few other ideas:
content.publish.php
that gets the$_GET
, instantiates the field, builds the$where
and$joins
that you need to pass toEntryManager::fetchByPage
We have a site at work with 250,000 entries in a Competition Entries section and downloading theCSV maxes the memory limit after about 2500 entries. So I built the AJAX output, which is still remarkably slow (about 40s for a page of 1000 entries). I wonder if you could speed that up somehow. Of course, adding the filtering would mean the client doesn't have to download all 250,000 entries; they could filter by competition first, and only download the entries they need.