Many people have told me there is a way to import thousands of products in one day using CSV. I have tried this and the format is never correct for the system. How is one able to do this with out having to rewrite thousands of lines of text?
Is there automation tools available? I am working with a client who is working with wholesalers and is importing the products.
You certainly want to NOT rewrite the CSV files. But you should be able to work with them by changing your code easily enough.
Chances are the CSV files are not using the default values for “delimiter”, “enclosure”, “escape” and you may need to specify a “length” and enable auto_detect_line_endings fgetcsv
Can you post a sample line and explain what’s breaking things for you?
yup – see the mysql LOAD DATA command
another option is to build your own ascii loader using php which you can customise to be as flexible as you like and to work how you like.
you could output any problem rows from the csv to a separate error file for processing later on.
Im not a programmer, are there any video tutorials available?
Another question is once the products are uploaded is there a way to update inventory on every product on a daily basis with out requiring to re-enter the custom pricing etc?
Thanks!
There is, but if I’m honest, it’s probably going to be beyond your current skill-set. Luckily for you, you’re in the best place to learn how to do it!
If you’re willing to learn, everyone will be more than happy to help you.
Anthony.
where are you trying to upload the CSV file to? is it a custom shopping cart or out of the box? You also have to consider how complicated the database design is too.
I do product imports all the time… its a pain… 
The system the client wishes to use is Magento, the other alternative the client has brought up was Virtuemart.
Magento has this functionality built-in.
Yes, I have tried that and in order for the CSV file to work it has to be completely re-written. Is there any automatic tools that re-write CSV files?
The problem you are going to have is that each wholesaler will have their own data storage format. Each new wholesaler is going to require a different translator to normalize the data to some type of format that is consistent throughout your stores databases. Its a lot of work, especially for someone who knows nothing about programming. Its probably impossible for someone who knows nothing about programming. It will probably be easier to rewrite each one, to consistent format in excel, as tedious as that sounds. Than it might be possible to run a single SQL, command but I still doubt it unless your going to duplicate a whole bunch of data. In most cases each row of the csv will need to be broken out across several tables to achieve business goals on the store front programming end.
Its never that easy. Normally products are represented in some of hierarchy that must be normalized across multiple tables when moving from csv to database storage. The logic to do so needs to to written specific to each different format of data being dealt with. Its a pain, because no one seems to provide similar, consistent data sets. Not to mention the unforeseen possibilities of additional data down the line, for specific products that one may not have thought of during initial development or was unplanned.
Though, if you post the details on here someone will most likely attempt to do it for you.
Just out of curiosity are you a designer? I suspect the project started like, I just need a store designed, than turned into I actually need a way to get these products in my store and will soon spiral out of control in a whole bunch of other programmatic custom modifications to the Magento core…
I recommend you set the ground now if you are only responsible for the design. Otherwise, you will probably end up neck deep in other programming assignments, that were outside the original agreement. All of this is just speculation of course.
Hello,
Thanks for the assistance, yes I am aware that the ground needs to be set and my client understands that I can only do design and marketing and will need to hire a programmer. They were asking if I would be able to do this function as well.
it’s not so much a pain if you implement the logic as a series of sql queries, starting with the uploaded csv table
yes, each series would have to be crafted to the peculiarities of each format received
but it’s almost always easier to do it with sql
if your programmer ends up building a customised ascii loader then once you have your “business rules” agreed with your client in terms of data format and variations of that format (flexibility) he/she should be able to code up an application where you input a file name, maybe specify some loading criteria and then the application does the rest - reads the text data, validates it, builds the sql query to run, runs the query to load the data and outputs any unloadable data rows into a separate output file and they can be looked at later.
Well yeah, ideally you would place everything in a landing table and move it to the final location using SQL. Still quite a bit of work, not to mention determining the database design that is appropriate for every different possible product.
just taking a 
over the fence to see where the noise is coming from.
are you talking to me or r937? I’m assuming it’s not me.
r937