Hey guys I'm new here and fairly inexperienced with website design and programming. Basically I have an idea for a site and no money to hire someone, but lots of time. I'm not sure if a marketplace is the right description either! I'm not going to get into the products at this point since they're not relevant just yet. This idea stemmed from all the headaches I've had trying to purchase refills or replacements for these products. Here's the story; I purchase product X from company A on a regular basis. Then I find out when I go to make my next purchase that they are out of stock. Now I have to decide how much time I want to invest into finding a new place to order from. This involves researching who is selling product X, their prices, availability, customer service, shipping etc. Even after that I still have to deal with the possibility that their quality, product, and/or services are not equivalent to product X from company A. This also stems from my rage at the internet for being the biggest haystack of needles!

So here is what I would like to do; make a site that is geared towards the consumer by consolidating as much information into one place as possible, make the information relative to each other, provide a place where consumers can leave reviews without fear of having their review removed because it reflects poorly on a product or company, and hopefully create a system for making product recomendations similar to how movies can be recommended by Blockbuster or Netflix. That last part would be really cool if it could compare users preferences and make recommendations based on users that are similar in their choices.

I would rant about my personal views and back story to all of this more but it doesn't help get this project done so I'll try to keep it minimal.

Considering I'm gearing this towards consumers, I'm sure that at least a few companies selling these products will not be happy with me when a user chooses to purchase elsewhere because they find that company B sells it for less and has a better customer service rating or any other reason that someone might jump ship.

The other goal of this as I mentioned briefly is the situation where the company is out of stock and you want to quickly find a good replacement even if you plan to continue using company A once they get more stock.

So far I have started but I'm not sure what the best way to do this is or where I should be starting. Since I'm going to assume that at some point there will be some company that isn't happy with their reputation on the site, I want to use a method to obtain the data without having to access their database. In a perfect world each of these companies could allow access to their databases to import the data into my database but even if that were the case, I'm sure there are many different formats that would have to be dealt with. So far I have used OutWit Hub to deal with the html source code and export what I need into Excel. I have also been messing with Excel directly to query the sites but so far none of them are nice for Excel to import.

This is the first big obstacle for me is the data collection. I need something that doesn't require special access or permission and reduces as much repetitive manual labor as possible. It would also be a plus if the set up labor can be minimized but my main concern is developing something that doesn't require tons of work to make changes to. Right now with either option I have used there will be lots of set up labor and then lots of labor to verify the data or find problems. In Excel I can create web queries for each page on each site and then visually look for the data I need and link those results to a different sheet which would be saved. This is slower than OutWit Hub for the moment but I think it might be faster in the long run. OutWit Hup Pro has macros which I think would be even better but I don't have money to purchase it. Either way both would require almost starting from scratch for a particular page or rule if the content I'm looking for changes or is moved and I would like my site to be able to update at least daily with all of these sites to verify the data is current and update if it's not.

The next thing is how to store the data, then get it onto my site.

Then things move on to site design and how to display the data with sorting and search functions.

The initial goal is to get the data and get it on my site where it is usable. Once traffic starts showing up I would be more concerned about user reviews and after that the recommendations thing. The important part in all of this is developing this project with these additions in mind so I'm not having to go back and change a bunch of stuff when I go to add a new feature.

If anyone is wondering, yes I do want to make some money off of this. My thoughts are to set up affiliate deals with as many sites as possible. I don't know exactly how that works either but I will not build this site to promote ANY PARTICULAR product. My goal is not to get in bed with some retailer but rather what I would like to see is my affiliate links on their specific product pages next to a direct link to that products page for that company. The idea is to let people know that I am not pushing any products but if they would like to help out, they can purchase through the affiliate link so I can receive some money. This way they still find what THEY want and THEY choose if they want to contribute to me by using my affiliate link. Personally I could care less if someone purchases from one company instead of another but I do appreciate when people show thanks when they feel I've helped them out. Hopefully some of the affiliate links I would be provided could offer a lower total cost to the user which I'm assuming would really give a user the incentive to contribute but it's just a hopeful thought.

Well thanks for reading and any input you guys can offer. Let me know if I should elaborate more on my goals etc.