2 Simple Concepts I'd Appreciate if Clarified

Databases
1

Hi,

I am looking to start a website that, basically, is a database-driven search engine. So basically, i will have a database of all the data (via MySQL) and i will have a search engine script that searches through this database and outputs relevant info.

Now, 2 issues i’d greatly appreciate if someone can clarify;

  1. The data that i want on the database is all on one website (www.yorku.ca). What im pondering about is how would i go on about extracting all that data- ie. the yorku.ca website has a listing of all their courses, how would i go on about getting only their courses and their course descriptions?

Do i have to do it manually? Or would i have to make some sort of bot? If a bot, are there any sources that can point me in the right direction for developing something like this?

  1. Now, the real issue that is more or less just based on experience (which i don’t have on this specific topic). So say i got all the data on my database. Now i need to make the search engine, what kind of search engine do i need (lets assume the data is not alot its just of about 100 000 entities).

Now i know the search engine is not like a Google-like script, what i need is a search engine that just searches through a database and nothing else. But after researching a bit, it was not only hard to find database-driven search engine scripts but kind of confusing. One i found was:

Arch Intranet Search Engine: http://www.atnf.csiro.au/computing/software/arch/

But i’m not quite sure if this is the best solution out there.

If anyone can give me resources or point me in the right direction, i’d greatly appreciate it. Thanks!

2

  1. ask them for it

  2. this doesn’t require a “script” so much as a fuly responsive web site, and for this you would either program it yourself or hire someone

3
  1. Ask them. They might sell it to you.
  2. Solr.