SitePoint Sponsor

User Tag List

Results 1 to 7 of 7
  1. #1
    SitePoint Zealot boballoo's Avatar
    Join Date
    Dec 2001
    Posts
    113
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Build a complex search function from scratch or plugin?

    I want to rebuild my 15-year old site from the ground up. I provide services to clients through freelancers who register their skills with the site. The search function will be the heart of it all so it has to be robust, but flexible enough to accept changes and additions after we find out what the client's needs are and how they are changing. It must be able to search the user info (profile text, projects info, etc. etc.) and make "decisions" about the relevant categories for each provider profile. If there is no search function such as this available as a site plugin, then I would want to develop it from scratch. I am surprised that I have not been able to find one already built as this is a common function for many sites. I have been told it will cost me an arm and a leg to develop it from scratch. Any hints or ideas on this would be appreciated.
    EditFast
    Any Document --> Any Time!
    Web Site Copy Editing & Proofreading


  2. #2
    SitePoint Wizard silver trophybronze trophy Cups's Avatar
    Join Date
    Oct 2006
    Location
    France, deep rural.
    Posts
    6,869
    Mentioned
    17 Post(s)
    Tagged
    1 Thread(s)
    I'd guess most of these type of services are built upon meta data.

    Coder A has skills PHP, JS, HTML5 etc

    And Client Z needs one or more of those matching skills.

    The trick is to have all Clients and all Coders only ever being to input skills (tags, meta data) which are clean and conform to your naming systems.

    The idea of tags then builds up to include Experience length, number of projects, projects on time, feedback and so on.

    Physical proximity may also be a factor.

    Doing this via pick-lists is easy enough, but that may not be what Clients want to sit and do.

    At some point you may then want to do some natural language processing to extract matching skills** with normal text input, as you may find in a word doc spec for example.

    **Matching skills may then open out to include synonyms. eg CVS may intone the skill GIT, git etc, so you'd either have to find a way of teaching it these, or a means of inferring these things through NLP and AI.

    I only worked on one system similar to this, and it seemed pretty easy to get reasonable results merely based on matching skills and proximity (show nearest qualified Coders on a map etc).

    You might be best to list all the factors you want to be taken into consideration - will that differ much from your existing site?

    Where does your existing site fall down then?

  3. #3
    SitePoint Zealot boballoo's Avatar
    Join Date
    Dec 2001
    Posts
    113
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by Cups View Post
    I'd guess most of these type of services are built upon meta data.
    Exactly. That is how I want to do it but I have to confess, I am not a programmer. In addition, the site is not a "coder" site. I am trying to prepare the site specifications document so that I can hire a programmer to do what I want. However, I still need to know how it is done and the best way to do it before I look for a programmer so your post is very helpful to me. I am happy to hear such information and thank you for your input. You are confirming my thoughts on the matter, and clarifying things.

    I also posted here to see if anyone knew of a plugin or some sort of pre-coded search function that would work as the base of the search function for my new site.

    Quote Originally Posted by Cups View Post
    At some point you may then want to do some natural language processing to extract matching skills** with normal text input, as you may find in a word doc spec for example.
    This is where I would start with the type of site I am operating. I have to import a ton of info into the new database from the old database via a specialized script but the new search function needs to be operational prior to this task so that it can parse the old profiles etc. and make recommendations to admin and to the users as to which tags/categories each profile should be tagged with.

    Quote Originally Posted by Cups View Post
    Matching skills may then open out to include synonyms.
    Yes, this will be an ongoing task to refine the tags and the meanings associated with them so as to provide the most efficient and useful search results for my clients. In addition, as you say, there will be some overlap into using synonyms that occur in the profile text as well as the tags associated with the profile. This is the key. I want the search function to "make suggestions" based on the text as to what categories the user should select or what categories the text is optimized for. And then we will need to define "optimized"

    Quote Originally Posted by Cups View Post
    You might be best to list all the factors you want to be taken into consideration - will that differ much from your existing site?
    Yes, the present site has nothing like that. I hand coded some of the present site (but I am not a coder) in 1998. Since then it has had dozens if not hundreds of additions, and programmers working on it. It is just barely functioning now as it has gotten too complex for anyone to figure out. Basically, it is old and outdated and I want to start clean. The search function now is based on FDSE which is coded in PERL. There is also another search function which confuses things.

    Quote Originally Posted by Cups View Post
    Where does your existing site fall down then?
    It is old and outdated. That is all.
    EditFast
    Any Document --> Any Time!
    Web Site Copy Editing & Proofreading


  4. #4
    SitePoint Wizard silver trophybronze trophy Cups's Avatar
    Join Date
    Oct 2006
    Location
    France, deep rural.
    Posts
    6,869
    Mentioned
    17 Post(s)
    Tagged
    1 Thread(s)
    Sounds as if "Search" in your terms actually means "Filtering" initially - filter by one or more skills and so on.

    I guess you could say you'd be pre-searching the body of documents, but that is more a one off job for data extraction I would have thought, rather than a full blown real-time search. (l'horreur!)

    Sticking with my programmers' website analogy for a moment.

    Document 1234 contains the words PHP, JS and GIT, and you extract those terms into a database so that when quizzed they return a pointer to Document 1234 (or its location on the hard drive).

    So what is the general domain, and does it have its own specialized vocabulary, as for example programming does?

    Are there recurring words or phrases which when extracted always mean roughly the same thing?

  5. #5
    SitePoint Zealot boballoo's Avatar
    Join Date
    Dec 2001
    Posts
    113
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by Cups View Post
    I guess you could say you'd be pre-searching the body of documents, but that is more a one off job for data extraction I would have thought, rather than a full blown real-time search. (l'horreur!)
    This 'real time" search will also be required as the clients will always have the option of searching the profile/text of the providers. In addition, when a new provider registers and submits the text of the profile it will be automatically searched and then provide the user with suggestions as to the correct tags to place on their profile.

    Quote Originally Posted by Cups View Post
    So what is the general domain, and does it have its own specialized vocabulary, as for example programming does?
    The domain is http://editfast.com which provides/will provide editing, writing, and proofreading services in 12 languages. (Not translating)
    Quote Originally Posted by Cups View Post
    Are there recurring words or phrases which when extracted always mean roughly the same thing?
    Yes, there are hundreds but a limited number so it is not impossible to use them all as a base for the search function and for the tags.
    EditFast
    Any Document --> Any Time!
    Web Site Copy Editing & Proofreading


  6. #6
    SitePoint Wizard silver trophybronze trophy Cups's Avatar
    Join Date
    Oct 2006
    Location
    France, deep rural.
    Posts
    6,869
    Mentioned
    17 Post(s)
    Tagged
    1 Thread(s)
    Sorry, my bad, by domain I actually meant your "field of endeavour" vs say, programming.

    I had to chuckle because I am working on a multilingual CMS myself at the moment, so the first thing I looked for was a "choose a language" button ...

    So the idea is that someone uploads a doc and from that you devine what the:

    language is
    subject area is

    and then suggest a list of ppl to work on it?

  7. #7
    SitePoint Zealot boballoo's Avatar
    Join Date
    Dec 2001
    Posts
    113
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by Cups View Post
    Sorry, my bad, by domain I actually meant your "field of endeavour" vs say, programming.

    I had to chuckle because I am working on a multilingual CMS myself at the moment, so the first thing I looked for was a "choose a language" button ...

    So the idea is that someone uploads a doc and from that you devine what the:

    language is
    subject area is

    and then suggest a list of ppl to work on it?
    Not quite. Clients come to the site and search for the editor writer, proofreader they want. They can/will search in any of 12 languages, they negotiate a price and payment is made to EditFast. When complete to the client's satisfaction, EditFast pays the editor. Clients can also submit the document directly to EditFast and let us take care of it.
    EditFast
    Any Document --> Any Time!
    Web Site Copy Editing & Proofreading



Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •