Is there no standards based search engine?

We have several search engines that drive the traffic on Internet, and many private networks (intranets). However, it is surprising to know that there are no standards prescribed for a search engine by any international standards organization such as ISO. Is it not time for having Open Standards for any Internet search? Are there any such standards already?

Elaborating on “Open Standard”, it should prescribe how the content on the Internet be indexed, how a page is weighed, what constitutes a spam or a scraper, what constitutes a duplicate content, etc.

Major search engines like Google, Bing, and Yahoo should take initiatives in Standardization of Search (Open Standards) of Internet.

Well, they are private companies, and exist to make a profit, so I don’t think we can dictate that. It would be nice to have an unbiased search service, but I doubt we’ll see it any time soon.

And then the spammers find a way to get to the first page, and before the search engines can change their algorithms they’ll have to wait for the SESO (Search Engine Standardisation Organisation) to agree on what to do :smiley:

SESO? nice abbreviation. There will be issues with Open Standards like security, exploitation of algos, etc. However, if you look at several software packages that are Open Source, the authors are able to prevent hacking or misuse effectively. For example, it is widely reported that Windows gets hacked more frequently than Linux. In the same fashion, it should be possible to prevent misuse of Open Standards. More over, companies/institutes that have domain knowledge will be happy to be part of the SESO. Usually, standards are set by these companies that have domain expertise and subsequently regularized. One of the important benefits of SESO is transparency and the ability to monitor the search engines (by independent organizations) for compliance with standards (if they claim to conform to the standards).

One need to look beyond spamming. What one may be losing is material that never seen light on the Internet. Just like unsung war heros. Unless there is a standards based search, it is difficult to index the Web properly and provide relevant, impartial search results.

It is surprising that none of the leading educational institutes initiated Open Standards for Internet Search! (or probably some work is already being done in this direction?)

First, open source does not mean standardization. All it means is that the source code is available to anyone. Someone could develop rotten code and release the code, and it would still be open source–even though it might not follow any standards or best practices.

Second, you’re also talking about an individual product controlled by one person or organization. Search engines are each owned, developed, and operated by a number of different organizations, so it’s not likely to see the same flexibility. Each organization considers their own search engine algorithms to be trade secrets.

Force Flow: Open Source software example is given just to highlight the fact that knowing the source code not necessarily makes the software vulnerable. Similar logic may be applied for SE logic. Like knowing what constitutes spam doesn’t necessarily makes an individual to beat logic.

As far as SESO is concerned, its more like a protocol compliance. Suitable protocols should be developed for Internet Search that provide transparency and compliance with standards.

Anything online can be hacked, period.

I doubt that. The way companies set themselves apart is offering different goods.

Then SESO would need a legal arm.

Impartial results would mean worthless. If there were no way to rank pages or sites, either there would be only one search engine or no matter what term(s) you enter you will always get the same hits. So, unless you were the first or second site onboard, don’t bother making an engine.

See above

Ryan B: It is not necessary that SESO is a legal arm, but it will be an independent organization that monitors compliance with search standards. What I mean by “impartial” and “transparent” is that the search engines do what they claim to be doing. There has to be a transparent way to measure the SE compliance to stated standards.

“I doubt that. The way companies set themselves apart is offering different goods.”

The companies with domain expertise will always like to join any such standardization effort, if the initiative is taken at appropriate level (such as a leading edu institute like Harward or by the Government). Most of the international standards that have been developed in the world today had the membership of leading corporations, edu institutes, or the Government. This is because, the companies that take lead role, will have the opportunity to influence their school of thought in the specifications formulation.

“Anything online can be hacked, period.”

The fear of hacking should not deter the intelligent community from formulating protocols and standards that are tough to crack. For example, it was said that any encryption could be cracked. But, how many are cracking an encrypted communication. Are we going to avoid encryption and moving towards manual communication?

It is surprising to see some members suggesting that it is NOT possible to evolve any kind of standards for Search Engines, considering that they are poised to drive the world economy and the knowledge base in the decades to come.

Nor does it mean it is not vulnerable. Your example does not match up. Furthermore, “Open” whatever is not a solution to every single problem. Do you see Coca-Cola and Pepsi forming a “Soda Standardization Organisation” ? I mean both are doing the same thing are they not? Search Engines DO NOT NEED to be standardized, nor will Google and Microsoft (and others) ever take part in such a thing. They have a hard enough time with HTML.

The closest you are going to get to an open standard with these guys: http://schema.org/ http://sitemaps.org/

One example where Open will work is Privacy Policy. One of the important issues is as to how a search engine collects, stores, uses, and disposes the privacy information of individual visitors. Sure that each of these SEs have Privacy Policy, and they claim to be following the policy, but it lacks verification. The standardization procedures may layout the policy clearly as to how a SE collects, stores, uses, and disposes individual user information, and methods to verify the compliance by independent organizations.

Privacy Policy is taken only as an example. So is every other aspect of SE operation such as a, Storage of information, b, Processing and Use of information, c, Display of search results, etc.

Please note that an SE may or may not want to be in compliance with a given standard according to its choice and let the visitors know about its compliance.