Expertise finding
Encyclopedia
The Oxford English Dictionary
Oxford English Dictionary
The Oxford English Dictionary , published by the Oxford University Press, is the self-styled premier dictionary of the English language. Two fully bound print editions of the OED have been published under its current name, in 1928 and 1989. The first edition was published in twelve volumes , and...

 defines "expertise" as follows:


a. Expert opinion
Opinion
In general, an opinion is a subjective belief, and is the result of emotion or interpretation of facts. An opinion may be supported by an argument, although people may draw opposing opinions from the same set of facts. Opinions rarely change without new arguments being presented...

 or knowledge
Knowledge
Knowledge is a familiarity with someone or something unknown, which can include information, facts, descriptions, or skills acquired through experience or education. It can refer to the theoretical or practical understanding of a subject...

, often obtained through the action of submitting a matter to, and its consideration by, experts; an expert's appraisal, valuation, or report. b. The quality or state of being expert; skill or expertness in a particular branch of study or sport. Oxford English Dictionary, second edition, 1989.


One could further say that expertise is the quality exhibited by people whom we believe demonstrate an above-average ability to perform a non-trivial task (see Expert for tongue-in-cheek examples). This can be restated by saying that when one is not an expert in a given field, the perception of expertise is often granted to a person based on what other people say is demonstrated by the presumed expert. That people often accept such claims prima facie is somewhat understandable, if ill-advised, and is just one of the many problems associated with assessing and quantifying human expertise. It can be argued that this behavior is partly due to a paucity of tools, metrics and software that focuses on characterizing whatever expertise they demonstrate in areas other than sports, the one domain where excellent tools are available.

This article addresses the issues and tools regarding the problem of finding and assessing individual expertise, with particular focus on scientific expertise.

Locating and assessing expertise, and why it matters

It can be argued that human expertise is the most valuable resource in the universe, more valuable than capital, means of production or intellectual property. Why? Contrary to expertise, all other aspects of capitalism are now relatively generic: access to capital is global, as is access to means of production for many areas of manufacturing. Intellectual property
Intellectual property
Intellectual property is a term referring to a number of distinct types of creations of the mind for which a set of exclusive rights are recognized—and the corresponding fields of law...

 can be similarly licensed. Furthermore, expertise finding is also a key aspect of institutional memory
Institutional memory
Institutional memory is a collective set of facts, concepts, experiences and know-how held by a group of people. As it transcends the individual, it requires the ongoing transmission of these memories between members of this group...

, as without its experts an institution is effectively decapitated. However, finding and “licensing” expertise, the key to the effective use of these resources, remain much harder, starting with the very first step: finding expertise that you can trust.

Until very recently, finding expertise required a mix of individual, social and collaborative practices, a haphazard process at best. Mostly, it involved contacting individuals one trusts and asking them for referrals, while hoping that one’s judgment about those individuals is justified and that their answers are thoughtful.

In the last fifteen years, a class of knowledge management
Knowledge management
Knowledge management comprises a range of strategies and practices used in an organization to identify, create, represent, distribute, and enable adoption of insights and experiences...

 software has emerged to facilitate and improve the quality of expertise finding, termed “expertise locating systems”. These software range from social networking systems
Social network service
A social networking service is an online service, platform, or site that focuses on building and reflecting of social networks or social relations among people, who, for example, share interests and/or activities. A social network service consists of a representation of each user , his/her social...

 to knowledge base
Knowledge base
A knowledge base is a special kind of database for knowledge management. A Knowledge Base provides a means for information to be collected, organised, shared, searched and utilised.-Types:...

s. Some software, like those in the social networking realm, rely on users to connect each other, thus using social filtering to act as “recommender systems”.

At the other end of the spectrum are specialized knowledge base
Knowledge base
A knowledge base is a special kind of database for knowledge management. A Knowledge Base provides a means for information to be collected, organised, shared, searched and utilised.-Types:...

s that rely on experts to populate a specialized type of database
Database
A database is an organized collection of data for one or more purposes, usually in digital form. The data are typically organized to model relevant aspects of reality , in a way that supports processes requiring this information...

 with their self-determined areas of expertise and contributions, and do not rely on user recommendations. Hybrids that feature expert-populated content in conjunction with user recommendations also exist, and are arguably more valuable for doing so (e.g., LinkedIn ).

Still other expertise knowledge bases rely strictly on external manifestations of expertise, herein termed “gated objects”, e.g., citation impact
Citation impact
Citation is the process of acknowledging or citing the author, year, title, and locus of publication of a source used in a published work. Such citations can be counted as measures of the usage and impact of the cited work. This is called citation analysis or bibliometrics...

s for scientific papers or data mining
Data mining
Data mining , a relatively young and interdisciplinary field of computer science is the process of discovering new patterns from large data sets involving methods at the intersection of artificial intelligence, machine learning, statistics and database systems...

 approaches wherein many of the work products of an expert are collated. Such systems are more likely to be free of user-introduced biases (e.g., ResearchScorecard ), though the use of computational methods can introduce other biases.

Examples of the systems outlined above are listed in Table 1.

Table 1: A classification of expertise location systems
Type Application domain Data source Examples
Social networking Professional networking User-generated
  • LinkedIn
    LinkedIn
    LinkedIn is a business-related social networking site. Founded in December 2002 and launched in May 2003, it is mainly used for professional networking. , LinkedIn reports more than 120 million registered users in more than 200 countries and territories. The site is available in English, French,...

     http://linkedin.com/
Scientific literature
Scientific literature
Scientific literature comprises scientific publications that report original empirical and theoretical work in the natural and social sciences, and within a scientific field is often abbreviated as the literature. Academic publishing is the process of placing the results of one's research into the...

Identifying publications with strongest research impact Third-party generated
  • Science Citation Index
    Science Citation Index
    The Science Citation Index is a citation index originally produced by the Institute for Scientific Information and created by Eugene Garfield in 1960, which is now owned by Thomson Reuters. The larger version covers more than 6,500 notable and significant journals, across 150 disciplines, from ...

     (Thomson Reuters)http://www.thomsonreuters.com/products_services/science/science_products/a-z/science_citation_index
  • Knowledge base Private expertise database User-generated
  • MITRE Expert Finder (MITRE Corporation)
  • MIT ExpertFinder (ref. 3)
  • MindServer Expertise (Recommind, Inc.)
  • Tacit Software
    Tacit Software
    Tacit Software was a company that provides expertise and knowledge search software for enterprises and consumers. It was acquired by Oracle Corp in November 2008, and its assets were rolled into the Oracle Beehive collaboration platform.,...

     (Oracle Corporation)
  • Knowledge base Publicly-accessible expertise database User-generated
  • Community of Science
    Community of Science
    Community of Science is a collection of online databases, providing research information to both the public and subscribers, and services for the research community...

     Expertise http://expertise.cos.com
  • ResearcherID
    ResearcherID
    ResearcherID is an identifying system for scientific authors. The system was introduced in January 2008 by Thomson Reuters.This unique identifier aims at solving the problem of author identification. In scientific literature it is common to cite name, surname, and initials of the authors of an...

     (Thomson Reuters)http://www.thomsonreuters.com/products_services/scientific/ResearcherID
  • SciLink.com (SciLink Inc.)
  • Knowledge base Private expertise database Third party-generated
  • MITRE Expert Finder (MITRE Corporation)
  • MIT ExpertFinder (ref. 3)
  • MindServer Expertise (Recommind, Inc.)
  • Tacit Software
  • Knowledge base Publicly-accessible expertise database Third party-generated
  • ResearchScorecard (ResearchScorecard Inc.)
  • authoratory.com
  • BiomedExperts (Collexis Holdings Inc.)
  • KnowledgeMesh (Hershey Center for Applied Research)
  • Community Academic Profiles (Stanford School of Medicine)
  • ResearchCrossroads.org (Innolyst, Inc.)
  • Blog search engine
    Search engine
    A search engine is an information retrieval system designed to help find information stored on a computer system. The search results are usually presented in a list and are commonly called hits. Search engines help to minimize the time required to find information and the amount of information...

    s
    Third party-generated
  • Technorati
    Technorati
    Technorati is an Internet search engine for searching blogs. By June 2008, Technorati was indexing 112.8 million blogs and over 250 million pieces of tagged social media...

     http://technorati.com/

  • Technical problems associated with expertise finding

    A number of interesting problems follow from the use of expertise finding systems:
    • The matching of questions from non-expert to the database of existing expertise is inherently difficult, especially when the database does not store the requisite expertise. This problem grows even more acute with increasing ignorance on the part of the non-expert due to typical search problems involving use of keywords to search unstructured data that are not semantically normalized, as well as variability in how well an expert has set up their descriptive content pages. Improved question matching is one reason why third-party semantically normalized systems such as ResearchScorecard and BiomedExperts should be able to provide better answers to queries from non-expert users.
    • Avoiding expert-fatigue due to too many questions/requests from users of the system (ref. 1).
    • Finding ways to avoid “gaming” of the system to reap unjustified expertise credibility
      Credibility
      Credibility refers to the objective and subjective components of the believability of a source or message.Traditionally, modern, credibility has two key components: trustworthiness and expertise, which both have objective and subjective components. Trustworthiness is based more on subjective...

      .

    Beyond expertise finding: Expertise ranking

    Means of classifying and ranking expertise (and therefore experts) become essential if the number of experts returned by a query is greater than a handful. This raises the following social problems associated with such systems:
    • How can expertise be assessed objectively? Is that even possible?
    • What are the consequences of relying on unstructured social assessments of expertise, such as user recommendations?
    • How does one distinguish authoritativeness
      Authority
      The word Authority is derived mainly from the Latin word auctoritas, meaning invention, advice, opinion, influence, or command. In English, the word 'authority' can be used to mean power given by the state or by academic knowledge of an area .-Authority in Philosophy:In...

       as a proxy metric of expertise from simple popularity, which is often a function of one's ability to express oneself coupled with a good social sense?
    • What are the potential consequences of the social or professional stigma associated with the use of an authority ranking, such as used in Technorati and ResearchScorecard)?

    Sources of data for assessing expertise

    Many types of data sources have been used to infer expertise. They can be broadly categorized based on whether they measure "raw" contributions provided by the expert, or whether some sort of filter is applied to these contributions.

    Unfiltered data sources that have been used to assess expertise, in no particular ranking order:
    • user recommendations
    • help desk tickets: what the problem was and who fixed it
    • e-mail traffic between users
    • documents, whether private or on the web, particularly publications
    • user-maintained web pages
    • reports (technical, marketing, etc.)


    Filtered data sources, that is, contributions that require approval by third-parties (grant committees, referees, patent office, etc.) are particularly valuable for measuring expertise in a way that minimizes biases that follow from popularity or other social factors:
    • patent
      Patent
      A patent is a form of intellectual property. It consists of a set of exclusive rights granted by a sovereign state to an inventor or their assignee for a limited period of time in exchange for the public disclosure of an invention....

      s, particularly if issued
    • scientific publications
    • issued grants (failed grant proposals are rarely know beyond the authors)
    • clinical trial
      Clinical trial
      Clinical trials are a set of procedures in medical research and drug development that are conducted to allow safety and efficacy data to be collected for health interventions...

      s
    • product launches
    • pharmaceutical drugs

    Approaches for creating expertise content

    • Manual, either by experts themselves (e.g., LinkedIn) or by a curator
    • Automated, e.g., using software agent
      Software agent
      In computer science, a software agent is a piece of software that acts for a user or other program in a relationship of agency, which derives from the Latin agere : an agreement to act on one's behalf...

      s (e.g., MIT's ExpertFinder and the ExpertFinder initiative) or a combination of agents and human curation (e.g., ResearchScorecard)

    Interesting expertise systems over the years

    In no particular order...
    • Autonomy's IDOL
    • AskMe
    • Tacit Knowledge Systems' ActiveNet
    • Triviumsoft's SEE-K
    • MIT’s ExpertFinder (ref 3)
    • MITRE’s (ref 1) Expert Finder
    • MITRE’s XpertNet
    • Dataware II Knowledge Directory
    • Thomson’s tool
    • Hewlett-Packard’s CONNEX
    • Microsoft’s SPUD project
    The source of this article is wikipedia, the free encyclopedia.  The text of this article is licensed under the GFDL.
     
    x
    OK