IBM Watson™ Discovery Service Ideas

We've moved...

You'll be redirected shortly, we've moved to our new idea portal: https://ibm-watson.ideas.aha.io

Discovery comprehensive handling of domain-specific dictionaries

Discovery is being positioned as replacement for R&R yet search (and not even WKS-based enrichment) cannot handle a comprehensive handling of domain-specific language, such as technical support. In R&R this was achieved by direct access to Solr configuration.

With Discovery, at this time, the only way to get close to text normalization based on domain-specific language is to use Conversation entities to get the relevant domain language for a doc and store it as a field in discovery. Then do the same to get domain language for each query and use the entities to expand your query ..while this gives you some matches, it completely prevents us from exploiting keyword proximity measures to improve ranking of most relevant records.

there several levels in which this can be achieved:

0. integrate dictionaries from WKS with Elastic Search

1. give full access to elastic search - which will give full benefit the powerful search engine on top of discovery enrichment - such as scoring function

2. ensure that synonyms in domain-specific language are applied to the enrichment output - as of now, they are completely disconnected

  • daniela rosu
  • Jun 12 2017
  • Future Consideration
Why is it useful?
Who would benefit from this IDEA? as a solution architect I need an effective and efficient approach for integration of domain-specific language
How should it work?
Idea Priority High
Priority Justification
Customer Name
Submitting Organization
Submitter Tags
  • Attach files
  • Holt Adams commented
    June 16, 2017 17:37

    The Watson Solution Delivery team also has many customer solutions that can be benefited by the support of Dictionaires and the normalization of mentions.

  • daniela rosu commented
    June 19, 2017 04:28

    do you have a timeline already for a solution? if not near future, we have to implement an alternative solution.

    Thank you

    daniela

  • Holt Adams commented
    June 20, 2017 19:38

    Customer for which the Watson Solution Delivery team is building solutions requires Domain Specific Dictionaries to better enable normalization.  A few examples are

    1. Mapping different strings to the same entity (e.g.Firm ->firm, Firm, firms, Firms),
    2. to support  phrases that are "terms of art"  that would be invalidated by general english normalization (e.g. Exchange Traded Funds if normalized could result in Exchange Trade Fund) and
    3. Domain acronyms conventions (mixed case)  e.g. messenger RNA  -> mRNA, transfer RNA -> tRNA would need to be normalized as the same concept with lowercase "m" and "t" maintained
  • TIM O'FLYNN commented
    October 02, 2017 14:06

    Interest in this topic area in particular level 2 mentioned above:ensure that synonyms in domain-specific language are applied to the enrichment output - as of now, they are completely disconnected