We need the ability to create custom tokenization dictionary for English language and eventually other languages.
As an example, my current client need to keep the following string as one token "930E-2". The query is not making difference between 930E-2 and 930E-4.
Why is it useful?
|Who would benefit from this IDEA?|
How should it work?