No announcement yet.

Trivial Word List and User Dictionary

  • Filter
  • Time
  • Show
Clear All
new posts

  • Trivial Word List and User Dictionary

    From: Ronni Rubenstein [mailto:[email protected]]
    Sent: Friday, April 07, 2006 11:45 AM
    To: Dean Fishman, [email protected]
    Subject: RE: Question regarding trivial word list and user dictionary

    Hi Dean,

    Auto-coding first looks for an exact match to the term in the dictionary. Since an exact match wasn't found, the compress algorithm removes special characters, sorts the words, removes spaces and creates one long string. Your term, in this case, would become the string, "hypertensionofworsening." This string is compared to the string that was created for every term in the dictionary during the conversion program. If there is still no match, then, depending on whether you selected to use the trivial word list, thesaurus, and perpetuate group, it searches further.

    With the trivial word list, the trivial word, in this case "of," would be removed, unless it appears in any dictionary term. If it’s not in any dictionary term, the compression algorithm runs and your term would be "hypertensionworsening." You said that the dictionary only has "worsened hypertension," or as a compressed string, it would be "hypertensionworsened" so of course, there is no match.

    "Worsening of Hypertension" is not in your thesaurus and you never coded it before for the perpetuate groups to be used, so no match there either.

    Call me if I need to clarify.
    Best regards,

    From: Dean Fishman
    Sent: Thursday, April 06, 2006 11:56 AM
    To: [email protected]
    Subject: Question regarding trivial word list and user dictionary

    I have a term in my User dictionary “Worsening Hypertension”. Another event was reported “Worsening of Hypertension”. The word “of” is in our trivial word list. When the auto-encoder runs and it searches the User dictionary first it does not remove any of the trivial words? Apparently it doesn’t but just wanted to double check.