Data Harmony update 2020 final

Editor's Notes

• #9: As a vendor to content producers of all kinds, shapes, and sizes here is what I want you to believe…
• #30: Data Harmony introduces the Deprecated status with the 3.14 release. Deprecated terms are used within thesauri to denote outdated or regressed terminology. The terms may include historical geographic states (e.g. “Soviet Union), may include outdated medical terminology (e.g. “Bright’s Disease” formally used to classify “Nephritis” of the kidneys), or outdated expressions of terms in which the taxonomy team wishes to retain but does not want to remove the term and add as a synonym. Indexers and archivists may want to preserve the outdated terminology for historical or legacy purposes, so we added an option to include them, yet limit the way they impact indexing future content. Functionally, the deprecated status acts similarly to the Candidate statuses. Both have dedicated views in Thesaurus Master in order for taxonomists to “resolve” the term within the vocabulary. Both statuses have MAI options to ignore or include within the MAI suggested terms postings, and both statuses can be changed with the click of a button in the term record pane. Finally, we added support for importing and exporting terms with Deprecated statuses . Terms with a Deprecated status, however, will not create identity rules for the term on import. Additionally, if a term is changed to Deprecated, users will not be allowed to edit or add rules which include a USE statement for the deprecated terms. These restrictions are to instruct taxonomy developers and managers to resolve future indexing of concepts which may have previously included the outdated vocabulary concepts yet must direct to a new term for future automated-indexing processes.
• #31: #1 on the upper left shows the radio button to change a term’s status in the thesaurus. To change the status, simply click Deprecated and the term record is saved as Deprecated. #2 displays the Deprecated Terms view, listing each term with the deprecated status. This will allow users to sort through a list of deprecated terms and resolve each of their rules to redirect to another term or to remove a rule outright. #3 shows the validation step in which a user attempts to save or add a rule which includes a deprecated concept. MAI will instruct the user that saving a rule including a deprecated term is invalid.
• #35: We have added multiple enhancements and rolled several smaller API calls into the suggest Terms API. This gives us and users more streamlined options to call on the Data Harmony suite for indexing content. Data Harmony now fully supports JSON calls alongside XML output. Weighting, or boosting, can now be performed at the section level of a document and adds a multiplier to the suggested returns value. For instance, terms discovered in the title would be skewed more heavily than terms in the body if the weightings were changed. Suggest Terms can specify the maximum limit of suggested terms returned without reconfiguring the project from the Admin module. Fullpath can specify the ideal-full-path of the descriptor as it appears in the taxonomy Highlighting options provide inline-tagging of the descriptors within the text
• #36: Here’s an example of how the boosting would appear in an API.
• #37: Special Character Extensions -Special characters such as single quotes, ampersands, greater than and less than symbols, etc. have not been allowed in the MAIstro syntax. They are however very important in chemical nomenclature, as they are in place names such as Washington, D.C. AI now allows import of most special characters Apostrophes, representing possession are now recognized by the MAI parser. MAI will now correctly parse terms, mainly entity names, containing multiple special characters including parentheses, commas, and periods. Wrote a best practices section in the DH User Guide to deal with the variation in the use of Periods and commas. As long as neither the periods nor commas are followed by a whitespace, MAI will correctly parse the text-to-match. Where they are followed by a space please see the section recommending changing the padding characters setting in the Data Harmony Administration Module.