A method for creating hierarchical classifiers of software components in a learning system, each software component comprising an identifier and a rule for processing an input message, the method comprising the steps of: receiving an input message by a first software component; parsing the input message to identify an input value; seeking a second software component having an identifier matching the identified input value; in the event that the seeking step fails to locate a match, creating a second software component and assigning an identifier to the created second software component, the identifier matching the identified input value.
|
1. A method for creating hierarchical classifiers of software components in a learning system for processing an input message, the method comprising the steps of:
receiving text in a first window of a hierarchical fact extraction tool;
building at least one rule from the received text comprising identifying a text structure in the received text, wherein the text structure comprises a word having multiple meanings in current usage of the word; and selecting a specific meaning from the multiple means based on a context of the text structure;
wherein the multiple meaning of the word comprise a calendar data and a name of a person, wherein the text structure comprises a grammatical element followed by the word, wherein the grammatical element is selected from the group consisting of a demonstrative, a verb, an adjective, a definite article, a noun, and a noun preposition, and wherein said selecting the specific meaning comprises selecting the calendar date;
transforming the at least one rule into a xml training message, said xml training message comprising output tags that define an output message and input tags that comprise at least one input value for generating the output message;
receiving the input message by a first software component, said first software component consisting of a cellular decision engine of a plurality of decision engines of a hierarchy, said input message consisting of the xml training message;
identifying at least one subcellular decision engine of the plurality of decision engines, each subcellular decision engine hierarchically related to the cellular decision engine as a child of the cellular decision engine;
parsing the input message to identify a first input value of the at least one input value;
ascertaining that no subcellular decision engine of the at least one subcellular decision engine has an identifier that is identical to the identified first input value;
responsive to said ascertaining, creating an additional subcellular decision engine in the hierarchy, said additional subcellular decision engine hierarchically related to the cellular decision engine as an additional child of the cellular decision engine;
assigning an identifier to the created additional subcellular decision engine, the identifier being identical to the identified first input value;
generating a conforming output message in conformity with the defined output message in the output tags and with inclusion of the first input value; and
transmitting the conforming output message to a requesting entity.
2. The method of
3. The method of
4. The method of
5. The method of
6. The method of
7. The method of
8. The method of
9. The method of
10. The method of
11. The method of
12. A data processing system comprising a processor and a computer readable memory unit coupled to the processor, said memory unit containing software code that when executed by the processor performs the method of
13. A computer program product, comprising a computer usable storage medium having computer software code that when executed on a computer causes the computer to perform the method of
14. The method of
|
|||||||||||||||||||||||||||
The invention relates to the field of artificial intelligence and in particular to a method and system for creating hierarchical classifiers of software components.
The functionality of any information system may be described as a series of information transformations. A simple database query can transform the contents of the database and the query information into a specific result set. Financial applications can transform the contents of a database into meaningful financial metrics or a number of graphical illustrations. The ability of being able to perform the information transformation is often carried out by artificial intelligence systems, which include, expert systems, neural networks and case based reasoning applications.
Another type of artificial intelligence system which is gaining in popularity is natural language processing. Natural language processing is concerned with designing and building software that will analyse, understand and generate languages that humans naturally use. This concept is by no means easy, as understanding what a language means, for example, what concepts a word or phrase stands for and knowing how to link those concepts together in a meaningful way is an incredibly difficult task.
A major challenge in any natural language application is to understand and appreciate the context of a word, sentence or phrase. This involves analyzing the sentence to try and determine the context of the sentence; particularly, as the context of a sentence is influenced by many factors including, preceding content of a sentence, the location of the speaker of the sentence, and the particular environment of interest. In order to understand the context of the two sentences below, the natural language application would have to analyse both sentences and determine the context of each sentence. For example looking at the two sentences below:
Therefore, when using natural language applications in a particular environment, for example, in a legal environment, to enable the natural language application to work successfully i.e. to produce better results, the natural language application needs to be customized to the environment that the application is working in. For example in a legal environment the difference between the use of the word ‘shall’ and ‘will’ is of great significance.
Another example of this is when, using a natural language application in a banking environment and the employees of the bank use particular terminology to describe various processes used throughout their day to day activities. The natural language application would need to be customized to comprise an understanding of the banking terminology, for example, the definition, and contextual meaning of terms, such as, debit, credit and overdraft etc.
In a business environment as more and more processes change or more and more different scenarios develop for a given situation; there is a need to customise the natural language application by adding new rules or modifying an existing training set. This becomes incredibly time consuming and can take days to write just one rule. Using conventional technologies, this creates a burden on the organizations that are using these types of applications and has hindered the development and adoption of natural language applications into common day practice.
Existing techniques for customizing natural language applications include the task of editing domain specific dictionaries and taxonomies to include the terminology expected in the final application. More advanced techniques include the development and implementation of specific language processing rules using systems, such as, VISUALTEXT®, from Text Analysis International, Inc,. VISUALTEXT provides the functionality for information extraction, natural language processing and text analysis for a variety of applications. For example, business events may be extracted from web sites and a searchable database can be built for searching for the latest business information.
Even with tools, such as, VISUALTEXT, the development of new language processing rules requires an in-depth expertise in the field of natural language.
Most types of artificial intelligence systems use a variety of mathematical techniques to automatically or semi-automatically discover patterns and develop rules. Whilst the specific form of these rules differs depending on the techniques used, the general principle of mapping some form of input pattern to some form of output pattern, is common across most techniques. For example, rule induction systems employ rule induction algorithms i.e. statistical modelling techniques, to perform recursive functions through a tree structure and construct sub structures and rules as a result. Rule induction system work better when the feature space that it is operating in is fixed and pre-defined. For example, when using a technique called N-Grams. N-grams are a separate technique from rule induction systems; however they both have the same limitation of working best with well defined and constrained feature spaces. Such techniques are described in the book: Programs for Machine Learning, by John Ross Quinlan, Morgan Kauffman, 1993.
Neural networks are modeled on a cellular structure; however the methods of specifying the cell structure are limited. For example, in some types of neuro-fuzzy systems it is possible to define a structure based on knowledge of the order of the underlying functions. Similarly, with lattice based neurons (e.g. CMAC) there are formal strategies or algorithms for knot placement. However, in many areas further work is still needed in understanding how to structure large systems with many thousands of cells. Many researchers have focused on the scale of large systems in the belief that self organizing rules will enable the emergence of structures. This has not proved to be the case. Garis's work on the emulation of billions of neurons focuses almost entirely on the processing load issues and not on the definition of the cell structure. This is still a problem that needs to be solved.
Finally, most artificial intelligence technologies are most successfully applied in constrained applications where a static feature space is formally defined and sufficient training data exists to fully populate that static feature space. Artificial intelligence technologies are less successful in areas where the feature space is dynamic and unconstrained; for example when new features are frequently introduced into the natural language system, or where a sub structure of a feature changes.
Therefore there is a need within the art for the above mentioned problems to be alleviated.
Viewed from a first aspect, the present invention provides a method for creating hierarchical classifiers of software components in a learning system, a software component comprising an identifier and a rule for processing an input message, the method comprising the steps of: receiving an input message by a first software component; parsing the input message to identify an input value; seeking a second software component whose identifier matches the identified input value; in the event that the seeking step fails, creating a second software component and assigning an identifier to the created second software component, the identifier matching the identified input value.
In an embodiment of the method for analyzing and processing unstructured text. The software components are capable of operating on an unconstrained feature space. For example, in a financial application a series of messages may be generated that describe “company profitability”. These messages could have different sub structures and do not need to be normalized onto a structured feature space (as is the case with neural and rule induction systems). Furthermore once an initial model has been constructed, new definitions of “company profitability” could be generated without the need to re-visit and amend the previous definitions. The cellular decision engines will also manage cases where counter examples are presented and adjust the model accordingly. The system used in the text analysis application operates entirely on a symbolic as opposed to a statistical basis. As generalization is not based on statistical normalization, the system is capable of taking into account very small, changes in a message that influence the overall meaning of the text (e.g. the existence of an apostrophe, the different spelling of a name, etc). Therefore the system is able to automatically evolve cell structures according to a set of symbolic processing rules as claimed above.
Preferably, the present invention provides a method wherein the created second software component comprises a rule for matching the identified input value of a subsequent received input message and determining whether an identifier of a third software component matches the identified input value.
Preferably, the present invention provides a method wherein the second software component further comprises a rule for matching the identified input value of a subsequent received input message, determining whether an identifier of a third software component matches the identified input value and placing an inhibit value of the identified input value.
Preferably, the present invention provides a method wherein the inhibit value stops a rule for triggering an action.
Preferably, the present invention provides a method wherein the input message is an XML message.
Preferably, the present invention provides a method wherein the received message comprises a plurality of input values forming a structure of a context.
Preferably, the present invention provides a method wherein the context is a sentence, a phrase, word or a numerical value.
Preferably, the present invention provides a method wherein the rule is a symbolic processing rule.
Viewed from a second aspect, the present invention provides a system for creating hierarchical classifiers of software components in a learning system, a software component comprising an identifier and a rule for processing an input message, the system comprising: a receiver for receiving an input message by a first software component; a parser for parsing the input message to identify an input value; a determination component for determining a second software component whose identifier matches the identified input value; a creator component for creating a second software component and assigning an identifier to the created second software component, the identifier matching the identified input value, in dependence of a negative determination by the determination component.
Preferably, the present invention provides a system wherein the created second software component comprises means for a rule for matching the identified input value of a subsequent received input message and means for determining whether an identifier of a third software component matches the identified input value.
Preferably, the present invention provides a system wherein the second software component comprises means for a rule for matching the identified input value of a subsequent received input message, means for determining whether an identifier of a third software component matches the identified input value and means for placing an inhibit value of the identified input value.
Preferably, the present invention provides a system wherein the inhibit value stops a rule from triggering an action.
Preferably, the present invention provides a system wherein the input message is an XML message.
Preferably, the present invention provides a system wherein the received message comprises a plurality of input values forming a structure of a context.
Preferably, the present invention provides a system wherein the context is a sentence, a phrase, word or a numerical value.
Preferably, the present invention provides a system wherein the rule is a symbolic processing rule.
Viewed from a third aspect, the present invention provides a computer program product loadable into the internal memory of a digital computer, comprising software code portions for performing, when said product is run on a computer, to carry out the invention as described.
Embodiments of the invention are described below in detail, by way of example only, with reference to the accompanying drawings in which:
The tool 200 comprises a number of windows 210, 205, each window performing a specific function. In
In window 210, an example is shown of a number of sentences under various headings. The headings of the text in this example are displayed for clarity and to help aid the readers understanding of the invention. The first heading 215, shows the text ‘it happened on the first day of June’. A user of the tool 200 is able to select parts of the sentence and classify the words as a verb, noun, proposition, a name, or a date etc, such that the entire sentence is classified into a known structure.
In this example, the word June is of significance and thus, the word June is classified as a date. Such that, when using the sentence ‘it happened on the first day of June’, which had previously been classified by noun, verb etc, the tool may build a rule which states every time you identify a structure with a demonstrative (it), verb (happened), adjective (on), definite article (the), noun (first), noun preposition (day), followed by a word, the date, in this example June, is a date.
Other type of rules can be built, for example, phrase rules 220, inhibit rules 225 and generalisation rules 230. In particular inhibit rules are built in order to restrict rules triggering an action under certain conditions. For example, in the sentence ‘it happened on the first day of June’. June has been classified as a date. But taking the sentence, on the 12th June, June agreed to participate’, June in this sentence is a name of a person and not a month. In this example, when a rule is faced with this structure, an inhibit value is placed on the rule to say in this situation don't trigger an action.
Generalization rules allow a rule to state that every time it is presented with a sentence in a particular structure, the structure should be classed as a reference number or the like. 1
The selection of the text allows every word within a piece of text to be correctly classified and correctly tagged with XML. Thus, when using the tool 200 for free text analysis for a number of documents stored in a data store, the text within the document may be classified in a meaningful manner and thus, by using XML, a hierarchical representation is created.
Window 205 comprises the core management functions of the tool 200. A user is able to select a number of functions including, the creation of rules, the application of rules, rule manipulation, rule management and configuration of the tool 200 etc. Window 205 further comprises a display window 235 for viewing the types of rules that may be applied to the text and the results of the application of the rules onto the text.
Once the chosen text has been classified, the tool 200 constructs an XML training message. The user many amend the XML message to either correct any errors or to specify the conversion of factual information into a standard representation schema.
The output of the tool 200 is a training message as follows:
<root>
<Scenario>
<output>
<traffic lights value = ‘stop’>
</output>
<input>
<red Value =’stop’/>
<green value = ‘go’/>
<amber value = ‘stop’/>
<input>
</scenario>
</root>
The training message comprises XML tags which define a scenario (in this example, the concept of traffic lights), output tags, which define an output message and input tags which define what inputs are required in order to get the required output. The training message is transmitted to a cellular decision engine for processing, which is explained below with reference to
XML is preferably used for the definition of messages for building training message sets, i.e. training the system to learn. XML provides syntax for developing specialized mark-up languages, to add identifiers, or tags to certain characters, words or phrases in a document so as to enable recognition and action during processing. Marking up of a document or data results in the formation of a hierarchical container that is platform, language and vendor independent and separates the content from the environment. Unlike HTML which uses tags to define the presentation of data, XML allows the user to define the identify of the data. Defining a mark up language involves defining the data elements, attributes and rules for their use. In XML, this is currently stored in a Document Type Definition (DTD). A well-formed and valid XML document conforms to the referenced DTD. XML provides a simple means of representing tree-structured data, because the DTD is a hierarchical.
Although XML is the preferred mark-up language of the present invention, it will be appreciated by a person skilled in the art that other forms of hierarchical data definition languages can be used.
Once the XML message has been defined by the tool, the XML message is passed to a cellular decision engine to process an output message. The present invention uses a cellular decision engine technique which provides for the automatic evolution of cell structures according to a set of symbolic processing rules.
A cellular decision engines is a software object that receives an input message, processes the message according to a number of internally held rules and in response to the processing step generates an output message. Whilst processing an input message, a cellular decision engine may construct a sub network of cellular decision engines and cascade elements of the input message to the cellular decision engines within the sub networks. The cellular decision engines within the sub networks process the cascaded message in an identical manner to their parent cells (i.e. the cells at the level above the cells within the sub network); this many include the potential construction of another sub network and the generation of output messages. In this manner, a network of cellular decision engines process received messages in a highly recursive manner which may emerge into a complex cellular topology.
With reference to
The processing of the input message 405 and the creation of the sub cellular decision engines 420, 425, 430 is determined by a set or pre-defined rules; each rule being unique to each class of cellular decision engine. For example, one set of rules may pertain to the class of vehicle and another set of rules may pertain to the sub class of cars etc.
Each cellular decision engine 415 on receiving an input message 405 processes the input message 405 according to the predefined rules. The pre-defined rules determine how the input message 405 is processed and whether any sub cellular decision engines 420, 425, 430 are created. The cellular decision engines process the input messages by performing the following steps:
For example, given the following simple XML input message 405 and with reference to
Input Message: Traffic Lights 1
<root>
<Scenario>
<output>
<traffic lights value = ‘amber’>
</output>
<input>
<red =’stop’/>
<amber value = ‘stop’/>
<green value = ‘go’/>
<input>
</scenario>
</root>
The cellular decision engine 415 on receiving the input message 405 cascades the input message 405 through the hierarchical structure to a second cellular decision engine 420. The second cellular decision engine 420 compares the value of the root tag (<Messagebody>) with a list of sub cellular decision engines within the hierarchical structure. On identification of a sub cellular decision engine 425, the input message is cascaded to the identified sub cellular decision engine 425. The predefined rules within the sub cellular decision engine 425 process the cascaded data elements of the input message 415 according to its rules. Once processed the output element of the message defines the message that should be generated whenever the scenario of lights is observed in the scenario.
The training message enables the definition of messages for training and application. Training messages comprise input and output elements, as shown above, and are used to provide a hierarchical definition of a concept. The input element of the example message defines the concept of ‘traffic light's’. In this case the concept comprises three sub concepts of red, amber and green. The output elements define the message that should be generated whenever this above scenario is observers in the training message.
Other rules within the XML input message state what action to perform if the traffic light is green. For example, if the same input message (input message traffic lights) was received by the cellular decision engine, but in this instance the data value ‘red’ was changed to ‘green’. The rule would be traversed until the action associated with the data element ‘green’ was located and so on. It is important to note that the rules pertaining to a data value within each cellular decision engine must be an identical match to a data value of the input message. For example the data value ‘red’ within the input message must match the data value ‘red’ of the rule, i.e. the letters that make up a word must be identical to the letters making up the word in which the comparison is taking place. Thus, performing symbolic processing and being able to detect any changes within a word or a sentence structure.
In order to create a sub hierarchy of cellular decision engines, the parent cellular decision engine identifies an input message for which there is no sub cellular engine in which to process the new context identified in the input message.
Taking the following input message and with reference to
<MessageBody type = application>
<Scenario>
<output>
<traffic lights value = ‘green’>
</output>
<input>
<red =’stop’/>
<amber value = ‘stop’/>
<green value = ‘go’/>
<input>
</scenario>
</MessageBody>
The root cellular decision engine 415 receives the input message 405 from the requesting entity and cascades the input message 405 to the next level of sub cellular decision engines (lights) 420. The sub cellular decision engines (lights) 420 compares the data values of the input message with the values of the sub cellular decision engines 420, 425 within the hierarchical structure 400 and attempts to locate a further cellular decision engine to process the data values. In this example, the input value is amber and although there is a sub cellular decision engine 425, its identifier is ‘green’ and not amber and therefore there is no match. Therefore a further cellular decision engine is created 430 (
Therefore, a hierarchy of cellular decision engines 415, 420, 425 and 430 are created which allows for the cascade of data elements of the input message to each of the cellular decision engines within the created sub networks. The cellular decision engines within these sub networks process the cascaded messages in a similar manner to their parent cells; this may include the potential construction of sub networks and the generation of output messages. In this manner, a network of cellular decision engines process received messages in a highly recursive manner and emerges into a highly complex cellular topology. As with natural systems, it is envisaged that there will be many different classes of cellular decision engines and that each of these classes will operate according to a unique set of processing rules. It is also envisaged that a cellular decision engine of one class may construct a sub network comprising cells of different classes. For example a parent class ‘vehicle’ may comprise child cellular engines, car, boat, truck and airplane. The present invention embodies two types of cellular decision engines, namely, match cells and exception cells.
Match cells represent specific lookup information, such as the fact that ‘red’ is a color.
Exception cells represent phrases where the meaning of a term is dependent on the context of a sentence. For example, “Luke spoke to John England about the situation”.
Both the match cells and the exception cells store the data represented in the XML input messages, i.e. the training messages.
Referring to
The input message (replicated here for the reader's convenience) is received by a match cell.
Input message from
<Root>
<Input>
<Noun>
<Name value=“England”/>
</Noun>
</Input>
<Output>
<Noun>
<Country ref=“1.1.1”/>
</Noun>
</Output>
</Root>
The match cells are classified by their class type, in the example of
The XML input message 500 is received by the root match cell 510 and cascaded to a sub cellular decision engine 520 and a comparison is performed to determine the input value. In this example the input value is determined as a noun. A further comparison is performed to compare the data values of the input message 500 with the identifier values of each of the match cells 520, 525 within the hierarchical structure. As a match is found between the data value ‘name’ and a match cell 525 whose identifier is also name, the input message 500 is cascaded to the name match cell 525 for processing. Once received by the name match cell, a further comparison is made by the name match cell 525 to determine if a match can be found for the data element ‘England’ 505. In this example a positive determination is made i.e. a match is found for ‘England’ 505 and an output message is generated stating that England is a country (according to the output definition of the input message).
A further example of the processing of an input message by a match cell can be seen in
The above examples illustrate how match cells are used to process a simple input message where a sub cellular decision engine exists in which to process the cascaded data elements and generate an output message.
As can be seen in
The above examples show how match cells process individual words. When processing sentences or phrases more complex hierarchical structures of cellular decision engines are created then previously discussed.
In order to process more complex contextual phrases, a second type of cellular decision engine is used, namely an exception cell. Exception cells store and process more complex contextual phrases such as:
In (1) above, it may be preferable to consider the ‘IBM Sports Club’ as a location, whilst in (2) is should be considered to represent an organization.
Referring to
Moving onto
After the relevant inhibit values have been stored, a new structure representing the new scenario is created, as illustrated with reference to
Finally referring to
If a positive determination is made control passes to step 1004 and the input values are stored. Moving back to decision 1003, if a negative determination in made control moves to step 1005 and the third unidentified cellular decision engine is created. This process is completed for all input values within the input message and further input messages.
Luke, James Steven, Hunter, Robert James
| Patent | Priority | Assignee | Title |
| 10198646, | Jul 01 2016 | International Business Machines Corporation | Hardware compilation of cascaded grammars |
| 10585987, | Oct 19 2015 | International Business Machines Corporation | Determining and discerning items with multiple meanings |
| 11328126, | Oct 19 2015 | International Business Machines Corporation | Determining and discerning items with multiple meanings |
| 8676618, | Dec 28 2006 | SAP SE | Systems and methods for automatic detection and processing of instances of business rules |
| 9519706, | Nov 29 2011 | International Business Machines Corporation | Multiple rule development support for text analytics |
| 9672207, | Oct 19 2015 | International Business Machines Corporation | System, method, and recording medium for determining and discerning items with multiple meanings |
| Patent | Priority | Assignee | Title |
| 3932859, | Mar 07 1974 | Anastasios, Kyriakides | Electronic dictionary with word length responsive decoder |
| 4989258, | Sep 09 1987 | International Business Machines Corporation | Character recognition apparatus |
| 5388196, | Sep 07 1990 | Xerox Corporation | Hierarchical shared books with database |
| 5559693, | Jun 28 1991 | HEWLETT-PACKARD DEVELOPMENT COMPANY, L P | Method and apparatus for efficient morphological text analysis using a high-level language for compact specification of inflectional paradigms |
| 5615378, | Jul 19 1993 | Fujitsu Limited | Dictionary retrieval device |
| 5765180, | May 18 1990 | Microsoft Technology Licensing, LLC | Method and system for correcting the spelling of misspelled words |
| 20010037346, | |||
| 20030018598, | |||
| 20040073874, | |||
| WO3003241, |
| Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
| Oct 14 2005 | HUNTER, ROBERT JAMES | International Business Machines Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 017051 | /0303 | |
| Oct 14 2005 | LUKE, JAMES STEVEN | International Business Machines Corporation | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 017051 | /0303 | |
| Oct 18 2005 | International Business Machines Corporation | (assignment on the face of the patent) | / |
| Date | Maintenance Fee Events |
| Mar 23 2010 | ASPN: Payor Number Assigned. |
| Sep 13 2013 | REM: Maintenance Fee Reminder Mailed. |
| Jan 30 2014 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
| Jan 30 2014 | M1554: Surcharge for Late Payment, Large Entity. |
| Sep 18 2017 | REM: Maintenance Fee Reminder Mailed. |
| Mar 05 2018 | EXP: Patent Expired for Failure to Pay Maintenance Fees. |
| Date | Maintenance Schedule |
| Feb 02 2013 | 4 years fee payment window open |
| Aug 02 2013 | 6 months grace period start (w surcharge) |
| Feb 02 2014 | patent expiry (for year 4) |
| Feb 02 2016 | 2 years to revive unintentionally abandoned end. (for year 4) |
| Feb 02 2017 | 8 years fee payment window open |
| Aug 02 2017 | 6 months grace period start (w surcharge) |
| Feb 02 2018 | patent expiry (for year 8) |
| Feb 02 2020 | 2 years to revive unintentionally abandoned end. (for year 8) |
| Feb 02 2021 | 12 years fee payment window open |
| Aug 02 2021 | 6 months grace period start (w surcharge) |
| Feb 02 2022 | patent expiry (for year 12) |
| Feb 02 2024 | 2 years to revive unintentionally abandoned end. (for year 12) |