Difference between revisions of "Vulcan/SystemStatus"
From Knowitall
(→Status) |
(→Inference) |
||
(15 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
+ | == Overview == | ||
+ | |||
+ | The input to our inference system is a natural language sentence that corresponds to an answer choice. (e.g., Iron nail is a good conductor of electricity). | ||
+ | The output is a score that represents the system's belief regarding the validity of the input sentence. | ||
+ | |||
+ | At a high-level the system performs the following steps: | ||
+ | |||
+ | <ol> | ||
+ | <li>Process sentence through Open IE 4.0 and convert it into a tuple. We refer to this query tuple as a proposition.</li> | ||
+ | <li>Find textual evidence that could support the input proposition. Find tuples and sentences that match the input proposition. Convert this textual evidence into axioms and rules in Tuffy's MLN format.</li> | ||
+ | <li>Run Tuffy with both textual and KB evidence. Output the marginal inference scores.</li> | ||
+ | </ol> | ||
+ | |||
The system consists of <b>Inference</b> (online) and <b>Axiom and Rule generation</b> (offline) components. | The system consists of <b>Inference</b> (online) and <b>Axiom and Rule generation</b> (offline) components. | ||
− | == Inference | + | == Inference == |
+ | ;Idea: | ||
+ | |||
+ | : Iteratively perform inference on new evidence until some termination criterion is met. | ||
+ | |||
+ | ; Procedure: | ||
+ | <pre> | ||
+ | 1. Select evidence for input proposition (T) using textual evidence finder. [This procedure is a bit hazily defined at the moment.] | ||
+ | -- Extract CNC and hypernym predicates from arguments in proposition. | ||
+ | |||
+ | 2. Run MLN inference. | ||
+ | |||
+ | 3. Check Termination criterion: Terminate if probability > threshold or if iterations exceed limit. | ||
+ | |||
+ | 4. For each newly inferred fact (N): | ||
+ | -- Compute textual matching score for N and T, and add a rule Score(N, T): N => T with the textual matching score as its weight. | ||
+ | -- Extract CNC and hypernym predicates from the new arguments. | ||
+ | |||
+ | 5. Compute textual transforms for T. For each T': | ||
+ | -- Find evidence for T' using textual evidence finder. | ||
+ | -- Compute textual matching score for T' with N and add a rule Score(N, T'): N => T' | ||
+ | |||
+ | 6. Go to step 2. | ||
+ | |||
+ | </pre> | ||
+ | |||
<!--;Terminology | <!--;Terminology | ||
: Axiom - Corresponds to a known fact. Has a probability to reflect the system's belief in it. | : Axiom - Corresponds to a known fact. Has a probability to reflect the system's belief in it. | ||
Line 8: | Line 46: | ||
[[File:Sys-arch-online.jpg|frame|center|alt=System Architecture|System Architecture: Online components]] | [[File:Sys-arch-online.jpg|frame|center|alt=System Architecture|System Architecture: Online components]] | ||
− | === | + | === Status === |
− | + | <span style="background-color:yellow">Last updated Aug 23, 2013.</span> | |
− | |||
− | |||
− | |||
− | |||
− | |||
{| class="wikitable" | {| class="wikitable" | ||
|- | |- | ||
Line 21: | Line 54: | ||
| Proposition Extractor || Extracts propositions to be verified by the system || Input: Sentence asserting the answer choice.<br/> Output: A proposition represented as a open IE tuple. || <b>Stub implemented: </b> Sentence piped into Open IE 4.0. Output tuples that fit a specified pattern are converted into a proposition. || Select the best tuple. Stem, normalize, extract polarity, handle n-ary tuples. | | Proposition Extractor || Extracts propositions to be verified by the system || Input: Sentence asserting the answer choice.<br/> Output: A proposition represented as a open IE tuple. || <b>Stub implemented: </b> Sentence piped into Open IE 4.0. Output tuples that fit a specified pattern are converted into a proposition. || Select the best tuple. Stem, normalize, extract polarity, handle n-ary tuples. | ||
|- | |- | ||
− | | | + | | Textual Evidence Finder || Provides evidence for the input proposition using <i>weak</i> text-based methods.|| Input: Proposition <br/>Output: Rules/Axioms that derive the proposition (in one-step) weighted according to the confidence provided by each method. || <b>Tuple matcher</b> implementation underway. Uses Open IE solr instance to find matching sentences/tuples.<br/><b>Pattern matcher</b> not implemented.<br/><!--b>Jena</b> evidence finder partly implemented. Produces a single derivation, which needs to be converted to axioms and rules. <span style="background-color:yellow"><b>Note: Not using Jena as a evidence finder anymore. Tuffy is optimized for efficient inference.</b></span--> || Implement pattern matching, improve tuple matching, add scores. |
|- | |- | ||
| MLN Instance Generator || Converts the evidence into MLN input format. || Input:Axioms/Rules from weaker methods.<br/>Output:Translated into Tuffy's input format.<br/> || <b>Stub implemented:</b> Does a faithful translation to MLN syntax. || Fix bug in predicate definitions. | | MLN Instance Generator || Converts the evidence into MLN input format. || Input:Axioms/Rules from weaker methods.<br/>Output:Translated into Tuffy's input format.<br/> || <b>Stub implemented:</b> Does a faithful translation to MLN syntax. || Fix bug in predicate definitions. | ||
Line 27: | Line 60: | ||
| End-to-End Driver || Command line application. || Input: Sentence asserting an answer choice. <br/>Output:Score || <b>Stub implemented</b>.|| Create web interface. | | End-to-End Driver || Command line application. || Input: Sentence asserting an answer choice. <br/>Output:Score || <b>Stub implemented</b>.|| Create web interface. | ||
|} | |} | ||
+ | |||
+ | === Textual Evidence Finder === | ||
+ | Greg is going to be developing this component. | ||
+ | |||
+ | [[File:weak-evidence-finder.jpg|frame|center|alt=Textual Evidence Finder Details|System Architecture: Textual Evidence Finder]] | ||
== Axiom and Rule Generation Components == | == Axiom and Rule Generation Components == |
Latest revision as of 20:29, 13 September 2013
Contents
Overview
The input to our inference system is a natural language sentence that corresponds to an answer choice. (e.g., Iron nail is a good conductor of electricity). The output is a score that represents the system's belief regarding the validity of the input sentence.
At a high-level the system performs the following steps:
- Process sentence through Open IE 4.0 and convert it into a tuple. We refer to this query tuple as a proposition.
- Find textual evidence that could support the input proposition. Find tuples and sentences that match the input proposition. Convert this textual evidence into axioms and rules in Tuffy's MLN format.
- Run Tuffy with both textual and KB evidence. Output the marginal inference scores.
The system consists of Inference (online) and Axiom and Rule generation (offline) components.
Inference
- Idea
- Iteratively perform inference on new evidence until some termination criterion is met.
- Procedure
1. Select evidence for input proposition (T) using textual evidence finder. [This procedure is a bit hazily defined at the moment.] -- Extract CNC and hypernym predicates from arguments in proposition. 2. Run MLN inference. 3. Check Termination criterion: Terminate if probability > threshold or if iterations exceed limit. 4. For each newly inferred fact (N): -- Compute textual matching score for N and T, and add a rule Score(N, T): N => T with the textual matching score as its weight. -- Extract CNC and hypernym predicates from the new arguments. 5. Compute textual transforms for T. For each T': -- Find evidence for T' using textual evidence finder. -- Compute textual matching score for T' with N and add a rule Score(N, T'): N => T' 6. Go to step 2.
Status
Last updated Aug 23, 2013.
Component | Description | I/O | Status | To Do |
---|---|---|---|---|
Proposition Extractor | Extracts propositions to be verified by the system | Input: Sentence asserting the answer choice. Output: A proposition represented as a open IE tuple. |
Stub implemented: Sentence piped into Open IE 4.0. Output tuples that fit a specified pattern are converted into a proposition. | Select the best tuple. Stem, normalize, extract polarity, handle n-ary tuples. |
Textual Evidence Finder | Provides evidence for the input proposition using weak text-based methods. | Input: Proposition Output: Rules/Axioms that derive the proposition (in one-step) weighted according to the confidence provided by each method. |
Tuple matcher implementation underway. Uses Open IE solr instance to find matching sentences/tuples. Pattern matcher not implemented. |
Implement pattern matching, improve tuple matching, add scores. |
MLN Instance Generator | Converts the evidence into MLN input format. | Input:Axioms/Rules from weaker methods. Output:Translated into Tuffy's input format. |
Stub implemented: Does a faithful translation to MLN syntax. | Fix bug in predicate definitions. |
End-to-End Driver | Command line application. | Input: Sentence asserting an answer choice. Output:Score |
Stub implemented. | Create web interface. |
Textual Evidence Finder
Greg is going to be developing this component.
Axiom and Rule Generation Components
Component | Description | I/O | Status |
---|---|---|---|
Axioms Extractor | Generates axioms from multiple knowledge sources. | Input: Output: |
None. |
Rules Generator | Uses second order rule templates to generate first-order inference rules. | Input: Output: |
None. |