Difference between revisions of "Vulcan/MeetingNotes/Aug09 2013"
(→Notes) |
(→To Do) |
||
(23 intermediate revisions by the same user not shown) | |||
Line 4: | Line 4: | ||
: 1. Inferencer stub using Jena. Stub takes in axioms and rules and outputs a derviation. | : 1. Inferencer stub using Jena. Stub takes in axioms and rules and outputs a derviation. | ||
+ | <blockquote> | ||
+ | * Jena API doesn't readily support multiple derivations.<br/> | ||
+ | * Ask Jena community to find out if this is possible or<br/> | ||
+ | replace with OWLIM (or Seasame) triplestore/inference systems.<br/> | ||
+ | |||
+ | </blockquote> | ||
: 2. Tested stub with axioms and rules that would help us solve the iron nail example. | : 2. Tested stub with axioms and rules that would help us solve the iron nail example. | ||
: 3. Built Proposition extractor stub that converts the answer assertions into propositions represented as <b>Open IE 4.0</b> tuples | : 3. Built Proposition extractor stub that converts the answer assertions into propositions represented as <b>Open IE 4.0</b> tuples | ||
− | |||
− | |||
; Resource collection | ; Resource collection | ||
Line 27: | Line 31: | ||
<b>1. Do we really needed an inference engine (Jena)? </b> | <b>1. Do we really needed an inference engine (Jena)? </b> | ||
− | We need | + | * We need an inference engine primarily to limit the set of groundings for BLP or MLN. |
+ | * However, the step by step derivation used by inference engines is a brittle approach and is likely to fail often. | ||
+ | * Instead, Mausam suggested looking into Knowledge Base Model Construction as an alternative for finding all relevant axioms and rules that can be fed into a MLN or KBP. | ||
<b>2. Discussed the [http://homes.cs.washington.edu/~niranjan/vulcan/aug09/stepsinvolved.docx inference steps involved] for 5 different questions. </b> | <b>2. Discussed the [http://homes.cs.washington.edu/~niranjan/vulcan/aug09/stepsinvolved.docx inference steps involved] for 5 different questions. </b> | ||
− | It was clear | + | It was clear we may not be able to have a complete derivation always. |
− | <b> What happens when deductive inference fails? </b> | + | <b>3. What happens when deductive inference fails? </b> |
− | <i>Approach A:</i> | + | * Add weak entailment rules that can help complete the inference. |
− | Identify axioms (tuples) that are highly "similar" to some node in the backward chained derivation graph. | + | * Use a "linguistically motivated pattern matcher" as a form of abductive reasoning. We can use this pattern matcher as a stand alone method as well. |
+ | |||
+ | <i>Approach A:</i> Identify axioms (tuples) that are highly "similar" to some node in the backward chained derivation graph. | ||
Add weak entailment rules (axiom -> derivation node) scored using edit distance. | Add weak entailment rules (axiom -> derivation node) scored using edit distance. | ||
− | <i>Approach B:</i> | + | <i>Approach B:</i> Find the most plausible answer. |
− | Find the | ||
<blockquote> | <blockquote> | ||
Line 54: | Line 61: | ||
From this we cannot <b>prove</b> that foxes use sense of smell to find food but it increases our belief in it. | From this we cannot <b>prove</b> that foxes use sense of smell to find food but it increases our belief in it. | ||
− | |||
</blockquote> | </blockquote> | ||
Line 68: | Line 74: | ||
</blockquote> | </blockquote> | ||
− | |||
</blockquote> | </blockquote> | ||
+ | This is a form of abductive reasoning using linguistically motivated templates. | ||
+ | |||
+ | <b>4. How to use this template matching module?</b> | ||
+ | |||
+ | * Use template matching first to find the most plausible answer. Then, use the inference engine to produce the best possible explanation. | ||
− | + | * Use template matching to add links in the inference graphs (whenever deductive inference fails). | |
+ | |||
+ | * Implement <i>Approach B</i> as a standalone method for answering questions.</b> | ||
− | |||
− | |||
<!-- == Agenda == | <!-- == Agenda == | ||
Line 98: | Line 108: | ||
--> | --> | ||
− | == To Do == | + | == To Do (Updated on Aug 16) == |
; System building | ; System building | ||
− | : 1. | + | : 1. Implement "template matching" using the ClueWeb corpus.<b>Pending</b> |
+ | <blockquote> | ||
+ | * URL for Open IE backend is available. | ||
+ | * For an assertion A, find sentences that have high overlap. Generate regex patterns for the proposition. Score sentences by how well they match the regex patterns. | ||
+ | </blockquote> | ||
: 2. Continue system building. | : 2. Continue system building. | ||
<blockquote> | <blockquote> | ||
− | * Create a derivation scorer stub. This will be replaced with a MLN or a BLP scorer. | + | * Create a derivation scorer stub. This will be replaced with a MLN or a BLP scorer. <b>Done.</b> |
* Test with iron nail example. | * Test with iron nail example. | ||
</blockquote> | </blockquote> | ||
Line 112: | Line 126: | ||
: 3. Jena API doesn't readily support multiple derivations. | : 3. Jena API doesn't readily support multiple derivations. | ||
<blockquote> | <blockquote> | ||
− | * Ask Jena community to find out if this is possible. | + | * Ask Jena community to find out if this is possible. <b>Done. Not possible.</b> |
− | * | + | * OWLIM as replacement. <b>Done. Doesn't look promising. No response from community.</b> |
</blockquote> | </blockquote> | ||
− | :4. Try out [http://hazy.cs.wisc.edu/hazy/tuffy/ Tuffy MLN] implemenatation. | + | :4. Try out [http://hazy.cs.wisc.edu/hazy/tuffy/ Tuffy MLN] implemenatation. <b>Done.</b> |
<blockquote> | <blockquote> | ||
* Use output of iron nail example | * Use output of iron nail example | ||
Line 122: | Line 136: | ||
</blockquote> | </blockquote> | ||
− | :5 Write evaluation code. | + | :5 Write evaluation code. <b>Vulcan has a good interface set up.</b> |
<blockquote> | <blockquote> | ||
− | * Check with Peter. | + | * Check with Peter. |
</blockquote> | </blockquote> | ||
− | ; | + | :6. Create a [[Vulcan/SystemArchitecture| system architecture]] page with a figure and overview of the main components. |
+ | <b>Created a [[Vulcan/SystemStatus| System status page]] instead.</b> | ||
+ | <blockquote> | ||
+ | * Created a figure. Added it to system design document. | ||
+ | * Need to create a wiki page for system architecture and overview. | ||
+ | </blockquote> | ||
+ | |||
+ | ; Experiments <b>Pending</b> | ||
+ | |||
+ | : 1. Run template matching approach as a baseline. | ||
− | + | : 2. Run inference system as a baseline. |
Latest revision as of 18:35, 16 August 2013
Update
- Focusing on system development.
- 1. Inferencer stub using Jena. Stub takes in axioms and rules and outputs a derviation.
- Jena API doesn't readily support multiple derivations.
- Ask Jena community to find out if this is possible or
replace with OWLIM (or Seasame) triplestore/inference systems.
- 2. Tested stub with axioms and rules that would help us solve the iron nail example.
- 3. Built Proposition extractor stub that converts the answer assertions into propositions represented as Open IE 4.0 tuples
- Resource collection
- 1. Gathered assertions from Peter/Phil. Each assertion corresponds to a single multiple choice answer.
- 2. Found RDF representations for WordNet and imported them into Jena.
- Analysis
- 1. Selected 10 propositions that are single Open IE tuples as starting targets.
- 2. Started to write down steps involved in verifying these propositions.
Notes
1. Do we really needed an inference engine (Jena)?
- We need an inference engine primarily to limit the set of groundings for BLP or MLN.
- However, the step by step derivation used by inference engines is a brittle approach and is likely to fail often.
- Instead, Mausam suggested looking into Knowledge Base Model Construction as an alternative for finding all relevant axioms and rules that can be fed into a MLN or KBP.
2. Discussed the inference steps involved for 5 different questions.
It was clear we may not be able to have a complete derivation always.
3. What happens when deductive inference fails?
- Add weak entailment rules that can help complete the inference.
- Use a "linguistically motivated pattern matcher" as a form of abductive reasoning. We can use this pattern matcher as a stand alone method as well.
Approach A: Identify axioms (tuples) that are highly "similar" to some node in the backward chained derivation graph. Add weak entailment rules (axiom -> derivation node) scored using edit distance.
Approach B: Find the most plausible answer.
Example 1: (x, helps, a fox find food), where x is one of {sense of smell, thick fur, long tail, pointed teeth}
We don't find sentences that directly state that "sense of smell helps fox find food".
However, several sentences say "sense of smell helps animals find food":"smell helps * find food" returns 7 million hits on Google.
"fur helps * find food" returns no hits.From this we cannot prove that foxes use sense of smell to find food but it increases our belief in it.
Example 2: (x, conductor of, electricity), where x is one of {iron nail, rubber boat, wax crayon, plastic cup}
We don't find sentences that directly state that "iron nail is a conductor of electricity" (or that it conducts electricity).
However, several sentences match the pattern "iron * is a conductor of electricity":"iron * is a conductor of electricity" returns 794,000 hits on Google.
"rubber * is a conductor of electricity" returns 196,000 hits.
This is a form of abductive reasoning using linguistically motivated templates.
4. How to use this template matching module?
- Use template matching first to find the most plausible answer. Then, use the inference engine to produce the best possible explanation.
- Use template matching to add links in the inference graphs (whenever deductive inference fails).
- Implement Approach B as a standalone method for answering questions.</b>
To Do (Updated on Aug 16)
- System building
- 1. Implement "template matching" using the ClueWeb corpus.Pending
- URL for Open IE backend is available.
- For an assertion A, find sentences that have high overlap. Generate regex patterns for the proposition. Score sentences by how well they match the regex patterns.
- 2. Continue system building.
- Create a derivation scorer stub. This will be replaced with a MLN or a BLP scorer. Done.
- Test with iron nail example.
- 3. Jena API doesn't readily support multiple derivations.
- Ask Jena community to find out if this is possible. Done. Not possible.
- OWLIM as replacement. Done. Doesn't look promising. No response from community.
- 4. Try out Tuffy MLN implemenatation. Done.
- Use output of iron nail example
- If easy to use write wrappers around Tuffy to hook into our system.
- 5 Write evaluation code. Vulcan has a good interface set up.
- Check with Peter.
- 6. Create a system architecture page with a figure and overview of the main components.
Created a System status page instead.
- Created a figure. Added it to system design document.
- Need to create a wiki page for system architecture and overview.
- Experiments Pending
- 1. Run template matching approach as a baseline.
- 2. Run inference system as a baseline.