Difference between revisions of "Vulcan/SystemPrototype"

Revision as of 21:35, 27 August 2013

Overview

The prototype is designed to work on three questions. We want the system to output the following:

• Score for the input proposition.

• New facts inferred.

• Facts and rules used in scoring.

Status

Ran Tuffy on three example questions. It failed on one question.

• Hand generated the input evidence for the propositions (one correct and one incorrect) for three questions.
• Hand generated the MLN rules (adapted from Stephen's rules)
• Ran Tuffy to obtain the inference probabilities on the propositions.
• The system also outputs:

• All inferred facts along with their probabilities.
• All rules that are reachable from the query fact. i.e., Clauses in the MLN that are relevant to the inference of the query fact.

See here for the set of MLN rules used.

• Does it work?

• Tuffy gets it right 2/3 questions. i.e., it assigns higher probabilities for the correct proposition.
• Facts inferred by larger number of steps have a lower score compared to those inferred by a smaller number of steps.

• Why does it fail on the one question?

• Looks like a bug in
• Both "iron nail" and "plastic cup" get similar weights (iron nail is slightly higher). I don't yet understand the scoring enough to explain this. Will dig in when I come back.

• What diagnostics do we NOT have?

• Connections between the clauses in the MLN.
• A reconstruction/visualization of the MLN network.

Working with Tuffy developers on this.

• What does this exercise suggest?

• Need to figure out how the weights on the MLN rules and evidence are use. [I assigned them arbitrarily for this round.]
• Use predicates with small arity. For example, avoid writing rules entire nested tuples as predicates.
• The only reason we'd need a nested tuple is for the purpose of computing the score. For now we can compute this from the score of its components: Score(nested_tuple) = Score(top tuple) * Score (nested).