Mulgara Project: Issueshttps://code.mulgara.org/https://code.mulgara.org/favicon.ico?15861924492012-02-07T21:37:21ZMulgara Project
Redmine Mulgara - Bug #213 (New): Error on ORDER BY variables projected away by SELECT clausehttps://code.mulgara.org/issues/2132012-02-07T21:37:21ZAlex Hall -alexhall@revelytix.com
<p>A SPARQL query that mentions a variable in an ORDER BY expression that appears in the WHERE clause but is projected away by the SELECT clause incorrectly causes an exception to be thrown. According to SPARQL, ordering is applied prior to projection, so this should be valid. This appears to be a case where TQL and SPARQL semantics are mis-aligned.</p> Mulgara - Bug #183 (In Progress): project() isn't considered with index selection.https://code.mulgara.org/issues/1832009-02-17T08:30:38ZAndrae Muys -andrae@netymon.com
<p>select $p from <graph:data> where $s $p $o; uses the GSPO index which requires a full external sort of the store to extract $p. If we use GPOS we get $p without the sort.</p>
<p>The problem is that project() call doesn't participate in index ordering within the query optimisation.</p>
<p>The solution is to introduce an Annotation that allows the project to specify a preferred order. Initially supporting this Annotation with <a class="wiki-page new" href="https://code.mulgara.org/projects/mulgara/wiki/StatementStoreResolution">StatementStoreResolution</a> will be sufficient to satisfy the query above. A general solution will also require supporting it with Join and Append tuples.</p> Mulgara - Feature #91 (In Progress): Refactor OperationContext to provide access to the Operation...https://code.mulgara.org/issues/912008-03-26T03:45:52ZAndrae Muys -andrae@netymon.com
<p>Currently we have multiple Operation classes performing their own explicit insertion routines. This duplication should be refactored into <a class="wiki-page new" href="https://code.mulgara.org/projects/mulgara/wiki/ModifyModelOperation">ModifyModelOperation</a>, and all other operations that require insertion functionality should do so via calls to Session::insert() and Session::delete().</p> Mulgara - Feature #86 (In Progress): Clean up XSD Resolver so it becomes suitable demo-resolverhttps://code.mulgara.org/issues/862008-03-17T01:36:49ZAndrae Muys -andrae@netymon.com
<p>The XSD-Resolver is currently in a very poor state of repair. The Resolver Interface also desperately requires detailed documentation, and this really requires an exemplar resolver.</p>
<p>Clean up the XSD-Resolver so that it can be used as an example of how to write a resolver.</p> Mulgara - Bug #83 (Feedback): Mulgara fails to start after abrupt terminationhttps://code.mulgara.org/issues/832008-03-07T06:07:03Zronald -ronald@foo.bar
<p>I killed mulgara abruptly, and subsequently it failed to start.<br />According to Andrae's initial analysis, the problem is that mulgara<br />assumes during recovery that certain deleted structures are internally<br />valid when in this case they aren't.</p>
<p>Attached is the db from after the crash. The failure to recover seems to<br />be reproducable only on 32-bit jvm's, and even then not consistently<br />(due to the timing between certain operations).</p> Mulgara - Bug #82 (In Progress): Read-close-write creates redundant phasehttps://code.mulgara.org/issues/822008-03-05T21:13:22ZPaula Gearon
<p>When reading and closing an Answer in a write transaction, a redundant micro-commit occurs. This leads to unbounded growth when regularly reading in large transactions.</p> Mulgara - Bug #81 (In Progress): Blank Node Assignment in Inserts inconsistent with autocommithttps://code.mulgara.org/issues/812008-02-28T06:23:04ZAndrae Muys -andrae@netymon.com
<p>It appears that the binding of variables to blank nodes when inserting <br />statements behaves differently depending on whether autocommit is turned <br />on for the session. If I have autocommit turned on and execute the <br />following iTQL commands:<br /><pre>
create <rmi://localhost/server1#test>;
insert <test:subj> <test:pred> $x $x <test:value> 'o1'
into <rmi://localhost/server1#test> ;
insert <test:subj> <test:pred> $x $x <test:value> 'o2'
into <rmi://localhost/server1#test> ;
select $s $p $o from <rmi://localhost/server1#test> where $s $p $o;
</pre><br />Then I see that the variable "x" is bound to different blank nodes in <br />each of the two insertions, and the resulting model has 4 statements. <br />This is the behavior (behaviour?) that I would expect.</p>
<p>However, when I turn autocommit off prior to the first insertion, and <br />turn it back on after the second insertion, then I see that the variable <br />"x" is bound to the <strong>same</strong> blank node in each of the insertions, <br />resulting in 3 total statements in the model, which came as a bit of a <br />surprise to me. Is this the expected behavior in that situation?</p> Mulgara - Feature #71 (New): Clean up DatabaseOperationContexthttps://code.mulgara.org/issues/712007-09-13T08:57:10ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">This is the next cleanup refactoring.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
We have:
<span class="nt"><br/></span>
<span class="nt"><br/></span>
[[ResolverFactory]] findModelResolverFactory(long model)
<span class="nt"><br/></span>
[[ResolverFactory]] findUncachedExternalResolverFactory(long model)
<span class="nt"><br/></span>
[[ResolverFactory]] findModelTypeResolverFactory(URI modelTypeURI)
<span class="nt"><br/></span>
Resolver obtainResolver(ResolverFactory resolverFactory)
<span class="nt"><br/></span>
URI mapToModelTypeURI(URI modelURL)
<span class="nt"><br/></span>
URI findModelTypeURI(long model)
<span class="nt"><br/></span>
String findProtocol(long n)
<span class="nt"><br/></span>
<span class="nt"><br/></span>
most of which do multiple overlapping tasks, often implemented slightly differently.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Most of these methods used to live in different classes, and have migrated here as their homes were refactored and tidied.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
[[DatabaseOperationContext]] and [[LocalQueryResolver]] are the last two dumping grounds in the system-core where we have left the cruft that was too difficult to clean up concurrently with previous refactorings; and [[DatabaseOperationContext]] is by far the worst of the two - so it's next on the list.
</code></pre> Mulgara - Feature #57 (New): Extend use of Annotations to improve transparency of deferral based ...https://code.mulgara.org/issues/572007-04-17T05:51:27ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">[[ReresolvableResolution]] needs to be removed and replaced with an Annotation.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
All Major Tuples that provide operations on Tuples need to be modified to provide aggregated access to their operands annotations when their operands provide them.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
As a longer term goal we should rephrase the annotations in terms of complexity estimates for reordering/sorting/etc operations as well as constraints on join ordering to permit more nuanced optimisation of joins - this however needs to wait until we are fully exploiting the capabilities of the existing annotations.
</code></pre> Mulgara - Bug #56 (Feedback): Distributed blank nodes not uniquely identifiedhttps://code.mulgara.org/issues/562007-04-15T05:55:17ZPaula Gearon
<pre>
<code class="html syntaxhl">Any blank nodes returned from a distributed resolver may contain the same gnode IDs as local blank nodes. This means they cannot be distinguished. Add a factory in the appropriate place to mark blank nodes from elsewhere with the IP of their source.
</code></pre> Mulgara - Bug #52 (New): Insufficient dereferencing in TransactionalAnswer.sessionClose leading t...https://code.mulgara.org/issues/522007-03-19T07:03:14ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">There is a need for the normal error-path to properly dereference and cleanup transactions so that exceptional failures within the error-path can be logged without being lost in 'noise'.
</code></pre> Mulgara - Feature #23 (New): We must find a more descriptive way of expressing the cost of iterat...https://code.mulgara.org/issues/232006-09-26T11:08:34ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">An increasing number of resolvers (relational, federating, distributed) cannot provide a meaningful upper-bound to their row-count.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Also row-count assumes a uniform cost of iteration, when iteration (as opposed to resolution) entails network latency this is unrealistic.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
How do we handle this? Do we need some sort of bogomips-like timing constant to provide a baseline for comparison? Can we use this to also provide selectivity data that would also improve our join performance?
</code></pre> Mulgara - Feature #18 (New): Provide a language independent interface to Mulgara compatible with ...https://code.mulgara.org/issues/182006-09-14T04:38:41ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">We need to support access to the entire Session API from languages other than java.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Both xml-rpc and REST have been proposed as alternatives. OTOH we should probably support both.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
XMLRPC because of it's wide language support and classic-rpc model.
<span class="nt"><br/></span>
REST because it is a better fit to W3C architecture, and provides benefits when working with loosely coupled distribution.
</code></pre> Mulgara - Feature #16 (In Progress): Add Rules for OWL DL/Full supporthttps://code.mulgara.org/issues/162006-09-14T04:28:54ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">Extend the new Rules Engine to support OWL Full.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Provide rule sets for OWL DL and OWL Full.
</code></pre> Mulgara - Bug #15 (In Progress): Product of Sum form has very poor performance.https://code.mulgara.org/issues/152006-09-06T04:30:23ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">[[OrderedAppend]] currently assumes that all it's operands share a common ordering. This is currently guarenteed by [[TuplesOperations]].append(), but the use of calls to project() on all operands not matching the first's variable list. This has signifigant performance ramifications as the desired ordering is unknown until any parent-join is optimised.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Without the projections currently done by [[TuplesOperations]].append(), we need new logic in [[OrderedAppend]] to handle mapping variables to operands columns. This is not a problem unless there the disjunction is subject to a join containing a left-bound prefix including mismatched variables.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Consider the following tuple expressions:
<span class="nt"><br/></span>
<span class="nt"><br/></span>
A[$z] ^ B( C[$z $y] v D[$y $z] )
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Note:
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>A[] will provide $z prefix to B[].
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>B[] currently passes this on blindly to C[] and D[].
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>This will bind $z for $y in D[], leading to an incorrect result.
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>Currently the non-union compatible
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>Fixing this will require either reordering D[], or filtering it; deciding between them is a performance optimisation issue.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
A[$y $z] ^ B(C[$y] v D[$z])
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Note:
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>Here there is a full prefix provided by A[] to B[].
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>The prefix needs to be decomposed for C[] and D[]
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>This is a case of non-union compatible disjunction, and will probably result in UNBOUND's.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Cases where this is probably a problem:
<span class="nt"><br/></span>
<span class="nt"><br/></span>
non-symmetric sum-of-products (ie. {$a <span class="ni">&lt;</span>foo<span class="ni">&gt;</span> $b ^ $b <span class="ni">&lt;</span>bar<span class="ni">&gt;</span> $c in <span class="ni">&lt;</span>m1<span class="ni">&gt;</span>} v {$b <span class="ni">&lt;</span>bar<span class="ni">&gt;</span> $c ^ $a <span class="ni">&lt;</span>foo<span class="ni">&gt;</span> $b in <span class="ni">&lt;</span>m2<span class="ni">&gt;</span>} )
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>Note: this is one of the key areas of concern. Differences between models can cause join-optimisation to generate a different ordering; This interferes with attempts to improve SOP performance which is required to support efficient views.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
symmetric disjunctions: (ie. $s <span class="ni">&lt;</span>foo<span class="ni">&gt;</span> $o v $o <span class="ni">&lt;</span>bar<span class="ni">&gt;</span> $s). - uncommon.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
non-union compatible disjunction (ie. $s <span class="ni">&lt;</span>name<span class="ni">&gt;</span> $name v $s <span class="ni">&lt;</span>email<span class="ni">&gt;</span> $email)
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>Note: this query is better phrased as a subquery anyway.
</code></pre>