Mulgara Project: Issueshttps://code.mulgara.org/https://code.mulgara.org/favicon.ico?15861924492009-02-17T08:30:38ZMulgara Project
Redmine Mulgara - Bug #183 (In Progress): project() isn't considered with index selection.https://code.mulgara.org/issues/1832009-02-17T08:30:38ZAndrae Muys -andrae@netymon.com
<p>select $p from <graph:data> where $s $p $o; uses the GSPO index which requires a full external sort of the store to extract $p. If we use GPOS we get $p without the sort.</p>
<p>The problem is that project() call doesn't participate in index ordering within the query optimisation.</p>
<p>The solution is to introduce an Annotation that allows the project to specify a preferred order. Initially supporting this Annotation with <a class="wiki-page new" href="https://code.mulgara.org/projects/mulgara/wiki/StatementStoreResolution">StatementStoreResolution</a> will be sufficient to satisfy the query above. A general solution will also require supporting it with Join and Append tuples.</p> Mulgara - Feature #94 (New): On insert/delete inferencing resolver (triggers)https://code.mulgara.org/issues/942008-03-28T04:22:27ZAndrae Muys -andrae@netymon.com
<p>Migrated from the Kowari RFE List</p>
<p>A Resolver that wraps the store and can read a basic<br />inferencing ruleset and insert/delete multiple<br />statements in/from the store in response to each<br />statement passed to modify.</p>
<p>This should include a <a class="wiki-page new" href="https://code.mulgara.org/projects/mulgara/wiki/ContentHandler">ContentHandler</a> that can parse<br />it's config file into rdf so the rules can be stored in<br />a ?def model in the system-store. It would be<br />important to be able to include by reference an<br />external config-file (accessed via the <a class="wiki-page new" href="https://code.mulgara.org/projects/mulgara/wiki/ContentHandler">ContentHandler</a>).</p> Mulgara - Feature #86 (In Progress): Clean up XSD Resolver so it becomes suitable demo-resolverhttps://code.mulgara.org/issues/862008-03-17T01:36:49ZAndrae Muys -andrae@netymon.com
<p>The XSD-Resolver is currently in a very poor state of repair. The Resolver Interface also desperately requires detailed documentation, and this really requires an exemplar resolver.</p>
<p>Clean up the XSD-Resolver so that it can be used as an example of how to write a resolver.</p> Mulgara - Bug #81 (In Progress): Blank Node Assignment in Inserts inconsistent with autocommithttps://code.mulgara.org/issues/812008-02-28T06:23:04ZAndrae Muys -andrae@netymon.com
<p>It appears that the binding of variables to blank nodes when inserting <br />statements behaves differently depending on whether autocommit is turned <br />on for the session. If I have autocommit turned on and execute the <br />following iTQL commands:<br /><pre>
create <rmi://localhost/server1#test>;
insert <test:subj> <test:pred> $x $x <test:value> 'o1'
into <rmi://localhost/server1#test> ;
insert <test:subj> <test:pred> $x $x <test:value> 'o2'
into <rmi://localhost/server1#test> ;
select $s $p $o from <rmi://localhost/server1#test> where $s $p $o;
</pre><br />Then I see that the variable "x" is bound to different blank nodes in <br />each of the two insertions, and the resulting model has 4 statements. <br />This is the behavior (behaviour?) that I would expect.</p>
<p>However, when I turn autocommit off prior to the first insertion, and <br />turn it back on after the second insertion, then I see that the variable <br />"x" is bound to the <strong>same</strong> blank node in each of the insertions, <br />resulting in 3 total statements in the model, which came as a bit of a <br />surprise to me. Is this the expected behavior in that situation?</p> Mulgara - Feature #71 (New): Clean up DatabaseOperationContexthttps://code.mulgara.org/issues/712007-09-13T08:57:10ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">This is the next cleanup refactoring.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
We have:
<span class="nt"><br/></span>
<span class="nt"><br/></span>
[[ResolverFactory]] findModelResolverFactory(long model)
<span class="nt"><br/></span>
[[ResolverFactory]] findUncachedExternalResolverFactory(long model)
<span class="nt"><br/></span>
[[ResolverFactory]] findModelTypeResolverFactory(URI modelTypeURI)
<span class="nt"><br/></span>
Resolver obtainResolver(ResolverFactory resolverFactory)
<span class="nt"><br/></span>
URI mapToModelTypeURI(URI modelURL)
<span class="nt"><br/></span>
URI findModelTypeURI(long model)
<span class="nt"><br/></span>
String findProtocol(long n)
<span class="nt"><br/></span>
<span class="nt"><br/></span>
most of which do multiple overlapping tasks, often implemented slightly differently.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Most of these methods used to live in different classes, and have migrated here as their homes were refactored and tidied.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
[[DatabaseOperationContext]] and [[LocalQueryResolver]] are the last two dumping grounds in the system-core where we have left the cruft that was too difficult to clean up concurrently with previous refactorings; and [[DatabaseOperationContext]] is by far the worst of the two - so it's next on the list.
</code></pre> Mulgara - Feature #57 (New): Extend use of Annotations to improve transparency of deferral based ...https://code.mulgara.org/issues/572007-04-17T05:51:27ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">[[ReresolvableResolution]] needs to be removed and replaced with an Annotation.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
All Major Tuples that provide operations on Tuples need to be modified to provide aggregated access to their operands annotations when their operands provide them.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
As a longer term goal we should rephrase the annotations in terms of complexity estimates for reordering/sorting/etc operations as well as constraints on join ordering to permit more nuanced optimisation of joins - this however needs to wait until we are fully exploiting the capabilities of the existing annotations.
</code></pre> Mulgara - Bug #53 (New): FileSystemResolver keeps exhausting heaphttps://code.mulgara.org/issues/532007-03-20T03:51:53ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">Regularly seeing heap exhaustion when running the [[FileSystemResolver]] unit test.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
I believe this resolver is memory bound and needs to be modified to use disk-backed structures.
</code></pre> Mulgara - Bug #52 (New): Insufficient dereferencing in TransactionalAnswer.sessionClose leading t...https://code.mulgara.org/issues/522007-03-19T07:03:14ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">There is a need for the normal error-path to properly dereference and cleanup transactions so that exceptional failures within the error-path can be logged without being lost in 'noise'.
</code></pre> Mulgara - Feature #27 (New): Eliminate all use of static variables from Mulgarahttps://code.mulgara.org/issues/272006-10-16T07:13:57ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">The use of static (ie. global) variables prevents Mulgara being restarted within a jvm. These are not necessary, and are being used to avoid considering where objects belong within the system.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Note this does not apply to static *constants* - being compile-time immutable values they are not an issue.
</code></pre> Mulgara - Feature #22 (New): Make from clause optionalhttps://code.mulgara.org/issues/222006-09-26T04:13:24ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">Consider the following query:
<span class="nt"><br/></span>
<span class="nt"><br/></span>
select $type
<span class="nt"><br/></span>
(select $s from <span class="ni">&lt;</span>server<span class="ni">&gt;</span> where $s <span class="ni">&lt;</span>rdf:type<span class="ni">&gt;</span> $type and $s <span class="ni">&lt;</span>p<span class="ni">&gt;</span> <span class="ni">&lt;</span>o<span class="ni">&gt;</span>)
<span class="nt"><br/></span>
from <span class="ni">&lt;</span>server<span class="ni">&gt;</span>
<span class="nt"><br/></span>
where $type <span class="ni">&lt;</span>mulgara:is<span class="ni">&gt;</span> <span class="ni">&lt;</span>test:Class<span class="ni">&gt;</span> ;
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Mandating from clauses is regularly redundant, and clutters the query.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
While from is the most common example, I have come across examples for every component of a itql query where they are redundant. Apart from mandating the keyword 'select', to identify it as a query, we should probably make the rest of the query should be optional.
<span class="nt"><br/></span>
</code></pre> Mulgara - Feature #21 (New): HybridTuples should defer sort and support prefix-define/reresolvehttps://code.mulgara.org/issues/212006-09-26T03:51:34ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">Currently [[HybridTuples]] performs it's sort when it is constructed. This forces the evaluation of its argument before join-optimisation has an opportunity to specify a desired prefix or reresolve. [[HybridTuples]] should defer it's sort until at least the first call to beforeFirst.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Considerations:
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Reresolve - If [[HybridTuples]] argument is Reresolvable [[HybridTuples]] should support passing this argument through.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
[[DefinablePrefix]] - If our argument is unordered then a projection is free, so we should expose this to the join optimisation. However note that if we are only sorting once, then we cannot pass this call through to the argument. Rather we should project the argument to maximise the prefix length available.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Single vs. Multiple sorts - A sort is O(nlogn), and a n from m prefix is O((0 from m)^((n - m) / m)). If we can increase the prefix length sufficiently to pay for the cost of the extra sorts we should resort on each call to beforeFirst(). Note that if the argument is itself prefix-definable then the cost of a sort is in terms of the estimated reduced length.
</code></pre> Mulgara - Feature #20 (New): Provide full client-side jrdf supporthttps://code.mulgara.org/issues/202006-09-14T04:54:08ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">Depends on fixing transactions to support read-only transactions.
</code></pre> Mulgara - Feature #18 (New): Provide a language independent interface to Mulgara compatible with ...https://code.mulgara.org/issues/182006-09-14T04:38:41ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">We need to support access to the entire Session API from languages other than java.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Both xml-rpc and REST have been proposed as alternatives. OTOH we should probably support both.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
XMLRPC because of it's wide language support and classic-rpc model.
<span class="nt"><br/></span>
REST because it is a better fit to W3C architecture, and provides benefits when working with loosely coupled distribution.
</code></pre> Mulgara - Feature #16 (In Progress): Add Rules for OWL DL/Full supporthttps://code.mulgara.org/issues/162006-09-14T04:28:54ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">Extend the new Rules Engine to support OWL Full.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Provide rule sets for OWL DL and OWL Full.
</code></pre> Mulgara - Bug #15 (In Progress): Product of Sum form has very poor performance.https://code.mulgara.org/issues/152006-09-06T04:30:23ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">[[OrderedAppend]] currently assumes that all it's operands share a common ordering. This is currently guarenteed by [[TuplesOperations]].append(), but the use of calls to project() on all operands not matching the first's variable list. This has signifigant performance ramifications as the desired ordering is unknown until any parent-join is optimised.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Without the projections currently done by [[TuplesOperations]].append(), we need new logic in [[OrderedAppend]] to handle mapping variables to operands columns. This is not a problem unless there the disjunction is subject to a join containing a left-bound prefix including mismatched variables.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Consider the following tuple expressions:
<span class="nt"><br/></span>
<span class="nt"><br/></span>
A[$z] ^ B( C[$z $y] v D[$y $z] )
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Note:
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>A[] will provide $z prefix to B[].
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>B[] currently passes this on blindly to C[] and D[].
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>This will bind $z for $y in D[], leading to an incorrect result.
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>Currently the non-union compatible
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>Fixing this will require either reordering D[], or filtering it; deciding between them is a performance optimisation issue.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
A[$y $z] ^ B(C[$y] v D[$z])
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Note:
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>Here there is a full prefix provided by A[] to B[].
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>The prefix needs to be decomposed for C[] and D[]
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>This is a case of non-union compatible disjunction, and will probably result in UNBOUND's.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Cases where this is probably a problem:
<span class="nt"><br/></span>
<span class="nt"><br/></span>
non-symmetric sum-of-products (ie. {$a <span class="ni">&lt;</span>foo<span class="ni">&gt;</span> $b ^ $b <span class="ni">&lt;</span>bar<span class="ni">&gt;</span> $c in <span class="ni">&lt;</span>m1<span class="ni">&gt;</span>} v {$b <span class="ni">&lt;</span>bar<span class="ni">&gt;</span> $c ^ $a <span class="ni">&lt;</span>foo<span class="ni">&gt;</span> $b in <span class="ni">&lt;</span>m2<span class="ni">&gt;</span>} )
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>Note: this is one of the key areas of concern. Differences between models can cause join-optimisation to generate a different ordering; This interferes with attempts to improve SOP performance which is required to support efficient views.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
symmetric disjunctions: (ie. $s <span class="ni">&lt;</span>foo<span class="ni">&gt;</span> $o v $o <span class="ni">&lt;</span>bar<span class="ni">&gt;</span> $s). - uncommon.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
non-union compatible disjunction (ie. $s <span class="ni">&lt;</span>name<span class="ni">&gt;</span> $name v $s <span class="ni">&lt;</span>email<span class="ni">&gt;</span> $email)
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>Note: this query is better phrased as a subquery anyway.
</code></pre>