Mulgara Project: Issueshttps://code.mulgara.org/https://code.mulgara.org/favicon.ico?15861924492007-04-15T05:55:17ZMulgara Project
Redmine Mulgara - Bug #56 (Feedback): Distributed blank nodes not uniquely identifiedhttps://code.mulgara.org/issues/562007-04-15T05:55:17ZPaula Gearon
<pre>
<code class="html syntaxhl">Any blank nodes returned from a distributed resolver may contain the same gnode IDs as local blank nodes. This means they cannot be distinguished. Add a factory in the appropriate place to mark blank nodes from elsewhere with the IP of their source.
</code></pre> Mulgara - Bug #53 (New): FileSystemResolver keeps exhausting heaphttps://code.mulgara.org/issues/532007-03-20T03:51:53ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">Regularly seeing heap exhaustion when running the [[FileSystemResolver]] unit test.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
I believe this resolver is memory bound and needs to be modified to use disk-backed structures.
</code></pre> Mulgara - Bug #52 (New): Insufficient dereferencing in TransactionalAnswer.sessionClose leading t...https://code.mulgara.org/issues/522007-03-19T07:03:14ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">There is a need for the normal error-path to properly dereference and cleanup transactions so that exceptional failures within the error-path can be logged without being lost in 'noise'.
</code></pre> Mulgara - Feature #49 (New): Create a SOAP endpoint on the serverhttps://code.mulgara.org/issues/492007-03-01T03:56:01ZPaula Gearon
<pre>
<code class="html syntaxhl">Currently, the most common way for non-Java clients to connect to Mulgara is via SOAP. This is done by wrapping the itql.jar in a SOAP endpoint, and calling that. This gives a topology of:
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Client <span class="ni">&lt;</span>---[SOAP]---<span class="ni">&gt;</span> Itql.jar <span class="ni">&lt;</span>---[RMI]---<span class="ni">&gt;</span> Mulgara
<span class="nt"><br/></span>
<span class="nt"><br/></span>
This double-network hop is horrible and inefficient. It also has an unfortunate dependence on RMI.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Until we have a good HTTP solution (via [[NetKernel]]) it would be far preferable to allow the following:
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Client <span class="ni">&lt;</span>---[SOAP]---<span class="ni">&gt;</span> Mulgara
<span class="nt"><br/></span>
<span class="nt"><br/></span>
This should not be too hard. The main thing that the server needs is a means of converting iTQL queries from the client into Query objects used by the session. Code already exists in [[ItqlInterpreter]] to do this, but it will to be decoupled from remote session creation (since it will now be done locally, and no finding of remote resources is necessary).
<span class="nt"><br/></span>
<span class="nt"><br/></span>
The other thing to do is:
<span class="nt"><br/></span>
- Provide SOAP access to the Answer object. This will help get it running, but is not suitable for real work (since SOAP calls would be needed to read every element of every line).
<span class="nt"><br/></span>
- Convert small Answers into a DOM for returning.
<span class="nt"><br/></span>
- Configure a streaming XML result set suitable to be parsed via SAX. Hopefully this can be done?
</code></pre> Mulgara - Bug #31 (New): resolveVariableModel does not check securityAdaptorListhttps://code.mulgara.org/issues/312006-11-10T07:01:02ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">This allows queries containing variable models to bypass the security layer.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Mulgara currently doesn't have any security modules in its distribution - so this code path is currently only used by various unit-tests. This bug must be fixed when we come to implement security modules.
</code></pre> Mulgara - Feature #27 (New): Eliminate all use of static variables from Mulgarahttps://code.mulgara.org/issues/272006-10-16T07:13:57ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">The use of static (ie. global) variables prevents Mulgara being restarted within a jvm. These are not necessary, and are being used to avoid considering where objects belong within the system.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Note this does not apply to static *constants* - being compile-time immutable values they are not an issue.
</code></pre> Mulgara - Feature #26 (New): Remove getElement() from the Constraint interface - An easy fix, no ...https://code.mulgara.org/issues/262006-10-03T06:30:07ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">Back in the distant past when the only type of Constraint supported was what we now refer to as [[ConstraintImpl]], getElement() was an ugly, but harmless way of obtaining the different elements of a constraint.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Now that we have dozens of different constraint types, the vast majority of which have to implement getElement() by throwing an exception, this is not so harmless.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
It would be great if someone could go through the codebase and:
<span class="nt"><br/></span>
<span class="nt"><br/></span>
1) Add getSubject(), getPredicate(), and getObject() to [[ConstraintImpl]]
<span class="nt"><br/></span>
2) Find each call to [[ConstraintImpl]]::getElement(), and replace with the appropriate call (element 0-<span class="ni">&gt;</span>subject, 1-<span class="ni">&gt;</span>predicate, 2-<span class="ni">&gt;</span>object; 3-<span class="ni">&gt;</span>model but that should already have been done).
<span class="nt"><br/></span>
3) Repeat for [[ConstraintNegation]] (and any other constraint type that implements getElement(), but IIRC there aren't any).
<span class="nt"><br/></span>
4) Remove getElement() from the Constraint interface and implementing classes thereby banishing this aberration.
</code></pre> Mulgara - Feature #23 (New): We must find a more descriptive way of expressing the cost of iterat...https://code.mulgara.org/issues/232006-09-26T11:08:34ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">An increasing number of resolvers (relational, federating, distributed) cannot provide a meaningful upper-bound to their row-count.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Also row-count assumes a uniform cost of iteration, when iteration (as opposed to resolution) entails network latency this is unrealistic.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
How do we handle this? Do we need some sort of bogomips-like timing constant to provide a baseline for comparison? Can we use this to also provide selectivity data that would also improve our join performance?
</code></pre> Mulgara - Feature #22 (New): Make from clause optionalhttps://code.mulgara.org/issues/222006-09-26T04:13:24ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">Consider the following query:
<span class="nt"><br/></span>
<span class="nt"><br/></span>
select $type
<span class="nt"><br/></span>
(select $s from <span class="ni">&lt;</span>server<span class="ni">&gt;</span> where $s <span class="ni">&lt;</span>rdf:type<span class="ni">&gt;</span> $type and $s <span class="ni">&lt;</span>p<span class="ni">&gt;</span> <span class="ni">&lt;</span>o<span class="ni">&gt;</span>)
<span class="nt"><br/></span>
from <span class="ni">&lt;</span>server<span class="ni">&gt;</span>
<span class="nt"><br/></span>
where $type <span class="ni">&lt;</span>mulgara:is<span class="ni">&gt;</span> <span class="ni">&lt;</span>test:Class<span class="ni">&gt;</span> ;
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Mandating from clauses is regularly redundant, and clutters the query.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
While from is the most common example, I have come across examples for every component of a itql query where they are redundant. Apart from mandating the keyword 'select', to identify it as a query, we should probably make the rest of the query should be optional.
<span class="nt"><br/></span>
</code></pre> Mulgara - Feature #21 (New): HybridTuples should defer sort and support prefix-define/reresolvehttps://code.mulgara.org/issues/212006-09-26T03:51:34ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">Currently [[HybridTuples]] performs it's sort when it is constructed. This forces the evaluation of its argument before join-optimisation has an opportunity to specify a desired prefix or reresolve. [[HybridTuples]] should defer it's sort until at least the first call to beforeFirst.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Considerations:
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Reresolve - If [[HybridTuples]] argument is Reresolvable [[HybridTuples]] should support passing this argument through.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
[[DefinablePrefix]] - If our argument is unordered then a projection is free, so we should expose this to the join optimisation. However note that if we are only sorting once, then we cannot pass this call through to the argument. Rather we should project the argument to maximise the prefix length available.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Single vs. Multiple sorts - A sort is O(nlogn), and a n from m prefix is O((0 from m)^((n - m) / m)). If we can increase the prefix length sufficiently to pay for the cost of the extra sorts we should resort on each call to beforeFirst(). Note that if the argument is itself prefix-definable then the cost of a sort is in terms of the estimated reduced length.
</code></pre> Mulgara - Feature #20 (New): Provide full client-side jrdf supporthttps://code.mulgara.org/issues/202006-09-14T04:54:08ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">Depends on fixing transactions to support read-only transactions.
</code></pre> Mulgara - Feature #18 (New): Provide a language independent interface to Mulgara compatible with ...https://code.mulgara.org/issues/182006-09-14T04:38:41ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">We need to support access to the entire Session API from languages other than java.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Both xml-rpc and REST have been proposed as alternatives. OTOH we should probably support both.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
XMLRPC because of it's wide language support and classic-rpc model.
<span class="nt"><br/></span>
REST because it is a better fit to W3C architecture, and provides benefits when working with loosely coupled distribution.
</code></pre> Mulgara - Feature #16 (In Progress): Add Rules for OWL DL/Full supporthttps://code.mulgara.org/issues/162006-09-14T04:28:54ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">Extend the new Rules Engine to support OWL Full.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Provide rule sets for OWL DL and OWL Full.
</code></pre> Mulgara - Bug #15 (In Progress): Product of Sum form has very poor performance.https://code.mulgara.org/issues/152006-09-06T04:30:23ZAndrae Muys -andrae@netymon.com
<pre>
<code class="html syntaxhl">[[OrderedAppend]] currently assumes that all it's operands share a common ordering. This is currently guarenteed by [[TuplesOperations]].append(), but the use of calls to project() on all operands not matching the first's variable list. This has signifigant performance ramifications as the desired ordering is unknown until any parent-join is optimised.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Without the projections currently done by [[TuplesOperations]].append(), we need new logic in [[OrderedAppend]] to handle mapping variables to operands columns. This is not a problem unless there the disjunction is subject to a join containing a left-bound prefix including mismatched variables.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Consider the following tuple expressions:
<span class="nt"><br/></span>
<span class="nt"><br/></span>
A[$z] ^ B( C[$z $y] v D[$y $z] )
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Note:
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>A[] will provide $z prefix to B[].
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>B[] currently passes this on blindly to C[] and D[].
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>This will bind $z for $y in D[], leading to an incorrect result.
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>Currently the non-union compatible
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>Fixing this will require either reordering D[], or filtering it; deciding between them is a performance optimisation issue.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
A[$y $z] ^ B(C[$y] v D[$z])
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Note:
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>Here there is a full prefix provided by A[] to B[].
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>The prefix needs to be decomposed for C[] and D[]
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>This is a case of non-union compatible disjunction, and will probably result in UNBOUND's.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
Cases where this is probably a problem:
<span class="nt"><br/></span>
<span class="nt"><br/></span>
non-symmetric sum-of-products (ie. {$a <span class="ni">&lt;</span>foo<span class="ni">&gt;</span> $b ^ $b <span class="ni">&lt;</span>bar<span class="ni">&gt;</span> $c in <span class="ni">&lt;</span>m1<span class="ni">&gt;</span>} v {$b <span class="ni">&lt;</span>bar<span class="ni">&gt;</span> $c ^ $a <span class="ni">&lt;</span>foo<span class="ni">&gt;</span> $b in <span class="ni">&lt;</span>m2<span class="ni">&gt;</span>} )
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>Note: this is one of the key areas of concern. Differences between models can cause join-optimisation to generate a different ordering; This interferes with attempts to improve SOP performance which is required to support efficient views.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
symmetric disjunctions: (ie. $s <span class="ni">&lt;</span>foo<span class="ni">&gt;</span> $o v $o <span class="ni">&lt;</span>bar<span class="ni">&gt;</span> $s). - uncommon.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
non-union compatible disjunction (ie. $s <span class="ni">&lt;</span>name<span class="ni">&gt;</span> $name v $s <span class="ni">&lt;</span>email<span class="ni">&gt;</span> $email)
<span class="nt"><br/></span>
<span class="ni">&nbsp;&nbsp;</span>Note: this query is better phrased as a subquery anyway.
</code></pre> Mulgara - Bug #8 (New): Temp directory managementhttps://code.mulgara.org/issues/82006-07-11T13:50:53Zbrian -brian@foo.bar
<pre>
<code class="html syntaxhl">A research group has reported seeing Kowari fill up temp directories and fall over. This was on Solaris, but it might be a more general problem to solve. Nothing to reproduce it yet, but I just wanted to capture the experience to potentially investigate this issue moving forward.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
The usage pattern was one big load and then mostly queries with the occasional insert.
</code></pre>