https://code.mulgara.org/https://code.mulgara.org/favicon.ico?15861924492006-07-11T14:39:58ZMulgara ProjectMulgara - Bug #8: Temp directory managementhttps://code.mulgara.org/issues/8?journal_id=992006-07-11T14:39:58ZPaula Gearon
<ul></ul><pre>
<code class="html syntaxhl">Queries create temporary files when constraint resolutions get too large to manage in memory. The result of a query may be small, but the results of individual constraints can be quite large, particularly when a lot of data has been loaded. So it may be these constraint resolution files at fault.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
It would be worth testing if these files are being removed in a timely manner. I suggest adding an environment variable to override the high watermark for in-memory processing to a lower level, and then resolve several constraints which exceed this level.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
The other thing to consider is how these files are being accessed. If they are managed through normal I/O calls, then the close() on the file should delete it (if if was opened as a <span class="ni">&quot;</span>temporary<span class="ni">&quot;</span> file). However, if they are being memory mapped, then we need to ensure that all references to the mapping are set to null. We can even use a System.gc() loop if we really need to make sure the file has gone (but that should be done as a last resort).
</code></pre> Mulgara - Bug #8: Temp directory managementhttps://code.mulgara.org/issues/8?journal_id=932006-07-11T14:47:44Zbrian -brian@foo.bar
<ul></ul><pre>
<code class="html syntaxhl">It wouldn't have been the case on Solaris, but I know that there are problems with temp files not be deleted on Windows until the VM exits without a fair amount of nonsense. We should probably build some temp dir management tests into the suite (or extend any that are there) so we can easily catch these issues.
</code></pre> Mulgara - Bug #8: Temp directory managementhttps://code.mulgara.org/issues/8?journal_id=872006-09-06T04:43:27ZAndrae Muys -andrae@netymon.com
<ul></ul><pre>
<code class="html syntaxhl">Some detailed bug reports would be useful, including the queries causing the trouble, and the count() attached to individual constraints.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
There are some places where we might be performing distinct() or sort() more agressively then we strictly need to - these being the operations that generate temporary files. However most of these calls are no-ops as once an intermediate result is sorted, subsequent calls to sort/distinct involving that data tend to become no-ops.
<span class="nt"><br/></span>
<span class="nt"><br/></span>
To track this down, enable logging in [[HybridTuples]] and log both the file and it's provenance for each instance (watch for clones, naturally they share the same file).
</code></pre>