(Patch) query speed improvement by 30% - 40%
Creating exceptions (instances of Throwable) is an expensive <br/> operation, mostly because of the stack trace. However, browsing the <br/> code I've seen a number of places where 'new Throwable()' was used <br/> unconditionally. Therefore I ran some experiments. <br/> <br/> Setup: a db with 1.4 million triples (production data), and 200 <br/> "production" queries (most of them unique). Each test was run by <br/> starting mulgara, running the first 100 queries, then timing the <br/> second 100 queries, and finally shutting mulgara down again. The <br/> log level was WARN, so that normally nothing gets printed. No <br/> explicit transactions were used (i.e. autoCommit=true). <br/> <br/> Some queries were simple (on the order of 50 ms or less to answer), <br/> some more complex (close to a second); the client was using SOAP to <br/> access the db, which introduces typically between 40 and 100 ms <br/> overhead per query, so these results are "worst case", i.e. the <br/> speed-up noted is probably slightly higher using RMI. <br/> <br/> Three sets of tests were run: straight rev <a href="http://mulgara.org/trac/changeset/192">192</a> ("with stack traces"), <br/> a version with all 'new Throwable()' removed ("without stack traces"), <br/> and a version with various 'new Throwable()' surounded by appropriate <br/> conditions ("partial stack traces"); the patch for this last set is <br/> the one attached to this issue. <br/> <br/> Each test was run with 1, 2, and 3 client threads hitting the db. <br/> <br/> Here are the results (the alignment of the table will probably be <br/> messed up in the html): <br/> <br/> stack-traces threads times (sec) avg min avg% min% <br/> <br/> with 1 29.2 31.4 29.0 29.9 29.0 100 100 <br/> with 2 18.7 19.5 18.1 18.8 18.1 100 100 <br/> with 3 17.7 17.6 19.2 18.2 17.6 100 100 <br/> <br/> without 1 17.7 17.7 18.1 17.8 17.7 60 61 <br/> without 2 13.0 12.5 13.1 12.9 12.5 69 69 <br/> without 3 12.6 12.2 12.5 12.4 12.2 68 69 <br/> <br/> partial 1 18.0 18.6 18.0 18.2 18.0 61 62 <br/> partial 2 12.5 12.9 13.3 12.9 12.5 69 69 <br/> partial 3 12.8 11.2 13.1 12.4 11.2 68 64 <br/> <br/> As you can see, the "partial" (which is the attached patch) gives <br/> around 30% - 40% reduction in query time. <br/> <br/> The attached patch is quite simple and quite safe. In a couple <br/> cases it makes the Throwable instantiation conditional on the log <br/> level at which it will get printed, and in the other two cases it <br/> removes unnecessary (i.e. duplicate) stack-trace creation. In <br/> short it does not remove any information from the logs. <br/> <br/> No tests are affected by this patch. <br/>
Updated by Andrae Muys - about 16 years ago
I won't have time to integrate this until next week, but I'll do it first-up if Paul hasn't had a chance to get to it first. <br/> <br/> Nice catch - I like those numbers.
Updated by ronald - about 16 years ago