Home » RDBMS Server » Performance Tuning » POWER function slows query way down? (9.2.0.8.0)
POWER function slows query way down? [message #523291] |
Fri, 16 September 2011 10:14 ![Go to next message Go to next message](/forum/theme/orafaq/images/down.png) |
![](/forum/images/custom_avatars/56289.jpg) |
Kevin Meade
Messages: 2103 Registered: December 1999 Location: Connecticut USA
|
Senior Member |
|
|
I have a query that without a few lines that use the POWER function, will complete in 3 minutes. If I add even one line using POWER, the query slows way down and takes 4+ hours to run. Is this a problem with the basic function?, a known bug?. Is there some kind of workaround you guys can think of? Anybody got ideas on what I might do? Is there an alternative to POWER for these compound interest calculations?
Kevin
with
rest_of_query as (
select
a.policy_number
, a.caseid
, a.funding_method_cd
, a.CLMEVNTID
, sum(a.PAID_AMT) paid_amt
, a.disability_dt
--
-- problem here with POWER
-- these lines cause the query to not finish
-- goes from 3 minutes to 4+ hours
--
, sum(a.PAID_AMT * POWER ( 1 + Y.GAAP, a.mnths_btwn/12 )) gaap_x
, sum(a.PAID_AMT * POWER ( 1 + Y.PRICING, a.mnths_btwn/12 )) pricing_x
, sum(a.PAID_AMT * POWER ( 1 + Y.STAT, a.mnths_btwn/12 )) stat_x
, sum(a.PAID_AMT * POWER ( 1 + Y.UW, a.mnths_btwn/12 )) uw_x
, sum(a.PAID_AMT * POWER ( 1 + Y.VALUATION, a.mnths_btwn/12 )) valuation_x
from
kevtemp1 a
, (select * from experience.EXP_DRIVING_MONTH where rownum = 1) e
, experience.dlr_discount Y
where a.RPLCTNDT_calculation <= e.driving_date
and y.YEAR = TO_CHAR (a.disability_dt,'YYYY')
group by
a.policy_number
, a.caseid
, a.funding_method_cd
, a.CLMEVNTID
, a.disability_dt
)
select *
from rest_of_query
/
|
|
|
|
Re: POWER function slows query way down? [message #523298 is a reply to message #523295] |
Fri, 16 September 2011 10:48 ![Go to previous message Go to previous message](/forum/theme/orafaq/images/up.png) ![Go to next message Go to next message](/forum/theme/orafaq/images/down.png) |
![](/forum/images/custom_avatars/56289.jpg) |
Kevin Meade
Messages: 2103 Registered: December 1999 Location: Connecticut USA
|
Senior Member |
|
|
Try this for a starter. This is the simplest example I could create. Notice how a simple count goes up in cost from .04 seconds to 5 seconds. All we did was a simple calculation using POWER.
create table kevtemp9
(
anumber number not null
, bnumber number not null
)
/
insert into kevtemp9
select mod(rownum,100)/10000,mod(rownum,33)
from dual
connect by level <= 20000
/
select * from kevtemp9 where rownum < 10
/
select count(*) from kevtemp9
/
select count(power(1+anumber,bnumber/12)) from kevtemp9
/
Here are my timings. You can adjust the number of rows as you want.
11:44:53 SQL> create table kevtemp9
11:44:53 2 (
11:44:53 3 anumber number not null
11:44:53 4 , bnumber number not null
11:44:53 5 )
11:44:53 6 /
Table created.
Elapsed: 00:00:00.01
11:44:53 SQL>
11:44:53 SQL> insert into kevtemp9
11:44:53 2 select mod(rownum,100)/10000,mod(rownum,33)
11:44:53 3 from dual
11:44:53 4 connect by level <= 20000
11:44:53 5 /
20000 rows created.
Elapsed: 00:00:00.04
11:44:54 SQL>
11:44:54 SQL>
11:44:54 SQL> select * from kevtemp9 where rownum < 10
11:44:54 2 /
ANUMBER BNUMBER
---------- ----------
.0091 31
.0092 32
.0093 0
.0094 1
.0095 2
.0096 3
.0097 4
.0098 5
.0099 6
9 rows selected.
Elapsed: 00:00:00.01
11:44:54 SQL>
11:44:54 SQL> select count(*) from kevtemp9
11:44:54 2 /
COUNT(*)
----------
20000
1 row selected.
Elapsed: 00:00:00.01
11:44:54 SQL>
11:44:54 SQL> select count(power(1+anumber,bnumber/12)) from kevtemp9
11:44:54 2 /
COUNT(POWER(1+ANUMBER,BNUMBER/12))
----------------------------------
20000
1 row selected.
Elapsed: 00:00:05.01
11:44:59 SQL>
Kevin
|
|
|
Re: POWER function slows query way down? [message #523302 is a reply to message #523298] |
Fri, 16 September 2011 11:19 ![Go to previous message Go to previous message](/forum/theme/orafaq/images/up.png) ![Go to next message Go to next message](/forum/theme/orafaq/images/down.png) |
ThomasG
Messages: 3212 Registered: April 2005 Location: Heilbronn, Germany
|
Senior Member |
|
|
Well, POWER of course needs more CPU than no calculation.
From 10.2.0.4:
PFK@test> select count(*) from kevtemp9
/
COUNT(*)
----------
20000
Elapsed: 00:00:00.04
select count(power(1+anumber,bnumber/12)) from kevtemp9
/
COUNT(POWER(1+ANUMBER,BNUMBER/12))
----------------------------------
20000
Elapsed: 00:00:01.38
TKPROF output:
select count(*)
from
kevtemp9
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.01 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 2 0.00 0.00 0 38 0 1
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 4 0.00 0.01 0 38 0 1
Misses in library cache during parse: 1
Optimizer mode: CHOOSE
Parsing user id: 42
Rows Row Source Operation
------- ---------------------------------------------------
1 SORT AGGREGATE (cr=38 pr=0 pw=0 time=2280 us)
20000 TABLE ACCESS FULL KEVTEMP9 (cr=38 pr=0 pw=0 time=114 us)
********************************************************************************
select count(power(1+anumber,bnumber/12))
from
kevtemp9
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 2 1.37 1.34 0 38 0 1
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 4 1.37 1.34 0 38 0 1
Misses in library cache during parse: 1
Optimizer mode: CHOOSE
Parsing user id: 42
Rows Row Source Operation
------- ---------------------------------------------------
1 SORT AGGREGATE (cr=38 pr=0 pw=0 time=1347071 us)
20000 TABLE ACCESS FULL KEVTEMP9 (cr=38 pr=0 pw=0 time=36 us)
The funny thing is this:
declare
i number;
x number;
begin
for i in 1..20000 loop
x := power (i,10);
end loop;
end;
/
PL/SQL procedure successfully completed.
Elapsed: 00:00:00.05
declare
i number;
x number;
begin
for i in 1..20000 loop
x := power (i,10.54146415412);
end loop;
end;
/
PL/SQL procedure successfully completed.
Elapsed: 00:00:03.09
So it seems power needs longer the more complicated the numbers get.
|
|
|
|
|
Re: POWER function slows query way down? [message #523311 is a reply to message #523306] |
Fri, 16 September 2011 12:13 ![Go to previous message Go to previous message](/forum/theme/orafaq/images/up.png) ![Go to next message Go to next message](/forum/theme/orafaq/images/down.png) |
ThomasG
Messages: 3212 Registered: April 2005 Location: Heilbronn, Germany
|
Senior Member |
|
|
He. Would be nice to know if the "build in" power function is completely C++, or if it's already implemented in PL/SQL.
Is it perhaps possible to pull the power call further up in the sum? Basically
sum(a * power(x,y))
be the same as
sum(a) * power (x,y)
IF x and y stay the same. Don't know if that is the case in the data model, though.
|
|
|
|
Re: POWER function slows query way down? [message #523332 is a reply to message #523312] |
Fri, 16 September 2011 14:44 ![Go to previous message Go to previous message](/forum/theme/orafaq/images/up.png) ![Go to next message Go to next message](/forum/theme/orafaq/images/down.png) |
ThomasG
Messages: 3212 Registered: April 2005 Location: Heilbronn, Germany
|
Senior Member |
|
|
Thanks for the flowers.
I just tried it doing a power calculation in perl, to have some sort of comparison.
tmp > cat test.pl
#!/bin/perl
for ( $i =0; $i < 20000 ; $i++ ) {
$e = $i ** 10.33535334646;
}
tmp > time perl test.pl
real 0m0.008s
user 0m0.003s
sys 0m0.003s
So it really might indeed be worthwhile to try if a java stored procedure that does a power is faster than the Oracle build in one. Since it really seems to be quite slow compared to other implementations on the same machine.
Or write an intermediate file and do the power calculation in perl.
|
|
|
Re: POWER function slows query way down? [message #523340 is a reply to message #523291] |
Fri, 16 September 2011 15:41 ![Go to previous message Go to previous message](/forum/theme/orafaq/images/up.png) ![Go to next message Go to next message](/forum/theme/orafaq/images/down.png) |
![](/forum/images/custom_avatars/56289.jpg) |
Kevin Meade
Messages: 2103 Registered: December 1999 Location: Connecticut USA
|
Senior Member |
|
|
Thanks everyone. I think I am all set. Here is an even smaller test case. I think my solution is simply to wait for the upgrade to 11gR2. I am headed for EXADATA in a couple of weeks and tests there are very encouraging. I even ran the original query on EXADATA and got the expected timings, not the forever timings.
9i database
SQL> with
2 some_data as (
3 select mod(rownum,100)/10000 anumber,mod(rownum,33) bnumber
4 from dual
5 connect by level <= 20000
6 )
7 select count(*) from some_data
8 /
COUNT(*)
----------
20000
1 row selected.
Elapsed: 00:00:00.02
SQL> with
2 some_data as (
3 select mod(rownum,100)/10000 anumber,mod(rownum,33) bnumber
4 from dual
5 connect by level <= 20000
6 )
7 select count(power(1+anumber,bnumber/12)) from some_data
8 /
COUNT(POWER(1+ANUMBER,BNUMBER/12))
----------------------------------
20000
1 row selected.
Elapsed: 00:00:05.03 11g EXADATA2
SQL> with
2 some_data as (
3 select mod(rownum,100)/10000 anumber,mod(rownum,33) bnumber
4 from dual
5 connect by level <= 20000
6 )
7 select count(*) from some_data
8 /
COUNT(*)
----------
20000
1 row selected.
Elapsed: 00:00:00.01
SQL> with
2 some_data as (
3 select mod(rownum,100)/10000 anumber,mod(rownum,33) bnumber
4 from dual
5 connect by level <= 20000
6 )
7 select count(power(1+anumber,bnumber/12)) from some_data
8 /
COUNT(POWER(1+ANUMBER,BNUMBER/12))
----------------------------------
20000
1 row selected.
Elapsed: 00:00:00.04
Quote:ALL HAIL THE EXADATA
As usual, OraFAQ has been faster than Oracle Service Requests.
Kevin
|
|
|
Re: POWER function slows query way down? [message #523364 is a reply to message #523340] |
Sat, 17 September 2011 02:54 ![Go to previous message Go to previous message](/forum/theme/orafaq/images/up.png) ![Go to next message Go to next message](/forum/theme/orafaq/images/down.png) |
John Watson
Messages: 8965 Registered: January 2010 Location: Global Village
|
Senior Member |
|
|
Kevin, is it possible for you to run the query again on the Exadata DB Machine? It would be interesting to see if the performance is because the cell offload processing is working, or if it is merely because Exadata can serve blocks more quickly than conventional storage. I have found several circumstances where offload processing is not as efficient as it should be.
Are you familiar with how to get this information?
John.
|
|
|
Re: POWER function slows query way down? [message #523369 is a reply to message #523364] |
Sat, 17 September 2011 04:17 ![Go to previous message Go to previous message](/forum/theme/orafaq/images/up.png) ![Go to next message Go to next message](/forum/theme/orafaq/images/down.png) |
John Watson
Messages: 8965 Registered: January 2010 Location: Global Village
|
Senior Member |
|
|
If you have the time, Kevin, what I would be interested in seeing is the results of a test something like the following, which tries to confirm whether the POWER function is in fact being evaluated more efficiently by the Exadata layer than it can be evaluated by the DB layer. Your performance improvement could be due to so many things: the 11.2 upgrade, Exadata offload processing, better DB server hardware, the Exadata storage, anything. I don't know that this test is valid, but it might be interesting. I'm trying to isolate some of the factors.
--make sure we do direct reads
alter session set "_serial_direct_read"=always
--check that the power function is offloadable
select offloadable from v$sqlfn_metadata where name='POWER';
--display whether the step COULD (not necessarily WOULD !) be offloaded
alter session set cell_offload_plan_display=always;
set autotrace on exp
select count(power(1+anumber,bnumber/12)) from kevtemp9;
select count(*) from kevtemp9;
set autotrace off
--check the timings with offload
set timing on
alter system flush shared_pool;
alter system flush buffer_cache;
alter session set cell_offload_processing=true;
select count(*) from kevtemp9;
alter system flush buffer_cache;
select count(power(1+anumber,bnumber/12)) from kevtemp9;
--did we actually get any offload?
select
sql_text,
physical_Read_bytes,
io_cell_offload_eligible_bytes,
io_cell_offload_returned_bytes
from v$sql
where sql_text='select count(power(1+anumber,bnumber/12)) from kevtemp9';
select
sql_text,
physical_Read_bytes,
io_cell_offload_eligible_bytes,
io_cell_offload_returned_bytes
from v$sql
where sql_text='select count(*) from kevtemp9';
--and now without offload
alter session set cell_offload_processing=false;
alter system flush shared_pool;
alter system flush buffer_cache;
select count(*) from kevtemp9;
alter system flush buffer_cache;
select count(power(1+anumber,bnumber/12)) from kevtemp9;
--did we actually get any offload?
select
sql_text,
physical_Read_bytes,
io_cell_offload_eligible_bytes,
io_cell_offload_returned_bytes
from v$sql
where sql_text='select count(power(1+anumber,bnumber/12)) from kevtemp9';
select
sql_text,
physical_Read_bytes,
io_cell_offload_eligible_bytes,
io_cell_offload_returned_bytes
from v$sql
where sql_text='select count(*) from kevtemp9';
|
|
|
|
Re: POWER function slows query way down? [message #523407 is a reply to message #523400] |
Sat, 17 September 2011 14:42 ![Go to previous message Go to previous message](/forum/theme/orafaq/images/up.png) ![Go to next message Go to next message](/forum/theme/orafaq/images/down.png) |
John Watson
Messages: 8965 Registered: January 2010 Location: Global Village
|
Senior Member |
|
|
I guess I didn't explain properly what I'm thinking.
The evaluation of the POWER function should be offloadable to the Exadata tier, so the database won't actually evaluate it at all: the data returned by the Exadata cell will be the final value, with the power already computed. The Exadata software is not an Oracle instance, so it would seem likely that the implementation of the power function is totally different. That MIGHT be why the performance is so much better.
But what I'm also thinking is that you could get a very different result with a more complex example where the computation of the power can't always be offloaded. There are a worrying number of circumstances (which can change from one run of the query to another) where offload can't be done, which can give seriously erratic performance. So I was hoping to determine just where the power is being evaluated.
If you can run my test, or variations thereof, I would really like to see the results.
(ps - I've just ordered a copy of this,
http://www.amazon.co.uk/Expert-Oracle-Exadata-Kerry-Osborne/dp/1430233923
I have a colleague who knows the guys that wrote it, and he tells me it is excellent)
|
|
|
Re: POWER function slows query way down? [message #523433 is a reply to message #523407] |
Sun, 18 September 2011 11:51 ![Go to previous message Go to previous message](/forum/theme/orafaq/images/up.png) |
![](/forum/images/custom_avatars/56289.jpg) |
Kevin Meade
Messages: 2103 Registered: December 1999 Location: Connecticut USA
|
Senior Member |
|
|
OK, I just bought it. 76$. Hoping it is more than just a recapitulation of the manuals. thanks buddy.
And I see what you are getting at. You are not telling me that the thing went faster on EXADATA. You are pointing at that if EXADATA did to the work, then I could someday end up with performance tanking on me in other situations where the work is not passed off to EXADATA. That is a good thing to point out.
Let me get on running the stuff you provided.
Kevin
OOPs just checked EBAY. There is a US seller offering it delievered for 25$less. Oh well...
[Updated on: Sun, 18 September 2011 11:55] Report message to a moderator
|
|
|
Goto Forum:
Current Time: Sun Feb 16 08:26:06 CST 2025
|