Table occupying max size [message #506397] |
Mon, 09 May 2011 02:03  |
Its_me_ved
Messages: 979 Registered: October 2009 Location: India
|
Senior Member |
|
|
Objective : To find solution to archieve data from 2 big tables which is occupying maximum size in the data base. With current data (From Jan 2005 to Sept 2011) it has records as mentioned below:
transaction - 41687927
trnansaction_dtl - 83945934
We need to load data and run monthly batches from October 2011 to current month which will increase this space.
1. Issue is there will not be having so much space.
2. Maintenance of such table is diffcult now.Also there is huge impact on performance. Can we think of partitioning the table base on date aswe query 1st table based on certain date range?
3. Most of reports use this table and creating performances issues
Need some advise from you all.
Regards,
Ved
|
|
|
|
Re: Table occupying max size [message #506412 is a reply to message #506406] |
Mon, 09 May 2011 02:50   |
Its_me_ved
Messages: 979 Registered: October 2009 Location: India
|
Senior Member |
|
|
Thanks for the reply.
Any suggesion regarding archieving data from 2 big tables which is occupying maximum size in the data base.there are some sqls where it is picking all data from transaction table till date ( trn_dt< sysdate)
Regards
Ved
|
|
|
|
|
Re: Table occupying max size [message #506451 is a reply to message #506437] |
Mon, 09 May 2011 06:20   |
Yasir Hashmi
Messages: 304 Registered: April 2006
|
Senior Member |
|
|
Also there is huge impact on performance.
You ask for performance.Mvies can help in performance as they store processed data.As you mentioned the table is huge,Mviews could help definitly.Do you have proper indexes?Are all queries related to those tables are slow or few ones? How much big is your table?
For table above 2gb,partitioning is an option.
|
|
|
|
Re: Table occupying max size [message #506458 is a reply to message #506452] |
Mon, 09 May 2011 06:49   |
Yasir Hashmi
Messages: 304 Registered: April 2006
|
Senior Member |
|
|
Michel,
The table is huge.If we create Mviews based on the queries that rae used in the application with ofcourse enable query rewrite enabled,wouldn't the response time decrease (fixed by Michel was "increase")?
The tables size is approximately 15gb and 30gb.[wild guess]
If there are space concerns,why not add new hard disks?
[Updated on: Mon, 09 May 2011 07:14] by Moderator Report message to a moderator
|
|
|
|
|
|
Re: Table occupying max size [message #506469 is a reply to message #506397] |
Mon, 09 May 2011 07:10   |
John Watson
Messages: 8965 Registered: January 2010 Location: Global Village
|
Senior Member |
|
|
Ved, I'm going to suggest a different approach to any suggested so far.
Basically, 80 million rows is not very many. To attempt to improve performance by throwing money at the problem (ie, partitioning or faster discs) may not be the best option: I would try to tune the SQL instead. Start with the reports. What is the code, the DDL, and the execution plans? What is current reponse time, and what response time do you need?
|
|
|
|
|
Re: Table occupying max size [message #506479 is a reply to message #506474] |
Mon, 09 May 2011 07:20   |
Yasir Hashmi
Messages: 304 Registered: April 2006
|
Senior Member |
|
|
Hmmm,
Well,obviously oracle only understands SQL.So there has to have sqls hidden inside reports.
So for a faster response, why can't mviews help?If not what is there purpose?
From the OP,it seems the queries were fast before but with increased data,they have become slow.So the query might be tuned to its full potential but yet the response time is high.
Why not take the advantage of Mviews? When if not now?
|
|
|
|
Re: Table occupying max size [message #506482 is a reply to message #506480] |
Mon, 09 May 2011 07:24   |
 |
Michel Cadot
Messages: 68737 Registered: March 2007 Location: Saint-Maur, France, https...
|
Senior Member Account Moderator |
|
|
Michel Cadot wrote on Mon, 09 May 2011 13:33Quote:...
Quote:there are some sqls where it is picking all data from transaction table till date ( trn_dt< sysdate)
How do you use Mview for this?
...
You didn't answer to that.
Regards
Michel
[Updated on: Mon, 09 May 2011 08:01] Report message to a moderator
|
|
|
Re: Table occupying max size [message #506485 is a reply to message #506482] |
Mon, 09 May 2011 07:59   |
Yasir Hashmi
Messages: 304 Registered: April 2006
|
Senior Member |
|
|
create materialized view m_mytable enable query rewrite
as select * from mytable ;
Won't it use m_mytable mview when queried as
select * from m_mytable where till_date<sysdate
It should use Mview and i hope the response time to be faster.
|
|
|
|
|
|
|
|
|
|
Re: Table occupying max size [message #506498 is a reply to message #506494] |
Mon, 09 May 2011 08:42   |
John Watson
Messages: 8965 Registered: January 2010 Location: Global Village
|
Senior Member |
|
|
Example:
orcl> create materialized view mv1 as select * from hr.regions;
Materialized view created.
orcl> select object_name,object_type from user_objects where object_name='MV1';
OBJECT_NAME OBJECT_TYPE
-------------------- -------------------
MV1 TABLE
MV1 MATERIALIZED VIEW
orcl> select segment_name,segment_type from user_segments where segment_name='MV1';
SEGMENT_NAME SEGMENT_TYPE
-------------------- ------------------
MV1 TABLE
orcl>
|
|
|
Re: Table occupying max size [message #506569 is a reply to message #506494] |
Tue, 10 May 2011 00:31   |
Yasir Hashmi
Messages: 304 Registered: April 2006
|
Senior Member |
|
|
I doubt the query would be as simple as select * from mytable.
Imagine this:
SQL> set autotrace traceonly statistics
SQL> select t_code,count(*) from pm group by t_code order by 1;
Statistics
----------------------------------------------------------
505 recursive calls
0 db block gets
1340 consistent gets
1114 physical reads
0 redo size
589 bytes sent via SQL*Net to client
469 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
5 sorts (memory)
0 sorts (disk)
1 rows processed
SQL> create materialized view m_pm enable query rewrite
2 as
3 select t_code,count(*) from pm group by t_code order by 1;
Materialized view created.
SQL>select object_name,object_type from user_objects where object_name='M_PM'
SQL> set autotrace off
SQL> /
OBJECT_NAME
--------------------------------------------------------------------------------OBJECT_TYPE
-------------------
M_PM
TABLE
M_PM
MATERIALIZED VIEW
SQL> select segment_name,segment_type from user_segments where segment_name='M_PM';
SEGMENT_NAME
--------------------------------------------------------------------------------SEGMENT_TYPE
------------------
M_PM
TABLE
SQL> alter system flush shared_pool;
System altered.
SQL> set autotrace traceonly statistics
SQL> select t_code,count(*) from pm group by t_code order by 1;
Statistics
----------------------------------------------------------
7555 recursive calls
0 db block gets
1532 consistent gets
17 physical reads
124 redo size
589 bytes sent via SQL*Net to client
469 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
84 sorts (memory)
0 sorts (disk)
1 rows processed
So you can see with Mview in action,the physical/logical I/O is decreased.
|
|
|
Re: Table occupying max size [message #506571 is a reply to message #506569] |
Tue, 10 May 2011 00:37   |
Yasir Hashmi
Messages: 304 Registered: April 2006
|
Senior Member |
|
|
A further execution:
SQL> set autotrace traceonly
SQL> /
Execution Plan
----------------------------------------------------------
Plan hash value: 4232873480
--------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
--------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 17 | 3 (34)| 00:00:01 |
| 1 | SORT ORDER BY | | 1 | 17 | 3 (34)| 00:00:01 |
| 2 | MAT_VIEW REWRITE ACCESS FULL| M_PM | 1 | 17 | 2 (0)| 00:00:01 |
--------------------------------------------------------------------------------------
Note
-----
- dynamic sampling used for this statement
Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
3 consistent gets
0 physical reads
0 redo size
589 bytes sent via SQL*Net to client
469 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
1 sorts (memory)
0 sorts (disk)
1 rows processed
SQL> ed
Wrote file afiedt.buf
1* select /*+ no_rewrite */ t_code,count(*) from pm group by t_code order by 1
SQL> /
Execution Plan
----------------------------------------------------------
Plan hash value: 3673671942
-------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time
|
-------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 5 | 11 (0)| 00:00:01 |
| 1 | SORT GROUP BY NOSORT| | 1 | 5 | 11 (0)| 00:00:01 |
| 2 | INDEX FULL SCAN | PM_PK | 232K| 1133K| 11 (0)| 00:00:01 |
-------------------------------------------------------------------------------------
Statistics
----------------------------------------------------------
1 recursive calls
0 db block gets
1111 consistent gets
0 physical reads
0 redo size
589 bytes sent via SQL*Net to client
469 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
1 rows processed
With mview the consistent gets are limited to 3 but 1111 otherwise!!
|
|
|
Re: Table occupying max size [message #506578 is a reply to message #506569] |
Tue, 10 May 2011 01:27   |
 |
Michel Cadot
Messages: 68737 Registered: March 2007 Location: Saint-Maur, France, https...
|
Senior Member Account Moderator |
|
|
Quote:I doubt the query would be as simple as select * from mytable.
So you didn't read what OP posted and I repeated (twice):
Michel Cadot wrote on Mon, 09 May 2011 14:24Michel Cadot wrote on Mon, 09 May 2011 13:33Quote:...
Quote:there are some sqls where it is picking all data from transaction table till date ( trn_dt< sysdate)
How do you use Mview for this?
...
You didn't answer to that.
...
Your post is completly irrelevant to the topic.
Regards
Michel
[Updated on: Tue, 10 May 2011 01:28] Report message to a moderator
|
|
|
|
|