DATA WAREHOUSING

• DATA WAREHOUSING

• DATA WAREHOUSE
• PROCESS FOR COLLECTING AND MANAGING DATA FROM VARIED SOURCES TO PROVIDE MEANINGFUL BUSINESS INSIGHTS.
• A DATA WAREHOUSE IS TYPICALLY USED TO CONNECT AND ANALYZE BUSINESS DATA FROM HETEROGENEOUS SOURCES
• THE PRIMARY CONCEPT BEHIND DATA WAREHOUSING IS THAT DATA STORED FOR BUSINESS CAN MOST EFFECTIVELY BE ACCESSED BY SEPARATING IT FROM DATA IN THE OPERATIONAL SYSTEM
• DATA WAREHOUSING IS NOT JUST THE DATA IN DATA WAREHOUSE BUT ALSO THE ARCHITECTURE AND TOOLS TO COLLECT,QUERY,ANALYZE AND PRESENT INFORMATION
• LIKE A HOSPITAL MAY CREATE A DATA WAREHOUSE EXTRACTING THE OPERATIONAL DATA IT HAS GATHER LIKE PATIENT INFORMATION,LAB RESULT AND USE AND LENGTH OF STAY AND ETC
• DATA WAREHOUSE
• START FROM EXTRACTING DATA FROM SOURCE SYSTEM ,TRANSFORMS AND CLEANSING IT BEFORE LOADING INTO THE REPOSITORY
• DATA IS ACCESSES,ANALYZED.MINED AND DASHBOARDED USING END USER TOOLS

1.     SOURCES : LIKE CORE PRODUCTION SYSTEM,CUSTOMER SERVICES SYSTEM,EXCEL FILES,FINANCIAL CASH MGMT SYSTEM ETC

2.     TRANSFORM : CONSOLIDATE,RECONCILE,CLEAN ,STANDARDIZE .ARGUMENT AND ENRICH

3.     DATA WAREHOUSE

4.     OLAP LAYER

5.     ACCESS

• APPLICATION OF DATA WAREHOUSE

1. DECISION SUPPORT SYSTEM
2. TREND ANALYSIS
3. FINANCIAL FORECASTING
4. CHURN PREDICTION FOR TELECOMS SUBSCRIBER
5. FINDING INSURANCE FRAUD
6. LOGISTICS AND INVENTORY MGMT
7. ALSO USEFUL IN AGRICULTURE

• FEATURES OF DATA WAREHOUSE

1. DESIGNED TO HELP THE ORGANIZATION ANALYZING DATA. FROM COMPANY’S SALES DATA: WE CAN KNOW WHO WAS OUR BEST CUSTOMERS. DATA WAREHOUSE IS SUBJECT ORIENTED
2. INTEGRATED : DIFFERENT SOURCES INTO CONSISTENT FORMAT
3. NONVOLATILE : ONCE ENTERED THE DATA INTO WAREHOUSE,WILL NOT CHANGE
4. TIME VARIANT “IN ORDER TO DISCOVER TREND IN BUSINESS ANALYSIS NEED LARGE AMOUNT OF DATA

• DATA WAREHOUSE ARCHITECTURE
• OPERATIONAL DATA “

1. MAINTENANCE OPERATIONAL DATA IN FIRST GENERATION HEIRARCHICAL AND NETWORK DATA BASE
2. DEPARTMENTAL DATA HELD IN PROPRIETARY FILE SYSTEM SUCH AS VSAM,RELATIONAL DATABASE MANAGEMENT SYSTEM
3. EXTERNAL SYSTEM SUCH AS INTERNET COMMERCIAL AVAILABLE DATA BASE ASSOCIATED WITH AN ORGANIZATION ‘S SUPPLIER OR CUSTOMERS

• OPERATIONAL DATA STORE :

1. REPOSITORY OF CURRENT AND INTEGRATED OPERATIONAL DATA USED FOR ANALYSIS
2. STRUCTURED AND SUPPLIED THE DATA
3. CREATED WHEN LEGACY OPERATIONAL SYSTEM FOUND TO BE INCAPABLE OF ACHIEVING THE INFORMATION
4. PROVIDE EASE OF USE OF A RELATIONAL DATABASE
5. HELPFUL INCREASING A DATA WAREHOUSE

• QUERY MANAGER :- ALSO CALLED BACKHEND COMPONENT

1. PERFORM ALL THE OPERATIONS ASSOCIATED WITH THE MANAGEMENT OF USER QUERIES
2. CONSTRUED USING VENDOR AND END USER TOOLS,DATA WAREHOUSE MONITORING TOOLS,DATA BASE FACILITIES AND CUSTOM BUILT PROGRAMS
3. DIRECTING QUERIES TO THE APPROPRIATE TABLES
4. QUERY MANAGERS ALSO GENERATES QUERY PROFILES TO ALLOW THE WAREHOUSE MANAGERS TO WHICH INDEXES AN AGGREGATION ARE APPROPRIATE

·         WAREHOUSE MANAGER

• MGMT OF THE DATA
• OPERATIONS PERFORMED BY :

1. RESPONSIBLE FOR CHECKING THE CONSISTENCY OF DATA
2. CREATION OF INDEXES
3. CREATES VIEWS FROM BASE TABLE TO IMPLEMENT THE CONCEPT OF DATA HIDING
4. AGGREGATION OF DATA
5. BACKING OF DATA

• LOAD MANAGER :FRONTED COMPONENTS : PERFORM EXTRACTION AND LOADING OF DATA INTO THE WAREHOUSE. THE FUNCTIONS PERFORMED BY THE LOAD MANAGER : SIMPLE TRANSFORMATION OF THE DATA TO PREPARE THE DATA FOR ENTRY INTO THE WAREHOUSE
• DETAILED DATA “ STORES ALL THE DETAILED DATA INTO THE SCHEMA
• SUMMARIZED DATA : STORES ALL THE PREDEFINED LIGHTLY AND HIGHLY AGGREGATED DATA AND OBJECTIVE IS TO SPEED PERFORMANCE OF QUERY
• BACKUP DATA :- DETAILED AND SUMMARIZED DATA. TRANSFERRED TO STORAGE ARCHIVES SUCH AS MAGNETIC TAPES OR OPTICAL DISK
• META DATA :_ THIS AREA OF THE WAREHOUSE STORES ALL THE META DATA DEFINITION BY ALL THE PROCESS. IT IS USED FOR EXTRACTION,LOADING,WAREHOUSE MGMT PROCESS AND META DATA IS USED TO DIRECT A QUERY TO THE MOST APPROPRIATE DATA SOURCE
• COMPONENTS OF DATA WAREHOUSE
• DATA BASE SERVERS :MUST BE EXTRACTED AND STORED INTO DATABASE. MOST COMPANIES USE A RELATIONAL DATA BASES STORED ON A MAINFRAME SERVER . LIKE ORACLE,SQL ETC
• QUERIES /REPORTS : ALL THE ACTIVITIES REQUESTING DATA FROM A WAREHOUSE FOR ANALYSIS. REPORTS ARE GENERATED TO DISPLAY THE RESULTS FOR SPECIFIED QUERY
• OLAP/MULTIDIMENSIONAL ANALYSIS :- RELATIONAL STORE DATA IN TWO DIMENSION FORMAT : TABLES OF DATA REPRESENTED BY ROWS AND COLUMNS. ONLINE ANALYTICAL PROCESSING ( OLAP) MULTIDIMENSIONAL VIEWS
• DATA MINING :PROCESS OF ANALYZING BUSINESS DATA IN THE DATA WAREHOUSE TO FIND FUTURE TRENDS AND BEHAVIOURS
• ADVANTAGE

1. DATA IS EXTRACTED,TRANSFORMED FROM MULTIPLE DATA SOURCES AND LOADED INTO SEPARATE DATA BASE CALLED DATA WAREHOUSE
2. CLEAN DATA
3. INDEXES MULTIPLE TYPES
4. QUERY PROCESSING
5. SECURITY : DATA AND ACCESS

• PROBLEMS OF DATA WAREHOUSE

1. UNDERESTIMATION OF RESOURCES FOR DATA LOADING
2. HIDDEN PROBLEM WITH THE SOURCE SYSTEMS
3. REQUIRED DATA IS NOT CAPTURED
4. INCREASED END USER DEMANDS
5. DATA HOMOGENIZATION
6. HIGH MAINTENANCE'S
7. COMPLEXITY OF INTEGRATION