Sunday, April 24, 2016

Basic understanding of Data Warehouse(DW) and Business Intelligence(BI)

We are familiar with different key words 'Business Intelligence', 'Data Warehouse', 'ETL' and so on. These keywords are interdependent, sometimes people call one but mean other, even in the job market those three keywords are often mix, e.g. 'ETL Developer is needed' post can be filled by BI or data warehouse developer.

So, the people who are new to the area, this post will give you basic understanding of the above keywords.

The folks who work with IT, they all know about database. As we know data driven applications/solutions store valuable data in the database. Mid size to big companies use different solutions through the organizations. Due to huge volume of unstructured data as well as  more than one solutions in the organization demands separate solution that can  produces intelligence decision output.

First look at below diagram where we know how software application, desktop/web based solution are built:

Fig 1: database driven application/solution



A Data Warehouse(DW) gathers information from a wide range operational systems from a company and it's external system. To get data from different operational systems and load that into EDW require a process called ETL (Extract, Transform and Load).


Fig 2: Data Warehouse basic diagram

We get the unstructured data from different solutions in the data warehouse. Now, we can make BI solutions on top of the data warehouse. Here we split between data warehouse and BI solutions. BI solutions can be built on Data warehouse, So first make data Warehouse and then built BI solutions by using Data Warehouse. To make BI solutions and data warehouse both require ETL( Extract, Transform, Load) tools.

                         
Fig 3: BI solutions top of Data Warehouse

A Small History:


Early '80s data warehouse was the place where aggregated data was placed into special data model; user presentation of dashboards, reports, scorecards, and graphical analysis was not present at that time.

In the ‘90s BI concept came along with the presentation layers where dashboards, reports, scorecards, and graphical analysis produces clear business presentation. By looking at the data presentation business can make future decisions. Over the time it became clear that BI could not exist without data warehouse and data warehouse is the foundation for BI.

My next post will be related to Technologies around BI and a simple guideline to become BI/Data Warehouse developer/Expert.