technologies > data warehousing

Data Warehousing

While databases have been traditionally used for information storage and retrieval within an organization, they have also started to be used for data analysis and decision making. To accommodate for such requirement however, massive amounts of data need to be processed and filtered in order to reach useful conclusions.

This has lead into databases with huge amounts of data optimized for analysis rather than fast retrieval, resulting in a “data warehouse”. Data warehouses are databases that need to scale and adapt to multidimensional data spanning across multiple time periods and allow the user to process it across different dimensions focusing only on the points of interest.

Data warehouses require specialized experts for their creation and maintenance compared to traditional databases and for a number of reasons.

The first important factor is the sheer amount of data. Importing and filtering large datasets is a time consuming process that has to be heavily optimized in order to appeal to end-users. The second factor is that a data warehouse is only useful as long as it is up-to-date. New datasets are always coming in and merging them into the core database is not always easy. Finally, as mentioned already, a data warehouse requires the presence of well-designed reporting facilities that will offer their users the views they need in order to successfully comprehend the data and make decisions.

You can consult the Wikipedia entry on data warehouses for more information on the subject.

Agilis has performed extensive research on data warehouses, with multiple scientific papers published in related conferences. This research has resulted in concrete implementations of software tools dealing with the processing and visualization of data warehouses.

 

Data Warehousing