BI tagged posts

Batch Data Processing Architecture for Visualization and Analytics

One of my municipality clients has a requirement to handle batch data from multiple data sources for visualization and analytics needs. The data source can be ERP system, sensors or internal databases. Sensors load data into time series database continuously. It requires BI dashboard show charts based on latest data in near real-time manner.

Let’s see how we design the architecture to satisfy the requirements.

Batch Data Processing for Visualization and Analytics

First, we have data sources listed in the left, including ERP, time series database-based backend systems, other databases and data flow from APIs ;

Second, we have ETL process to load data from data sources to DVA platform. You may choose any ETL tool you like, but we highly recommend Lionsgate Software’s LiveSync Au...

Read More

Case Study: Data Modeling Transition for BI

One of our utility clients uses MS Visio as its logical data modeling tool. MS Visio is designed as a drawing dmtool,not as a professional data modeling tool by Microsoft official statement. To create DDL from MS Visio diagram is not straightforward; usually it takes lots of efforts.

We prefer to use UML modeling instead of ERD modeling. Since to adopt object-oriented design in database field is better aligned with object-oriented application design. In order to achieve this goal, we did some research on quite a few tools in the market, and we found Visual Paradigm is an ideal UML modeling tool in data architectural design.

Below is a case study regarding how we transit client’s data model from Visio to Visual Paradigm in their data warehouse project, and how we utilize Visual Paradigm in...

Read More