Category Database

September 16, 2020 Posted in Blogs, Data Migration, Database, GIS, Oracle, Spatial, SQL Server

I spent many years on spatial data modeling and migration on ESRI ArcGIS and ArcSDE, served nature resources sector. Spatial data migration is common when we get involved in map-related applications.

I worked on a project which requested spatial data migration from Oracle to SQL Server. I would like to give you a quick walk-through on how it can be done correctly.

First, let’s introduce some basic concepts:

• WKT & WKB: Well-known Text & Well-known Binary, text markup language defined by Open Geospatial Consortium (OGC), supported by Oracle 9i+, MS SQL Server 2008 R2+, PostgreSQL PostGIS M2, etc.
• Geometry & Geography: Geometry: Euclidean (flat) coordinate system; Geography: Round-earth coordinate system.
• Spatial Data Types: consist of simple types and collection types as illustr...

June 7, 2020 Posted in Blogs, Database

Often we need work on existing database schema. How to quickly master the database structure can be a challenge since we might not be able to get the data model ERD diagram from project repository.

However, as long as you get access to the database, then you should be able to generate the data model using SchemaSpy. SchemaSpy is LGPL-based, developed by John Currier in 2004. It has been improved constantly since then.

SchemaSpy is written in Java and deployed as a jar file. Current version is 6.1. It’s a command line tool. The output is a folder with html and other files which outline the database tables, views and their relationships. But before you run, you need have JDBC driver and GraphViz ready...

December 15, 2019 Posted in BI, Blogs, Data Visualization and Analytics, Database, LiveSync Automation

One of my municipality clients has a requirement to handle batch data from multiple data sources for visualization and analytics needs. The data source can be ERP system, sensors or internal databases. Sensors load data into time series database continuously. It requires BI dashboard show charts based on latest data in near real-time manner.

Let’s see how we design the architecture to satisfy the requirements.

First, we have data sources listed in the left, including ERP, time series database-based backend systems, other databases and data flow from APIs ;

Second, we have ETL process to load data from data sources to DVA platform. You may choose any ETL tool you like, but we highly recommend Lionsgate Software’s LiveSync Au...