Challenges of Data Consolidation Part III: Normalizing the Data

The last installment of this three part series deals with data normalization and the alignment process of taking one piece of data and loading it to a destination. This topic, along with our previous conversations on network connectivity and data transfer technologies, rounds out our series on the Challenges of Data Consolidation.

Data normalization in this case involves aligning two different data sources in areas of naming, units and scaling. Most of this alignment is taken into account with what is typically referred to as Extract Transform Load or ETL process. ETL is taking data from one source, mapping or modifying the data and then loading it to a destination. Effective planning for mapping and modifying the data is key to a successful integration.

The end goal is to format the data to accommodate the visualization system in use or any preferences on presenting the data. The planning must include a good understanding of any unit conversions. These might include simple scaling such as watts to kilowatts or calculations such as converting BTU to kilowatts. In some cases, data in one system do not align well with one another due to a missing value. If a system has the parameters to calculate that value, the ETL transform process could perform the calculation and align the results. Proper name mapping is something best done in a spreadsheet initially to make sure you have all the data identified with a source and destination name.

Some software platforms such as a BMS or DCIM have this ETL capability natively. If it is native, the software is typically configured with a target IP and an authentication parameter. This defines the connection to the standard data exchange of the other system. Native support sounds ample, but typically leaves out critical parameters due to customization or data that does not translate easily. Without effective native support, a quality data integration tool is required with some expert consulting that understands details about how each platform stores and shares data.

In summary, effective planning for mapping and modifying the data is key to a successful integration. The end goal is to format the data to accommodate the visualization system in use or any preferences on presenting the data.

This wraps my three part series on data consolidation. Remember, data consolidation has to be considered as a project and not just a simple task. The planning and break down of tasks will help you understand the scope of work. There are many decisions to make throughout the process of data consolidation, and understanding if this is a feat you want to tackle internally or with the help of a vendor is really the first question that needs to be answered.

Jason is a part of Schneider Electric’s Data Center Software Solutions Team.  More information on our software solutions can be found here. Or visit Schneider Electric DCIM support.

The post Challenges of Data Consolidation Part III: Normalizing the Data appeared first on Schneider Electric Blog.