To accelerate the adoption of Metal Additive Manufacturing (MAM) for production, an understanding of MAM process-structure-property (PSP) relationships is indispensable for quality control. A multitude of physical phenomena involved in MAM necessitates the use of multi-modal and in-process sensing techniques to model, monitor and control the process. The data generated from these sensors and process actuators are fused in various ways to advance our understanding of the process and to estimate both process status and part-in-progress states. This paper presents a hierarchical in-process data fusion framework for MAM, consisting of pointwise, trackwise, layerwise and partwise data analytics. Data fusion can be performed at raw data, feature, decision or mixed levels. The multi-scale data fusion framework is illustrated in detail using a laser powder bed fusion process for anomaly detection, material defect isolation, and part quality prediction. The multi-scale data fusion can be generally applied and integrated with real-time MAM process control, near-real-time layerwise repairing and buildwise decision making. The framework can be utilized by the AM research and standards community to rapidly develop and deploy interoperable tools and standards to analyze, process and exploit two or more different types of AM data. Common engineering standards for AM data fusion systems will dramatically improve the ability to detect, identify and locate part flaws, and then derive optimal policies for process control.