Development of ETL pipeline for Electronic Health Record to support Machine Learning based Approaches for Security and Prediction - Saraswat, Birendra Kumar; Varshney, Neeraj; Vashist, P. C. (International Journal of Intelligent Systems and Applications in Engineering, 2024)
Electronic Health Records (EHRs) contain a wealth of information about a patient’s medical history, treatments, and health outcomes. However, the data in EHRs is often unstructured and scattered across multiple systems, making it challenging to extract meaningful insights. Developing an extract, transform, and load (ETL) pipeline for EHRs is crucial to address this challenge. This pipeline will enable the efficient integration and transformation of EHR data into a standardized format that can be used for machine learning-based approaches for security and prediction. This paper uses an ETL pipeline to identify the data sources and types of data to be extracted. These can range from structured data, such as diagnosis codes and lab results, to unstructured data, such as doctors’ notes and imaging reports. Once the data sources are identified, the pipeline needs to be designed to extract the data from these sources securely and efficiently. The proposed model transforms the extracted data into a standardized format that can be used for machine learning algorithms. It involves cleaning the data, dealing with missing values, and converting it into a structured form. The proposed model obtained 96.59% accuracy, 96.36% precision, 95.64% recall, 97.56% f1-score, 96.66% false positive rate, 93.39% false negative rate.