Tsinghua Science and Technology  2019, Vol. 24 Issue (2): 195-206    doi: 10.26599/TST.2018.9010074
Parallel ADR Detection Based on Spark and BCPNN
Li Sun, Shan Sun, Tianlei Wang, Jiyun Li*, Jingsheng Lin
∙ Li Sun, Shan Sun, Tianlei Wang, and Jiyun Li are with School of Computer Science and Technology, Donghua University, Shanghai 201620, China. E-mail: sli@dhu.edu.cn; sunshan_1128@126.com; 576138877@qq.com.
∙ Jingsheng Lin is with the Ruijin Hospital Affiliated to Shanghai Jiao Tong University, Shanghai 200020, China. E-mail: jasonlin@rjh.com.cn.

Abstract

Adverse Drug Reaction (ADR) is one of the major challenges to the evaluation of drug safety in the medical field. The Bayesian Confidence Propagation Neural Network (BCPNN) algorithm is the main algorithm used by the World Health Organization to monitor ADRs. Currently, ADR reports are collected through the spontaneous reporting system. However, with the continuous increase in ADR reports and possible use scenarios, the efficiency of the stand-alone ADR detection algorithm will encounter considerable challenges. Meanwhile, the BCPNN algorithm requires a certain number of disk I/O, which leads to considerable time consumption. In this study, we propose a Spark-based parallel BCPNN algorithm, which speeds up data processing and reduces the number of disk I/O in BCPNN, and two optimization strategies. Then, the ADR data collected from the FDA Adverse Event Reporting System are used to verify the performance of the proposed algorithm and its optimization strategies. Experiments show that the parallel BCPNN can significantly accelerate data processing and the optimized algorithm has a high acceleration rate and can effectively prevent memory overflow. Finally, we apply the proposed algorithm to a dataset provided by a real medical consortium. Experiments further prove the performance and practical value of the proposed algorithm.

Received: 16 August 2017      Published: 29 April 2019
Corresponding Authors: Jiyun Li