Please wait a minute...
Tsinghua Science and Technology  2021, Vol. 26 Issue (4): 548-557    doi: 10.26599/TST.2020.9010016
    
MetaOJ: A Massive Distributed Online Judge System
Miao Wang(),Wentao Han(),Wenguang Chen*()
Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China.
Download: PDF (1298 KB)      HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

Online Judge (OJ) systems are a basic and important component of computer education. Here, we present MetaOJ, an OJ system that can be used for holding massive programming tests online. MetaOJ is designed to create a distributed, fault-tolerant, and easy-to-scale OJ system from an existing ordinary OJ system by adding several interfaces into it and creating multiple instances of it. Our case on modifying the TUOJ system shows that the modification adds no more than 3% lines of code and the performance loss on a single OJ instance is no more than 12%. We also introduce mechanisms to integrate the system with cloud infrastructure to automate the deployment process. MetaOJ provides a solution for those OJ systems that are designed for a specific programming contest and are now facing performance bottlenecks.



Key wordsonline judge      programming test      distributed systems     
Received: 13 May 2020      Published: 12 January 2021
Corresponding Authors: Wenguang Chen     E-mail: miaowang@tsinghua.edu.cn;hanwentao@tsinghua.edu.cn;cwg@tsinghua.edu.cn
About author: Miao Wang received the bachelor degree from Tsinghua University in 2017. He is pursuing the master degree at Tsinghua University. His research interests include distributed systems and networking.|Wentao Han received the bachelor and PhD degrees in computer science from Tsinghua University in 2008 and 2015, respectively. He is currently an assistant research fellow at the Department of Computer Science and Technology, Tsinghua University. His research interest includes parallel and distributed computing, especially in big data processing systems and neuromorphic systems.|Wenguang Chen received the bachelor and PhD degrees in computer science from Tsinghua University in 1995 and 2000, respectively. He was the Chief Technology Officer (CTO) of Opportunity International Inc. in 2000-2002. Since January 2003, he has joined Tsinghua Univeristy. He is now a professor at the Department of Computer Science and Technology, Tsinghua University. His research interest includes parallel and distributed computing, programming model, and mobile cloud computing.
Cite this article:

Miao Wang,Wentao Han,Wenguang Chen. MetaOJ: A Massive Distributed Online Judge System. Tsinghua Science and Technology, 2021, 26(4): 548-557.

URL:

http://tst.tsinghuajournals.com/10.26599/TST.2020.9010016     OR     http://tst.tsinghuajournals.com/Y2021/V26/I4/548

Fig. 1 Structure of typical OJ systems.
8] model of the front-end service.
">
Fig. 2 Entity-Rleationship (ER)[8] model of the front-end service.
Fig. 3 Structure of OJ instances and OJ manager.
Fig. 4 ER model of OJ manager.
Fig. 5 Delay of responses under extreme pressure.
StatisticsMinimumMedian0.95- quantile0.99- quantileMaximum
Before1.44.212.132.268.3
After1.54.313.533.978.0
Table 1 Statistics of delay under normal high workload. (ms)
Fig. 6 Distribution of delay under normal high workload.
Fig. 7 Creation of a new OJ server.
[1]   Kurnia A., Lim A., and Cheang B., Online judge, Comput. Educ., vol. 36, no. 4, pp. 299-315, 2001.
[2]   Wang G. P., Chen S. Y., Yang X., and Feng R., OJPOT: Online judge & practice oriented teaching idea in programming courses, Eur. J. Eng. Educ., vol. 41, no. 3, pp. 304-319, 2016.
[3]   Wasik S., Antczak M., Badura J., Laskowski A., and Sternal T., A survey on online judge systems and their applications, ACM Comput. Surv., vol. 51, no. 1, p. 3, 2018.
[4]   Wu J. H., Chen S. P., and Yang R. R., Development and application of online judge system, presented at 2012 International Symposium on Information Technologies in Medicine and Education, Hokodate, Japan, 2012, pp. 83-86.
[5]   Revilla M. A., Manzoor S., and Liu R. J., Competitive learning in informatics: The UVa online judge experience, Olympiads in Informatics, vol. 2, pp. 131-148, 2008.
[6]   Kosowski A., Malafiejski M., and Noinski T., Application of an online judge & contester system in academic tuition, in Proc. 6th International Conference on Web Based Learning, Edinburgh, UK, 2007, pp. 343-354.
[7]   Luo Y. W., Wang X. L., and Zhang Z. Y., Programming grid: A computer-aided education system for programming courses based on online judge, in Proc. 1st ACM Summit on Computing Education in China on First ACM Summit on Computing Education in China, Beijing, China, 2008, pp. 1-4.
[8]   Chen P. P., The entity-relationship model—Toward a unified view of data, ACM Transactions on Database, .
doi: 10.1145/320434.320440
[9]   Maggiolo S. and Mascellani G., Introducing CMS: A contest management system, Olympiads in Informatics, vol. 6, pp. 86-99, 2012.
[10]   GitLab, GitLab.com database incident, , 2017.
[11]   Han J., E H. H., Le G., and Du J., Survey on NoSQL database, presented at 2011 6th International Conference on Pervasive Computing and Applications, Port Elizabeth, South Africa, 2011, pp. 363-366.
[12]   Josefsson S., RFC 4648: The base16, base32, and base64 data encodings, , 2006.
[13]   Jones M., Bradley J., and Sakimura N., RFC 7519: JSON Web Token (JWT), , 2015.
[14]   Moriarty K., Kaliski B., and Rusch A., RFC 8018: PKCS #5: Password-based cryptography specification version 2.1, , 2017.
[1] . Performance Improvement of Distributed Systems by Autotuning of the Configuration Parameters[J]. Tsinghua Science and Technology, 2011, 16(4): 440-448.