“How can this computer beat me so heavily in this chess game?”, this was the first question that I asked my sister after playing against the Medium bot of Win98’s Chess Game. My sister, a graduate student in computer science, introduced me to some basics of computer systems so that I knew a computer could be programmed to be a chess player. She told me that I could train the computer for any of my purposes if I was good enough.
For example, the computer in her project could be a tool to construct house model for sustaining severe environmental impacts, which was so interesting for me. In the eyes of a 6 years-old boy, it sparked my interests in computing. Since there, I wanted to know how to create a program, the theory behind computers and the methodology of how they work. Choosing Computer Science as my major in undergraduate, I wanted to further my computing knowledge so that I could fulfill my childhood dream as a well-known computing scientist. The excitement of knowing more about computing continues during my undergraduate at Catholic University of America (CUA). By participating in many research opportunities in CUA, I have not only refined my research interest but also developed my programming skills. Thanks for many interesting computer science courses offered by CUA, I am familiar with Python and MATLAB, which I have learned to use for programming. I have also learned about the computing theory such as Stored Program Concept, Fetch-Execute Cycle, and the Database Management.
In addition, I have advanced my programming skills such as using PyQt to create my user interface or furthering my techniques including Object and Event orientated programming languages. By my junior year, I had refined my research interest to the study of Machine Learning. My independent research involved analyzing the efficiency of Deep Reinforcement Learning method compared to other informed-database algorithms in creating an autonomous bot for Chess Game.
First, I tried to create a database of chess’s configuration. However, due to its enormous state space and multi-players property, it was impossible for me to create such database storing all chess’s configuration. Even if I had a good database, the time required for finding an optimal move was large due to the problem’s complexity. The next step in my research was testing the performance of deep learning algorithm – Alpha Go, provided by Google DeepMind. The algorithm was based on game theory, sophisticated search techniques, domain-specific adaptation and handcrafted evaluation function. With no other knowledge other than the rule of the game, the program would self-play to learn about chess’s configurations. Within 24 hours, starting from random plays, Alpha Go was able to beat other expert bots, which were built on an informed database, in the chess game.
Through this research, it helped me to understand why Artificial Intelligence was the current technology’s trend and confirmed my passion for this field. Exciting about the new knowledge, I am willing to spend more effort on it. Since undergraduate level does not provide specialized knowledge in this area, pursuing an advanced study will be my choice in exploring the new and challenging field – Machine Learning. Another benefit of pursuing an advanced study is practicing self-study and implementing knowledge to solve the real-life problem.
I am currently in my second-year of working in Dr. Liu’s communication lab, using reinforcement learning to allocate resource in a base station to increase the performance of millimeter wave (5G) network. First, I have created a theoretical system model and trained our hypothesis on the model using simulation. By analyzing the simulation results, our lab proposed that using reinforcement learning method can minimize the average latency, contributing to the improvements of communication channels. Based on the simulation results and verified theoretical models, we are constructing experiments to check our proposal with real-life conditions. By tracking the traffic signal at each received modems, I can analyze the effects of noise, channels’ state-condition to channels’ data rate, then modify the theoretical model in our algorithm.
We are now able to demonstrate a significant improvement of average latency, which each channel experience higher data-rates and lower queue stacked using machine learning. A paper on this study which I will be listed as a co-author is currently undergoing peer-review.