My Research Journey

I always wanted to work in AI since starting my career in 2016. However, it wasn't until 2023 that I got the opportunity.

One interesting fact is all my previous jobs were at startups, with founders from diverse backgrounds and cultures spanning Myanmar, the US, Hong Kong, Malaysia, India, and Singapore. I was what the kids today call a "Founding Engineer". :)

I didn't have a formal education in computer science, so I put in long hours to catch up. I told myself, "I have to be three times better than my peers to be considered equal, due to lack of credentials."

But my heart yearned for AI. I was fortunate enough to meet Dr. Htet, who became my mentor. My journey in AI research began in 2023.

Modular neural networks [1] were the first thing I worked on, inspired by the human brain's modularity. The brain has neural pathways for different tasks. So, what if we could create neural networks with pathways inside the weights?

Then, we discovered continual learning [2], a research area where modular networks would be promising. Continual learning is teaching AI to learn new things without forgetting existing knowledge. It's not in the mainstream research. But catastrophic forgetting may come back to haunt current state-of-the-art models.

At this point, I took a career break to focus full-time on research, shifting my focus to more popular topics with better career prospects. It was about the Mixture of Experts (MoE) in Large Language Models. The problem I identified with MoE is that the experts are not the actual experts in anything. It's like a misnomer for the general public although it has significant origin in the research literature.

So, I was exploring how to make the experts more specialized by inventing novel routing algorithms. In reality, no one really cared about expert specialization. I began to doubt my research. Then, I discovered Deepseek cared about the problem [3]. It was the first time in my life I felt like I was working on something meaningful in AI research. The biggest challenge was getting compute resources. If you want to prove your routing algorithm is better than the state-of-the-art, you have to pretrain LLMs. Alright, bye-bye.

Enough is enough. I would just follow my passion, Reinforcement Learning, regardless of career prospects. I started learning the fundamentals of RL. I implemented a few RL algorithms from scratch. I reproduced some of the state-of-the-art RL. The most difficult research I reproduced was Meta RL with TransformerXL [4]. The architecture was so complex that I spent weeks understanding and engineering it. On a side note, Meta Reinforcement Learning is the most fascinating topic in AI research for me. [5]

The achievement of pursuing RL was getting 14th place in the NeurIPS 2024 RL competition. We could have done better, but I was in the honeymoon phase with RL. I romantically thought I could solve all the problems in RL with my novel ideas instead of taking proven approaches. Looking back, I have learned valuable lessons from this experience.

Now, I am focusing on LLM reasoning and RL. We are tackling the ARC AGI challenge. To be honest, it is a daunting task. But you learn the most when you try to tackle the hardest problems. And, I hope everything I learn will be coming together.

I am just getting started in AI research. I have a long way to go. The meaning of life for me is to create intelligence. The world could use some more intelligence.