I am Harsh Raj, currently pursuing a M.S. in Computer Science at Northeastern University, Boston. I am also the Co-Founder of the open-source organization Ontocord AI. Previously, I worked as an Applied Scientist at VijilAI.
My research interests focus on language agents, language model evaluation, and software engineering. Over the past few years, I have been fortunate to grow as a researcher with the support of wonderful collaborators and mentors.
Most recently, I led the MixtureVitae project at Ontocord AI alongside Huu Nguyen, developing state-of-the-art pretraining corpora for LLMs. Our work produced the first permissible dataset achieving results comparable to or even surpassing non-permissible sources (paper coming soon).
In addition, I am the co-author and maintainer of Terminal-Bench and collaborate with Professor Ludwig Schmidt’s lab.
News and Timeline
2025
- August: Released MixtureVitae, the first permissibly licensed pretraining corpus to achieve performance comparable to or even surpassing some non-permissive sources. (paper coming soon)
- June: Led the Swiss AI Large Grant proposal in collaboration with Prof. Robert West. Secured CHF 61K and 200K GPU hours on the Alps Supercomputer.
- May: Released terminal-bench.
- April: Started as a Research Assistant at the Interpretable Neural Networks Lab advised by Prof. David Bau.
- January: Our work on Improving Consistency in Large Language Models through Chain of Guidance was accepted at Transactions of Machine Learning Research (TMLR).
2024
- December: Our work on Mitigating Unsafe Feedback with Learning Constraints got accepted for poster presentation at AAAI-25 Workshop on Artificial Intelligence for Cyber Security.
- September: Published my first Lesswrong blog on Interpreting the effects of Jailbreaking in LLMs.
- June: Released the preprint of our work on Reverse Preference Attack, led by Domenic.
- January: Joined VijilAI as an Applied Scientist.
- January: Our work on Vision and Language Navigation ranked 3rd on the R2R leaderboard. Team Name: MLR_Lab_DTU.
2023
- December: Presented our work on robustness transfer at NeurIPS 2023 in New Orleans.
- May: Our work on robustness transfer accepted to NeurIPS 2023. Led by Laura and mentored by Yash.
2022
- November: Our work on consistency evaluation won the Best Paper Award with a cash prize of $5000.
- April: Two papers accepted to NeurIPS Workshop 2023: one on consistency evaluation and another on evaluating the robustness of biomedical concept normalization.