I am Harsh Raj, currently pursuing a M.S. in Computer Science at Northeastern University, Boston. I am also the Co-Founder of the open-source organization Ontocord AI. Previously, I worked as an Applied Scientist at VijilAI.

My research interests focus on language agents, language model evaluation, and software engineering. Over the past few years, I have been fortunate to grow as a researcher with the support of wonderful collaborators and mentors.

Most recently, I led the MixtureVitae project at Ontocord AI alongside Huu Nguyen, developing state-of-the-art pretraining corpora for LLMs. Our work produced the first permissible dataset achieving results comparable to or even surpassing non-permissible sources (paper coming soon).

In addition, I am the co-author and maintainer of Terminal-Bench and collaborate with Professor Ludwig Schmidt’s lab.

News and Timeline

2025

• August: Released MixtureVitae, the first permissibly licensed pretraining corpus to achieve performance comparable to or even surpassing some non-permissive sources. (paper coming soon)
• June: Led the Swiss AI Large Grant proposal under Prof. Robert West. Secured CHF 61K and 200K GPU hours on the Alps Supercomputer.
• May: Released terminal-bench.
• April: Started as a Research Assistant at the Interpretable Neural Networks Lab advised by Prof. David Bau.
• January: Our work on Improving Consistency in Large Language Models through Chain of Guidance was accepted at Transactions of Machine Learning Research (TMLR).

2024

• December: Our work on Mitigating Unsafe Feedback with Learning Constraints got accepted for poster presentation at AAAI-25 Workshop on Artificial Intelligence for Cyber Security.
• September: Published my first Lesswrong blog on Interpreting the effects of Jailbreaking in LLMs.
• June: Released the preprint of our work on Reverse Preference Attack, led by Domenic.
• January: Joined VijilAI as an Applied Scientist.
• January: Our work on Vision and Language Navigation ranked 3rd on the R2R leaderboard. Team Name: MLR_Lab_DTU.

2023

• December: Presented our work on robustness transfer at NeurIPS 2023 in New Orleans.
• May: Our work on robustness transfer accepted to NeurIPS 2023. Led by Laura and mentored by Yash.

2022

• November: Our work on consistency evaluation won the Best Paper Award with a cash prize of $5000.
• April: Two papers accepted to NeurIPS Workshop 2023: one on consistency evaluation and another on evaluating the robustness of biomedical concept normalization.