I am a Research Fellow at Microsoft Research India, where I work with Dr. Aditya Kanade, Dr. Nagarajan Natarajan and Dr. Abhijeet Awasthi in the AI4CODE team.
My primary interest lies in comprehensively exploring the strengths and limitations of modern AI systems, particularly Large Language Models (LLMs), across multiple dimensions. My goal is to harness this knowledge to craft robust and interpretable AI solutions that tackle a wide range of challenges. My research interests aligns with two key areas: 1. Architectural Innovations - I am passionate about uncovering how these models function through approximate mathematical modeling and developing tools that help people to understand these models. I aim to enhance the performance and efficiency of these models by modifying their architecture. 2. AI4CODE - I also aim at improving these AI models for code generation. I wanted to incorporate human-like reasoning abilities into LLMs, enabling them to engage in more robust and structured problem-solving before arriving at target code.
I graduated with B.Tech in Mathematics and Computing from IIT Goa, India in 2023. For more details about my background, refer to my CV. If you'd like to chat with me about my work or research in general, feel free to reach out!
Experience
-- Released NoFunEval benchmark for testing non-functional aspect of code like latency and designed new metric diffBLEU.
-- Fine-tuning LLMs for code-editing tasks and creating pipeline for synthetic data-generation using GPT-4.
-- Automated process of allocating drivers optimally to Metro trains by formulating constraints in Gurobipy solver.
-- Restructured the problem using Max Flows reducing timetable preparation time from few days to a few seconds.
-- Deployed the algorithm in Bengaluru Metro Rail Corporation Limited (BMRCL).
-- Worked on the backend of Questa compiler in C-language for optimizing coverage calculations.
-- Designed comprehensive test cases and resolved JIRA issues, contributing to overall software stability and performance.
NoFunEval: Funny How Code LMs Falter on Requirements Beyond Functional Correctness
Manav Singhal, , Abhijeet Awasthi, Nagarajan Natarajan, Aditya Kanade
COLM'24 PDF
Robust Learning of Diverse Code Edits
, Swayam Singh, Abhijeet Awasthi, Aditya Kanade, Nagarajan Natarajan
DL4C workshop ICLR'25, Under Review at ICML'25 PDF
Language Models' Factuality Depends on the Language of Inquiry
, Kumar Tanmay, Ayush Agrawal, Kumar Ayush, Hamid Palangi, Paul Pu Liang
Under Review at ACL'25 PDF
PASS: Presentation Automation for Slide Generation and Speech
, Aarohi Bhand
Under Review at ACL'25 PDF