Siddharth Parekh

Hi! I’m Siddharth, a student at Carnegie Mellon University where I’m completing my Fifth-Year Master’s in Computer Science. I am currently focused on interpretability - specifically, uncovering the internal circuits and representation geometry that LLMs use to parse and reason over structured data.

As a part of Professor Carolyn Rosé’s group at CMU’s Language Technologies Institute - I collaborated with Armineh Nourbakhsh on developing graph-based models for form processing, and robust evaluation metrics for document visual question answering.

I am also fascinated by the intersection of game theory and AI — particularly, how optimal play emerges in complex environments. I find that viewing deep learning through this lens — as players discovering complex strategies in a massive game - makes for a compelling perspective driving interpretability research.

Feel free to reach out to me to chat about my research or any shared interests!

news

Aug 25, 2025	I’m starting my Fifth-Year Master’s at CMU!
May 08, 2025	I graduated from CMU with University and SCS College Honours!
Jan 22, 2025	Our work, Where is this coming from? Making groundedness count in the evaluation of Document VQA models, has been accepted to NAACL 2025 Findings!
Sep 20, 2024	Our work, AliGATr: Graph-based layout generation for form understanding, has been accepted to EMNLP 2024 Findings!

selected publications

AliGATr: Graph-based layout generation for form understanding

Armineh Nourbakhsh, Zhao Jin, Siddharth Parekh, and 2 more authors

In Findings of the Association for Computational Linguistics: EMNLP 2024, Nov 2024

Abs DOI

Forms constitute a large portion of layout-rich documents that convey information through key-value pairs. Form understanding involves two main tasks, namely, the identification of keys and values (a.k.a Key Information Extraction or KIE) and the association of keys to corresponding values (a.k.a. Relation Extraction or RE). State of the art models for form understanding often rely on training paradigms that yield poorly calibrated output probabilities and low performance on RE. In this paper, we present AliGATr, a graph-based model that uses a generative objective to represent complex grid-like layouts that are often found in forms. Using a grid-based graph topology, our model learns to generate the layout of each page token by token in a data efficient manner. Despite using 30% fewer parameters than the smallest SotA, AliGATr performs on par with or better than SotA models on the KIE and RE tasks against four datasets. We also show that AliGATr’s output probabilities are better calibrated and do not exhibit the over-confident distributions of other SotA models.