Siddharth Parekh

Hi! I’m Siddharth, a student at Carnegie Mellon University where I’m completing my Fifth-Year Master’s in Computer Science. My current research interests lie in factuality and how knowledge is learned and represented within Large Language Models.

I’ve worked extensively with Professor Carolyn Rosé’s Document Understanding Group at CMU’s Language Technologies Institute - collaborating with Armineh Nourbakhsh on developing graph-based models for form processing, and robust evaluation metrics for document visual question answering.

Zooming out, I often find myself digging into the weeds of deep neural networks hoping to uncover something interesting.

Feel free to reach out to me to chat about my research or any shared interests!

news

Aug 25, 2025	I’m starting my Fifth-Year Master’s at CMU!
May 08, 2025	I graduated from CMU with University and SCS College Honours!
Jan 22, 2025	Our work, Where is this coming from? Making groundedness count in the evaluation of Document VQA models, has been accepted to NAACL 2025 Findings!
Sep 20, 2024	Our work, AliGATr: Graph-based layout generation for form understanding, has been accepted to EMNLP 2024 Findings!

selected publications

AliGATr: Graph-based layout generation for form understanding

Armineh Nourbakhsh, Zhao Jin, Siddharth Parekh, and 2 more authors

In Findings of the Association for Computational Linguistics: EMNLP 2024, Nov 2024

Abs DOI

Forms constitute a large portion of layout-rich documents that convey information through key-value pairs. Form understanding involves two main tasks, namely, the identification of keys and values (a.k.a Key Information Extraction or KIE) and the association of keys to corresponding values (a.k.a. Relation Extraction or RE). State of the art models for form understanding often rely on training paradigms that yield poorly calibrated output probabilities and low performance on RE. In this paper, we present AliGATr, a graph-based model that uses a generative objective to represent complex grid-like layouts that are often found in forms. Using a grid-based graph topology, our model learns to generate the layout of each page token by token in a data efficient manner. Despite using 30% fewer parameters than the smallest SotA, AliGATr performs on par with or better than SotA models on the KIE and RE tasks against four datasets. We also show that AliGATr’s output probabilities are better calibrated and do not exhibit the over-confident distributions of other SotA models.
Where is this coming from? Making groundedness count in the evaluation of Document VQA models

Armineh Nourbakhsh, Siddharth Parekh, Pranav Shetty, and 3 more authors

In Findings of the Association for Computational Linguistics: NAACL 2025, Apr 2025

Abs

Document Visual Question Answering (VQA) models have evolved at an impressive rate over the past few years, coming close to or matching human performance on some benchmarks. We argue that common evaluation metrics used by popular benchmarks do not account for the semantic and multimodal groundedness of a model‘s outputs. As a result, hallucinations and major semantic errors are treated the same way as well-grounded outputs, and the evaluation scores do not reflect the reasoning capabilities of the model. In response, we propose a new evaluation methodology that accounts for the groundedness of predictions with regard to the semantic characteristics of the output as well as the multimodal placement of the output within the input document. Our proposed methodology is parameterized in such a way that users can configure the score according to their preferences. We validate our scoring methodology using human judgment and show its potential impact on existing popular leaderboards. Through extensive analyses, we demonstrate that our proposed method produces scores that are a better indicator of a model‘s robustness and tends to give higher rewards to better-calibrated answers.