Paul Soulos

I am a PhD student studying Computational Cognitive Science at Johns Hopkins University and a collaborating researcher with the Microsoft Research Deep Learning Group. My main areas of focus are compositionality, generalization, and disentangled representations in neural networks. My PhD advisors are Paul Smolensky and Leyla Isik.

Before entering graduate school, I worked as a software engineer at Google and Fitbit. My focus at these companies was on wearables and health technology.

CV  / GitHub  /  Google Scholar

profile photo

Research

My work intersects Cognitive Science, Computational Linguistics, Machine Learning, and Neuroscience.

project image

Compositional Generalization Across Distributional Shifts with Sparse Tree Operations


Paul Soulos, Henry Conklin, Mattia Opper, Paul Smolensky, Jianfeng Gao, Roland Fernandez
Thirty-seventh Conference on Neural Information Processing Systems, 2024
Spotlight Award
Spotlight oral presentation at System 2 Reasoning Workshop, NeurIPS 2024
paper / poster /

project image

Recurrent Transformers Trade-off Parallelism for Length Generalization on Regular Languages


Paul Soulos, Aleksandar Terzić, Michael Hersche, Abbas Rahimi
The First Workshop on System-2 Reasoning at Scale, NeurIPS'24, 2024
paper /

project image

Toward Compositional Behavior in Neural Models: A Survey of Current Views


Kate McCurdy, Paul Soulos, Paul Smolensky
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024
paper /

project image

Disentangled Face Representations in Humans and Machines


Paul Soulos, Leyla Isik
PLOS Computational Biology, 2024
arxiv / paper / code / poster /

project image

Differentiable Tree Operations Promote Compositional Generalization


Paul Soulos, Edward Hu, Kate McCurdy, Yunmo Chen, Roland Fernandez, Paul Smolensky, Jianfeng Gao
Proceedings of the 40th International Conference on Machine Learning, 2023
arxiv / paper / code / poster /

project image

Structural Biases for Improving Transformers on Translation into Morphologically Rich Languages


Paul Soulos, Sudha Rao, Caitlin Smith, Eric Rosen, Asli Celikyilmaz, R. Thomas McCoy, Yichen Jiang, Coleman Haley, Roland Fernandez, Hamid Palangi, Jianfeng Gao, Paul Smolensky
Proceedings of the 4th Workshop on Technologies for MT of Low Resource Languages (LoResMT2021), 2021
arxiv /

project image

Enriching Transformers with Structured Tensor-Product Representations for Abstractive Summarization


Yichen Jiang, Asli Celikyilmaz, Paul Smolensky, Paul Soulos, Sudha Rao, Hamid Palangi, Roland Fernandez, Caitlin Smith, Mohit Bansal, Jianfeng Gao
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021
arxiv /

project image

Discovering the Compositional Structure of Vector Representations with Role Learning Networks


Paul Soulos, R. Thomas McCoy, Tal Linzen, Paul Smolensky
Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, 2020
Spotlight oral presentation at Workshop on Context and Compositionality in Biological and Artificial Neural Systems, NeurIPS 2019
arxiv / code /

project image

Learning to generalize like humans using basic-level object labels


Joshua C Peterson, Paul Soulos, Aida Nematzadeh, Thomas L Griffiths
Journal of Vision, 2019
arxiv /





Design and source code from Jon Barron's website