Model selection in hybrid quantum neural networks with applications to quantum transformer architectures

Quantum machine learning models generally lack principled design guidelines, often requiring full resource-intensive training across numerous choices of encodings, quantum circuit designs and initialization strategies to find effective configuration. To address this challenge, we develope the Quantum Bias-Expressivity Toolbox ($QBET$), a framework for evaluating quantum, classical, and hybrid transformer architectures. In this toolbox, we introduce lean metrics for Simplicity Bias ($SB$) and Expressivity ($EXP$), for comparing across various models, and extend the analysis of $SB$ to generative and multiclass-classification tasks. We show that $QBET$ enables efficient pre-screening of promising model variants obviating the need to execute complete training pipelines. In evaluations on transformer-based classification and generative tasks we employ a total of $18$ qubits for embeddings ($6$ qubits each for query, key, and value).

We identify scenarios in which quantum self-attention variants surpass their classical counterparts by ranking the respective models according to the $SB$ metric and comparing their relative performance.