Improving antibody binding to an antigen without antibody–antigen complex structures or antigen-specific training data is a central challenge in therapeutic protein design. We introduce AffinityEnhancer, a framework for one-shot antibody affinity improvement with strong generalization: given a single lead sequence, we propose variants that increase affinity without fine-tuning on the lead and without using antigen information, epitope/paratope labels, or the lead’s structure in complex with the antigen. During training, AffinityEnhancer leverages a pan-antigen dataset of diverse binding environments (antigens) and constructs paired examples of related sequences with higher vs. lower measured binding. A shared, structure-aware module learns to transform low-affinity sequences toward high-affinity ones, distilling consistent, causal features associated with improved binding across environments. By combining pretrained sequence–structure embeddings with a sequence decoder, AffinityEnhancer generalizes to entirely unseen antibody seeds. Across multiple held-out internal and public leads, AffinityEnhancer concentrates mutations on the rim of the paratope, outperforms existing structure-conditioned and inpainting baselines, and achieves substantial in silico affinity gains in true one-shot experiments, despite never observing antigen-specific data at test time.