The porous geopolymer aggregate (GPA) and the improved GPA-matrix interfacial bonding gave geopolymer aggregate concrete (GPAC) distinct mechanical behavior. The dynamic compressive behavior of GPAC was investigated under different specimen size based on the split Hopkinson pressure bar (SHPB) test. The corresponding analysis was conducted from the dynamic stress-strain curves, digital image correlation, fragment size distribution, and energy evolution. Results showed that GPAC exhibited a layered splitting failure pattern dominated by multiple sub-horizontal cracks under impact loading, accompanied by local inclined cracks and fragment spalling. The dynamic size effect of GPAC was opposite to its static size effect, and its critical value was approximately 1–50 s−1. The improved GPA-matrix interfacial bonding and aggregate strength dominated the strength and integrity of GPAC under dynamic loading. When the specimens were too short, the enhanced end constraint effect induced bursting failure. The energy dissipation mechanisms of GPAC were governed by the synergistic effects of porous aggregate compaction, the improved GPA-matrix interfacial bonding, and crack propagation. The combined influence of size and strain-rate effect on GPAC strength was quantified using dynamic increase factor and coupled response models, respectively. Subsequently, the effects of incident waveform and inertia effect were weakened based on 3D mesoscale simulation of GPAC. A unified static-dynamic size effect law of GPAC was presented and validated, which enabled the relative decoupling of the size and strain-rate effect at the mechanism level. The research findings provided experimental evidence, numerical methods, and a theoretical basis for applying GPAC to protective and resilient structures.