With the non-periodicity and discrete nature, and unnecessary of phase unwrapping process, digital speckle correlation method shows its significant advantages in three-dimensional (3D) shape measurement. Combining with the spatial correlation and temporal correlation method in the digital speckle correlation, a spatio-temporal digital speckle correlation was developed in this paper, which can improve the accuracy of 3D shape measurement and effectively reduce the number of the recorded speckle images for restoring the corresponding 3D shape at the same time. In the experiment, only 5 frames of the required speckle images was needed to reconstruct the 3D shape of a complex object with spatio-temporal digital speckle correlation method, and its accuracy was same as the result when 20 frames speckle images were used in temporal correlation method.