A large number of studies over different experimental sites globally has successfully shown the utility of ICESat-2 LiDAR sensor-based datasets for the evaluation of topography (elevation) and water level. The current study evaluates the different filtering strategies for the selection of ATL08 footprints from the Advanced Topographic Laser Altimeter System (ATLAS) instrument based on the terrain uncertainty available in the ATL08 data. The openly accessible digital elevation models (DEMs), namely ASTER GDEM V003, CartoDEM V3 R1, TanDEM-X EDEM Global 30m, and SRTM 1 Arc Second Global, were evaluated for the Madurai Region. The footprints with unknown (null) uncertainties were removed first, and thereafter different datasets were generated for DEM assessment based on uncertainties of <10m, < 5m, < 2.5, < 1, <0.75, <0.5m, <0.25m, and <0.1m. It is observed that the CartoDEM performance is better than ASTER among the optical photogrammetrically derived DEMs, whereas TanDEM-X is better than the SRTM among the SAR Interferometry (InSAR)-based DEMs. Overall, the accuracy of TanDEM-X EDEM was found to be best upon comparison of RMSE among the four openly accessible DEMs used in the analysis. For example, with the filtering strategy of footprints selected with <0.75m uncertainty, the RMSE values for ASTER GDEM V003, CartoDEM V3 R1, TanDEM-X EDEM Global 30m, and SRTM 1 Arc Second Global were 7.14m, 3.68m, 1.52m, and 3.16m, respectively. It is also observed that reducing the uncertainties beyond an uncertainty level does not provide clear estimates due to inherent qualities and errors in different DEMs as well as techniques employed in DEM generation. The results may vary from one DEM to another, thus indicating that a careful selection of DEMs shall be performed before using in any application.