Aluminosilicate binders such as Portland cement or alkali-activated materials (slag and fly ash) are generally considered electrical insulators. In order to decrease their electrical resistance, electrically conductive fillers are added. This brings new application possibilities such as the self-sensing and self-monitoring of smart structures. Three different aluminosilicate composites with the same amount of fine graphite filler (6% with respect to the basic aluminosilicate raw material) were tested for resistance- as well as capacitance-based self-sensing properties in this study. Portland cement and two different alkali-activated binders were used as basic matrices for the conductive composites. The composites were tested for self-sensing properties in repeated compression in the elastic area, static mechanical properties and microstructure by means of scanning electron microscopy and mercury intrusion porosimetry. The results showed that alkali-activated materials are less stiff compared to the Portland cement composite but they provide better self-sensing properties regardless of measured electrical parameter. The best self-sensing properties were achieved with the blended alkali-activated slag/fly ash composite. On the other hand, this composite showed the worst mechanical properties, which was mainly due to the increased porosity in the range of large capillary pores (1–10 μm).