Annatto dye is a natural organic dye belonging to carotenoids, whose main components are bixin and norbixin. Due to its low stability, it is convenient to protect the dye molecules with other materials. The use of clay minerals is an alternative, which are phyllosilicates with attractive physico-chemical properties, such as high specific surface area, cation exchange capacity, mechanical/chemical stability and non-toxicity. The main purpose of this work was to develop hybrid materials, using annatto dye and clay mineral modified with different inorganic cations, and then, to evaluate the stability of the new pigments. The process of preparing the modified clay minerals involved mixing a synthetic montmorillonite in solutions containing the precursor salts of the metal cations. Subsequently, the dye was dissolved in a solution containing water and alcohol, followed by filtration and mixed with the modified clay, giving rise to the hybrid pigments. Through the characterizations, it was noted that a variety of colors was obtained, and the sample containing aluminum was the one that most adsorbed the dye and showed a significant increase in stability at high temperatures. This hybrid material was better than to the dye in its pure form. Therefore, the bixin/montmorillonite pigments are promising for replacing artificial colors in practical applications such as in the cosmetics, food or pharmaceutical industries.