Large Language Models (LLMs) have demonstrated remarkable capabilities in modeling complex sequential data by learning long-range dependencies. Although originally developed for natural language processing, their autoregressive architecture is inherently domain-agnostic, making them suitable for applications well beyond text generation. This observation opens new opportunities for leveraging LLMs in engineering design and optimization. This work investigates a GPT-based framework, built on the GPT-2 architecture, for the topology optimization of planar truss structures. The classical design problem is reformulated as a sequential construction process, where design actions are governed by predefined grammar rules that ensure structural feasibility. These actions are encoded symbolically and mapped into text-like strings, allowing each truss configuration to be represented as a tokenized sequence. Using this formulation, a pretrained model is fine-tuned on a dataset of structurally meaningful designs. Structural performance is accounted for through a mechanically informed loss function, weighting training according to structural stiffness. This strategy effectively biases the model toward high-performing configurations while preserving diversity in the design space. The proposed approach is validated across six benchmark cases, achieving performance levels ranging from 82% to 100% of the corresponding global optima. The model also demonstrates the ability to generate novel, mechanically sound topologies. These results highlight the potential of LLM-based generative frameworks as complementary tools for exploring large and complex design spaces in structural optimization.