The migration of T cells and access to tumor antigens is of utmost importance for the induction of protective anti-tumor immunity. Once having entered a malignant site, T cells encounter a complex environment composed of non-tumor cells along with the extracellular matrix (ECM). It is now well accepted that a deregulated ECM favors tumor progression and metastasis. Recent progress in imaging technologies has also highlighted the impact of the matrix architecture found in solid tumor on immune cells and especially T cells. In this review, we argue that the ability of T cells to mount an antitumor response is dependent on the matrix structure, more precisely on the balance between pro-migratory reticular fiber networks and unfavorable migration zones composed of dense and aligned ECM structures. Thus, the matrix architecture, that has long been considered to merely provide the structural framework of connective tissues, can play a key role in facilitating or suppressing the antitumor immune surveillance. A new challenge in cancer therapy will be to develop approaches aimed at altering the architecture of the tumor stroma, rendering it more permissive to antitumor T cells.