Esophageal cancer is a prototypic squamous cell cancer that carries a poor prognosis, primarily due to presentation at advanced stages. We used human esophageal epithelial cells as a platform to recapitulate esophageal squamous cell cancer, thereby providing insights into the molecular pathogenesis of squamous cell cancers in general. This was achieved through the retroviral-mediated transduction into normal, primary human esophageal epithelial cells of epidermal growth factor receptor (EGFR), the catalytic subunit of human telomerase (hTERT), and p53^R175H, genes that are frequently altered in human esophageal squamous cell cancer. These cells demonstrated increased migration and invasion when compared with control cells. When these genetically altered cells were placed within the in vivo-like context of an organotypic three-dimensional (3D) culture system, the cells formed a high-grade dysplastic epithelium with malignant cells invading into the stromal extracellular matrix (ECM). The invasive phenotype was in part modulated by the activation of matrix metalloproteinase-9 (MMP-9). Using pharmacological and genetic approaches to decrease MMP-9, invasion into the underlying ECM could be suppressed partially. In addition, tumor differentiation was influenced by the type of fibroblasts within the stromal ECM. To that end, fetal esophageal fibroblasts fostered a microenvironment conducive to poorly differentiated invading tumor cells, whereas fetal skin fibroblasts supported a well-differentiated tumor as illustrated by keratin “pearl” formation, a hallmark feature of well-differentiated squamous cell cancers. When inducible AKT was introduced into fetal skin esophageal fibroblasts, a more invasive, less-differentiated esophageal cancer phenotype was achieved. Invasion into the stromal ECM was attenuated by genetic knockdown of AKT1 as well as AKT2. Taken together, alterations in key oncogenes and tumor suppressor genes in esophageal epithelial cells, the composition and activation of fibroblasts, and the components of the ECM conspire to regulate the physical and biological properties of the stroma.