Hair differentiates from follicle stem cells through progenitor cells in the matrix. In contrast to stem cells in the bulge, the identities of the progenitors and the mechanisms by which they regulate hair shaft components are poorly understood. Hair is also pigmented by melanocytes in the follicle. However, the niche that regulates follicular melanocytes is not well characterized. Here, we report the identification of hair shaft progenitors in the matrix that are differentiated from follicular epithelial cells expressing transcription factor KROX20. Depletion of Krox20 lineage cells results in arrest of hair growth, confirming the critical role of KROX20⁺ cells as antecedents of structural cells found in hair. Expression of stem cell factor (SCF) by these cells is necessary for the maintenance of differentiated melanocytes and for hair pigmentation. Our findings reveal the identities of hair matrix progenitors that regulate hair growth and pigmentation, partly by creating an SCF-dependent niche for follicular melanocytes.