Recently reported contrast in phase images of human and animal brains obtained with gradient-recalled echo MRI holds great promise for the in vivo study of biological tissue structure with substantially improved resolution. Herein we investigate the origins of this contrast and demonstrate that it depends on the tissue “magnetic architecture” at the subcellular and cellular levels. This architecture is mostly determined by the structural arrangements of proteins, lipids, non-heme tissue iron, deoxyhemoglobin, and their magnetic susceptibilities. Such magnetic environment affects/shifts magnetic resonance (MR) frequencies of the water molecules moving/diffusing in the tissue. A theoretical framework allowing quantitative evaluation of the corresponding frequency shifts is developed based on the introduced concept of a generalized Lorentzian approximation. It takes into account both tissue architecture and its orientation with respect to the external magnetic field. Theoretical results quantitatively explain frequency contrast between GM, WM, and CSF previously reported in motor cortex area, including the absence of the contrast between WM and CSF. Comparison of theory and experiment also suggests that in a normal human brain, proteins, lipids, and non-heme iron provide comparable contributions to tissue phase contrast; however, the sign of iron and lipid contributions is opposite to the sign of contribution from proteins. These effects of cellular composition and architecture are important for quantification of tissue microstructure based on MRI phase measurements. Also theory predicts the dependence of the signal phase on the orientation of WM fibers, holding promise as additional information for fiber tracking applications.