Quantum mechanics is our most successful physical theory. However, it raises conceptual issues that have perplexed physicists and philosophers of science for decades. This book develops a new approach, based on the proposal that quantum theory is not a complete, final theory, but is in fact an emergent phenomenon arising from a deeper level of dynamics. The dynamics at this deeper level are taken to be an extension of classical dynamics to non-commuting matrix variables, with cyclic permutation inside a trace used as the basic calculational tool. With plausible assumptions, quantum theory is shown to emerge as the statistical thermodynamics of this underlying theory, with the canonical commutation/anticommutation relations derived from a generalized equipartition theorem. Brownian motion corrections to this thermodynamics are argued to lead to state vector reduction and to the probabilistic interpretation of quantum theory, making contact with recent phenomenological proposals for stochastic modifications to Schrodinger dynamics.