This monograph concentrates on reasoning techniques for intelligent, autonomous agent systems. In particular, it focuses on planning techniques for both single and multi-agent systems acting in uncertain domains. In modeling these domains, two types of uncertainty are considered: (i) the outcomes of agent actions are uncertain and (ii) the amount of resources consumed by agent actions is uncertain and only characterized by continuous probability density functions. For solving planning problems modeled in these domains the monograph proposes a class of efficient algorithms that provide guarantees on the solution quality. The experimental evaluation of the proposed algorithms shows up to three orders of magnitude speedups in solving single agent planning problems and up to one order of magnitude speedup in solving multi-agent planning problems. Additionally, it is demonstrated how the proposed algorithms allow for more efficient rescue operation in a large-scale disaster simulation.