Stochastic models are everywhere. In manufacturing, queuing models are used for modeling production processes, realistic inventory models are stochastic in nature. Stochastic models are considered in transportation and communication. Marketing models use stochastic descriptions of the demands and buyer's behaviors. In finance, market prices and exchange rates are assumed to be certain stochastic processes, and insurance claims appear at random times with random amounts. To each decision problem, a cost function is associated. Costs may be direct or indirect, like loss of time, quality deterioration, loss in production or dissatisfaction of customers. In decision making under uncertainty, the goal is to minimize the expected costs. However, in practically all realistic models, the calculation of the expected costs is impossible due to the model complexity. Simulation is the only practicable way of getting insight into such models. Thus, the problem of optimal decisions can be seen as getting simulation and optimization effectively combined. This text does not try to touch all work done in this area. Instead, many concepts are presented and treated with mathematical rigor and necessary conditions for the correctness of various approaches are stated. It is suitable as a text for a graduate level course on stochastic models or as a secondary text for a graduate level course in operations research.