Exploration portfolio management is crucial for oil companies operating deep-water assets, whose portfolios include hundreds of exploration prospects with varying risk profiles and hydrocarbon volumes and dozens of producing platforms with varying excess capacity available for new tie-back projects. In this paper, we describe a methodology for portfolio planning and optimization (PPO) that uses reservoir simulators to model production of exploration scenarios, and the Markov decision process (MDP) to optimize the portfolio development plan. In the PPO methodology, prospects, existing platforms, and segments of the export pipeline network (that connect platforms to markets onshore) are modeled as wells and nodes in the reservoir simulator. As such, field management logic typically used for surface facilities is adapted to model the export network on a regional level. The exploration plan is modeled as an MDP to capture the impact of outcomes of earlier events on later decisions. Optimization is performed on a specially parameterized policy to allow for the optimization of exploration timing, tie-back options, and project sequences. The adapted simulation model is integrated with an external engine that evaluates the cost and economics of each scenario. The optimizer orchestrates the large number of runs needed for multi-objective optimization under uncertainty. This implementation is deployed to portfolio planners and applied to portfolios with more than 200 prospects and 70 platforms. Results showed that field management logic in the reservoir simulator is adequate to model all oil and gas capacity constraints in the platforms and export path on a regional level. Three optimization approaches are investigated with a synthetic portfolio example. It is shown that the black-box bi-objective optimization (BOO) can improve portfolio values, but the improvement is modest if the number of functions evaluated is small. Proxy accelerated optimization (PAO) allows for a larger population size and more iterations, producing solutions that are superior with similar computational costs. Finally, an automatic hierarchical optimization (AHO) approach is also examined, which leverages the structure of the problem to cut options step-by-step. AHO is shown to provide superior results to BOO and PAO in the example case but requires very problem-specific logic that may be hard to generalize. As far as we know, this is the first-ever adaptation of reservoir simulation for modeling productions from an exploration portfolio. The MDP formulation of the portfolio optimization problem and the policy-based parameterization are also first in the literature. While the methodology is introduced with applications to the deep-water asset class, the modeling approach applies to generic portfolio problems in other asset classes and other parts of the world.