Game equipment strategy optimization

Research question

Equipment selection couples a limited gold budget with interactions between item attributes. I modeled purchases as sequential decisions and used a simulated combat objective to study how a DQN policy allocates resources across a six-step selection process.

My contribution

• Built a custom equipment-selection environment with accumulated item attributes, remaining budget, and a six-step purchase horizon.
• Implemented rewards combining intermediate simulated damage, a terminal outcome, and penalties for unaffordable choices; connected the environment to DQN training.
• Organized item processing and sensitivity-analysis code to examine how modeling assumptions affect equipment choices.

Technical approach

• Treat each equipment purchase as a discrete action, updating remaining gold and accumulated combat attributes.
• Score the combination using a simulated opponent model. Combine intermediate damage feedback with a terminal reward and penalties for unaffordable choices.
• Connect the resulting transitions to DQN training and use sensitivity-analysis tooling to study the relationship between input attributes and strategy selection.

Results & outcomes

• Built a resource-constrained learning workflow linking item data, environment transitions, reward shaping, and DQN training.
• Under the simulated opponent setting, the project makes budget allocation and attribute combinations explicit, with sensitivity-analysis tooling to examine how input assumptions influence the selected strategy.

Modeling decisions and analysis

The central question is how to express a one-off combinatorial choice as a sequential learning problem. Selecting one item at a time keeps the action space manageable, while the state must retain the selected combination, remaining budget, and information relevant to later purchases.

I considered terminal-only rewards and intermediate reward shaping. Terminal damage directly reflects the final objective; intermediate feedback makes credit assignment denser, but rewarding purchase speed can encourage cheap items rather than better combinations.

An alternative is to choose the whole set in one action. With N available items and six slots, an ordered action space with repetition grows as N⁶. This motivates the sequential formulation and analysis of whether state aggregation preserves the information needed for decision-making.