Using Aspect-Oriented Programming to Record DRL Agents' Data
Subscribe
Apple
Spotify
Overcast
Stitcher
RSS
-->
About<br>Home
Background<br>Playtesting is one of the most important processes in game development. It helps with finding bugs, evaluating the game's UX, balancing the game, and, most importantly, assessing how fun it is. This is where a developer determines whether a game idea is good and discovers flaws to fix in their game. It is an iterative process that needs to be ongoing throughout the project, but it can be very time-consuming, and there are no reliable tools that automate some of this process.<br>PEAK aims to help with that problem, but by no means does it aim to replace playtesters. The idea behind the project is to develop a game engine with integrated deep reinforcement learning (DRL) agents that a designer can dispatch to test their game at any point.<br>GUI Dashboard for PEAKThe workflow of a designer using PEAK would be:<br>Design a new level using the level editor within PEAK.<br>Train DRL agents on that level, choosing the agent playstyle you want to test with.<br>The agent would gather data, and based on your defined thresholds, it would tell you how balanced the level is and point out if there are issues within it, such as a difficulty spike on one of the platforms.<br>Balancing the Pipeline of PEAKPEAK is developed by Cristiano Politowski, Al Shifan, and Kevin Chua at Ontario Tech University. I only joined this project at a later stage. It currently only handles 2D platformer games, maybe even Metroidvanias, since they share many similarities. That scope can be expanded later on, but for now, 2D platformers are used as a proof of concept.<br>The Problem<br>My focus on this project is on data collection, aggregation, and visualization for the user. Basically, I am responsible for the step where an agent collects data while playing and uses that data to tell whether a game is balanced or not (just as an example).<br>Sounds easy, right? If the number of jumps is part of the criteria, I can just add a tracked_jumps variable and add processing to it in the jump method in the module that has it, right?<br>Well, some requirements and issues make it not so simple:<br>1- This is not a mature project yet. In the first few months I worked on it, it went through 2 major refactors.<br>2- This is an engine that supports multiple games with no architecture that ties them together, so the way I collect data should be independent of any game. Otherwise, we will end up with a lot of repeated code.<br>3- Adding new data to track and process should involve minimal changes to the codebase. Otherwise, there will be no separation between core game code and data collection code.<br>Adding variables directly violates all requirements! To solve that, I use the Observer Pattern and Aspect-Oriented Programming.<br>But What is Aspect-Oriented Programming?<br>Before this project, I knew about Object Oriented Programming, and a little bit of Data Oriented as well as Functional Programming, but I did not know what an aspect even is, and how to orient programming to it.<br>Don't worry, I did not know I was using AOP while I was using it! I only knew later when the professor pointed it out in a meeting. An aspect is the functionality we want to add to the code, but cannot directly add to it; otherwise, we tangle multiple responsibilities in the same module. So for this specific project, the data I want to collect and process are the aspects. In AOP, we specify the parts of the project we want to add our aspects to. This specification is called a pointcut, and the points of the code where aspects can be applied are called join points. A developer would only be concerned with what to run, when to run it, and how to specify the join points in AOP.<br>The textbook example for using AOP is loggers. You don't want those into your core files; you either violate the single responsibility principle or end up with high coupling.<br>Using Decorators<br>Python 2.4 introduced this really cool feature where you can define wrappers for a method that can be applied by just annotating the target method(s).<br>The image below shows an example of a decorator. Here, the green arrow points to the original function, the red arrow points to the decorator, and the orange arrow points to the annotation. With this simple definition, we can define what to do before and after the function call by just annotating it.<br>Geeks4Geeks Example of a decorator functionThe way I used decorators is as follows:<br>1- I created a class called stats_observer that holds the variables of the data of interest, with a record method for each variable to determine how we are recording them (is it a simple increment or do we need to do other checks?).<br>2- I defined a generic track decorator that takes the record method name as a parameter, and through reflection calls that method after the annotated method call.<br>3- With that, all you have to do is define record_something in stats_observer, and add...