Human-Machine
Guitar Hero
Independent Study
May-11-2024- 1/5
Abstract
This topic raises several research questions. First, “Can we approach the concept of cooperativeness in a limited co-op game setting and expand the discoveries found here to provide new ideas for conceptually modeling cooperative agency and scenarios?” Second, “If so, will this model lead to genuine technical and computational approaches for realizing Cooperative AI and AI Alignment?”
Asking these questions contributes to related fields in two ways. First, by narrowing down the broad concept of “cooperation” to specific settings, relevant researchers and developers can critically reassess the notion of cooperation. Second, cooperative AI researchers and developers can incorporate game designers’ perspectives into the discourse on cooperative AI, not only in terms of conceptual foundations but also in developing computational methods and algorithms.
The sections of this paper are as follows: First, the context revisits ideas and perspectives on AI and machine agency from the computational design field, where early pioneers engaged with the concept of human-computer interaction long before HCI became a formal discipline. The author then positions this research within one of today’s most vibrant areas, “Cooperative AI”, where various research approaches to solving cooperation-related problems are being explored. Then, the definition of “co-op games” and the explicit cooperative qualities derived from it will be elaborated. Following this, a detailed explanation of the design of the Human-Machine Guitar Hero will be provided. Finally, the project will be concluded with a discussion of its expected contributions and limitations.
- 2/5
The Game : Human Machine Guitar Hero
- 4/5
Agents’ Learning and Cooperation Model
The agent’s cooperation model in this game can be analyzed using the terminology of First-order systems, Second-order systems, and Conversing systems, which are combinations of the first two types, from cybernetics. According to What Is Interaction? Are There Different Types? [1], a first-order system is a simple self-regulating system that cannot adjust its own goal; its goal can be adjusted only by something outside the system. For example, a thermostat adjusts temperature based on the external environment. [1]
An interesting phenomenon occurs here: by simply stacking two first-order systems, we can create a second-order system. Second-order systems are enhanced versions of the former because the newly added component measures the first system’s impact on the environment and adjusts the objectives to align(or choose not to align) with the initial purpose. (Dubberly, 2009) This concept is analogous to meta-planning, where the agent plans the plan. Importantly, this is where true learning takes place—moving beyond merely reacting to the environment based on pre-set goals. Similarly, the mental and behavioral models of players can be described using these two systems.
First-order system: The human and machine agents individually measure the game environment and play accordingly. Both agents initiate their actions by reacting to the environment.
Second-order system: The two agents devise a strategy, or next action plan, aimed at achieving a higher score. They evaluate the outcome of their actions and modify their action plan accordingly.
We can even design an even more advanced system: a conversing system. This is done by parallelizing two Second-order systems, where the output of one learning system becomes the input for another. Borrowing words from the literature above, “Furthermore, the systems learn from each other, not just by discovering which actions can maintain their goals under specific circumstances (as with a standalone second-order system), but by exchanging information of common interest. They may coordinate goals and actions.” [1]
This Conversing system plays a crucial role in this game and serves the objective of this prototype. The system is embedded in the gameplay, where the human and machine provide feedback to one another regarding the next action plans that their counterpart should take to achieve a higher score.
- 5/5
Modeling Cooperative Machines
1. What to learn
According to the previous four expected qualities of cooperative machines, the machine agent should learn about the game environment. However, for this prototype game, this learning is pre-programmed. Rather, the project focused on the second part. To learn its human counterpart. This is done by the machine player holding attention to the human player’s every game action. This will let the machine player decide the human player’s playstyle.
2. How to model decisions?
Then, how could we model such decisions? The agent uses two numeric values. Skill level is a human player’s adeptness in hitting the correct beat at the precise timing, and the managing level is a human player’s capacity to plan a balanced contribution between the human and machine players.
Then, the machine will map these features into the playstyle quadrant. The x-axis represents the skill level, and the y-axis denotes the human’s game managing level. The machine player plots the human player’s gameplay features on the quadrant and decides whether the player is Master, Strategic, Novice, or Solo.
3. How to model actions?
| Machine’s decision | Machine’s behavior |
|---|---|
| A: Master | Follows the human player’s action plan. |
| B: Strategic | Minimize human play when difficult beat patterns are coming up. Otherwise, follow the human player’s management decision. |
| C: Novice | Leads play session, by minimizing human player’s play. |
| D: Solo | Suggest string configuration that maximizes human player’s play. |
One last piece remains. how to model actions? Based on the previous decision, the machine player will act based on its action table. For example, if the machine player decides the human player is a Novice, then it will change the string configuration in a way that human play is minimized. On the other hand, when the player is a Solo, then the machine will let the human player be engaged actively ensuring the contribution level is maintained. When the player is a Master, the machine will follow its counterpart’s action plan.
References
[1] H. Dubberly, P. Pangaro and U. Haque. “What Is Interaction? Are There Different Types?” ACM Magazine Interactions, vol.16, pp. 69-75, Jan. 2009. Accessed: Sep.15, 2024. DOI: 10.1145/1456202.1456220. [Online]. Available: https://dl.acm.org/doi/10.1145/1456202.145622
Presentation Slides