KNOCKNOCKNIK




Little Cooperative Machines v1.0

Interim Master’s Thesis

Dec-16-2024



0/4“How can machine agents be designed to learn and cooperate with human players in real-time, emulating the nuanced and dynamic experience of co-op video games?”
  • 1/4

  • Abstract


         To Develop Little Cooperative Machines: Building to Interacting is a research project addressing the question: “How can machine agents be designed to learn and cooperate with human players in real-time, emulating the nuanced and dynamic experience of co-op video games?” This project focuses on building actual games and developing machine agents that interact and cooperate with their human partners.

        Supporting the high fidelity of cooperative human-machine interaction represents one of the next great challenges in the Cooperative AI research field. As J. C. R. Licklider envisioned in his 1960 paper “Man-Computer Symbiosis”, human and machines' symbiotic partnership—intellectual cooperation between agents to make decisions, solve problems, and control situations, [1] is now becoming a reality driven by advancements in AI and machine learning.

        Despite this progress, a significant gap remains in how we approach cooperative AI. Current academic efforts predominantly focus on theoretical definitions of cooperation and mathematical modeling of cooperative behaviors. While these contributions are valuable, this study shifts the emphasis to the experiential qualities of cooperation with machine agents. Grounded in the hypothesis that rich cooperative interactions can not only be emulated with machine agents but also that an empirically grounded approach will advance the development of robust and genuine cooperative AI algorithms, this research aims to bridge the gap between theory and practice.

        To test this hypothesis, the research introduces a game called “Paint Your Partner” and develops a little cooperative machine agent within the game. Games provide an ideal environment for machine agents to learn game environments and align with their human partners. It also allows designers to carefully devise interactions embedded in gameplay. The concept of little reflects a deliberate design constraint in this research: the agents should be approachable, explainable to users, and computationally lightweight. While Reinforcement Learning (RL) is a key technology in this process, the research also explores intersections with embodied intelligence, robotics, and psychology. Once the gameplay system is operational, the performance and qualities of cooperative interactions will be analyzed to evaluate the initial hypothesis.
  • 2/4

  • Research Questions


  1. How can machine agents be designed to learn and cooperate with human players in real-time, emulating the nuanced and dynamic of the co-op video game experience?
  2. How does an interaction-focused approach facilitate the genuine crossbreeding of machine learning paradigms in designing explainable, contextual, and lightweight cooperative machines?


    • 3/4

    • The Game : Paint Your Partner

            The win condition is simple: the human and machine players must reach the goal tile together, with both players matching their body color to the color of the goal tile. To achieve this, they can paint themselves and each other. The game tracks the number of color changes, which serves as a cooperative performance metric for evaluation. Additionally, a feature to promote cooperation: one player can be frozen during gameplay, and the other player can unfreeze their partner using a fire item.

    Paint Your Partner learning and interaction field diagram


    • 4/4

      • The Agent: Little Cooperative Machines

            A nested layered learning system with reinforcement learning as a foundational technology was introduced to support the high resolution of LCM's behavior and its online learning. Layered learning (LL) is a hierarchical machine-learning system, where each layer directly maps input to output. Using this model, even a highly complicated task could be decomposed into subtasks. The ability to perform subtasks is a necessary condition to carry out high-rank tasks that build on top of those subtasks. [5] The concept of LL has been actively discussed in Robotics. According to the literature “Intelligence Without Representation” or “A Robust Layered Control System for a Mobile Robot" Rodney Brooks suggested a type of LL called Subsumption Architecture. Here, the author views each layer of the LL model as an “activity-producing” layer. Interestingly, he argues that the machine out of Subsumption Architecture is merely a collection of “competing behaviors”. [6][7] 

            On the other hand, Reinforcement Learning (RL) is concerned with agent-based decision-making to maximize rewards in a given environment. Unlike other deep learning applications where a dataset is provided for training a model, RL algorithms instead must learn unsupervised through sparse reward signals [8].

    Outer layer of LCM agent


    Inner cooperative layer of LCM agent


    Pseudocode to define Cooperative LL block
    # To Define Cooperative Layered Learning Block
if (partner not movable):
    u ← Make partner movable
else if (self not movable):
    u ← Make itself movable
else if (partner not match color):
    u ← Match partner’s body color to goal tile
else if (self not match color):
    u ← Match its body color to goal tile 
else:
    u ← Move to the goal tile


    References
    [1] J. C. R. Licklider. Man-Computer Symbiosis. IRE Transactions on Human Factors in Electronics, volume HFE-1, pp. 4-11, March. 1960. Accessed: Dec. 04, 2024. [Online]. Available:https://groups.csail.mit.edu/medg/people/psz/Licklider.html
    [5] P. Stone and M. Veloso. “Layered learning”. In Proceedings of the 11th European Conference on Machine Learning (ECML'00). Springer-Verlag, Berlin, Heidelberg, 369–381. 2000. Available: https://doi.org/10.1007/3-540-45164-1_38
    [6] R. Brooks. “Intelligence without representation”. Artificial Intelligence 47, 139-159, 1991.
    [7] R. Brooks. "A robust layered control system for a mobile robot". IEEE Journal on Robotics and Automation, vol. 2, no. 1, pp. 14-23, March 1986.
    [8] V. Mnih, K. Koray, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller. “Playing Atari with Deep Reinforcement Learning.” arXiv, December 19, 2013. https://doi.org/10.48550/arXiv.1312.5602.

    Presentation Slides






    Check out more cool stuffs:
    Proceed to Arch + Nik

    Nik Kim

    GitHub
    ML Blog
    YouTube


    A New Turing Test

    I envision machines that pass my own version of the Turing test: “Can we play with them?” I remember the moments of playing co-op video games with my little brother under a shabby tent made of chairs and blankets. I cherish these small, precious memories—they’re still alive within me.     Favourite...

    - Fungus: Mushrooms
    - Animal: Dogs
    - Series: Arcane (Jinx is my #1)
    - Movie: EEAT
    - Game: To The Moon     - Instrument: Guitar