Spatial Social Simulation for Neurodivergent Learners

@A.Project Lab - startup 

​

Executive Summary:

Before my doctoral studies, I co-founded A.Project Lab to develop an MVP of "Pally," a pioneering mixed reality (MR) simulation platform for neurodivergent learners. Developed on the HoloLens 1 and funded by the Verizon 5G EdTech Challenge, the project transitioned social-emotional learning from static exercises into immersive, room-scale simulations. This project served as the empirical catalyst for my Ph.D. research, proving that spatial computing could bridge the "theory-to-practice" gap in behavioral training.

Core Strategic Pillars

​

1. Cognitive Scaffolding & Constraint-Based UX

Designing for the HoloLens 1 required a radical focus on Cognitive Load Theory to ensure the hardware's technical limitations (limited field of view and basic gesture recognition) did not overwhelm the neurodivergent user.

​

• Head-Gaze Navigation: I designed a "Gaze-and-Commit" interaction model that utilized the user’s natural head orientation as the primary cursor, reducing the motor-coordination load required by manual gestures.

​

• Visual Anchor Optimization: To account for the limited hardware "sweet spot," I architected the spatial placement of non-player characters (NPCs) to remain within the user’s optimal field of vision, preventing disorientation and maintaining the "Social Presence" necessary for learning.

​

2. Deterministic Logic & Semantic Triggers

Because the project preceded modern LLMs, we utilized a Deterministic State Machine to govern the flow of the social simulation with high reliability.

​

• Keyword-Driven Transitions: I defined the interaction logic where the AI character’s responses were triggered by specific verbal keywords. This created a predictable, "failure-tolerant" environment where learners could test different social approaches.

​

• Linear & Branching States: We architected decision trees that moved the conversation through specific social "checkpoints," ensuring the pedagogical goals were met regardless of the user's verbal variance.

3. Behavioral Engineering: The ABC Framework

I integrated core principles of Applied Behavior Analysis (ABA) into the simulation’s architecture to automate social-emotional reinforcement.

​

• Antecedent-Behavior-Consequence (ABC): The simulation was built on this loop—the NPC provided an Antecedent (e.g., crying), the user provided a Behavior (e.g., asking "How can I help?"), and the system provided an immediate Consequence (e.g., the character smiling and thanking them).

​

• Positive Reinforcement Loops: We utilized AR-native rewards (visual and auditory cues) to provide immediate feedback, a critical component in helping users with autism generalize these skills to real-world scenarios.

4. The Research Genesis: Pre-Doctoral Foundations

Pally was the direct precursor to my Ph.D. research at Columbia University, establishing the link between Social Agency Theory and immersive technology.

​

• The "Persona Effect" in AR: I observed that the spatial presence of a 3D avatar significantly increased user motivation compared to 2D video-based tools, a finding that became a cornerstone of my later work on AI Pedagogical Agents.
  
  In addition, to create realistic avatars, we used a volumetric studio to capture a real person and create a 3D duplicate. By recording someone in 360 degrees, we were able to generate a digital version that moved and looked like the original person. This technology, available at the time, produced close-to-reality avatars, which are different from the hyper-realistic human avatars generated by AI today.

​​

• Empirical Validation: The success of this platform—validated through 5G lab testing and national media attention—confirmed that "Agentic" environments are the future of personalized, scalable education.

Impact & Recognition

​

• Theory-to-Experience: Translated Applied Behavior Analysis (ABA) frameworks into system-logic evaluations. By utilizing the ABC (Antecedent-Behavior-Consequence) model as a rubric for AI output, I ensured that virtual agents provided consistent positive reinforcement loops, increasing the generalization of social skills in simulated environments."

​

• Innovation Awards: Winner of the Verizon 5G EdTech Challenge; 2nd Place LG U+ XR Challenge.

​

• Media & Outreach: Featured on CBS Mission Unstoppable; ISTE Young Educators Ignite Talk.

​

• Strategic Evolution: This foundational work in deterministic social logic informed the advanced "Agentic AI" architectures I now lead for large-scale enterprise training.

​