Learning in Immersive Worlds

Thought I'd share this great report titled, "Learning in Immersive Worlds." It's directly relevant to several of our serious games projects. Some of the best stuff starts on the bottom of page 11:

...this report adopts a more general approach considering learning through play, with games and with simulations as part of a more general process of learning in immersive worlds. Immersive worlds are taken here to mean ‘microworlds’ that is the space of the game, the game-world. See call out box on microworlds. Realising that some games and simulations may be puzzles, board games or adventure-style games, this report focuses rather upon games and simulations as facilitators of virtual experiences rather than as sources of data or information which can be assimilated. Learning in immersive worlds itself is a process of learning, of flow or of activity (Csikszentmihalyi, 1992) that is a set of interactions or constructed reasoning that may or may not be transferred into lived experiences, expressions or outputs.

Microworld
The concept of ‘microworlds’ was introduced by Minsky and Papert (1971). Microworlds are given domains or environments which may be explored in a non-linear way by users or learners, the environment includes artefacts and objects, and learners may learn through exploring the environment and its objects in a relatively open-ended way.

For learning to be effective in immersive worlds, or indeed any contexts, a relationship needs to be made between what is learnt and how it is applied in practice. This broadly follows Kolb’s experiential learning cycle (Kolb, 1984) although connections are not always made between what is learned and how it is applied in practice. This is the reason why post-exercise reflection and correct embedding of the ‘virtual experience’ into a learning context is so important. The phrase ‘exploratory learning’ will be used in this report therefore to mean the learning process that takes place in an immersive, virtual context, that may (or may not) then be transferred into lived experience in the real-world, a real-world experience, or abstract reasoning or imagining at a later time or in another similar or usually dissimilar context. Exploratory learning is in a sense taken to mean play as rehearsal or ‘pattern formation’ or in a neurological sense as a ‘mapping of types of maps’ (Edelman, 1992: 109)

de Frietas, S (2006). “Learning in Immersive Worlds: A Review of Game Based Learning.” JISC, retrieved January 15, 2006 from http://www.jisc.ac.uk/media/documents/programmes/elearning_innovation/gaming%20report_v3.3.pdf

JHotDraw

JHotDraw (http://www.jhotdraw.org/) is one of the best open source Java drawing tools available (not that there are many). It covers the basics of vector drawing with the types of tools that people expect (see image below).

The main reason that I'm interested in it is that I like to draw. But I don't just like to draw for other people, I like to draw pictures as inputs to programming. A framework like JHotDraw gives me most of what I want from a drawing tool in a form that I can easily customize to create visual programming languages or semantic images. More about that to come in later posts.


If you're interested in getting started with JHotDraw, you should be very comfortable with Java. Although the code is pretty readable, there is a lot of it, there is very little up-to-date documentation and the included examples are fairly complex. To help you get started, I'm posting my implementation of the simplest possible JHotDraw application - HelloJHotDraw. This code is based on the latest version of JHotDraw at the time of this writing: v7.4.1 You can find the JavaDoc at http://www.randelshofer.ch/oop/jhotdraw/.

-- code --

import org.jhotdraw.app.SDIApplication;
import org.jhotdraw.samples.draw.DrawApplicationModel;
import org.jhotdraw.samples.draw.DrawView;

public class HelloJHotDraw extends SDIApplication {

public HelloJHotDraw() {

}

/**
* @param args
*/
public static void main(String[] args) {
HelloJHotDraw app = new HelloJHotDraw();
DrawApplicationModel model = new DrawApplicationModel();
model.setViewClass(DrawView.class);
app.setModel(model);
app.launch(args);
}

}

Combat Sim Wargames

I've been looking for realistic, modern US Army computer-based wargames from a commander's perspective. No first-person shooters. My emphasis is on commercially available games that use realistic Army terminology, symbology and tactics.

I found some good background information on Wikipedia under Wargaming. Another website to check out: DoD Game Community.

Here is the list of games that I found, organized by size of echelon. Several of these are commercial versions of training games that are being used by the military:

• Close Combat - Modern Tactics (Company Level)
• Full Spectrum Command (Company Level) - Not Commercially Available
  
• TacOps (Battalion Level)
• Armored Task Force (successor to Brigade Combat Team) (Battalion & Brigade Level)
• Decisive Action (Division & Corps Level)

I also looked at The Operational Art of War (Battalion, division, and corps combat), which looks fun, but not particularly realistic.

Daily Practice

On a bad weather day a few months back, my boys were suffering a bit from cabin fever. I started thinking and researching activities that could burn up some of their energy without destroying our house.

About the same time, I was watching the Super Soccer Skills DVD by Dan Metcalfe. In this video, Dan demonstrates a bunch of drills that can be done in the living room in front of the TV. The drills are all footwork drills, dribbling in place. The DVD fits right in with the fundamental skill of soccer as shown in the Coerver pyramid below, ball mastery. In particular, beginning players (like my kids) really benefit from spending lots of time with the ball at their feet, learning the feel of the ball and how it responds to various types of dribbling, turning and passing.


Coerver Pyramid of Player Development (coerversoccer.com)

Now the really cool thing about the drills in Dan's video is that its the kind of thing that can be practiced every day. It doesn't require good weather, special equipment, etc. The problem with these drills is that they are a bit too hard for young kids to do and also they are quite boring (according to my boys). Also, my boys already have so many daily requirements such as chores, piano practice, daily reading and math that I hesitate to add another requirement. (Interesting how Scott's first grade class has gone to daily, at home, extra practice of fundamental skills like reading and math.)

But, I really like the concept of trying to work in extra soccer practice when the kids would be watching TV anyway. The key, I think, is to make practicing fun enough that they'll want to do it. So I put two and two together and came up with a set of quick and simple activities that kids can do indoors to work on their soccer footwork (with minimal damage to the furniture). And, I'm working on an idea for incorporating these activities into some type of game, to make the drills less mechanical and more fun. So far, the game is coming together nicely. I'll post back here if anything comes of it.

Education and Practice

I've been thinking a lot about education lately. My wife is getting her teaching certificate, my three sons are going through preschool and elementary school, I'm coaching soccer, I'm learning new skills and I've been working on several computer-based training projects.

I'm just starting to dive deeper into educational theory, so much of what I'll be posting will be from my limited point-of-view. On first glance, there seems to be dozens or hundreds of different educational theories or systems in the literature. A large percentage of these seem to have very little in terms of independent validation.

With regards to teaching a new skill, the following elements seem to be widely accepted:

• Explain
  
• Demonstrate
• Practice
• Assessment
  
• Feedback

From my limited reading and experience, practice is king. From everything I've seen and read, if you want to improve performance, you need practice -- the more the better (almost always). The other four bullets are about setting the student up to practice the right thing in the right way. I'll discuss this more in my next post, but for now I'll lay out a series of references that I've come across recently that show the importance of practice.

Power Law of Learning
I first learned of the power law of learning when I was taking Dr. John Anderson's cognitive psychology class at Carnegie Mellon University. Also called the power law of practice, this law states that memory performance improves as a power function of practice and is one of the most validated findings in education. Basically, the more you practice, the better you get and this improvement follows a logrithmic power curve -- faster improvement at the beginning followed by slower improvement as you reach peak performance. This curve is one of the origins of the term, "learning curve." The following graph from Dr. Anderson's cognitive psychology textbook demonstrates this effect.



The 10,000 Hour Rule
The next reference comes from a very popular recent book called Outliers by Malcom Gladwell. In this book, Gladwell researches and interviews successful people on the extreme outer edge of what is statistically possible and tries to determine how they got there. He finds that in almost all cases it is an accumulation over time of high-quality practice and repetition. Read the wikipedia review of Outliers for a more complete description.

Sports
Not only is this effect true in academics and business skills, but it also applies to sports and physical skills. Gladwell's book documents the case for Canadian youth hockey. Similarly, I remember an article about Russian youth tennis camps based on the same premise (but I can't find it right now). Coaching Youth Soccer by John McCarthy has this nice little nugget, "The more you get your players drilling their skills, within reason, the better your team will be. That's the surest thing about any sport."

Google Maps for Knowledge Elicitation

I'm experimenting with Google Maps "My Maps" drawing tools for knowledge elicitation. In this case, I'm specifically interested in knowledge elicitation for ground combat. My plan is to create a custom map someplace on the globe and use the drawing tools to draw up a combat situation using the MIL-STD 2525 notation. My Maps has very simple drawing tools, but I think with just lines, polygons and some custom icons like the ones below, I can do some interesting things.



Here's my second set. This time I made them a bit smaller, added red platoon icons and some icons to indicate that the red units are not visible. I also made the icons a standard size 50x50 pixels so that they line up better.



Here's an example of what you can get out of My Google Maps just using these icons and its standard drawing tools.



I'll write up my procedure and results soon.

Interface Design Methods for Intelligent & Complex Systems I: Context-based Programming

I’ve spent a good chunk of the last few years doing research and design around adaptive user interfaces and interfaces that include or connect to systems with some significant AI or complex system component. Over that time I’ve noticed some recurring themes and methods popping up again and again. Over the next few weeks I’m going to try to get some of them written up. Since this is my first time trying to capture the core ideas in a document feedback is very much appreciated.

Context-Based Programming
The idea of context-based programming comes from the observation that many applications, more all the time, change core behavior based on changes in their context. While it’s easy to get hung up on precisely defining ‘behavior changes’ and ‘context’, I’ve developed what I think of as a simple clarifying design method that helps me think about both issues.

Start with Data-Driven Programming

I see context-based programming as a natural extension of data-driven programming, an idea with a rich history in computer science / software engineering. Stated briefly, and from my own perspective, data-driven programming is an approach to software design that favors moving program control information from the ‘hard-coded’ language of the program to a data-structure populated from some other source (constant definitions, config file, user parameters, database, or external message). There are a lot of implications and analysis questions that go into decided how much a given program should be data-driven but they usually boil down to issues of

• Time: how rapidly (over what time period) will a given variable need to be changed relative to other changes in the source code. For example, a user preference for large or small text in a browser should be modifiable without going back to the original programmers.

• Difference in deployment environment: How will one deployment of the software legitimately differ from other deployments. For example, online-game players will want to choose a game server close enough on the net to insure fast response.

• Difference in development responsibilities: Separating out development responsibilities often leads to the realization that significant differences occur in expertise, representation requirements, and time cycles. Computer games typically create level editors (data file editors) to allow artists and game designers to work a level that is comfortable to their skills and that better represents the game abstractions. The files from this editor are then input as data into the game.

There are lots of practical impacts on code reuse, software complexity, software usability, and development time that go into deciding how much of a system should be data driven, but it’s a trade off space that is (or should be) comfortable to a professional software engineer.

To sum up, data-driven programming is about paying a price for reading control data elements from a structure external to the source code in order to improve some needed system capability.

Extending Data-driven programming into Context-based programming

If a program can read in control data elements, than it can read in more than one set of control elements at different times. We can think of each set of control elements as a policy. User A (big text) or Deployment A (server 1, game level 1) has a different policy than User B (small text) or Deployment B (server 2, game level 2). If the system can load multiple policies there must be a mechanism for loading the policy. This mechanism may be as simple as a configuration file loader or data base query and happen once at program load or may involve continual update by an interactive user or by back end functions. Regardless of how it is implemented, conceptually each program that supports policies must provide some code that acts as a policy manager. The pattern then is from fully internalized source code we move some variables or constants out to external policies that are injected by a policy manager.

What about the policy manager, though? Isn’t it just another block of code that can be implemented (as needed) in a data-driven manner? What if in one circumstance, an event (system start, user action, external data update) causes the instantiation of Policy A and under another circumstance that same event should cause the instantiation of Policy B?

For example, imagine the ringer function in a cell phone software application. Typical cell phones allow the user to define a policy that includes the volume of the ringer output and the specific file to play for each member of a contact list (and incoming calls not in the contact list). This ringer policy allows each cell phone to reflect the personality of the phone owner, an important consideration in the world of consumer electronics. This policy, though is fairly rigid. My cell phone is configured to play, moderately loudly, the rowdy goofy song Yakety Sax, from the Benny Hill show when either of my two brothers call. I love this because it captures, for me, an important part of my relationship with my brothers and how I imagine them. It would, however, be embarrassing if one of them were to call while I was in a serious meeting. (Something that has happened at least once recently). To avoid this, most phones allow the default policy to be modified on the fly. At any time I can easily lower the phone volume or change the ring tone. But what if the phone designer wanted to do better? Today’s phones often have the phone owner’s calendar available to them. What if I could create my default policy (which contains Yakety Sax) and a second “meeting” policy which overrides Policy A by setting the ringer to “vibrate - audio volume = 0”. The phone would then select which policy to apply based on the externalized calendar data.

This is the essence of context-based programming. In context-based programming, the policy manager itself is designed according to data-driven programming principles. This allows the system to provide different behaviors at different times based on a set of observable conditions. This then requires a set of meta-policies that define the conditions under which the primary policies are applied. We’ve now entered the world of AI and user-adaptive systems.

A design method for Context-Based Programming

While is absolutely true that the calendar-based ringer manager described above could be designed and implemented by a skilled software engineer without ever cracking open an AI reference book (AI code is just code, after all), it is equally true that this kind of pattern-matching of observable conditions to dynamically modify system behaviors has been far more studied and described in the AI community than in the traditional software community. Particularly for meta-policies that use non-trivial conditions. What if the ringer manager needed select a ringer policy by including both the calendar, the location of the phone owner (driving or not), the users current phone use, the output modes available (volume, vibrate, screen) and the background noise level? Each of these elements, and probably a lot more, are part of the policy manager’s context and could be included in the meta-policy. The bottom line is that the larger the set of context elements available and the larger the set of possible system configurations possible, the more context-driven programming pushes for the inclusion of AI techniques including Dialog Management, Agent Based Systems, Formal Logic, Heuristic Search, Constraint-Based Reasoning, Probabilistic Reasoning, and Decision Trees.

Before I worry about whether to apply an AI technique or which AI technique to apply, I start with analyzing the system from the perspective of policies and meta-policies.

1. What are the core changes (policy elements) that should be defined in data and how will moving these policy elements into data impact the system. What can we really do here that makes sense. (Users want to set ringer sound on their cell phone. While the cost of this may be high and involve adding memory to the phone, new management UIs, network performance issues, and server design issues, the feature will be popular enough to drive additional fee and/or ringtone sales to justify the effort)
2. What is the life cycle of these core changes? Which of these core changes will be well supported by a set of static (not-conditional) policies. Which ones require dynamic (conditional) policies? (Users would like their ringer policies to change reflect their calendar; for other reasons phone designers are already investing in providing calendar data on phones).
3. What are the conditions (context) that drive changing from one policy to another and what are the algorithms for organizing the context, selecting the policy, and applying the policy.
   

My next essay will expand on this notion by describing the adaptive-system loop and how it impacts and reflects context-driven design. Stay tuned.