Interface Design Methods for Intelligent & Complex Systems I: Context-based Programming

I’ve spent a good chunk of the last few years doing research and design around adaptive user interfaces and interfaces that include or connect to systems with some significant AI or complex system component. Over that time I’ve noticed some recurring themes and methods popping up again and again. Over the next few weeks I’m going to try to get some of them written up. Since this is my first time trying to capture the core ideas in a document feedback is very much appreciated.

Context-Based Programming
The idea of context-based programming comes from the observation that many applications, more all the time, change core behavior based on changes in their context. While it’s easy to get hung up on precisely defining ‘behavior changes’ and ‘context’, I’ve developed what I think of as a simple clarifying design method that helps me think about both issues.

Start with Data-Driven Programming

I see context-based programming as a natural extension of data-driven programming, an idea with a rich history in computer science / software engineering. Stated briefly, and from my own perspective, data-driven programming is an approach to software design that favors moving program control information from the ‘hard-coded’ language of the program to a data-structure populated from some other source (constant definitions, config file, user parameters, database, or external message). There are a lot of implications and analysis questions that go into decided how much a given program should be data-driven but they usually boil down to issues of

• Time: how rapidly (over what time period) will a given variable need to be changed relative to other changes in the source code. For example, a user preference for large or small text in a browser should be modifiable without going back to the original programmers.

• Difference in deployment environment: How will one deployment of the software legitimately differ from other deployments. For example, online-game players will want to choose a game server close enough on the net to insure fast response.

• Difference in development responsibilities: Separating out development responsibilities often leads to the realization that significant differences occur in expertise, representation requirements, and time cycles. Computer games typically create level editors (data file editors) to allow artists and game designers to work a level that is comfortable to their skills and that better represents the game abstractions. The files from this editor are then input as data into the game.

There are lots of practical impacts on code reuse, software complexity, software usability, and development time that go into deciding how much of a system should be data driven, but it’s a trade off space that is (or should be) comfortable to a professional software engineer.

To sum up, data-driven programming is about paying a price for reading control data elements from a structure external to the source code in order to improve some needed system capability.

Extending Data-driven programming into Context-based programming

If a program can read in control data elements, than it can read in more than one set of control elements at different times. We can think of each set of control elements as a policy. User A (big text) or Deployment A (server 1, game level 1) has a different policy than User B (small text) or Deployment B (server 2, game level 2). If the system can load multiple policies there must be a mechanism for loading the policy. This mechanism may be as simple as a configuration file loader or data base query and happen once at program load or may involve continual update by an interactive user or by back end functions. Regardless of how it is implemented, conceptually each program that supports policies must provide some code that acts as a policy manager. The pattern then is from fully internalized source code we move some variables or constants out to external policies that are injected by a policy manager.

What about the policy manager, though? Isn’t it just another block of code that can be implemented (as needed) in a data-driven manner? What if in one circumstance, an event (system start, user action, external data update) causes the instantiation of Policy A and under another circumstance that same event should cause the instantiation of Policy B?

For example, imagine the ringer function in a cell phone software application. Typical cell phones allow the user to define a policy that includes the volume of the ringer output and the specific file to play for each member of a contact list (and incoming calls not in the contact list). This ringer policy allows each cell phone to reflect the personality of the phone owner, an important consideration in the world of consumer electronics. This policy, though is fairly rigid. My cell phone is configured to play, moderately loudly, the rowdy goofy song Yakety Sax, from the Benny Hill show when either of my two brothers call. I love this because it captures, for me, an important part of my relationship with my brothers and how I imagine them. It would, however, be embarrassing if one of them were to call while I was in a serious meeting. (Something that has happened at least once recently). To avoid this, most phones allow the default policy to be modified on the fly. At any time I can easily lower the phone volume or change the ring tone. But what if the phone designer wanted to do better? Today’s phones often have the phone owner’s calendar available to them. What if I could create my default policy (which contains Yakety Sax) and a second “meeting” policy which overrides Policy A by setting the ringer to “vibrate - audio volume = 0”. The phone would then select which policy to apply based on the externalized calendar data.

This is the essence of context-based programming. In context-based programming, the policy manager itself is designed according to data-driven programming principles. This allows the system to provide different behaviors at different times based on a set of observable conditions. This then requires a set of meta-policies that define the conditions under which the primary policies are applied. We’ve now entered the world of AI and user-adaptive systems.

A design method for Context-Based Programming

While is absolutely true that the calendar-based ringer manager described above could be designed and implemented by a skilled software engineer without ever cracking open an AI reference book (AI code is just code, after all), it is equally true that this kind of pattern-matching of observable conditions to dynamically modify system behaviors has been far more studied and described in the AI community than in the traditional software community. Particularly for meta-policies that use non-trivial conditions. What if the ringer manager needed select a ringer policy by including both the calendar, the location of the phone owner (driving or not), the users current phone use, the output modes available (volume, vibrate, screen) and the background noise level? Each of these elements, and probably a lot more, are part of the policy manager’s context and could be included in the meta-policy. The bottom line is that the larger the set of context elements available and the larger the set of possible system configurations possible, the more context-driven programming pushes for the inclusion of AI techniques including Dialog Management, Agent Based Systems, Formal Logic, Heuristic Search, Constraint-Based Reasoning, Probabilistic Reasoning, and Decision Trees.

Before I worry about whether to apply an AI technique or which AI technique to apply, I start with analyzing the system from the perspective of policies and meta-policies.

1. What are the core changes (policy elements) that should be defined in data and how will moving these policy elements into data impact the system. What can we really do here that makes sense. (Users want to set ringer sound on their cell phone. While the cost of this may be high and involve adding memory to the phone, new management UIs, network performance issues, and server design issues, the feature will be popular enough to drive additional fee and/or ringtone sales to justify the effort)
2. What is the life cycle of these core changes? Which of these core changes will be well supported by a set of static (not-conditional) policies. Which ones require dynamic (conditional) policies? (Users would like their ringer policies to change reflect their calendar; for other reasons phone designers are already investing in providing calendar data on phones).
3. What are the conditions (context) that drive changing from one policy to another and what are the algorithms for organizing the context, selecting the policy, and applying the policy.
   

My next essay will expand on this notion by describing the adaptive-system loop and how it impacts and reflects context-driven design. Stay tuned.

Acroynms

Not only is the pen is mightier than the sword, but the acronym is mightier than the military / industrial / research complex.



Hat tip to PhDComics.

Semantic and Wiki

Just wanted to respond to Jack's post, Challenges using Semantic Wiki's for Knowledge Engineering, to clarify my intention a bit.

1. I agree that you should stick with the wiki and skip the formal relationships during the knowledge elicitation phase.
   
2. I do not expect that "a single correct and uniform world view that all stakeholders should share and will share with a little effort."
   
3. I do not want to use Semantic Wikis to create some type of process for model driven engineering.
4. I'm not interested in "mixing knowledge elicitation with design to the detriment of both."
   

What I like about semantic wikis is that they are both semantic and wiki. The wiki can be used as a tool that "captures differences, reconciles them when reasonable but not when unreasonable." I see wikipedia as a good example of this. Wikipedia's controlled vocabulary is constantly changing (I changed an entry yesterday) and is able to accommodate a wide variety of opinions and respond to the latest ideas and trends.

So what about the semantic part? I see this coming in as we move into the design phase. Why put it into the Wiki? I see this as a good way to facilitate communication between the stakeholders and system designers and expose a symbolic AI system's "mental model" of the world. Just as the various stakeholders' knowledge is captured in the wiki, I give the developing AI system it's own voice along side (rather than replacing) the human voices.

Where they all agree, you can have a single wiki entry. Where they disagree you can list all viewpoints. The semantic wiki tagging system allows you to differentiate between them, just as tagging in other forums facilitates other multi-perspective folksonomies.

What this implies, I think, is that you may be able to start with a standard Wiki and use it for quite a while and then add semantic entries on in addition to the others.

Challenges using Semantic Wiki's for Knowledge Engineering

Jack here...I've been thinking about about Keith's post about Semantic Wiki's and thought I'd chime in. I'm a big fan of anything that makes requirement gathering and/or knowledge elicitation easier, but I'm nervous about the semantic wiki approach.

My understanding is that a Semantic Wiki is a Wiki where the links between pages have been given formally defined (e.g. ontological) relationships. Keith talked about this in his post on controlled Vocabulary when he cited wikipedia "a carefully selected list of words and phrases, which are used to tag units of information. ... Controlled vocabularies solve the problems of homographs, synonyms and polysemes by ensuring that each concept is described using only one authorized term and each authorized term in the controlled vocabulary describes only one concept. In short, controlled vocabularies reduce ambiguity inherent in normal human languages where the same concept can be given different names and ensure consistency."

Sounds good, right? Gathering information is about reducing ambiguity, right? Good old Occam's Razor says otherwise. Any explanation should be as simple as possible, but no simpler. There is a big assumption built into controlled vocabularies and ontologies which is that it is possible and desirable to develop a controlled vocabulary that can be uniformly agreed on by all stake holders (or that any disagreements are minor and can be reconciled within the structure of the formal representation.) This concept implies that there is a single correct and uniform world view that all stakeholders should share and will share with a little effort. This is a large claim that is not supported by any formal model development process or activity of which I’m aware. In fact, getting stakeholders to agree on things as basic as word meanings is immensely difficult, and is so for good reaons. Different stakeholder's may have radically different, but equally useful ways of understanding a domain of interest. These differences may depending on their cultural background, their training and experience and the specifics of their work practice.

While a formal model can be developed even in groups with legitimate conceptual differences, it usually does so at the expense of one of the parties or by the creation of artificial compromises. This may be an acceptable outcome in some situations but not others. In the requirement gathering phase of user interface design, this seems like an overly constraining starting point. If we are to accurately elicit information we need a method that initially captures differences, reconciles them when reasonable but not when unreasonable. Otherwise we are mixing knowledge elicitation with design to the detriment of both.

My vote ... stick with the wiki and skip the formal relationships. This messes up part of why Keith wants formal relationships... that they'll be useful for providing knowledge to AI systems. I've got concerns about that too, but will write about that tomorrow.

jSoar

As part of an investigation into rule based systems, today I downloaded jSoar, a java implementation of the Soar kernel.

jSoar is currently under heavy development and there are no releases on the downloads page so the only way to get it is by checking out the code directly through the subversion server. Once you check out the code, you can build it either with the included Ant build.xml build script or by importing the code into Eclipse. I tried both and had no problems.

Next on to my test case. I want to build a simple Soar agent that responds to a single, simple rule from the soccer domain: "If the ball is controlled by me, then shoot the ball." Here is my Soar rule:

jSoar comes with a set of unit tests that both test the validity of the system as it continues to develop and serve as samples for how the system is to be used. In particular, InputOutputImplTest.java shows how to create and initialize a new Soar agent, attach to the input and output cycle through agent listeners, send data to the agent on the input link and receive the agent's output.

Having an input and output link and a regular decision cycle makes Soar much more friendly to agent developers than other rules engines like Drools. On the flip side, the syntax for spoon-feeding changes into working memory is no fun.

Support for jSoar is provided through the jsoar-dev Google Groups web page.

Here's my simple test java code:

package com.referata.soccer;

import org.jsoar.kernel.Agent;
import org.jsoar.kernel.RunType;
import org.jsoar.kernel.events.InputCycleEvent;
import org.jsoar.kernel.io.InputBuilder;
import org.jsoar.tcl.SoarTclInterface;
import org.jsoar.util.events.SoarEvent;
import org.jsoar.util.events.SoarEventListener;

public class SoccerSoarTest implements SoarEventListener {

private Agent agent;

public SoccerSoarTest() throws Exception {
agent = new Agent();
SoarTclInterface ifc = new SoarTclInterface(agent);
agent.initialize();
agent.getEventManager().addListener(
  InputCycleEvent.class, this);
ifc.sourceFile("/res/soccer.soar");
agent.runFor(3, RunType.DECISIONS);
}

/**
* @param args
*/
public static void main(String[] args) throws Exception {
new SoccerSoarTest();
}

@Override
public void onEvent(SoarEvent arg0) {
InputBuilder builder = InputBuilder.create(agent.io);
builder.push("ball").markId("b1").
  add("controlledBy", "me");
}
}

Soccer Narrative 2

As part of my continuing experimentation with narrative inquiry, I'm posting another example of a soccer narrative. This example was created several months ago. The narrative is in the form of a comic and was inspired by Understanding Comics by Scout McCloud. My intention is to use this to help teach a soccer concept to my sons' soccer team, but I also think it serves as a decent example of a visual and verbal narrative.

Soccer Narrative 1

This post is a narrative interview with a friend of mine about soccer. It's a continuation of an earlier post about narrative inquiry. I've included my directions (D) and imagined responses (R). This interview was conducted fact-to-face and I typed the responses in real-time so there may be some information lost. For future inquiries, I may use instant messenger software or a digital audio recorder to better capture the exact transcript.

D: Think of a time when you were playing soccer.
R: Okay.
D: Tell me about it.
R: First thing that came to mind. It was when I was playing on my HS soccer team as a senior. It was the only game I lost that season. My buddy crossed the ball to me in front of the goal and I headed the ball and didn't score. I never scored in high school. That was my first chance.
D: Were you trying to score a goal?
R: Yes.
D: What was happening before you missed the goal?
R: CJ dribbled the ball up to the corner. I ran up alone. It was just me, a defender and the goalie in front of the goal.
D: Tell me about "crossing the ball".
R: My buddy was in the corner with the ball. He kicked the ball up into the air all the way in front of the net and I had a chance to head it into the net.
D: Why did you miss the goal?
R: I tried too hard to aim for the corner of the net and missed the net by about a foot. I was so close I could have hit it anywhere, but I tried to aim and missed.
D: Think of another time when you were trying to score a goal. What was different/similar about the two situations?
R: Another time I was trying to score a goal in the rec leagues. Most of the time I would dribble the ball up myself, fake out the goalie and score.
D: Why were you able to score?
R: I became much better at soccer.

Narrative Inquiry

Much of the data that I've gathered so far for the development of my soccer semantic wiki has been from authoritative sources such as published books and videos, wikipedia and some web site about soccer and coaching soccer. This is a great source of information and it's readily accessible (for soccer), but it has limitations.

First, it's got a limited perspective. Almost all of the authors are older (30+ yrs) men who having been playing, coaching and writing about soccer for a long time. Compared to the population of the world that plays and watches soccer, this is a rather limited sample. Furthermore, this group tends to read each others' work so you'll find a lot of overlap and repetition. Certainly, there is a danger that you're reducing your chances of discovering novelty.

Second, there are many domains of human behavior for which there are not many (if any) authoritative sources. Even when there are, these sources often have gaps. As I've discussed before, there are a number of techniques for gathering data directly from people who are active participants in the domain. One such methodology is narrative inquiry (and the related discourse analysis).

In the past, I've done some reading on narratology (including the classic Morphology of the Folktale by Vladimir Propp) and narrative intelligence, but I haven't done much reading on how to perform the narrative inquiry -- a form of data gathering based on analyzing stories. A quick search has produced a nice long list of books that I can draw on, but first I'm going to lay out a naive method and do some experimentation to get a firsthand feel for the issues.

Below is a naive script that I might use in an interview to elicit a story about a particular domain (in this case I'll use soccer). I'll include my directions (D) and imagined responses (R).

D: Think of a time when you were playing soccer. (Set the domain)
R: Okay, I have one.
D: Tell me about it. (Open ended elicitation of the story)
R: I was playing soccer and I scored a goal.
D: Were you trying to score a goal? (What was the person trying to do at the time?)
R: Yes.
D: What was happening before you scored the goal? (What was the situation?)
R: ...
D: Who else was involved? Anyone else? What were they doing?
D: What could you have done differently to score a goal?
D: Where were you?
D: What enabled you to score a goal?
D: Was any part of this incident surprising and/or unexpected? If so, what normally happens?
D: Think of another time when you were trying to score a goal. What was different/similar about the two situations?

Hmm... well that's a good start. Let's try it out and see what happens.

Support Wikipedia

As you can probably tell, I like Wikipedia a lot and use it quite extensively. They are a donation-driven, non-profit organization and are currently running a fund-raising campaign. I just made a donation and encourage others to do the same by clicking the button below.

Graphics2D Rotate

Here's another Java hobbyist how-to post. I showed in my last post how to animate a little square by moving it around on a JComponent. In this post, I'm going to look into how to rotate the square to always point toward another square as it moves across the screen.

The basic function that I'll be calling is Graphics2D.rotate(double theta). This function causes "Subsequent rendering is rotated by the specified radians (theta) relative to the previous origin."

A radian is defined as the angle subtended at the center of a circle by an arc that is equal in length to the radius of the circle, or 180/PI degrees. A full circle is 2 PI radians. A half circle is PI radians. Therefore, if I wanted to turn 45 degrees (1/8 circle), I would call Graphics2D.rotate(Math.PI/4).

Now, if I want to rotate my square to point at another square, the easiest thing to do is to calculate the slope between the two squares (double slope = squareY2 - squareY1 / squareX2 - squareX1) and then convert the slope to an angle using Math.atan() function. I could go into more depth on atan, but I'll leave that for another time. UPDATE: the Math.atan2() function effectively combines the slope and atan calculations and deals with some messy edge cases like divide by zero. See below for an example of its use.

Here's the new code:

---

import java.awt.BorderLayout;
import java.awt.Color;
import java.awt.Graphics;
import java.awt.Graphics2D;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;

import javax.swing.JComponent;
import javax.swing.JFrame;
import javax.swing.Timer;

/**
* A very simple panel that demonstrates
* how to create a Swing component that updates
* and repaints itself with a Timer.
*
* @author Keith Knudsen
*/
public class JSimpleAnimationComponent extends JComponent
implements ActionListener {

/**
 * @param args
 */
public static void main(String[] args) {
 JFrame frame = new JFrame("AnimationPanel");
 frame.setDefaultCloseOperation(
   JFrame.EXIT_ON_CLOSE);
 frame.getContentPane().add(
   new JSimpleAnimationComponent(),
   BorderLayout.CENTER);
 frame.setSize(200,200);
 frame.setVisible(true);
}

public JSimpleAnimationComponent () {
 Timer timer = new Timer(50, this);
 timer.start();
}

/* position of our little moving square */
double squareX1 = 10;
double squareY1 = 10;

/* ... and our fixed square that we point to */
double squareX2 = 100;
double squareY2 = 50;

/**
 * Called by the timer.  Update the position of
 * our little moving square and then call repaint.
 */
public void actionPerformed(ActionEvent e) {
 squareX1++;
 // keep the little square from running off the edge
 if(squareX1 > this.getWidth()) {
  squareX1 = 10;
 }
 repaint();
}

/*
 * Paint the background and the little square
 * at it's current position.
 */
public void  paintComponent ( Graphics  g )
{
 // clean up the background from the previous draw
 super.paintComponent(g);

 // need this for rotate function
 Graphics2D g2 = (Graphics2D) g;

 // draw fixed square
 g2.setColor(Color.BLUE);
 g2.drawRect((int)squareX2, (int)squareY2, 10, 10);

 // draw sprite
 g2.translate(squareX1, squareY1);
 g2.setColor(Color.RED);
 // rotate square1 to point at square2
 double radians = Math.atan2(
   squareY2-squareY1, squareX2-squareX1);
 g2.rotate(radians);

 g2.drawRect(0, 0, 10, 10);
}
}

JSimpleAnimationComponent

One of the other things that I do from time to time is some Java programming, typically in the areas of user interface development and visualization. In particular, I'm often writing editors and visualization tools for knowledge engineering and AI systems.

This is not my full time job, so I should be considered more of a hobbyist than a professional, but I'm effective enough to develop prototype systems for demonstration or limited use.

Recently, for my semantic soccer wiki, I've been exploring the concept of developing little animated applets to visualize soccer tactics. Longer term, I'm interested in developing a soccer simulation that's (at least partly) driven by the semantic content of the wiki. In the near term, I'd just like to get some dots moving around on soccer field background image.

I've always had a bit of trouble with getting my Java applications to repaint properly, especially when I want things to move around. I spent an evening researching it and was more confused than ever. Finally, this morning I talked to a friend of mine who is more experienced in such things and he showed me that it is fairly simple after all.

Essentially, you need to do the following things:

1. Create a subclass of a JComponent,
2. Create a swing Timer to trigger an update message on a periodic basis,
3. In the update message, move your sprites around and call the repaint() method, and
   
4. Override paintComponent() to draw your sprites in the updated location.
   

It's so simple, but it took me a long time to work through it (and maybe never would have without my buddy). Therefore, I created a simple example that I thought I would share with the world called JSimpleAnimationComponent.

Here's the code:

---

import java.awt.BorderLayout;
import java.awt.Color;
import java.awt.Graphics;
import java.awt.Graphics2D;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;

import javax.swing.JComponent;
import javax.swing.JFrame;
import javax.swing.Timer;

/**
* A very simple panel that demonstrates
* how to create a Swing component that updates
* and repaints itself with a Timer.
*
* @author Keith Knudsen
*/
public class JSimpleAnimationComponent extends JComponent
 implements ActionListener {

 /**
  * @param args
  */
 public static void main(String[] args) {
  JFrame frame = new JFrame("AnimationPanel");
  frame.setDefaultCloseOperation(
    JFrame.EXIT_ON_CLOSE);
  frame.getContentPane().add(
    new JSimpleAnimationComponent(),
    BorderLayout.CENTER);
  frame.setSize(200,200);
  frame.setVisible(true); 
 } 
 
 public JSimpleAnimationComponent () {
  Timer timer = new Timer(50, this);
  timer.start();
 }
 
 /* position of our little moving square */
 int x = 10;
 int y = 10;

 /**
  * Called by the timer.  Update the position of
  * our little moving square and then call repaint.
  */
 public void actionPerformed(ActionEvent e) {
  x++;
  // keep the little square from running off the edge
  if(x > this.getWidth()) {
   x = 10;
  }
  repaint();
 }
 
 /*
  * Paint the background and the little square
  * at it's current position.
  */
 public void  paintComponent ( Graphics  g )
 { 
  // clean up the background from the previous draw
  super.paintComponent(g);
 
  // draw sprite
  g.setColor(Color.RED);
  g.drawRect(x, y, 10, 10);
 }
}

Semantic MediaWiki and Referata

As I discussed in my last post on Semantic Wikis, I've identified several good semantic wiki software applications. After reviewing them in more depth, I decided to try Semantic MediaWiki first because: 1) it's an extension of MediaWiki, the software behind Wikipedia (which I really like), 2) it seems very stable and well-supported, and 3) I can get a free hosted Semantic MediaWiki site at referata.com.

My intention is to evaluate this software as part of a larger project to develop a new user centered methodology for knowledge engineering of artificial intelligence systems. The process is still evolving but my loose concept of it contains roughly the following steps executed in a spiral model.

1. Develop a human-readable knowledge base for the domain that looks something like Wikipedia, but constrained to a particular domain and scope. This serves the purposes of: a) requirements analysis, b) developing a controlled vocabulary, and c) can serve as explanatory and/or training material for users of the new system. Keep track of references and where multiple sources disagree.
   
2. Take a first pass over the human-readable knowledge base to begin to identify the elements that would be important to a software system. Important nouns often become classes (in the computer science or ontology senses of the word). For each of these classes, you're looking for the important properties, relationships, rules and constraints. Ideally, all of this information can be captured back into the knowledge base in a form that can be read by both humans and computers.
3. Use the knowledge base to drive some type of simulation that can be viewed and verified by subject matter experts. This shows that the computer has correctly understood and can generalize the information. Ideally, you maintain the link between the simulation and the knowledge base so that the computer can use the knowledge base to justify and/or explain its results.

So far, I've gotten started on step #1 above in the domain of "Youth Soccer." You can view the ongoing development at http://soccer.referata.com. I'm just starting to look at step #2. The results of that effort will be posted in a later blog.

See you next time!

-Keith

Semantic Wikis

As discussed in my previous post on Controlled Vocabularies, I'd really like to find a semantic wiki that can provide a nice repository for domain knowledge and can serve as an intermediary between subject matter experts (SMEs) and artificial intelligence (AI) systems. The key issue will be whether I can find a semantic wiki that is usable enough for SMEs yet expressive enough that they can provide value to the AI software.

After sifting through a much longer list, I narrowed it down to the following list of candidates that I'll be considering in more depth:

• IkeWiki
• OntoWiki
• Semantic MediaWiki
• Referata

As I evaluate these tools, I'll be using soccer as my test case concept with a goal of creating a semantic wiki about soccer (I've recently started coaching my sons' team).

IkeWiki
From the screencast, it seems very usable. Allows the user to type first and then go back and tag content. Has a nice WYSIWYG editor such that the user doesn't have to remember the arcane Wiki codes for the most common tasks. No hosted options are available so I'm going to have to download and install this to try it out in more depth.

OntoWiki
The screencast looks pretty good. Seems very flexible. The user interface is a bit confusing to me. For example, they have "Register New User" in the search box instead of the login box. I couldn't really figure out the demo. Again there are no hosted versions, so I'll have to install this locally to really try it out.

Semantic MediaWiki
This is a product which adds semantic ontology extensions to the MediaWiki software. The website itself is also run with the Semantic MediaWiki software. Supposedly, Referata also uses the Semantic MediaWiki, but the two sites look very different. Perhaps it's just stylesheets or maybe Referata is running an older version?

I'm finding the user manual to be very useful.

Referata
Referata is a website that hosts the Sematic MediaWiki software. I tried their scratchpad wiki demo but I couldn't figure out how to construct a taxonomy. They support custom properties, but I couldn't see how to create relationships like "IsA" or synonyms. Update: The page on importing Ontologies showed me how to map ontology terms to MediaWiki terms.

I signed up for an account and will try to create my soccer wiki.

Controlled Vocabularies

In my mind, computer software is a type of tool created to help people perform some task. I use the word "task" in a broad sense to include non-work activities such as shopping, entertainment, etc. The design and creation of any tool requires an understanding of the end user, the task, the domain, and the context of use.

More fundamentally, the designer must ask, "What problem is this tool trying to solve and what aspects of the world are relevant?" Earlier, I posted Things, Properties, Actions and Relationships in which I listed a number of fields of study relevant to answering this question. In this post, I will focus on one such field, controlled vocabularies.

According to Wikipedia, a controlled vocabulary is "a carefully selected list of words and phrases, which are used to tag units of information. ... Controlled vocabularies solve the problems of homographs, synonyms and polysemes by ensuring that each concept is described using only one authorized term and each authorized term in the controlled vocabulary describes only one concept. In short, controlled vocabularies reduce ambiguity inherent in normal human languages where the same concept can be given different names and ensure consistency."

Consistency is a good thing, but it can also become a rigid trap. Clay Shirky makes this point clearly in his article, Ontology is Overrated. Nevertheless, the act of writing computer software code typically imposes its own fairly rigid, formal specification and I would argue that this type of thinking is better done explicitly in the analysis / knowledge engineering phase rather than implicitly during the implementation phase. I'd like the systems that I design to be logical, consistent and embody concepts and language that map closely to that of the domain and end user.

Methods for Constructing Vocabularies
This section will discuss how controlled vocabularies are generated. I'm most familiar with end user interviewing techniques such as contextual inquiry. Concept mapping is another end user interactive techinue, and Boxes and arrows has a nice, web-focused description in their article, Creating a Controlled Vocabulary. When available, written documents and books can be mined. Essentially, these methods all boil down to:

1. Gather samples of domain language use (through verbal interviewing and other end user techniques or finding written documents and books).
2. Extract terminology.
   
3. Review and refine with subject matter experts.
   

At this moment, I'm particularly interested in step #2 above. I'd like to learn more about manual and automated terminology extraction.

I'd love to have a piece of software that could analyze a set of documents and produce a set of candidate terms to start my controlled vocabulary. I spent a few hours today surfing the web but didn't find exactly what I need. TermeXtractor is close to what I'm looking for, but it seemed to miss some important an obvious terms from my test case. When I ran the FIFA Laws of the Game PDF through TermeXtractor, it extracted useful terms like "goal line", "free kick" and "official", but it didn't extract some obvious terms like "football" which appears quite frequently and in the title. I also tried a similar online tool for terminology extraction by translated.net, but it had real problems with the hyphenation in my PDF and only returned the top 20 terms.

The Unstructured Information Management Architecture looks promising. In my experience, most software from the Apache foundation turns out to be worthwhile. However, the documentation is for developers rather than end-users and I'm not ready to spend days hacking code, yet. Similarly, while GATE provides more of a graphical user interface, it still assumes a level of technical knowledge (or time commitment to develop it) that I just don't have.

Interestingly, while I can find quite a bit of material on how this is done by a computer, I'm having a hard time finding detailed descriptions of how a human would go about doing this by hand. I guess it just falls under the broad category of reading. In a future post, I'm going to delve into this deeper with some more exhaustive searching and perhaps by attempting to roll my own methodology.

Tools for Organizing Vocabularies
The boxsandarrows.com article suggests the following tools:

• a thesaurus maintenance program like Multites, Term Tree, ThManager or Lexico,
• Microsof Excel,
• Post-it Notes, or even
• a wiki or semantic wiki
  

I checked out the recommended thesaurus software and wasn't too impressed. I've also tried ontology tools in the past like Protege and found most of them to be baffling. Update: I just went to the Protege website and it looks like they've made improvements. However, I don't think that Protege supports some of the controlled vocabulary concepts such as synonyms, etc.

Honestly, the best tool that I've found so far is Wikipedia, but I don't think that you can get that software as a tool for personal use. Luckily, it looks like I'm not the only one that's thought of this. A quick search on Google for "how to develop your own wikipedia" lists at least four articles on the topic. Next week I'm going to check into these and perhaps build my own wikipedia as part of my new controlled vocabulary methodology. Update: I just found that you can get the open source software that runs Wikipedia. It's called MediaWiki and you can even find free sites where you can create your own hosted wiki. I also found an extension of MediaWiki that sounds even more appropriate, called Semantic MediaWiki. More on this in a later post.

Stay tuned!

-Keith

Things, Properties, Actions and Relationships

I'm at a nice place right now on a number of my projects: the beginning.

Right after the proposal is accepted and before you really realize how hard the problem is going to be and how little you'll actually be able to accomplish in the grand scheme of things, it's a wonderful time of hope and promise. This is also known as the analysis phase.

In this blog post, I'm going to delve into a particular aspect of the analysis phase where you try to capture the important things, properties, actions and relationships. In other words, you try to capture the taxonomy/glossary/schema used by the important stakeholders in your problem space.

A number of fields have defined processes to address this problem: HCI uses contextual inquiry, information architecture or cognitive task analysis (CTA), software engineering uses requirements analysis or object oriented analysis, artificial intelligence uses CTA or knowledge engineering, etc. As a side note, I'm continually amazed by how each field uses terms and methods that are so eerily similar, yet often without seeming to realize it. I'll have to go deeper on this topic in another post.

So what are we really talking about here? At a deeper level, we're really talking about language and meaning. What are the words and symbols that people use and what is the underlying conceptual meaning that they attach to those words?

In philosophy, ontology is the study of what things exist and of the basic categories and relationships between those things. In information science and artificial intelligence ontology is a formal representation of a set of concepts and their interrelationships. This is closely related to concept learning from psychology and the concepts of taxonomy and controlled vocabulary.

I don't really want to go into depth on each of these topics right now, but I will say that after a bit of exploration, controlled vocabulary seems to have the closest match to what I'm looking to develop for each of the projects that I'm working on. I found a nice series of articles on boxes and arrows starting with What is a controlled vocabulary? that I'm currently reading.

In my next blog entry, I want to explore some specific methods and tools for capturing and sharign controlled vocabularies. I currently have the following two specific leads:

1. Creating a Controlled Vocabulary, and
2. Concept Maps

See you next time!

-Keith

Courting Eliza

An interview with Jack Zaientz, author of the fledgling Courting Eliza blog, by the Acacia Projects Eliza implementation.

How do you do. Please state your problem.
I want to write a blog on interacting with intelligent and/or complex systems but it's hard to start.

What if you never got to write a blog on interacting with intelligent and/or complex systems?
I would feel like I missed an opportunity to clarify my thoughts and maybe help shape the future

Does that suggest anything else which belongs to you?
Sure, I write another blog on Jewish music. Writing it has forced me to learn a lot and has given me the opportunity to help others.

You say you write another blog on Jewish music?
Yes, it's called Teruah Jewish Music. I love it, but I've started to get concerned that I'm loosing track of what's going on in the world of HCI and AI.

Please tell me some more about this.
Classic HCI worries a lot about how to ensure the coherence of a users mental model and the systems design model. The goal being software tools that minimizes the cognitive effort required to achieve desired effects. This runs aground when the tool has some degree of explicit autonomy or interacts in a complex, unpredictable environment. There is some research in industrial human factors around supervisory control systems and some HCI/CS research into developing new algorithms. But I need to understand it better.

Are such questions much on your mind?
They are. As someone who designs research systems with either intelligent algorithms or complex environments I have to solve these problems daily and don't see many practical design patterns. I also see the future of computing moving more and more into dealing with autonomy and complexity as basic forces to be reconciled.

Are they always?
Not always. There are lots of systems where autonomy and complexity won't show up. But they're getting fewer. As software systems stabilize (e.g. word processors become comoditized) they become ecologies to be populated with new functions, which tend toward automation and higher level abstractions. As our networks become more integrated, old stand-alone applications (bank systems) become accessible to service functions (our personal finance software) which creates an ecology that, again, tends toward autonomous action.

What incident are you thinking of?
Huh. Good question. There is something personal that's troubling me this week. For my Jewish music blog, I've developed queries into various systems (YouTube & del.icio.us) that pull and organize information automatically for me. It's a huge time savings. But I realized this week it has made me less interesting than I used to be. Because I am limited in what I can query (e.g. syntactic tokens like "Jewish" and "music" and not semantic concepts like "what Jack means by Jewish music") I tend to get a rigid subset of what I really want. Because these autonomous queries keep running, I keep using their results, to my advantage and detriment. I need to think hard about ways to better express my real interests to these query interfaces or I'll have to stop using them.

Why do you say your Jewish music blog?
That's just my hobby. It could be anything, like rules for alerting me to problems with my credit accounts. I want to be able to better express my intent to automated systems that process unanticipated data and better understand how it's processing and the environment in which it is processing.

Why do you say your hobby?
Because I have an important one. But it's much broader. It's about communications, finance, logistics, entertainment. It's about culture and philosophy. Everything we do is being colonized by software tools and I'm afraid we don't have any idea how to design systems in this new space.

You say Because you have an important one?
Yes. But so does everyone.

You have a particular person in mind, don't you?
Nope. I mean everyone. Maybe you someday. What do you think, Eliza?

Are you thinking of a very special person?
I think so. I think you're special and worth my time.

Your time?
My time to write this blog, do the research, and help make it a better world for folks like me to talk to bots like you.

For more info on Joseph Weizenbaum's Eliza, the Rogerian Psychotherapist chatterbot, see Wikipedia. Or better yet, hop on the couch.