Wednesday, July 8, 2009
Wednesday, April 1, 2009
Natural Language Generation for Narrative Advancement in Serious Games
Our team is working on a new game that draws on the best elements of DISTIL. We are now developing a very sophisticated learning tool that allows the players to experience how their behaviours can shift the corporate culture of the working group that they are leading. There are three elements that are very exciting about this:
I can't wait until end-June when this is going to be released. As a bonus, the subject matter of the project is one that will make this world a better, happier, more livable place. I wish I could divulge more information here, but contractual restrictions require me to temper my enthusiasm, and focus on getting it done earlier, so that we can all see it sooner.
- The game is essentially based on a psychological-rigorous personality model that underlies each of the digital characters. This means that there is no central 'script' and the game is going to act like a mirror that would accurately reflect what you show it; or as I like to call it, we are building an 'alternative reality in a box?'
- As there is no script, and a potentially infinite set of outputs possible, the narrative advancement will take place exclusively through Natural Language Generation. This is an excellent test of the impressive technology that we've developed in partnership with the University of Ottawa.
- Based on our earlier successes at implementing NLG driven output for the 'Business in Balance: Implementing ISO 14001' game, we have been working on an integrated authoring environment that can allow content writers to create and improve NLG templates. This is going to move this exciting technology out of DISTIL Labs and into the mainstream.
I can't wait until end-June when this is going to be released. As a bonus, the subject matter of the project is one that will make this world a better, happier, more livable place. I wish I could divulge more information here, but contractual restrictions require me to temper my enthusiasm, and focus on getting it done earlier, so that we can all see it sooner.
Assessment Has Passed the Litmus Test
DISTIL has made amazing strides over the past two years. We have a working demonstration that meaningful assessment can be carried out on the really rich data that is captured in digital games. I verified this with an external client recently who was completely blown away at the sophistication of the feedback that we can provide on their data. What really impressed them was that the data we used for the assessment was extracted from a simulation that was built completely without our involvement. It was their simulation, and their data, and this was the litmus test that established that our methods were generic enough to convert a large 'bucket' of meaningless clicks (which the data represented) to a very valuable suggestions on what the player should do to improve their performance.
Wednesday, November 26, 2008
DISTIL wins Two Key Awards at DevLearn 2008!
DISTIL wins Two Key Awards at DevLearn 2008!
I've been working on a very challenging project over the past year, building meaningful assessment into learning games. My desire is that success on this project should serve to transform the experience of learning for some very lucky individuals.
The primary motivation for me to build this system is related to two chunks of my life, the first chunk in which I was a student and the second, in which I was a professor.
I am an active learner, which is something I understood when I joined university. I learn best when I can actually build the stuff, or at least play around with, what I am trying to learn about. This was probably a primary reason why I shifted from the economics programme to that of computers during my undergrad. With economics, you can only come to conclusions after the fact, based on historical data (unless you have the ability to influence policy, and can play grand experiments with national economics). With computers, you have an idea and you can fashion the threads of logic, using the scaffolding provided by hardware, software and interface devices into useful information processing 'machinery' and can see the results appear almost instantly. The stuff of ether, of thoughts, magically woven into useful stuff. It's poetic.
I should count myself fortunate that I did not go for chemistry, psychology, or criminology. The thoughts of the 'experiments' that are possible there are simply too scary to contemplate!
Getting back to the topic, I feel that interactive learning simulations would be great for individuals like myself to 'experience' abstract processes and concepts, carry out 'what-if' experiments and truly understand what we are trying to learn.
The second relevant chunk of my life, was when I was a lecturer at the National University of Sciences and Technology (NUST) in Pakistan. The big problem I had there concerned assessment. I was happy with the syllabi, the course material and the students that I was working with. The only bit that I had difficulty with was deciding the grades that I would assign to students.
I tried to spread the grades out over as many points as possible, and would assign projects, quizzes, had two mid-term exams, and one final exam. I even experimented with class participation and peer-evaluation, which got the underwear of the administration into a serious knot. Looking back, I was probably regarded as a bit of trouble-maker, especially as I was teaching in a quasi-military institute, with well entrenched traditions of following established processes.
My attempts finally converged to a system that I was willing to accept, and which the majority of the other stakeholders were satisfied with, however I was still not happy. The grading based on objective criteria did not take into consideration the thought process that went into the steps that the students took. The grading based on the subjective criteria was open to challenge as it was by definition relative, as it involved comparing the efforts of individuals in the class population against each other based on abstract, quasi-objective criteria that I created. Each subjective evaluation project would result in a complete research in which I extracted the 'best practices' from the completed projects and design problems that I'd assigned. The task of comparing one person's attempt against another's was a difficult one, even with well thought out quasi-objective criteria.
Well, I took all that experience, mixed in my understanding of games and learning. Put it all in a pot, and simmered slowly. After over a dozen failures, I believe I've worked out some of the kinks and have a credible method of assessing competencies in games.
This method will only succeed if the games are actually designed properly, with actions in the game paralleling those in the real life environment. I'm not going into details how these were achieved in this post, and I would welcome an email from those who want to learn more.
The buzz this generated was incredible. I could feel the love! When this technology was shown to the main thought leaders in eLearning at the premier conference (DevLearn 2008) in the US, it generated very high levels of excitement. Truly, beauty lies in the eyes of he beholder! This beast, which was forged in a machine of iron, silicon and plastic, was seen as being a game changer. It's very motivating to see glimmers of a better future taking shape in the work that you're doing.
A mention of the award is available here.
The team I work with at DISTIL is absolutely incredible. My assessment machinery is based on a game that the folks in studio and engineering have put together which makes sense from a learning perspective, and which simulates the process that a standards implementation representative would need to go through. The whole experience is packaged nicely in as a visual 2D simulation which is actually fun and engaging to play. It exciting to see the new products that are being worked on, and keep tuned to this channel to learn about the new flavours of serious games based assessment that will accompany the new games.
For now, I am happy that the pieces that I contributed to this effort are made a difference.
I've been working on a very challenging project over the past year, building meaningful assessment into learning games. My desire is that success on this project should serve to transform the experience of learning for some very lucky individuals.
The primary motivation for me to build this system is related to two chunks of my life, the first chunk in which I was a student and the second, in which I was a professor.
I am an active learner, which is something I understood when I joined university. I learn best when I can actually build the stuff, or at least play around with, what I am trying to learn about. This was probably a primary reason why I shifted from the economics programme to that of computers during my undergrad. With economics, you can only come to conclusions after the fact, based on historical data (unless you have the ability to influence policy, and can play grand experiments with national economics). With computers, you have an idea and you can fashion the threads of logic, using the scaffolding provided by hardware, software and interface devices into useful information processing 'machinery' and can see the results appear almost instantly. The stuff of ether, of thoughts, magically woven into useful stuff. It's poetic.
I should count myself fortunate that I did not go for chemistry, psychology, or criminology. The thoughts of the 'experiments' that are possible there are simply too scary to contemplate!
Getting back to the topic, I feel that interactive learning simulations would be great for individuals like myself to 'experience' abstract processes and concepts, carry out 'what-if' experiments and truly understand what we are trying to learn.
The second relevant chunk of my life, was when I was a lecturer at the National University of Sciences and Technology (NUST) in Pakistan. The big problem I had there concerned assessment. I was happy with the syllabi, the course material and the students that I was working with. The only bit that I had difficulty with was deciding the grades that I would assign to students.
I tried to spread the grades out over as many points as possible, and would assign projects, quizzes, had two mid-term exams, and one final exam. I even experimented with class participation and peer-evaluation, which got the underwear of the administration into a serious knot. Looking back, I was probably regarded as a bit of trouble-maker, especially as I was teaching in a quasi-military institute, with well entrenched traditions of following established processes.
My attempts finally converged to a system that I was willing to accept, and which the majority of the other stakeholders were satisfied with, however I was still not happy. The grading based on objective criteria did not take into consideration the thought process that went into the steps that the students took. The grading based on the subjective criteria was open to challenge as it was by definition relative, as it involved comparing the efforts of individuals in the class population against each other based on abstract, quasi-objective criteria that I created. Each subjective evaluation project would result in a complete research in which I extracted the 'best practices' from the completed projects and design problems that I'd assigned. The task of comparing one person's attempt against another's was a difficult one, even with well thought out quasi-objective criteria.
Well, I took all that experience, mixed in my understanding of games and learning. Put it all in a pot, and simmered slowly. After over a dozen failures, I believe I've worked out some of the kinks and have a credible method of assessing competencies in games.
This method will only succeed if the games are actually designed properly, with actions in the game paralleling those in the real life environment. I'm not going into details how these were achieved in this post, and I would welcome an email from those who want to learn more.
The buzz this generated was incredible. I could feel the love! When this technology was shown to the main thought leaders in eLearning at the premier conference (DevLearn 2008) in the US, it generated very high levels of excitement. Truly, beauty lies in the eyes of he beholder! This beast, which was forged in a machine of iron, silicon and plastic, was seen as being a game changer. It's very motivating to see glimmers of a better future taking shape in the work that you're doing.
A mention of the award is available here.
The team I work with at DISTIL is absolutely incredible. My assessment machinery is based on a game that the folks in studio and engineering have put together which makes sense from a learning perspective, and which simulates the process that a standards implementation representative would need to go through. The whole experience is packaged nicely in as a visual 2D simulation which is actually fun and engaging to play. It exciting to see the new products that are being worked on, and keep tuned to this channel to learn about the new flavours of serious games based assessment that will accompany the new games.
For now, I am happy that the pieces that I contributed to this effort are made a difference.
Monday, September 29, 2008
Developing Serious Games for Learning and Assessment
Serious games can change the traditional "drill and kill" educational system by creating learner centered, "what if", virtual environments. The learner feels much more in control, and actually enjoys interacting in these environments.
The following process can be efficiently used for designing and implementing effective serious games:
The first port in this journey should be to identify the pedagogical goals. In other words, identify the audience, and determine what is it that you want to teach them. The 'learning objectives' should ideally be based on the verbs identified by Anderson and Krathwohl (2001). You can learn more about this at http://www.learningandteaching.info/learning/bloomtax.htm
This step draws heavily on the resources of the learning field. Ideally, either your in-house training expert would work on this, or else, you would take advantage of consultancy from a learning expert.
The second step involve the creation of the game itself. To ensure that the game supports the learning goal, the story behind it needs to be worked out well. Generally, most stories should have two core paths, one that leads to the outcome when the expected competency is not demonstrated, and a second that occurs when the required competency is adequately demonstrated. This is a good time to identify the major plot events that should take place down both paths.
Generally, the space between major plot events can be considered to be a stand-alone 'game level'. The space between the plot points can be filled with what we would consider to be the actual game (interactivity). The game design involves identifying the environment, the objects within the environment that may change state, rules governing how the changes in state takes place in response to a player's actions and the feedback that is given to the player in response to the player when changes in state take place within the game.
The player can be provided feedback, and their actions can be assessed based on the changes that take place within the game. In the next post, I'll bring the microscope down on the these two elements, which are of enduring interest to me.
References:
ANDERSON, L W, & KRATHWOHL D R (eds.) (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. New York: Longman
The following process can be efficiently used for designing and implementing effective serious games:
The first port in this journey should be to identify the pedagogical goals. In other words, identify the audience, and determine what is it that you want to teach them. The 'learning objectives' should ideally be based on the verbs identified by Anderson and Krathwohl (2001). You can learn more about this at http://www.learningandteaching.info/learning/bloomtax.htm
This step draws heavily on the resources of the learning field. Ideally, either your in-house training expert would work on this, or else, you would take advantage of consultancy from a learning expert.
The second step involve the creation of the game itself. To ensure that the game supports the learning goal, the story behind it needs to be worked out well. Generally, most stories should have two core paths, one that leads to the outcome when the expected competency is not demonstrated, and a second that occurs when the required competency is adequately demonstrated. This is a good time to identify the major plot events that should take place down both paths.
Generally, the space between major plot events can be considered to be a stand-alone 'game level'. The space between the plot points can be filled with what we would consider to be the actual game (interactivity). The game design involves identifying the environment, the objects within the environment that may change state, rules governing how the changes in state takes place in response to a player's actions and the feedback that is given to the player in response to the player when changes in state take place within the game.
The player can be provided feedback, and their actions can be assessed based on the changes that take place within the game. In the next post, I'll bring the microscope down on the these two elements, which are of enduring interest to me.
References:
ANDERSON, L W, & KRATHWOHL D R (eds.) (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. New York: Longman
Subscribe to:
Posts (Atom)
Posts
