Instructional Design and Technology - Masters and Doctoral Program
Personal Education Journal for IU Instructional Systems Technology Program
AECT Proposal Revisions: Back in the Summer Saddle
Summer classes are over and my summer "to-do" list includes completing revisions to two conference papers which were accepted for the fall 2009 AECT conference. Both are currently in the gobbled (not saying what I want them to say) stage, but I am hopeful that will change as I devote the needed attention to both ... attached are both drafts ...
blech ...
Generative Strategies - Notes of Lecture by Dr. Gary Morrison
What is good instruction? Answer based on learning and instructional theories that form your theory and philosophy.
Mathemagenic Model (Rothkopf, 1970)
Meaning not implicit in instruction
Designer controls elaborations - orient, show what is relevant, transfer to LT memory
Emphasis - controlling learning, by controlling learner
Neo-behaviorist approach
Learning outside of learner
Control by expert information
Result: shallow learning (memorization)
Generative Model Wittrock (1974):
learner constructs meaning based upon prior learning - relating old to new the basis!
Semantic processing (deep) versus phonemic (superficial)
instruction activates prior learning.
learning inside learner - meaning making by learner
Active creation of knowledge
Process: a) Selective processes, b) Constructive processes (note taking, paraphrasing, summarizing, generating questions, cognitive mapping, creating images)
Sorting and retrieving from memory based on schema
4 key ideas 1) meaninful learning from Ausabel (relating old to new!!), 2) assimilation encoding theory (Mayer), 3) levels of processing (Craik and Lockhart), 4) schemata and scripts (guide us)
Newman, 2001: Study comparing instructional methods to achievement
Cognitive Strategies:
more connections, more likely to retain the material
Working and long term memory
Cognitive Approach - CANNOT BE CONTROLLED BY DESIGNER!
view learner as active
attend to stimuli
access / recall prior knowledge
relate new to old
encode to LT
Learning Strategies:
What are they? See Rigney in H.F. O'Neil (1978). Learning strategies. New York: Academic Press.
Mental operations or procedures that the student may use to acquire, retain, and retrieve different kinds of knowledge and performance.
Jonassen - New form of educational technology 1) recall, 2) integration (with schema), 3) organizing, 4) elaboration (add information from existing).
Two Type: 1) Detached: can be used in any class (probably used the most often) 2) Embedded not separate from the content (manipulation strategies)
2 Components: 1) Cognitive orienting: direct learner to activity to process content 2) cogntiive strategy: a) recall ( rehearsal, review, mnemonic), b) integration (paraphrase, metaphors, covert practice), c) organizational (analysis of key ideas and elaborate, categorization, outlining), d) elaboration (relate to what they already know (analogies, synthesis, create images)
2 Readings: Kourilsky and Wittrock - Applied Research:
Wittrock - literative review
IU P540
Spring 2006: P540 Learning and Cognition in Education IUB: Section 28488; IUPUI: Section 28627
E. Article Summaries/Blogging (60 Points—40 pts for personal blog and reflection paper plus 20 pts for responding to your critical friend):
Adult Learning Strategies
Jackie Dobrovolny presents a model for adult learning in Learning Strategies, an October 2003 article in Learning Circuits. Dobrovolny highlights how adult learners use 5 key learning strategies (outlined below) and how instructional design can support these key adult learning strategies.
Learning Strategies:
Metacognition (defined as self-assessment and self-correction): Though self-assessment and reflection, learners evaluate their progress and modify their learning strategies in order to find their own "preferred" learning strategy.
Reflection: Learners use reflection to visualize, see the "big picture", compare and recall information and experiences.
Prior experiences: Learners frequently "compare and contrast" course content with their own experiences. While this can cause confusion if the information conflicts or difficulty in learning if they lack prior experiences, prior experiences can help to validate new information.
Conversations: Leaning through discussions or conversation helps learners review and extend their understanding of the content, gain confidence in their mastery of the content and learn by explaining to others.
Authentic experiences: Authentic experiences allow learners to apply what they have learned, modify their skills based on their environment (that may not mirror the learning environment) and improve job performance.
Instructional Design Implications: Dobrovolny provides instructional design techniques to match these key adult learning strategies, including: To Self-assess / Self-correct:
Frequent "self-checks, practice exercises, and / or hands-on simulations" along with feedback,
Clearly defined goals so that learners can continuously self-assess their progress to the goals, and
Content that can be re-read, easily searched and printed for future off-line use (or as a job aid).
To use Reflection:
Provide examples (case studies, simulations, hands-on experiences) that show ways to apply the content (make it relevant),
Require learners to create their own examples,
Pose Rhetorical questions,
Use "building block analogy" to show how pieces of content relates (to each other and to the whole), and
Allow access to course material after the course is over.
To use Prior Experiences:
Identify range and type of prior experiences during course analysis, and
Continuously create, compare and contrast links between course content and learner's prior experiences.
To use Conversations:
Encourage discussion of the course content - with everyone (inside and outside of the class).
Discussions with novices helps the learner break down the material while discussions with co-workers or experts provides an understanding of skill or knowledge gaps.
To use Authentic Experiences:
Provide examples of situations where learners can apply their knowledge,
Conclude training with reflection questions, and
Describe or demonstrate multiple approaches or procedures.
Knowledge Transfer: Boomer Brain Drain?
The linked article appeared in T + D Magazine and addresses the issue of transferring the knowledge and skills of one generation of workers to the next as the Baby Boomer generation nears retirement. The article highlights the demographic issues that are causing what the author views as a Baby Boom Brain Drain. By 2010, the number of 55 to 64 year olds (those retiring out of senior level positions) is expected to grow by 52 percent, while the number of 35 to 44 year olds (those moving into senior level positions) is expected to decline by 10 percent. Companies are reacting to this anticipated dramatic loss of experienced workers (and the corporate knowledge they posses) by finding ways to facilitate "intergenerational learning". Suggested practices to transfer the knowledge and expertise of senior employees to the next generation of workers include:
Utilizing retirees (or soon to be retirees) as trainers and coaches,
Embracing communities of practice and other informal networks to generate ideas, to accelerate communication and to solve problems,
Expanding repositories of content,
Preparing video archives to preserve not only information, but also the "context" within the job processes,
Utilizing retirees to assist in the identification and mapping of key job competencies,
Enhancing mentor programs to pair retiring employees and other workers in "learning-based" teams
Be a good girl, have a good time and learn a lot
Linked here is an Educause article entitled "Engage Me or Enrage Me". Lately, I have been drawn to reading articles like this about educating the so-called "Net Generation". Like others, this article makes a case that we need to edu-tain these students. The main premise of these articles is that education (in this era of the Xbox and the iPod) needs a complete overhaul in order to engage students so that they are no longer "bored" with school. In this article, the author states that there is a group of student who are "...convinced that school is totally devoid of interest and totally irrelevant to their life. In fact, they find school much less interesting than the myriad of devices they carry in their pockets’ and backpacks. These kids are used to having anyone who asks for their attention . . . work really hard to earn it. When what is being offered isn't engaging, these students truly resent their time being wasted . . . The motto for this group? 'Engage me or enrage me'."
This morning, I sat here for an hour trying to articulate what makes me so angry about this viewpoint. I have mulled over this article (and others like it) for weeks. I have started outline after outline trying to express why this viewpoint is a cop out that lowers the bar (and our expectations) for students. This view is a threat to the basic expectation that students come to school prepared to work (yes, school is work) for their education. Student "engagement" and motivation is fascinating. This article (and others like it), place the responsibility (and blame) for student motivation solely on the shoulders of the school and the instructional design which the author notes is "boring" to this generation of students.
The author asserts that at school "it is so boring that the kids, used to this other life, just can't stand it." Further, he cites that the old school curriculum is to blame. Does this include the old school curriculum standbys of reading, writing and arithmetic that adults need in order to function in society? Unfortunately, what is missing in most articles like this is an analysis of the students' responsibility. What about the level of attention and engagement we must demand of them? The author notes, "They certainly don't have short attention spans for their games, movies, music or internet surfing." Of course not - those activities are all low effort leisure pursuits. It is not plausible to compare the level of personal involvement needed to watch a movie to the attention and engagement required of a student to actively listen and participate during lectures, to write papers and to study for exams in school. Further, it is not practical to compare the rewards. While games, movies and internet surfing all provide instant gratification, it is nothing compared to the long term benefits of a hard-earned education. It is lowering the bar to defend students who do not put forth the effort and rise to the challenge. There is no way around it. You must work (hard) for your education. We must instill in students that education is work and work has its own rewards far beyond the immediate gratification of the current leisure time pursuits.
Which brings me to the title of this post . . . Every time (and I mean every time) I leave my mother (from the time I can remember through today), she says, "Be a good girl, have a good time and learn a lot." These three little phrases sum up a wonderful approach to life - and to education. The underlying theme is that life is what YOU make it - not what someone else creates for you. It is up to YOU to be good. It is up to YOU to have a good time. It is up to YOU to learn a lot. While I firmly believe that it is the responsibility of all educators to walk out the door and prepare to educate students with this same mantra running though their heads, I return to my argument that it is a cop out if we do not expect (and demand) the same of students. While I firmly agree that it is our responsibility to make education as relevant and engaging as possible, we must instill in learners that it is their education and it is up to them to be good, have a good time and learn a lot.
The highlighted article (available via this link with an IU network ID and password) discusses the use of "communities of practice" (CoP) as a learning model for corporate training. The author refers to a CoP as a group connected by "shared competence" and "mutual interest" where experience and knowledge is shared (and created) among team members versus conveyed by an instructor. The author asserts that a CoP is an effective way to not only convey knowledge, but to also produce creative and valuable knowledge that is relevant to the employees and the company.
The report briefly discusses a CoP model used as an employee training program at Samsung Electronics in Korea and includes results of a survey of the Samsung CoP members. While the Samsung HR department facilitates participation and provides support (materials), the teams run independently with an elected leader and shared goals established by the team members. The survey results provide insight into conditions that contribute to CoP success. In order of rank, the following were listed as the top factors impacting the CoP activity:
Team member learning motivation
Creation of work-related knowledge and sharing of expertise in CoP participation
Relationship between outcome of CoP and performance
Unfortunately, after reading the article, I do not have a clear understanding of the problem nor the author's suggestions for a solution. While I do believe that teachers have a responsibility to find every way possible to reach and engage students, I feel this article puts to much emphasis on how education needs to change in order to accommodate a student’s short attention span (a problem that is in no way the sole domain of the Net Generation).
As an example, the author notes that the Net Generation will get bored if not challenged properly. “Bored” is a word used a lot in our culture. It implies someone is not doing something to “entertain”. However, there are a few problems with the author’s argument: 1) members of all generations have been “bored” from time to time in school (the Net Generation is not the first to experience that life is not always fun and games), 2) learners often need to work very hard to stay engaged and challenged, regardless of the learning environment, and 3) education should never be confused with entertainment.
I look forward to reading the other articles in this series. However, at this point, I am still not convinced that this generation learns any differently than prior generations. While technology used in education will continue to advance (as it is in most industries), I ague that the process of learning is not different from what it was in prior generations. In addition, I am troubled that this article implies that the responsibility for student motivation falls solely on educators and the educational system. While there are many new tools available to reach and accommodate different learning styles, in the end, learning takes hard work, focus, concentration and dedication from both the teacher and the student. Whether instruction is delivered from a professor in a lecture hall or on a message board, the responsibility to learn also falls on the student. However, this is likely a lesson to be learned through experience and with increased maturity.
Evaluation of "Learning Communities" in an Online Master's Degree Program
The linked article, Discovering the Meaning of Community in an Online Master's Degree Program, examines an online learning community from the perspective of adult learners in an online master's degree program in instructional design and technology (sound familiar?) at California State University - Fullerton. The paper was prepared by several instructors in the program and summarizes a survey of students conducted by the university. The purpose of the survey was to evaluate online learning community development within the program.
The authors conclude that an online learning community creates "foundations for transformative learning." Further, they cite prior findings that, "strong feelings of community increase the flow of information among all learners, the availability of support, commitment to group goals, cooperation among members and satisfaction with group efforts". The authors report that the following program characteristics assist in online learning community development:
- Positive peer interactions: The survey supported prior findings that student satisfaction "is strongly related to the amount of active interaction with other learners" and that "small group activities can enhance learning motivation". However, it is important to note that while peer interactions contributed to satisfaction, only 26% felt that the most meaningful learning in the program came through interactions with others (!!!) which supports other cited research.
- Positiveinstructor interactions: The survey also supported prior research that positive teacher / student interactions positively influence 1) motivation, 2) course engagement and 3) learning achievement. The lack of teacher involvement had the opposite effect. Noted important instructional factors included teacher:
responses to students' contributions in discussion topics,
constant presence on the discussion boards,
supportive phone calls,
good response time and
exhibited "thoughtfulness and caring".
- Adopting constructivists approaches to teaching and learning: The authors highlight the distinction between a "teaching" curriculum and a "learning" curriculum (their objective) in which "knowledge is co-created by members of the learning community, with each person contributing his or her additional resources to the curriculum of the course." Say it together now, "social constructivists"! The results of the survey indicate that 86% felt they engaged in constructivist learning activities and that the interactions were beneficial to their learning.
- Cognitive growth through effectively facilitated critical discourse: 80% agreed that online discussions challenged them to think critically and provided a "safe" venue to express ideas.
- Accessibility and reliability of web-based technologies: Not real surprisingly, the study validated that technical difficulties diminished the "sense of community". However, the variety and selection of the appropriate online tools (discussion boards, e-mail, etc) were shown to be important to students' success. - Institutional collaboration: This section of the survey addressed how the university "support" functions viewed the online learning community. While all agreed they "contributed to the development of the online learning community", only 54 strongly agreed or agreed that the online community as an extension of the campus community! Coincidentally, I just received an "invitation" to an interesting on campus seminar some hundreds of miles away that would not be made available to distance students :( - Members' prior technological proficiency: Students in the program "recognized and valued" each member's experience and contribution within the community.
My response as a student: As a student in a similar program, I agree with the major findings of this student survey. I agree with the importance of creating a sense of community within an online learning setting. The community (peer, teacher, institution) provides invaluable steering, insight, motivation and new ideas for me to ponder. Online learning can be a very lonely place . . . hello, hello . . . is anyone out there and the learning community can help you feel connected. Working for hours on the computer, I often think of the old "if a tree falls" story . . . I'm working away, but is anyone out there to hear me (validate?) my work . . . However, as I have been reminded elsewhere this semester, does it matter to my education if they are or not?
Yet, as the students in the survey noted, collaboration does not provide my primary source of "meaningful learning". While I truly benefit from the constructive feedback of my peers and I learn a tremendous amount from reading their work, collaboration has a downside, too. Beyond the headache of dealing with schedule coordination, collaboration on some projects leaves me feeling educationally short-changed. I sometimes feel I miss pieces of the puzzle when I am in a jointly collaborating on assignments. While teamwork and project management skills can be gained in a collaborative environment (don't adults get enough of that practice at work?), I feel that attaining such experience can stand in the way of learning. I learn the most when I think things through (construct) on my own. Granted, in life most "projects" are usually best done when tasks are shared and delegated. However, I don't view learning in the same way as completing a "project". Unlike a project, my education is not something that needs to "get done" in a speedy and efficient manner.
Finally, I found the the comments on the connection with the "institution" and the accessibility of the web-based technologies to ring true, as well. While the technology does not make the "education", it certainly is a primary facilitator. If the system is down, you can't get to your learning community or most of your resources or materials. If the system is poorly designed, you can't find things and time and opportunities are wasted. In an online setting, you must not only rely on technology to facilitate your education, but to also provide a connection to the institution. It is amazing how many things you take for granted in a f2f learning setting, such as using the library, getting a student ID card and attending ad hoc seminars with fellow students and professions. Also, when your only interaction with the institution is mediated by a piece of technology, you must work within the confines of the system's capabilities to get and stay connected with information gatekeepers and the resources they control.
Job Aid for Online Learning
Job Aid for Online Learning
In a 2004 Sloan-C report created by Karen Swan, Kent State University and the Sloan-C Editor for Effective Practices in Learning Effectiveness, Relationships Between Interactions and Learning In Online Environmentsprovides an overview of research "about interaction online and its implications for practitioners". The report (effectively a job aid for those who teach or design in an online learning environment), highlights research findings regarding learner interaction in an online learning environment and the corresponding implications for those responsible for creating and managing it. Some examples include:
Interaction with Content:
Findings: "Online discussion may be more supportive of ... experimentation, divergent thinking ... than F2F discussion", whereas "Online discussion may be less supportive of ... convergent thinking, instructor directed inquiry ... than F2F discussion"
Implication: To support divergent thinking, encourage multiple perspectives through open-ended discussion questions while encouraging diverse points of view. To support convergent thinking, use other activities, such as group collaboration and written assignments.
Interaction with Instructors:
Findings: Teaching presence (as well as quality, timeliness and quantity of the interaction and feedback) is linked to student learning.
Implication: Frequent, timely and supportive teacher interaction and feedback (both private and public) must be encouraged.
Interaction with Classmates:
Findings: Learning occurs socially within "communities of practice" and is related to the quantity and quality of discussion posts, including the perceived value that the instructor places on them.
Implications:Discussion participation must be highly valued within the instructional design and by the teacher. In addition, other community-building activities must be incorporated and encouraged.
Interaction with Course Interfaces:
Findings: The student’s impression and interaction with the course interface impact learning. Narration and animation can provide better learning than on-screen text alone, as can the learner-controlled pace of the presentation.
Implications: Selection of best course platform and interface to support learning is crucial, including consistent interfaces for all courses.
Motivating Adult Learners
I came across an interesting article about motivating adult learners. It highlights some key characteristics of adult learners and how course design should contemplate their learning styles. While much of what is presented could be considered "common sense", it does do a good job of providing an overview of the unique characteristics of an adult learner, along with strategies to accommodate their learning and motivational needs.
Central to the article is the definition of an adult learner - which could include college students. This is important as it sets the stage for a discussion of how to establish a learning setting and motivation strageties for a more mature and self-motivated student who has more life experience, established habits and tastes, many obligations outside of class and (most importantly?) options. For the purpose of the article, the definition includes an individual who:
"performs roles associated by our culture with adults", and
"perceives himself / herself to be responsible for his/her own life".
Given the maturity level of an adult learner, the author proposes a learning environment that is rich with activities that "allow maximum participation by learners so they can invest their expereince and values in the learning process." Suggestions include estabishing a learning environment that:
encourages past experiences
is collaborative between instructor-student and student-student
incoroprates "experiential activities"
The role of the instructor is viewed as managing "the process", but not managing "the content" where the instructor primarily facilitates versus lectures. Instructors should offer clarity, empathy and expertise, while always showing enthusiasm. All of this builds credibility in the eyes of the adult learner. Additionally, adult learners benefit from being "actively involved in establishing the learning objectives." While the author warns that instructors should not allow students to "call the shots", instructors should provide students the opporutnity to take a stake in setting their objectives.
Suggested strategies to help motivate the adult learner include:
presenting material in "chunks" and building on material using a "whole-part-whole" concept that begins with a wide view (to show the relevance, followed by the details, then a summary based on the whole
making the material relevant (such as relating to the learner's current job) and explaining why the material is relevant (adult learners aren't always an easy "sell")
providing a lot of documentation and options to explore the material
establishing group interactions
incorporating flexibility (1) to allow learners to try new things and (2) to accommodate their outside work and home life
Online Collaboration and Knowledge Construction
The linked paper is authored by members of the Florida State University Instructional Systems Program and published in Educational Technology & Society. It is an excellent evaluation of the effectiveness of computers in an online learning environment to "function as cognitive tools . . . that extend student's capacity to learn and create knowledge." Their observation is that current course management systems largely fall short of this goal. They propose improved design to address (1) social structure (shared context) and (2) collaboration (knowledge construction) in an online learning environment.
Design support to create a shared social context: The authors propose that a shared social context is needed for "online learners to socialize, learn and create knowledge." They note that it can be established by providing:
Virtual environments to foster social relationships: They point to research indicating "design of the virtual environment has to have the impact of being real and concrete" to produce a feeling of "place" with the goal of increased and sustained user involvement in the online activities. Suggestions include student lounges, space for personal information, virtual office hours, lecture halls and enhanced discussion forums.
Informational support: This includes support for navigation and use within the learning environment itself (for example, providing assistance in selecting "features of interest").
Multiple information channels and tools to support collaboration: Learners need to be "aware" in order to act and react. Awareness includes: social awareness (who is involved?), action awareness (what is happening?) and activity awareness (how is it going?). The authors note that support for social and action awareness is often present, but activity awareness is rarely adequately addressed. Yet, they propose that activity awareness (involving goals, plans, resources, understanding of others' actions, assessment of status) "is necessary for the successful implementation of complex and long-term tasks". They suggest improved "joint workspace", enhanced notification systems and other tools to address gaps in awareness.
Design support that facilitates online communication: A distinction is made between systems that "allow" collaboration and systems that "support" collaboration. It is a fine distinction, but an important one as the ultimate goal of collaboration should be "knowledge construction". The results of studies on knowledge construction in collaborative learning environments indicate that "the students' ability to conduct effective interactions with others was limited." It was found that the focus was primarily on "observational or procedural matters" versus "knowledge construction where each member of the team contributed" to a consensus solution. The authors propose both design intervention (for example, "communication scaffolds", "embedded communication tools", Gantt charts to display project status), but also intervention by online facilitators to support both communication, as well as to facilitate knowledge construction.
Teaching and Learning: Media Accumulation and Review
The linked article (Web 2.0: A new wave of innovation for teaching and learning) by Bryan Alexander, Director of Research at the National Institute for Technology, was published in the March / April 2006 Educause Review. It is a comprehensive media accumulation and review of "Web 2.0" technologies for use in teaching and learning settings -- see also my prior media accumulation post regarding Feedback and Collaboration tools.
While there is no single agreement on a definition for "Web 2.0", it is widely used as the term to describe emerging web based technologies. As outlined in this article, many of the Web 2.0 technologies fall into a category of "social software", including podcasts, blogs, wikis and other social networking tools. These tools transform web content from static pages to dynamic content "blocks" or "microcontent" can be "saved, summarized, addressed, copied, quoted and built into new projects." The collaboration aspect reflects the notion of "wisdom of crowds" where microcontent contributed by many ebbs and flows to form meaning in new and ever changing ways contributing to online communities of practice. The author notes that these applications satisfy the desires to "discover, publish and share". While these applications are not exclusive to learning settings, these desires are central to learning in general.
The media accumulation includes references to social bookmarking, social writing, social commentary and feedback, content aggregation and content search that can all assist in the teaching and learning process. The bookmarking and content aggregation / search applications support discovery learning. The writing applications allow for learner reflection and practice, as well as teacher / learner commentary and feedback.
Round 2 of ?: Personal Theory of Learning and Instruction
Here is my most recent shot at assessing my personal theory of learning and instruction (in IU P540) . . . it will be interesting to see how this morphs over time . . .
To begin, I am miles away from where I was when I answered this question at the beginning of this semester. I entered the class with no prior learning theory exposure, little experience as a trainer and even less experience as an online student. I feel my personal theory of learning has evolved due to not only the material presented in the class, but also by the process of being a learner in an online learning environment. In my first response during week one, my idea of "learning" was pretty much: a) hear a lecture, b) read a text, c) take a test to repeat back a. and b. Clearly, that format is not what happened in this class and, yet, I feel I have really "learned" the material while completely changing the way I view the learning and instruction process.Throughout the semester, I found myself constantly reflecting and challenging the validity of the material (from my perspective as a student). As I read the material and completed the assignments, I would ask myself "Does this make sense to me as a learner? Is this how I learn? Would this work for other adults in a corporate training setting?"
Therefore, it terms of my personal theory of learning, I put self-reflection quite high on the list of important things. It seems to be what separates rote learning from meaningful learning. Additionally, I don't feel you can fully reflect on what you are learning if aren't motivated to do so. Therefore, the section on motivation and self-regulation in learning probably resonates the most with me. It also builds a bridge between learning theory to instructional theory (how to create and sustain an effective learning environment to help others become and stay motivated to learn).
I am quite sure at the beginning of the semester, I would have been a huge Gagne fan -"Ahh, here is how to drill that information into them." However, now I am not so sure. I'm not sure that the objective should be to drill information. I now see how learning is more meaningful if you spend time seeking it out, trying it on for size, considering other options and opinions- all far more constructivist ideas. However, in my into instructional systems technology class, we've engaged in some good conversations on how / if you can mix constructivist approaches with Gagne's more lock step instructional methods. I have a feeling that this will be a key question running in my mind for many semesters to come in my IST program.
At present, I do thing there is room for both. For example, while I think learners need to take a stake in their learning objectives and outcome, I feel the teacher plays a vital role in this process, too. I agree with our prior discussion that a teacher should be a coach helping to scaffold learning. In this role, they need to continuously monitor progress and help steer the ship. However, this is in sharp contrast to my prior belief that the teacher is there to "instruct".
And finally . . . interaction (with students, the teacher, the learning institution). I came into this class with a fairly bad opinion of collaboration in learning - likely way too many late nights bailing out a team in "group project" work. However, I now feel that "group project" work is only a small (and often misused) subset of what I now define as collaborative learning. My definition now includes all kinds of interactions and activities among students, teachers and the learning institution. I feel the interaction and the give / take of working with others is crucial. For example, in our blog project writing down my thoughts for review and critique by my peers and reading and critiquing the thoughts of my peers was a wonderful experience for me. It is a great way to see, hear and think about things far beyond what one might read in the text or hear in a lecture. It not only informs, but also motivates, challenges and inspires!
Super Summary / Quiz Option
C. Super Summary (30 Points) / Quiz Option
Quiz Option: Grade 40/40
IU R511 Syllabus
IST Core R511: Instructional Technology Foundations I
Per introduction: "Dr. Frick gives an orientation to using the IU computing environment, with lots of examples of valuable resources available through the University and the School of Education websites."
Dr. Ted Frick - Orientation of IU Computing Environment - Week 2 - Journal Entry
Dr. Ted Frick - Orientation of IU Computing Environment - Week 2Dr. Ted Frick - Orientation of IU Computing Environment - Week 2Dr. Frick delivered an orientation of the IU computing environment. One of Dr. Frick's roles at IU is Director of web services for School of Education. He hightlighted the following areas during his presentation:
Calendars: Events happening in School of Education and IST Department
Search Education: Browse or Search the IU Web Site; Department of Ed spends time and effort to ensure well designed pages that are "search optimized" giving high success rates in finding what you want
Important Web Addresses: Lists url's for key sites at IU, including the IU "Knowledge Base" and the IU Libraries
UITS: University Information Technology department that adminsters IT for IU, including Help Desk.
IU Webmail: IU's own mail client. However, can use any other mail client, as well.
Computer Labs and Technology Center: UITS labs around campus, as well as IST Lab.
CFS: Personal file storage option 100MB
Web Publishing options: Personal web pages - already have an account. Also, have other web hosts available for students. "Mentor" account (unlimited space).
Telnet and FTP is disabled on most computers, but can download Secure Shell software from IU ware for free.
Connecting to IU (when off campus): Virtual Private Network connection to get into secure resources - a download from IU ware - and is needed for distance student.
Courses: Many workshop and self-study options Some are free, some are not. NETg is a good eLearning option (reduced fee for "bundle"). STEPS Classes (most are free)
Libraries: Dr. Frick highlighed some important resources, including: IUCAT - Library Catalog; Databases - Books24x7 ITPro; Encyclopedia; Ingenta.
Onstart: Presentation "cut out" on discussion of OneStart?
IUWare Online: Reviewed various software offers available from the IUWare site.
Hardware "deals" for IU students available, as well.
Distribution Lists: Subscribe to them from IST web site.
Dr. Molenda and Dr. James Pershing - week 3
Per introduction, "In the Fall of 2004 Dr. Molenda and Dr. James Pershing, an IST professor who is very involved in ISPI, talked about professional associations, primarily AECT and ISPI. The audio quality is not great, but it is audible if you use headphones."
Dr. Molenda and Dr. Pershing - Week 3 - Journal Entry
Audio often very weak and difficult to hear and at several points someone is talking over the recording? Topics discussed include:
(inaudible) ISPI Association
(inaudible) Performance Technology
ASTD: Primarily focused on corporate training and education; international association; largest organization on the list; broad occupational category; learning and performance improvement; commercially oriented - closer to a trade associational with a professional operating staff
AECT: Roughly 3,000 international members; Umbrella association covering many topics within field; innovating approaches to educational delivery; members have access to important publications and scholarly journals which make it a worthwhile membership for IST students;
AERA: Most professors in IST belong; very large; many publications; methodology and new ideas in research; reports finding of research; divisions and special interests an important part of organization
ISTE: Closest to AECT for computers in education - tightly related to k-12; typically "part timers" who take care of technology issues for schools; took lead in "certification"
AACE: Founded in computing in education (like ISTE); major conferences by sub-specialty
Other Publications: Most widely subscribed journal in IST field; for profit "one man" Larry Lipsitz; Programmed Instruction
Visiting lecturer from the Massachusetts Institute of Technology (MIT) - week 4
Per introducation, "This video lecture is part of the MIT OpenCourseWare initiative. In this presentation, Ms. Toby Woll, the Learning Technology Initiatives Director at the Sloan School of Management at MIT, talks about the ongoing educational technology initiatives in business education in the Sloan School."
MIT - Toby Woll - Week 4 - Journal Entry
Presented an outline for a "purpose of education" with examples of how Sloan has structured courses to meet these purposes. Explicit Purpose (school age and adult):
convey content
allow students to practice
learn from each other (community of learners)
Implicit Pupose:
(both) socialization
(childgren) baby sitting purpose
(adult) networking, friendships that can be used later beyond classroom, time out away from other responsibilities
Technology Considerations:
Facilitative vs. NonFacilitated
Synchonous / Asynchronous
Electronic / Face to Face
Customized / Off the shelf
Interactive / One way
How does technology impact education?
School Age Students: Enriches and engaging; multidimentional;
Adult Students: Frees us in terms of time and space, but jeopardize (implicit) socialization and timeout; problem: no hiatus
What has Sloan done? Some expamples:
Content: Pre-MBA package as an enrichment; knowledge updates (to current research)
Learning Communities: Sloan space: student centric portal - which portlets they want open with they come to the space
Meryl Lynch investments course - Addressed educational purposes as follows:
content: advanced theory class on CD with hard copy notebook (due to quantitative nature of "lectures");
application / practice: built a project with groups within class;
community: questions in discussion groups (pre and post lecture space) with TA and learning director would get involved in the discussion when needed; synchronous "office hours" text chat room;
socialization: in-person component in NY to build rapport; graduation party;
time-out: violated this principle, but . . . they felt this was fine as the students were high potential and at risk to leave, so this was a "nomination" highly selective "honored pool"
MIT (continued) - week 5
Per introduction, "We will continue with two visiting lecturers from the Massachusetts Institute of Technology (MIT), part of the MIT OpenCourseWare initiative. The topic of their presentations is not directly related to Performance Technology, but rather to the uses of instructional technology in school and university settings. There are two short video lectures: #4. Bonnie Bracey on "Media, Education, and Technology." (16 minutes). Ms. Bracey is mentor teacher who works with technology integration projects. She focuses on the uses of technology in classrooms, on which she is a nationally recognized leader. #5. Steven Lerman on "Educational Uses of Technology." (32 minutes). He is currently the Chair of the MIT Faculty. He also is the Director of the Center for Educational Computing Initiatives (CECI), the research unit of an MIT-wide research center devoted to studying the application of computational and communication technologies to teaching and learning. He describes the strategic directions of MIT regarding uses of technology for education. http://ocw.mit.edu/OcwWeb/Comparative-Media-Studies/CMS-930Media--Educat...
(Part 1) Bonnie Bracey and (Part 2) Steven Lerman- Week 5 - Journal Entry
Part 1: In this video from MIT, Bonnie Bracey argues for support for use of media in education. The presentation is primarily geared to k-12 education. Bracy proposes the following benefits of media and technology:
To students: Media "speaks" to children and helps them create, analyze and go places they otherwise couldn't go. Bracey refers to the "weightless goods" of information that is passed across the internet.
To teachers: Technology allows teachers to bring in resources and experts from around the world and to find new and interesting content
Bracey proposes that teaching and learning can be changed by creating "engaged learning" that allows teachers to explore and create a "learning landscape". She defines a "learning landscape" as a way in which children learn and explore that provides many forms of resources to learn. In order to create a learning landscape, teachers need access and support to learn the technology (how does it work?, how do we use it?). Construction of a learning landscape includes creation of the product, project, objective and assessment where the teacher is the guide. Bracey argues that such "transformational learning" will create a path to lifelong learning.
Finally, Bracey calls on stakeholders (the public, teachers and administrators) to consider new:
Ways to explore technology (via technology roundtables),
Ways to share technology (to know the available resources for students and teachers),
Technologies (to know the technologies of the future).
Part 2. Steven Lerman (Note: Could not access the "slides" referenced in the video?) -- Need to finish this section after Lori confirms whether slides available?
Opportunities:
Too much is passive; need active participation
Scheduled learning to "learn on demand"
Set of materials given to students to students working toward some goal
Enabling Technologies
Applications (applied research): build things that will help students
Evaluation:
Dr. Bonk - week 6
Per introduction, "For this week we tap another great resource available through the Web, a series of "brown bag" faculty development video sessions recorded at the Indiana University School of Education. For this week we have an information-packed and fast moving presentation by Prof. Curt Bonk, of our very own IST Department. He is an award winning instructor and an international expert on pedagogical methods for distance learning. His title, "Addressing Student Motivation and Learning Styles with Different Blends of Technology," only begins to suggest the wild ride he takes us on. He discusses a broad range of effective instructional methods for Web-based instruction and various blends of media. He is really giving a wealth of examples drawn from a range of learning theories and instructional theories."
Dramatic increase in online learning in higher education all across the world, including "blended" learning (roughly 2/3 of online classes are blended in US vs fully online). Seems to more support in public learning institutions. What is blended learning? Is it just face to face vs. online? Or are we overusing this term as almost all classes now blend some form of technology? Some see it as a combination of modalities / media in instruction, tasks and delivery. Sloan Definitions: Traditional 0% online; Web Facilitated 1-29% online; Blended/Hybrid 30-79% online; Online: 80% of content is online. Advantages: Access, flexibility, convenience, increased learning, cost effective, effective pedagogy Disadvantages: Time, procrastination, resistance, overwhelming, trying too much
Examples: Assessments/reviews online; follow up in communities of practice; reference material; office hours online; mentoring and coaching tool; access experts live online
Auditory learners: Videostreamed Lectures Reflective and Observational Learners: Blogs, live conf. feeds, portfolios Visual Learners: Pictures, film, demonstrations Authentic Data: Interactive news and exploration SimuLearn
Need to decide what is best for your learners.
Dr. Terry Ochoa - week 8 (hiatus week 7)
Per introduction, "For this week we tap another great resource available through the Web, a series of "brown bag" faculty development video sessions recorded at the Indiana University School of Education. For this week we will look at the presentation by Prof. Terry Ochoa, a faculty member in the School of Education who specializes in emotional and behavioral disorders. Here she talks about her R & D work related to Problem Based Learning, which is one of the favorite instructional formats of Constructivists. Her title is "Simulations in Teaching: Special Education Multimedia PBL Cases." Notice that her focus is on how to increase the on-job transfer of skills learned in the classroom, an issue faced in corporate training as well as university education."
Theresa Ochoa - Week 8
Simulations in Teaching: Special Education Multimedia PBL Cases. Gap between classroom and outside world activities. Developed simulations in problem based framework. Presented 2 separate modules based on her area of research:
Education with students with limited English
Students with behavioral disorders
PBL Instructional Approach:
"messy", real problems that may or may not have answers
anchored constructivism where students assume major responsibility for their learning
students assigned to project teams
evaluation is shared (students, professor, practitioner)
MUSE Case Example: Takes problem and puts it in a simulation for student to solve within group:
Phase 1: Narrative Phase: Seek out information with multimedia examples (to show multiple perspectives)
Phase 2: Role Strands discuss and review options (i.e. write goals, meet with Child Study Team, modify classroom schedule)
Teaching Results:
PBL simulations are well regarded
Students feel real life nature of simulations will prepare them
Student engagement is genuine and significant
Dynamics within groups varies (some better than other)
Instructors need group management skills
Evaluation is time consuming, but working well
Issues:
Not "real" students highlighted
Hard for instructors to "be quite" and not control session - have to become a listener
Need to realize that not all groups function at same level - some get more out of it, so "grading" becomes an issue
Dr. Charles Reigeluth - week 9
Per introduction, "In the Fall semester of 2004 Prof. Charles Reigeluth presented a colloquium session entitled, "A New Paradigm of Education and Training." He begins by explaining why he feels that the current schooling structure is out of step with the Information Age, and he goes on to suggest a new paradigm for formal education. In the latter part of the 90-minute presentation he focuses on new paradigms for instructional theory and instructional design. This part is directly related to this week's issues, and the advice is equally applicable to formal education and corporate training. At the very end he explains his own perspective, which strikes many of the same chords as the constructivist perspective. Reigeluth is a professor in Instructional Systems Technology at Indiana University and is one of the most widely cited researchers in the field. His book, Instructional-Design Theories and Models (two volumes), is one of the standard works in the field."
Week 9: Charles Reigeluth: Systemic Change in Education
Recent interest in school reform. Discussed a new paradigm for education and training. Two types of change:
piecemeal
systemic, paradigm shift, replacing
Only need systemic change when environment fundamentally changes. Toffler's three great waves of change: Agriculture, industrial and information revolution -> each brought paradigm shift in family, business and transportation. So, how has education changed in the information age.
Systemic change: Fundamental transformation; big changes in society cause (require) systemic changes in all societal systems; driven by pull and push.
What might it look like - what are the emerging features?
Look to other societal changes from industrial to information age as clue (for example, centralized control vs. autonomy, mass production vs. customized, compliance vs. innovative, conformity vs. diversity).
People learn at different rates (education now a sorting process vs. learning process), so need to instead focus on attainment-based, resource-based and person-based constructivist approaches. Technology plays a role in the systemic change, but not just as a tool in current learning.
Examples:
Mastery learning
Continuous progress
Personal learning plans
Performance-based assessment and learning
Teacher as coach or facilitator
Thinking skills and meaning making
Interpersonal Skills focus
Implications for Instructional Designers:
Customized, learning-focused instruction
New ISD process (linear not suited for this complex environment that involves the learners in their own instruction)
Per introduction, "Once again we will take advantage of the resources on the Internet and view a video presentation by Dr. Robert Metcalfe, entitled "The Next Big Thing: Video Internet." #3 Bob Metcalfe is a venture capitalist at Polaris Venture Partners in Waltham, Massachusetts. While an engineer-scientist (1965-1979), Metcalfe helped build the early Internet. In 1973, at the Xerox Palo Alto Research Center, he invented Ethernet, the international local-area networking standard on which he shares four patents. While an entrepreneur-executive (1979-1990), Metcalfe founded 3Com Corporation, the billion- dollar networking company where at various times he was Chairman, CEO, division general manager, and vice president of engineering, marketing, and sales. In this presentation, Metcalfe traces the evolution of the Internet and speculates on what will be "the next big thing" on the Internet, which he predicts to be video over some sort of network. He expects this new development to have significant implications for education."
Robert Metcalfe - Week 11 - Video Internet - the Next Big Thing
Note: This video is getting quite "dated" as it was made in 2001 and some of the "statistics" and current state of technology has changed. Some of his "predictions" have already come true today - broadband, video on demand, BitTorrent).
Metcalf runs through history of "networks". Internet was originally envisioned as "applications" on Internet accessed anywhere on dumb terminals (hey, maybe we are coming full circle?). FTP arose to transfer data; e-mail to transfer message; newsgroups to collect messages and categorize by categories. In 1990's, WWW brought about web publishing and brought about a new kind of content 1) editorial; 2) advertising; (new) 3) community. By late '90's went from publishing into commerce. What is next big thing now? Convergence of Internet, telephone and television networks.
Goals vs Reality of Convergence:
Internet people: Goal: To bring Internet to "everyone", so started on phone networks. What they got? Newbies and slow connections.
Telephone people: Goal: To get a lot of long distance minutes! What they got? Short distance hours that they couldn't bill, as well as cannibalization of telephone via Internet.
Television people: Goal: Interactive television. What they got? An entirely different business model where Internet content and carriage is not the same. TV networks lost their limited competition as the Internet offers millions of choices for viewers vs. limited channels on TV.
Three kinds of television conversion.
TV as cheap Internet "monitors" (like WebTV)
Use of cable modems to access Internet.
Carry TV over the Internet.
The problems:
current technology makes video shaky
deployment of broadband access (via cable, DSL)
hardware of Internet TC / IP makes it very slow to transmit video
applications that will drive (for example video on demand)
Predicted Phases from Old to New Technologies:
Arbitrage: old technology is more expensive (tariffs, etc)
Actual cost savings
Brings about new ways to do things
Convergence of old and new
Emergence of "standards" to facilitate infrastructure development
Implication for education?
Improved production quality
Education at home
Also, Metcalf notes he is on Board of Avistar (linked here)
Dr. Ted Frick - week 12
Per introduction, "This week's colloquium session comes from the Fall of 2004. Dr. Ted Frick, an IST professor, talked about "Web design praxiology." He and Prof. Elizabeth Boling have written a book on Webpage design, and this presentation captures some of the major principles that should guide our practice. This session is not intended to be related closely to this week's topic. The purpose is to introduce you to a timely topic and acquaint you with an IST professor who you might want to seek out in your later studies in IST."
Dr. Frick - Web Design - Week 12
Dr. Frick discussed his research interests, including:
Web Design Praxiology - How to design web based instruction. Beyond the tools to the process: Set of practices to create effective and efficient to design web sites. Practical Web Development includes:
Needs assessment and analysis (talking to stakeholders)
Paper prototype
Making / revising computer prototype
Analysis of test results
Building and checking the site
Maintaining the site
Conducting ongoing reviews of the site
* In instruction, have to contemplate assessment and evaluation, too.
Per introduction, "David Berque, PhD, associate professor of computer science at DePauw University visited the IU campus in November 2004 to demonstrate and discuss the remarkable interactive hardware-software system he has developed, DyKnow VISION, a pen-based groupware system that allows students and instructors to share free-hand information during class. For more information see http://people.depauw.edu/dberque/projects.html"
Week 13: David Berque
This presentation displays how new technological innovations are used in classroom to foster various learning activities. His motivation was lectures in math and sciences, but it has application in many other learning / lecture settings.
How it works: Teacher and students writes on tablet PC or electronic whiteboard using Dyknow Vision software.
Pedagogy Examples: 1) Problem examples tried by students immediately; 2) Students can reply with examples
Implications for learning: Enable participation by all students and get a more diverse range of views, more time on a task; bring in content from other places.
(Optional - not part of Journal) - Dr. Elizabeth Boling
Per introduction, "This is the last colloquium session for this semester, and it is optional. If you plan to fulfill the IST portfolio requirement for a Master's degree, you will not want to miss Prof. Elizabeth Boling's session on Do's and Don'ts for portfolio building. She learned about portfolios in her career as an artist and instructional design manager. She has many tips and concrete examples to offer. You are not expected to include this presentation in your colloquium report. It is here only to help you meet the portfolio requirement."
Grades
Assignments and Grading Criteria:
Assignment
Percent of Final Grade
Team weekly deliverable
40%
Peer Evaluations
10%
Whole Class Discussion participation
10%
Colloquium Journal Report
15%
Final Paper
25%
Total Points
100%
Gradebook
Assignment Title
Grade
Comments
R511 Spring 2005
DELVERABLE WK 2
8.5
Please see the feedback sheet in your group space.
R511 Spring 2005
INDIVIDUAL DELIVERABLE WK 3
10/10
Great! You fulfilled all of the requirements of the assignment nicely!
R511 Spring 2005
DELIVERABLE WK 3
10/10
Your team covered all of the questions well within the 2 paragraph limit. Please see general feedback via the announcement about this assigment.
R511 Spring 2005
DELIVERABLE WK 4
10+
WOW! This was EXCELLENT! Nice graphic that coordinates with group you are working with. Good overall document design. Great introduction which sets up the piece nicely. Nice explanation of the IDI model and why you are choosing it for your group (ie. you explained the benefits in context of your needs). Good conclusion and references all in good shape. One small piece of advice for next time... It might look a little bit better to leave a bit more space between your bolded headers. I think that`s it - well done!
R511 Spring 2005
DELIVERABLE WK 5
10
Looked good, everything in good shape (references, document design etc). The most important thing though is that your memo outlined the difference between IST and PT via what kinds of "interventions" might be determined or used. This was the main point of the assignment.
R511 Spring 2005
DELIVERABLE WK 6
9.0
Good overall. Needed to keep focus on ONE instructional theory instead of trying to tie together three theoretical perspectives.
R511 Spring 2005
DELIVERABLE WK 7
9.0
You received the 9.0 because you did a good job of describing the various methodologies associated with behaviorism (although some things could have been elaborated upon, for example, what constitutes and "enrichment activity", what is meant by "consisten" instruction". I took a point off because although you described the basic tasks for your organization (pass motor vehicle test, CPR etc), you did not tie the specific task to a specific methodology and then persuade or provide evidence for why this would be a good idea. At one point you simply wrote "It works!`, but there was no evidence really to support this and since one of the tasks of the deliverable was to "convince your mangager" to use behavioristic methodologies, I`m not quite sure that this component shone through... Also, you provided a powerpoint presentation via html when all that was needed was the powerpoint. Perhaps it was an effort to make the project even better, but it backfired a bit because it seemed unnecessary to do so, and required extra steps on my part to open the files. Finally, when you reference a powerpoint from an online course, you need to provide the URL for the course. I think that`s it, I`m getting nit-picky because we`re past the middle of the semester, but I didn`t take off really for the last two things... just sort of an "FYI".
R511 Spring 2005
DELIVERABLE WK 8
10
Thanks to Chuck for resending the document with the rest of the nine events... all looked good. Can tell that you have a good understanding of a contructivist perspective from this document.
R511 Spring 2005
DELIVERABLE WK 9
10
Very nice! Your target audience, descriptive theory and instructional theory sections were all VERY clear and made good sense. Historically, military instruction/training was very behavioristic in nature, in fact military training was one of the main reasons the field of instructional design was born. You make a good case however, for using constructivist theory in military instructional settings and perhaps the training in that area is moving toward a more constructivist approach - Chuck would probably know more about the current situation for that particular audience than I do at this point...anyway, nice, tight, meaningful paper.
R511 Spring 2005
DELIVERABLE WK 11
10+
Sucessfully fulfilled the assignment requirements. Integrated the readings into your task which was appropriate for your given audience. References looked good. Best components - overall design (look) of the powerpoint and level of detail/elaborative nature of the presentation!
R511 Spring 2005
DELIVERABLE WK 12
10
Very nice! Good document design and flow. References were in good shape. Fulfilled the requirements of the assignment well in that you explained both the positives and negatives. Good recommendation/conclusion section. Well done!
R511 Spring 2005
WEEK 13 DELIVERABLE
9.0
Nice introduction which included right away the "underlying values" of IST. You included both hard technology and conceptual development within your various categories. The one thing that appeared to be missing was the link or connection between the hard technolog/conceptual developments and the values of effeciency, effectiveness etc. The linkage/connections were not made explicit under each of your categories. The conclusions section suggested that they were "evident", but it seemed as if the reader would have had to "read between the lines" to make the necessary connections. Other than that, nice overall summary!
R511 Spring 2005
Week 14 Deliverable
9.0
Hi Jennifer, mostly I just want to thank you for being such a great participant and role model for the class (which has been a bit interesting this semester to say the least). I had to take off one point however for this assignment since the requirement is to both give your take on things AND to comment on what the other folks are saying. You only had one posting (this time - you have been very prolific in the past which will show up in your partipation grade, so not to worry in this case).
R511 Spring 2005
COLLOQUIA JOURNAL
15/15
Jennifer, you have captured the essence of what the colloquia series is all about in your summary. Your individual entries were the easiest to read (out of the whole group) due to the way they were set up and document design techniques used. Thanks for being such a positive influence (both in quality of work and personality) this semester!
Nearly 100% of assignments were group project work. Therefore, assignments for this course will not be posted here as representation of individual work.
Shrock, Sharon and Coscarelli, William (2000). Criterion-Referenced Test Development, 2nd ed. Washington, DC: International Society for Performance Improvement. Available at www.ispi.org.
Phillips, J. (1997). Handbook of Training Evaluation and Measurement Methods (Improving Human Performance Series) 3 rd Edition. Butterworth-Heinemann. ISBN: 0884153878
You should already have: Morrison, Gary R., Kemp, Jerrold E., & Ross, Steven M. (2001) Designing Effective Instruction (4th edition). New York: John Wiley & Sons. [required text – core]
Van TiemVan Tiem, Darlene M., Moseley, James L., Dessinger, Joan Conway (2004). Fundamentals of Performance Technology: A Guide to Improving People, Process, and Performance, 2nd Edition. Washington , DC : International Society for Performance Improvement. Available at www.ispi.org.
Readings Materials Available in Education Electronic Reserves or Linked from the Course Web site:
Andersen [in electronic reserves]. Luebke, J. F. & Bumpass, S. E. (2000). Managing the information of an executive development program. In J. J. Phillips (Ed.), Performance analysis and consulting (pp. 167-179). Alexandria, VA: ASTD.
Birnbrauer, H. (1996). Improving evaluation forms to produce better course design. Performance & Instruction 35:1, 201-204.
Bragg, D.D. (1995). Working together to evaluate training. Performance & Instruction 34:10, 195-199.
Brethower, D. & Smalley, K. (1998). Integrating Evaluation into Performance-based Instruction. In Performance-Based Instruction. San Francisco: Jossey-Bass. Chapter 12, pp. 156-181.
Brinkerhoff, R. O. & Dressler, D. (2002). Using evaluation to build organizational performance and learning capability: A strategy and a method. Performance Improvement 41:6, 14-21.
Burkman, E. (1987). Factors Affecting Utilization. In R.M. Gagne (ed.) Instructional Technology: Foundations. Hillsdale, NJ: Lawrence Erlbaum Associates. pp..429-455.
Caudron, S. (1997, February). The human side of a technology launch. Training & Development 52:2, 21-24.
Dormant, D. (1986). The ABCDs of Managing Change. In Introduction to Performance Technology. Washington: National Society for Performance and Instruction. Chapter 17, pp. 238-256.
Duck, J. D. (1998). Managing Change: The Art of Balancing. In Harvard Business Review on Change. Boston: Harvard Business School Publishing, pp. 55-81. (originally published in November/December, 1983 issue of Harvard Business Review).
Garland, K.P. (1995). Diffusion and Adoption of Instructional Technology. In G.J. Anglin (Ed.) Instructional technology: Past, present, and future, 2nd ed. Englewood, CO: Libraries Unlimited, pp. 282-287.
Kaufman, R., Keller, J, and Watkins, R. (1995). What works and what doesn’t: Evaluation beyond Kirkpatrick. Performance & Instruction 35:2, 205-209.
Kifer, E. (1995). Evaluation: A General View. In G.J. Anglin (Ed.) Instructional technology: Past, present, and future, 2nd ed. Englewood, CO: Libraries Unlimited, pp. 384-392.
Kirkpatrick, D. L. (1998). The Four Levels: An Overview. Ch. 3 in Evaluating Training Programs: The Four Levels, 2nd ed. San Francisco: Berrett-Koehler.
Kirkpatrick, D. L. (1998). Evaluating Reaction. Ch. 4 in Evaluating Training Programs: The Four Levels, 2nd ed. San Francisco: Berrett-Koehler.
Korth, S. J. (2001). Consolidating needs assessment and evaluation. Performance Improvement 40:1, January, 38-43.
Kotter, J. P. (1998). Leading Change: Why Transformation Efforts Fail. In Harvard Business Review on Change. Boston: Harvard Business School Publishing, pp. 1-20. (originally published in March/April, 1995 issue of Harvard Business Review)
Lee, S.H. and Pershing, J. A. (1999). Effective reaction evaluation in evaluating training programs: Purposes and dimension classification. Performance Improvement 38:8, September, 32-39.
Leshin, C.B., Pollock, J., and Reigeluth, C.M. (1992). Develop Performance Measures for Each Task. In Instructional Design Strategies and Tactics, pp. 54-72.
Molenda, M. & Pershing, J. A., (in press). The strategic impact model: An integrative approach to performance improvement (PI) and instructional systems design (ISD). TechTrends.
Newby, Tony (1992) Training Evaluation Handbook. San Diego, CA: Pfeiffer. (Chapters 1 - 4). [This book is out of print and not available, unfortunately.]
Pershing, J. A. & Pershing, J. L. (2001). Ineffective reaction evaluation. Human Resource Development Quarterly 12: 1, 73-90.
Pershing, J.L. (2002). Using document analysis in analyzing and evaluating performance. Performance Improvement 41:1, January, 36-42.
Peterson, B. D. & Bothell, T. W. (1999). Measuring the Impact of Learning and Performance: The Franklin Covey Company. In J. J. Phillips & T. K. Hodges (Eds.), Measuring Learning and Performance (pp. 115-134). Alexandria, VA: ASTD.
Pershing, James A. & Lee, Sung Heum. Analyzing Needs for Performance Improvement: Processes and Core Competencies. Draft, 1999. Sections 9, 10, 11, 12, 13, 14, 16, 17, 18, 19, 21, & 22. Available at Collegiate Copies. (these are linked from each lesson as PDFs).
Rogers, E. M. (1995). Diffusion of Innovations, 4th Edition. New York: Free Press. Chapter 1 – pp. 1-37 and Chapter 5 – pp.161-203.
Strebel, P. (1998). Why Do Employees Resist Change? In Harvard Business Review on Change. Boston: Harvard Business School Publishing, pp. 139-157. (originally published in May/June, 1996 issue of Harvard Business Review)
Wang, C. (2000). How to grade essay examinations. Performance Improvement 39:1, January, 12-15.
Yeh, S. S. (2001). Tests worth teaching to: Constructing state-mandated tests that emphasize critical thinking. Educational Researcher 30:9, December, 12-17.
Recommended Textbooks:
Mager, Robert F. (1997) Preparing Instructional Objectives. Atlanta, GA: Center for Effective Performance Press. Available at www.ispi.org.
Pyrczak, Fred (2003). Making Sense of Statistics: A Conceptual Overview, 3rd ed. Los Angeles: Pyrczak Publishing. Available at www.pyrcazk.com.
Rossi, Peter H., Lipsey, Mark W. & Howard E. Freeman. (2004). Evaluation: A Systematic Approach, 7th Edition. Thousand Oaks: Sage.
Van TiemVan Tiem, Darlene M., Moseley, James L., Dessinger, Joan Conway (2004). Fundamentals of Performance Technology: A Guide to Improving People, Process, and Performance, 2nd Edition. Washington , DC : International Society for Performance Improvement. Available at www.ispi.org.
<!-- rwe 1/28/02 show links to attachments of parent message if there are any -->
Our rule of thumb is one engagement activity per hour of instruction.
Author:
PCH
Posted:
1/27/2007 5:44:05 PM
Subject:
School of Hard Knocks
<!-- rwe 1/28/02 show links to attachments of parent message if there are any -->
The
rule originally came from Darryl Sink's Course Developers Workshop. It
is something that we've taught (and used) for years. Darryl developed
while working with faculty in a community college. It has evolved from
experience. And if you use it, the best way is to say "at least one
involvement activity per hour" -- as it is prudent to perhaps have more.
Jennifer Maddrell Indiana University <!--[if !supportLineBreakNewLine]--> <!--[endif]-->
R511: Instructional Technology Foundations
Week 15 Deliverable
Professor Hubbard-Welsh
24 April 2006
Colloquium Journal Summary
The Colloquium serves as an introduction to the Instructional Systems Technology (IST) field, as well as the Indiana University (IU) IST program, and supplements course materials by presenting important people, ideas, trends and issues impacting the field. The following summarizes key elements within the presentations that provide greater insight into both (a). The IU IST Program and IST Career Field and (b). IST Ideas / Trends / Issues.
The IU IST Program and IST Career Field:
People in IST:New distance students in the IU IST program typically lack face-to-face interactions with faculty.The Colloquium provides a unique opportunity for distance students to not only put a face with a name, but also to learn about the faculty’s interests, research and current projects (see below).Further, it is valuable to hear from those outside IU who are instrumental in shaping the field, including those taking part in the groundbreaking Massachusetts Institute of Technology (MIT) Open Course Ware initiative in which MIT course material is made available (without charge) to anyone with Internet access.
IST Resources:The Colloquium introduces students to IST resources both inside and outside the IU IST program.Dr. Frick’s review of the IU computing environment and other school resources provides a good introduction to the many resources available to distance students (listservs, library and research sources, software). In addition, Dr. Bonk and Dr. Berque highlight new and emerging media technologies that can aid in the instructional process.Further, each presenter includes additional references and resources that encourage and facilitate further review on the topic.
IST Career Perspective:Students enter the IU IST program with varied career backgrounds.While some may already work in the field, specific job and professional experiences may be very different.By hearing working professionals discuss their career interests, backgrounds and projects within the Colloquium presentations, an IST student gains a better perspective on the diversity of the field, as well as the potential career opportunities.For example, some presenters, such as Dr. Reigeluth and Bonnie Bracey, address their diverse interests within a K-12 educational setting, while Dr. Pershing focuses on his work in the area of Performance Technology within a corporate setting.Further, information regarding professional associations supplements course material and provides history and background on IST as a professional practice.
IST Ideas / Trends / Issues:
IST Research and Projects: The Colloquium introduces students to important research and projects related to the field. For example, Dr. Reigeluth’s current research focuses on what he views as a required “paradigm shift” in education to bring instructional processes in line with the “information age”.The outcome of his research is the recommendation for customized and learner-focused instruction that he notes will require a new instructional design process to meet the “complex environment that involves learners in their own instruction.” In addition, Dr. Frick presents his research in designing web-basedinstruction, as well as the status of his “SimEducation” project.Dr. Frick proposes that design must go “beyond the tools” to the process of effective and efficient design.
IST Media Technologies: The opportunities to integrate media technology into education are discussed in most presentations and many technologies to support learning and instruction are presented.Bonnie Bracey shares her perspective on the importance of incorporating media technology within a learning setting to engage students and create a rich “learning landscape” which provides students with information and experiences far beyond the classroom.Dr. Bonk reviews the growth of online learning and presents numerous emerging technologies to facilitate online and distance education, while Dr. Berque presents technology that he developed to better facilitate face-to-face learning.In addition, Dr. Metcalf highlights the positive impact that Internet innovations (the convergence of Internet, telephone and television networks) may have on the quality and access to educational information.
IST Practices:Those working in the field present their instructional design ideas and instructional approaches based on their direct experience. Many stress the importance of engaging students and of incorporating social learning and constructivist approaches, as does Dr. Ochoa in her presentation of instructional methods for Problem Based Learning.Steve Lerman proposes that too much learning is “passive” and “scheduled” and advocates that instruction needs to be designed to incorporate active participation and “learning on demand”.In highlighting the instructional design process and approach used by MIT in the Sloan School of Business, Toby Wall presents an approach that incorporates significant application and practice, socialization and building community with the instructional design of their programs.
810 Interaction in Group-Based and Individualized Instruction
This paper compares six types of group-based and individualized instructional approaches on the basis of planned opportunities for learner interaction. Three types of interaction are suggested as crucial components of the education process (Anderson, 2003; Moore, 1989) and frame this comparison, including (a) learner-content interaction, (b) learner-instructor interaction, and (c) learner-learner interaction. The following considers how these six groupbased and individualized instructional approaches distribute the instructional load among the three interaction types and suggests that the differences in interactional emphasis across the approaches reflects a value judgment regarding the relative advantage of each type of interaction. However, it is further suggested that additional research is needed to evaluate whether a relative advantage exists or whether the perceived advantage relates to the efficiency of instructional delivery rather than the effectiveness of the instructional strategy to support the processing of the to-be-learned material.
810 Interaction in Group-Based and Individualized Instruction
Interaction in Group-based and Individualized Instruction 1 Running head: INTERACTION IN GROUP-BASED AND INDIVIDUALIZED INSTRUCTION Interaction in Group-based and Individualized Instruction Jennifer Maddrell Old Dominion University IDT 810 Trends and Issues in Contemporary Instructional Design Dr. Gary Morrison April 14, 2009 Interaction in Group-based and Individualized Instruction 2 Instructional Load of Interaction Types This paper compares six types of group-based and individualized instructional approaches on the basis of planned opportunities for learner interaction. Three types of interaction are suggested as crucial components of the education process (Anderson, 2003; Moore, 1989) and frame this comparison, including (a) learner-content interaction, (b) learner-instructor interaction, and (c) learner-learner interaction. The following considers how these six groupbased and individualized instructional approaches distribute the instructional load among the three interaction types and suggests that the differences in interactional emphasis across the approaches reflects a value judgment regarding the relative advantage of each type of interaction. However, it is further suggested that additional research is needed to evaluate whether a relative advantage exists or whether the perceived advantage relates to the efficiency of instructional delivery rather than the effectiveness of the instructional strategy to support the processing of the to-be-learned material. Distribution of Instructional Load by Interaction Type Group-based Instruction Traditional classroom. While it is impossible to generalize the interaction that exists across all face-to-face and virtual classrooms, some critics of the traditional classroom characterize the instruction as dominated by the learner-teacher interaction where learner-content and learner-learner interaction play smaller supporting roles (Hannafin, Land, & Oliver, 1983). In such a classroom, the teacher-learner interaction focuses on teacher presentation, guidance and help during learner practice, and feedback following practice. Learner-content interaction incorporates standardized forms of content, such as textbooks and hand-outs, prepared for and utilized by all learners in the group. While learner-learner interaction includes classroom discussion, research suggests a very small percentage of classroom time is spent in learnerlearner discussion (Nunn, 1996). Figure 1 suggests a distribution of the instructional load based on this view of the traditional group-based classroom. Figure 1. Distribution of Instructional Load – Traditional Classroom Group-based learning environments. In contrast to the traditional classroom described above, some advocate group-based learning environments in which the learner-teacher interaction shifts from a mediating to a scaffolding role the instruction (Hannafin et al., 1983). As suggested in Figure 2, the group-based learning environment places significant emphasis on the learner-content interaction. While the learner-learner interactions are recognized as being Interaction in Group-based and Individualized Instruction 3 supportive of the learner-content interaction, learner control over the learner-content interaction is paramount . Figure 2. Distribution of Instructional Load – Group-based Learning Environments Group-based communities of inquiry (CoI). The objective of a CoI model is to support critical thinking and critical discourse though a mix of learner-learner, learner-content, and learner-teacher interactions designed to optimize (a) teacher presence, (b) social presence, and (c) cognitive presence (Garrison, Anderson, & Archer, 1999). As suggested in Figure 3, the CoI places high value on all three forms of interaction. Figure 3. Distribution of Instructional Load – Community of Inquiry Individualized Instruction The Keller Plan. As depicted in Figure 4, Keller (1968) suggests a personalized system of instruction (PSI) which incorporates learner-content and learner-tutor interaction where the tutor (or proctor) can be a peer who has mastered the material. As such, the learner-tutor interaction is a hybrid of the previously described learner-teacher and learner-learner interaction as the peer has already successfully completed the instructional material. In Keller’s approach, learnercontent interaction is the critical design consideration. Learners work independently and at their own pace working toward personal mastery of the presented instructional content. Learner-tutor interaction is considered as motivational and administrative rather than for the delivery of critical information (Grant & Spencer, 2003). Interaction in Group-based and Individualized Instruction 4 Figure 4. Distribution of Instructional Load – Keller’s PSI Distance education independent study. Early distance education approaches were based on independent study (Garrison & Shale, 1987; Keegan, 1996). Wedemeyer (1981) characterized independent study as a teaching-learning arrangement which allows learners the freedom and opportunity to self-direct their learning within their own environment. Unlike more recent group forms of distance education which incorporate expanded learner-learner interaction, independent (or private) distance education approaches rely on significant learner-content interaction with added support for two-way learner-teacher interaction (Garrison & Shale). Figure 5. Distribution of Instructional Load – Distance Education Independent Study Personal learning environment. The April 2008 special edition of the Interactive Learning Environments journal was dedicated to a discussion of the personal learning environment (PLE). Concurrent with the explosion of web-based communication technologies, two views of technology enabled PLEs have emerged, including (a) a learner-centered but provider-driven approach, and (b) a learner-driven approach where the role and control of the institution (as provider of education) is diminished (Johnson & Liber, 2008). Within the learnercentered provider-driven approach, personal web-based communication and interaction tools, such as instant messaging, content aggregation and management, and authoring tools, enable personalized learning activities within the institution’s virtual environment (Johnson & Liber, 2008; Severance, Hardin, & Whyte, 2008; Van Harmelen, 2008). In contrast, a learner-driven PLE approach challenges the centralization and institutional control and ownership of instructional tools and content and shifts the instructional focus to life-long learning beyond the classroom and to individualized construction of portable instructional artifacts which the learner Interaction in Group-based and Individualized Instruction 5 retains and maintains over time (Severance et al., 2008; Wilson, 2008). In either form, a PLE is generally conceived of as a technology-enabled network which connects the learner with people (inside and outside the classroom) and resources (Wilson, 2008). As such, the learner-content and learner-peer interactions dominate the PLE instructional approach with the learner-teacher interaction playing a supporting role, as depicted in Figure 6. Figure 6. Distribution of Instructional Load –Personal Learning Environments The Quest for the Right Interaction Mix Value Judgment It is suggested that the interaction within the noted designs represents the beliefs of the designers regarding the relative value of the interaction types. As suggested in Table 1 these beliefs manifest themselves within the design of the instruction with the three interaction types being either (a) emphasized, (b) viewed as necessary, but emphasis neutral, or (c) deemphasized. Table 1. Interaction Type Emphasis within Instructional Approach LearnerContent Group-based Instruction Traditional Classroom Learning Environments Community of Inquiry Individualized Instruction Keller PSI Distance Education - Independent Study Personal Learning Environments LearnerLearner LearnerTeacher = Emphasized; = Recognized as needed, but emphasis neutral; = De-emphasized Given the range in interactional emphasis across these instructional approaches, it is suggested that interaction is not value neutral across instructional designers. Inherent in the highlighted group-based and individualized instructional approaches is a value judgment regarding the right (or optimal) interactional mix. An important question for future study is Interaction in Group-based and Individualized Instruction 6 whether the use and mix of interaction types within the design of instruction makes a difference in terms of instructional effectiveness? Or, can learning occur as effectively through any combination of learner-learner, learner-content, learner teacher interaction? If so, is the primary interactional consideration instructional efficiency versus effectiveness? Interaction as Instruction Delivery Mode or Instructional Strategy To conduct such an evaluation, it is necessary to consider whether interaction is a way to facilitate instructional message delivery (as in an instructional delivery mode) or if interaction is a method to facilitate the processing of the to-be-learned material (as in an instructional strategy). As is suggested within research regarding the comparative ability of various media to effectively deliver instruction (Clark, 1983), it is conceivable that the ability of various interaction types to deliver the instructional load is equivalent. For example, is it just as effective for a learner to independently read instructional content in a book (learner-content interaction) as it is to have a teacher present the same content in a lecture to the class (learner-teacher interaction)? Anderson (2003) suggests within his equivalency theorem that a designer can substitute one type of interaction for another. If this is the case, then selection and mix of learnercontent, learner-learner, and learner-teacher interaction in the delivery of instruction should not impact instructional effectiveness and the design consideration centers on efficiency. However, if interaction is conceived of as something other than a means to deliver the instruction, but rather as an instructional strategy which supports the learner’s processing of the instructional material, is there a difference in effectiveness across these interaction types? Some suggest a significant difference in the opportunities for critical thinking in learner-content interaction involving unresponding course material and critical discourse in two-way learnerlearner and learner-teacher interaction (Garrison, 1990). Summary The field has forwarded a variety of group-based and individualized instructional approaches incorporating a range of learner-content, learner-learner, and learner-teacher interactions. Inherent in these forwarded approaches is a value judgment regarding the relative advantage of one type of interaction over another. However, it is suggested that further study is needed to evaluate whether there is an advantage of one form over another or if the perceived advantage relates to the efficiency of instructional delivery rather than instructional effectiveness. Additional research is needed to consider whether there a comparative difference in terms of how the interaction types effect the processing of the to-be-learned material or whether any form of interaction be an equivalent substitute for another to deliver instruction. Interaction in Group-based and Individualized Instruction 7 References Anderson, T. (2003). Getting the Mix Right Again: An Updated and Theoretical Rationale for Interaction. The International Review of Research in Open and Distance Learning; Vol 4, No 2 (2003). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/149. Clark, R. E. (1983). Reconsidering the research on media. Review of Educational Research, 53(4), 445-459. Garrison, D., & Shale, D. (1987). Mapping the boundaries of distance education: Problems in defining the field. American Journal of Distance Education, 1, 4-13. Garrison, D. R. (1990). An Analysis and Evaluation of Audio Teleconferencing to Facilitate Education at a Distance. American Journal of Distance Education, 4(3), 13-24. Garrison, D. R., Anderson, T., & Archer, W. (1999). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2-3), 87-105. Grant, L., & Spencer, R. (2003). The Personalized System of Instruction: Review and Applications to Distance Education. The International Review of Research in Open and Distance Learning; Vol 4, No 2 (2003). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/152/705. Hannafin, M., Land, S., & Oliver, K. (1983). Instructional-design Theories and Models. In C. M. Reigeluth (Ed.), Instructional-design Theories and Models (Vol. 2, p. 728). Johnson, M., & Liber, O. (2008). The Personal Learning Environment and the human condition: from theory to teaching practice. Interactive Learning Environments, 16(1), 3-15. doi: 10.1080/10494820701772652. Keegan, D. (1996). Foundations of distance education (3rd ed.). London: Routledge. Keller, F. (1968). "Goodbye teacher...’. Journal of Applied Behavior Analysis, 1, 79-89. Moore, M. (1989). Three types of interaction [Electronic version]. The American Journal of Distance Education, 3(2). Retrieved April 8, 2008, from http://www.ajde.com/Contents/vol3_2.htm#editorial. Nunn, C. E. (1996). Discussion in the College Classroom: Triangulating Observational and Survey Results. The Journal of Higher Education, 67(3), 243-266. Severance, C., Hardin, J., & Whyte, A. (2008). The coming functionality mash-up in Personal Learning Environments. Interactive Learning Environments, 16(1), 47-62. doi: 10.1080/10494820701772694. Van Harmelen, M. (2008). Design trajectories: four experiments in PLE implementation. Interactive Learning Environments, 16(1), 35-46. doi: 10.1080/10494820701772686. Wedemeyer, C. (1981). Learning at the back door : reflections on non-traditional learning in the lifespan. Madison: University of Wisconsin Press. Wilson, S. (2008). Patterns of Personal Learning Environments. Interactive Learning Environments, 16(1), 17-34. doi: 10.1080/10494820701772660.
833 Social Presence in Synchronous CMC Research Proposal
The purpose of this concurrent mixed methods study is to examine the effect of competing parallel synchronous computer-mediated communication on learners' perceptions of social presence.
Social Presence in Synchronous Computer-mediated Communication Running head: SOCIAL PRESENCE IN SYNCHRONOUS CMC 1 Social Presence in Synchronous Computer-mediated Communication Jennifer Maddrell Old Dominion University ELS 833 Advanced Research Design Dr. Duggan April 27, 2009 Social Presence in Synchronous Computer-mediated Communication Social Presence in Synchronous Computer-mediated Communication As of the fall 2007 semester, an estimated 3.9 million college students, roughly 22% of all students enrolled in degree-granting U.S. higher education institutions, were taking at least one online course which represents a 12.9% increase over the fall 2006 semester (Allen & Seaman, 2008). This growth in online course enrollment is significantly higher than the 1.2% increase in overall higher education enrollment over the same period (Allen & Seaman). During 2 the 2006-07 academic year, 61% of U.S. higher education institutions offered online courses and of those institutions 75% utilized some form of synchronous computer-based media to facilitate live online instruction at a distance (Parsad & Lewis, 2008). The latest synchronous technologies used by educators include options for parallel voice, video, and text based synchronous communication as found in leading online conferencing systems such as Elluminate Live and Adobe Connect (Schullo, Hilbelink, Venable, & Barron, 2007). While many studies have examined asynchronous computer-mediated communication (CMC) in distance education, relatively little research has been conducted on learners’ experiences with synchronous CMC (Park & Bonk, 2007). In addition, no studies have examined the impact of parallel communication occurring within synchronous online conferencing systems. While some learners may perceive a positive benefit from the additional opportunities for real-time peer and teacher support, the parallel channels of communication may also pose a negative disorienting distraction. Purpose of Study The purpose of this concurrent mixed methods study is to examine the effect of competing parallel synchronous computer-mediated communication on learners' perceptions of social presence. In this study, a survey of college students will be used to measure and compare the learners’ perceptions of social presence between two methods of synchronous CMC; one Social Presence in Synchronous Computer-mediated Communication method utilizing only a single main channel of audio and video communication and the other method utilizing an additional text-based channel for simultaneous parallel communication with the main audio and video channel. The nature of the parallel text-based communication among participants will be explored through content analysis of text-chat transcripts from two class sessions in the course. The reason for combining both quantitative and qualitative data within this mixed methods study is to better understand this research problem by considering both quantitative survey data regarding the relationship between the parallel communication and learners’ perceptions of social presence and qualitative transcript analysis data offering insight into the 3 nature of the learners’ text-based parallel communication. This study will focus on the following research questions: 1. What effect does the parallel CMC channel communication have on the learners' perceptions of social presence? 2. To what degree (if any) is the parallel communication supportive of learners’ perceptions of social presence? 3. What is the nature of the parallel text-chat communication and what aspects make the learners feel more (or less) connected to communication in the main channel? 4. How can a parallel text-based channel be used to gauge and foster the learners’ presence with the main channel communication? It is predicted that the parallel synchronous computer-mediated communication will have a significant effect on learners’ social presence. However, it is unclear whether the effect will be positive or negative across learners. While learners may perceive a benefit from the additional Social Presence in Synchronous Computer-mediated Communication peer and teacher support, the parallel text-chat channel of communication may also pose a disorienting distraction. Synchronous Computer-Mediated Communication The set of available synchronous communication tools in online conferencing systems, including public and private text-chat, video and audio interfaces, web browsers, polling tools, application sharing, and whiteboards, offer instructors and learners expanded opportunity for interaction, communication, and content sharing (Shi & Morrow, 2006). While audio and video 4 communication tends to dominate the main channel instructional presentation in the synchronous online conferencing environment, the text-chat feature often supports spontaneous and unfacilitated parallel (backchannel, sidebar, or side-talk) exchanges among participants. However, little research has been conducted on learner experiences in these online conferencing environments (Shi & Morrow). Therefore, the effect of the competing parallel synchronous textchat communication on the learners' perceptions of social presence is unknown. Social Presence Social presence theory builds upon the concept of social presence from the work of Short, Williams, and Christie (1976) in technology-mediated communication and is often used as a theoretical framework in the study of asynchronous computer-mediated communication (De Wever, Schellens, Valcke, & Keer, 2006). Social presence within the context of a computermediated classroom is the degree to which learners present themselves and are perceived socially and affectively as real people in mediated communication (Garrison, Anderson, & Archer, 2000). Research on social presence in asynchronous computer-mediated learning environments has moved beyond an evaluation of the medium’s effect on social presence to an evaluation of how Social Presence in Synchronous Computer-mediated Communication social presence can be cultivated through instructional methods to support critical thinking and critical discourse within the computer-mediated environment . Some suggests social presence is related to student satisfaction and learning (Garrison & 5 Arbaugh, 2007; Gunawardena, 1995; Gunawardena & Zittle, 1997; Rourke, Anderson, Garrison, & Archer, 1999; So & Brush, 2008). Others argue that while social presence alone will not ensure the development of critical discourse, it is difficult for such discourse to develop without it (Garrison & Cleveland-Innes, 2005). Overall, research suggests that (a) interactivity impacts social presence, (b) patterns of communication and perceptions of social presence change over time, and (c) social presence can be impacted by the social context, the design of the instruction, and the support of the instructor (Garrison & Arbaugh; Gunawardena; Gunawardena & Zittle; So & Brush). However, notably missing from research on social presence in the computer-mediated classroom are studies involving synchronous CMC. Methods Mixed Method Research Design Mixed methods research combines both quantitative and qualitative forms of inquiry and allows a comprehensive understanding of the research problem through the collection and analysis of multiple sources of data (Creswell, 2009). A mixed methods research design approach is selected for this study as the quantitative survey analysis will examine the relationship between the parallel synchronous computer-mediated communication and learners’ perceptions of social presence while qualitative transcript analysis will offer insight into the nature of the learners’ communication. As depicted in Appendix A, a concurrent triangulation strategy will be utilized in this study in which the quantitative survey data and qualitative content analysis data will be Social Presence in Synchronous Computer-mediated Communication concurrently collected and analyzed with the results subsequently compared to examine similarities and differences in the findings (Creswell, 2009). The survey data collection and quantitative analysis will offer a comparison of social presence between two methods of synchronous CMC; one method including only a single channel of audio and video communication and the other incorporating an additional text-based channel for parallel communication with the main channel. The nature of the communication among students within the parallel text-chat channel will be explored through content analysis of text-chat transcripts. As shown in Appendix B and described below, both survey and text-chat transcript data will be collected during the fall 2009 semester and will be analyzed in the three months that follow the end of the semester. Within the final results comparison, the findings from the quantitative analysis will be compared to the qualitative text-chat transcript analysis. While this concurrent mixed method approach will allow in a shorter data collection 6 period than if the quantitative and qualitative approaches were done separately or sequentially, it is possible that discrepancies in the results may arise that cannot be resolved with the data collected. For example, the quantitative survey data may suggest that learners perceived overall high levels of social presence, but the qualitative content analysis may suggest relatively few indications of social presence. In contrast, the reverse may occur and the survey data may suggest low perceived levels of social presence with relatively high levels of interaction and communication among learners in the text-chat. Such a discrepancy in results may require future study with additional quantitative or qualitative analysis. Participants Participants in this study will be enrolled students in distance education courses at a large public university in the United States. While the university offers distance courses in a range of Social Presence in Synchronous Computer-mediated Communication 7 formats, eligible courses will include only those distance courses in which (a) seven or more live synchronous computer-mediated online sessions are scheduled during each 16 week semester, and (b) 15 or more students are enrolled. The eligible classes will be stratified into two groups based on whether the existing online conferencing interface used to facilitate the course includes the opportunity for parallel text-chat communication. Currently, parallel text-chat is available in synchronous courses coded in the university’s course catalogue as a video streamed instructional method, but is not available in synchronous courses coded as a two-way audio and video instructional method. From within each group (the two-way audio and video group and the video streamed group), three classes will be randomly assigned to the study. The three courses assigned from the video streamed group will be the experimental group while the three classes from the two-way audio and video group will be the control group. Quantitative Data Collection and Analysis - Survey of Student Perceptions Gunawardena (1995) and Gunawardena and Zittle (1997) utilized a survey instrument to solicit learner perceptions of their experience with asynchronous CMC, including satisfaction, social presence, participation, reactions to training, and attitudes toward the CMC. Suggesting that previous survey methods failed to capture a thorough perception of social presence, Tu (2002) devised the Social Presence and Privacy Questionnaire (SPPQ) which measured students’ perceptions of the social context, online communication, interactivity, and privacy. So and Brush (2008) subsequently combined the social presence scale items of Tu’s SPPQ with the satisfaction measures used in the survey instrument developed by Gunawardena and Zittle (1997). The resulting Collaborative Learning, Social Presence, and Satisfaction (CLSS) questionnaire measured general learner characteristic information, as well as learners’ perceptions regarding satisfaction, collaboration, and social presence. Social Presence in Synchronous Computer-mediated Communication An adapted version of the CLSS questionnaire will be utilized in this study. The adapted 8 version includes similar questionnaire items, but is presented within the context of a synchronous CMC environment, as shown in Appendix C. A link to the online version of the questionnaire will be sent via e-mail to all enrolled students in both the experimental and control groups after the last live synchronous session of the semester. Mean score comparison. For each student, an overall profile score for satisfaction, collaboration, and social presence will be calculated based on the student’s average scores for each category. To examine whether there is a statistically significant difference in the mean satisfaction, collaboration, and social presence scores between the experimental and control groups, separate independent samples t tests of mean differences between the experimental and control group will be calculated. Where a significant difference is suggested, a Cohen’s D effect size will be calculated. Based on the calculated effect size and the overall standard deviation for each measure, the estimated difference in average scores between the groups for each measure will be estimated. Correlation analysis. Using the analysis approach taken by So and Brush (2008), Pearson bivariate correlation coefficients will be calculated to analyze the relationships among the measured satisfaction, collaboration, social presence, and learner characteristics (age, gender, computer competency, distance education experience) measures. In addition, partial correlations will be calculated to control for the type of synchronous discussion (either utilizing or not utilizing the parallel text-chat) and each of the general demographic variables to allow an analysis of the impact of these variables on satisfaction, collaboration, and social presence. As shown in Appendix D, the resulting bivariate and partial correlations, as well as coefficients of Social Presence in Synchronous Computer-mediated Communication determination, will be presented. To evaluate statistical significance, a standard level of p < .05 will be used. Qualitative Data Collection and Analysis – Text-chat Content Analysis The purpose of the qualitative text-chat data collection and analysis is to examine the nature of the learners’ conversation within the parallel text-chat. What are the learners saying to 9 each other? What are their patterns of communication? In what respect is the conversation on- or off-task with the conversation in the main audio and video channel? What are the indicators of social presence within the dialogue? While a range of content analysis methods have been used to measure social presence within asynchronous CMC, present a content analysis categorization for examining social presence from the transcripts of an asynchronous computer-mediated environment which has been used in several subsequent studies (Rourke & Anderson, 2004). Based on defined categories and indicators of social presence, including (a) emotional expression seen in affective responses, (b) open communication seen in interactive responses, and (c) group cohesion seen in cohesive responses, messages in asynchronous text-based transcripts are assigned to one of the three categories to assess the relative existence of social presence (Rourke et al.). However, Shi, Mishara, Bonk, Tan, & Zhao (2006) argue content analysis methods for asynchronous computermediated communication must be modified to address the nature of synchronous text chat which is characterized by disrupted, fragmented, and often parallel threads of discourse. To conduct a qualitative analysis of the text-chat transcripts, the present study will include both (a) the threaded discourse analysis method suggested by to examine the nature of the threads of conversation and (b) the content analysis categorization forwarded by to examine the nature of individual text-chat posts. For the three synchronous courses incorporating parallel Social Presence in Synchronous Computer-mediated Communication 10 text-chat discussions, the text-chat transcripts for both the third and the final live sessions will be analyzed independently by two researchers using the coding protocols described below. Interrater reliability will be calculated using Holsti’s calculation for percent agreement (Holsti, as cited in . Analysis of threads of conversation. Using the method of analysis recommended by Shi et al. (2006) to address the often non-sequential and non-linear patterns of synchronous text-chat sessions, the individual text-chat posts for the session will be rearranged in chronological order in a best estimate of related conversations creating a series of separate continuous threads of discussion. The threads will be compared on a common timeline which will allow analysis of the parallel nature of the conversation within the text text-chat itself. To protect the anonymity of the participants, student login names will be replaced with a coding indicator. The qualitative analysis will examine (a) the number of threads of communication an individual participated in within the session, (b) the degree to which the individual is participating in simultaneous threads of conversation, and (c) the relative level of interaction of the individual participant within the text-chat communication. In addition, each thread will be categorized based on a judgment by the raters of whether the thread is either (a) on-task or (b) off-task with the subject of the communication in the main instructional channel. Analysis of individual text-chat posts. As an additional level of analysis, separate textchat posts will be analyzed based on evidence of the three social presence indicators forwarded by . Unlike the analysis described above encompassing the entire threaded conversation, the unit of analysis will be each separate text-chat post. Any individual text-chat post displaying either an affective, interactive or cohesive indicator will be coded as such based on the respective social presence category, as shown in Appendix E. While Rouke et al. used their categorization approach to support a quantitative analysis resulting in a calculation of social presence density Social Presence in Synchronous Computer-mediated Communication within the context of the whole class communication, such a quantitative calculation and comparison to prior findings will not be made here as the analyzed text-chat is a parallel communication channel occurring simultaneously to the main channel audio and video 11 conversation. Instead, the categorization of individual text-chat posts in this study will aid in the qualitative analysis. By extracting text-chat post where indicators of social presence are suggested, the nature of the conversation and displays of social presence can be explored. Preparation and Dissemination of Results Both survey and text-chat transcript data will be collected during the fall 2009 semester and will be analyzed in the three months that follow the end of the semester. Within the final results preparation in early 2010, the findings from the quantitative analysis will be compared to the qualitative text-chat transcript analysis. The details of the research, the suggested findings, and a discussion by the researcher will be released within a paper to be submitted to an academic journal in mid-2010. Social Presence in Synchronous Computer-mediated Communication References Allen, I. E., & Seaman, J. (2008). Staying the Course Online Education in the United States, 2008. Sloan Survey of Online Learning (p. 23). 2008 Sloan Survey of Online Learning, Babson Survey Research Group and the Sloan Consortium. Retrieved March 1, 2009, from http://www.sloan-c.org/publications/survey/pdf/staying_the_course.pdf. Creswell, J. (2009). Research design: qualitative, quantitative, and mixed methods approaches (3rd Ed.). Thousand Oaks Calif.: Sage Publications. De Wever, B., Schellens, T., Valcke, M., & Keer, H. V. (2006). Content analysis schemes to analyze transcripts of online asynchronous discussion groups: A review. Computers & Education, 46(1), 6-28. doi: 10.1016/j.compedu.2005.04.005. 12 Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2-3), 87-105. Garrison, D. R., & Arbaugh, J. (2007). Researching the community of inquiry framework: Review, issues, and future directions. Internet & Higher Education, 10(3), 157-172. doi: Article. Garrison, D. R., & Cleveland-Innes, M. (2005). Facilitating Cognitive Presence in Online Learning: Interaction is Not Enough. American Journal of Distance Education, 19(3), 133. Gunawardena, C. N. (1995). Social Presence Theory and Implications for Interaction and Collaborative Learning in Computer Conferences. International Journal of Educational Telecommunications, 1(2), 147-166. Social Presence in Synchronous Computer-mediated Communication 13 Gunawardena, C. N., & Zittle, F. J. (1997). Social Presence as a Predictor of Satisfaction within a Computer-Mediated Conferencing Environment. American Journal of Distance Education, 11(3), 8. Park, Y. J., & Bonk, C. J. (2007). Synchronous learning experiences: Distance and residential learners’ perspectives in a blended graduate course. Journal of Interactive Online Learning, 6(3), 245-264. Parsad, B., & Lewis, L. (2008). Distance Education at Degree-Granting Postsecondary Institutions: 2006-07. National Center for Education Statistics, U.S. Department of Education. Retrieved March 1, 2009, from http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2009044. Rourke, L., & Anderson, T. (2004). Validity in quantitative content analysis. Educational Technology Research and Development, 52(1), 5-18. doi: 10.1007/BF02504769. Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (1999). Assessing Social Presence in Asynchronous Text-Based Computer Conferencing. Journal of Distance Education, 14(2), 50-71. doi: Article. Schullo, S., Hilbelink, A., Venable, M., & Barron, A. (2007). Selecting a Virtual Classroom System: Elluminate Live vs. Macromedia Breeze (Adobe Acrobat Connect Professional). Journal of Online Learning and Teaching, 3(4). Retrieved March 22, 2009, from http://jolt.merlot.org/vol3no4/hilbelink.htm. Shi, S., & Morrow, B. V. (2006). E-Conferencing for Instruction: What Works? Educause Quarterly, 29(4), 42. Social Presence in Synchronous Computer-mediated Communication Shi, S., Mishara, P., Bonk, C. J., Tan, S., & Zhao, Y. (2006). Thread Theory: A Framework Applied to Content Analysis of Synchronous Computer Mediated Communication Data. International Journal of Instructional Technology and Distance Learning, Vol. 3(No. 3). Retrieved August 26, 2008, from http://www.itdl.org/Journal/Mar_06/article02.htm. Short, J., Williams, E., & Christie, B. (1976). The Social Psychology of Communications. London: John Wiley. So, H., & Brush, T. A. (2008). Student Perceptions of Collaborative Learning, Social Presence 14 and Satisfaction in a Blended Learning Environment: Relationships and Critical Factors. Computers & Education, 51(1), 318-336. Tu, C. (2002). The Measurement of Social Presence in an Online Learning Environment. International Journal on E-Learning, 1(2), 34-45. Appendix A Figure A1. Concurrent Triangulation Design 15 Source: (Creswell, 2009) Appendix B Study Timeline 16 Appendix C Sample Questionnaire This sample questionnaire is adapted from the Collaborative Learning, Social Presence, and Satisfaction Questionnaire (So & Brush, 2008). 17 Instructions: This questionnaire is designed to measure your perceptions on the level of collaborative learning, social presence and satisfaction. There is no right or wrong answer for each question. However, it is important for you to respond as accurately as possible by checking the most appropriate response. Section 1: General Information 1. What is your gender? ___ ___ ___ Female Male Not applicable 2. What is your age? ___ ___ ___ ___ ___ ___ Under 18 18-25 26 - 35 36 - 45 Above 45 Not applicable 3. Please estimate your level of computer expertise? ___ ___ ___ ___ ___ No experience Novice Intermediate Expert Not Applicable 4. How many distance courses have you taken at any institution prior to this course? Please circle the number. 0 1 2 3 4 5 6 7 8 9 10 More than 10 Appendix C Section 2: Satisfaction 18 Please read each statement carefully and then indicate the degree to which you agree or disagree with the statement. Strongly Disagre e 1 1 1 1 1 1 1 1 1 1 1 Disagre e 2 2 2 2 2 2 2 2 2 2 2 Neutral 3 3 3 3 3 3 3 3 3 3 3 Agree 4 4 4 4 4 4 4 4 4 4 4 Strongly Agree 5 5 5 5 5 5 5 5 5 5 5 1 2 3 4 5 6 7 8 9 10 11 I was able to learn from our live class discussions I was stimulated to do additional readings or research on topics discussed in our live discussions The live discussions assisted me in understanding other points of view As a result of my experience with this course, I would like to take another distance course in the future This course was a useful learning experience The diversity of topics in this course prompted me to participate in the live discussions I put a great deal of effort to learn the online conferencing system to participate in this course My level of learning that took place in this course was of the highest quality Overall, the learning activities and assignments of this course met my learning expectations Overall, my instructor for this course met my learning expectations Overall, this course met my learning expectations Section 3: Collaboration Strongly Disagre e 1 1 1 1 1 1 1 1 Disagre e 2 2 2 2 2 2 2 2 Neutral 3 3 3 3 3 3 3 3 Agree 4 4 4 4 4 4 4 4 Strongly Agree 5 5 5 5 5 5 5 5 1 2 3 4 5 6 7 8 Collaborative learning experience in the computermediated environment is better than in a face-toface environment I felt part of a learning community I actively exchanged my ideas during the live class sessions I was able to develop new skills and knowledge from other class members I was able to develop problem solving skills through peer collaboration Collaborative learning was effective Collaborative learning in my group was timeconsuming Overall, I am satisfied with my collaborative learning experience in this course Appendix C Section 4. Social Presence 19 Please read each statement carefully and then indicate the degree to which you agree or disagree with the statement as it relates to live online conferencing sessions in this class. Strongly Disagre e 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 Disagre e 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Neutral 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 Agree 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 Strongly Agree 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Computer-mediated discussions are social forms of communication Computer-mediated discussions convey feelings and emotions Computer-mediated discussions can be private and confidential Computer-mediated discussions are personal Computer-mediated discussions are a pleasant way to communicate with others The language people use to express themselves in online communication is stimulating It is easy to express what I want to communicate through computer-mediated discussions during class The language used to express oneself in online communication is easily understood I am comfortable participating, even when I am not familiar with the topics The online conferencing system is technically reliable Computer-mediated discussions allows relationship to be established based upon sharing and exchanging of information Computer-mediated discussions allows me to build more caring social relationships with others It is unlikely that someone might obtain personal information about me from the computer-mediated discussion Where I access the online conference (home, office, computer lab, public areas) does not affect my desire to participate. Where I access the online conference (home, office, computer lab, public areas) affects my ability to participate. Computer-mediated discussions permit the building of trust relationships The amounts of discussion in class does not inhibit my ability to communicate Appendix C 20 Appendix D 21 Results Presentation Variables Collaboration Social Presence Age Computer Competency Type of Synchronous Discussion Number of Distance Courses Satisfaction .00 .00 .00 .00 .00 .00 Collaboration -.00 .00 .00 .00 .00 Social Presence --.00 .00 .00 .00 Appendix E Categories for Assessment of Social Presence Affective Expression of emotions Use of humor Interactive Continuing a thread Quoting from others’ messages 22 Cohesive Vocatives: Addressing to participants by name Addresses or refers to the group using inclusive pronouns Social greetings, salutations Self-disclosure Referring explicitly to others’ messages Asking questions Complimenting, expressing, appreciation Expressing agreement
System Analysis 1 Running head: SYSTEM ANALYSIS OF ATHABASCA UNIVERSITY System Analysis of Athabasca University Jennifer Maddrell Old Dominion University System Analysis 2 Focus and Purpose Institution Typology Athabasca University was formed as a distance education university by the Government of Alberta Canada in 1970. While Athabasca provides distance education course offerings for colleges and universities throughout Canada through inter-institution course transfer credit, it continues to operate as an autonomous degree granting distance learning university. With liberal transfer of credit options within the Canadian college and university system, credit for prior learning, rolling enrollment, and admission provisions that allow undergraduate admission to anyone over 16 years of age without regard to prior academic achievement, Athabasca classifies itself as an Open University. The government remains a major force behind Athabasca. In 2007, the Province of Alberta provided $31,064,000 (CAD) in grant funding which represented over 30% of the university’s operating revenue. Further, the university’s governance is dictated by Alberta Regulation 50/204, the Post-secondary Learning Act, which establishes the powers and duties of the university’s administration by the Athabasca University Governing Council. As of March 31, 2007, the Governing Council, headed by an Executive Officer (also the President of Athabasca University), included one nonacademic staff member, one tutor member, two academic staff members, two student members, nine appointed public members, and one alumni member. Mission and Mandate Since its inception, Athabasca University’s stated mission has been to offer distance education to residents of Alberta, the rest of Canada, and the world. As presented within the 2007Annual Report, the university’s mission is to 1) remove barriers that restrict access to university level studies, 2) increase equality of educational opportunities for adult learners System Analysis 3 worldwide, 3) commit to excellence in teaching, research, scholarship, and public service, and 4) focus on distance education and the associated learning technologies. Athabasca’s mandate is restated in the 2007 Annual Report and calls for the publicly funded university to offer undergraduate degree programs in natural and pure sciences, humanities, social sciences, interdisciplinary studies, administrative studies, commerce, nursing, and allied professional fields, as well as graduate degree programs in distance education, health studies, and business administration. Strategic University Plan for 2006 – 2011. A new strategic plan was drafted in 2006 and is presented as an appendix to the 2007 Annual Report. The plan outlines specific goals intended to achieve Athabasca’s continued commitment to open access and the delivery of high quality distance education, as well as a renewed focus on research. Features Open Admissions and Enrollment As noted, edibility for admissions to undergraduate courses at Athabasca is liberal compared to other degree granting universities in North America. Students age 16 or older are admitted throughout the year regardless of their previous educational experience or achievement. From 1997 to 2007, total course enrollment increased a dramatic 415%. Currently, 34,000 students are enrolled in undergraduate courses and 3,000 are enrolled in graduate level courses which Athabasca reports as a full load equivalent of 5,930 undergraduate students and 1,263 graduate students. Of the total number of students enrolled, 35% are residents of Alberta. System Analysis 4 As is common in other distance education programs, nearly all enrolled students work while attending classes. During a recent survey conducted by the university and cited in the 2006 Annual Report, 94% of graduates reported working while completing their coursework. Course and Degrees During the 2006 - 2007 fiscal year, 68,284 individual courses were taken representing an increase of 6.4% over the prior year. Over that same time period, 780 undergraduate degrees and 208 graduate degrees were conferred. Athabasca currently offers 11 undergraduate degrees, 20 certificate programs, and 8 graduate degrees. In the fall of 2008, Athabasca will begin a new doctoral program which will grant a Doctor in Distance Education (EdD). The undergraduate degree programs with the highest current enrollment include Bachelor of Arts with 2,413 enrolled, Bachelor of Nursing with 2,122 enrolled, Bachelor of Commerce with 1,760 enrolled, and Bachelor of Professional Arts with 1,614 enrolled. The graduate degree programs with the highest current enrollment include Master of Arts with 632 enrolled, Master of Business Administration with 835 enrolled, Master of Distance Education with 369 enrolled, Master of Health Studies with 465 enrolled, Master of Nursing with 529 enrolled. Despite the growing admission figures and the increasing number of degree programs, graduation rates are low compared to other Canadian universities. Powell and Keen (2006) report that while hundreds of thousands of students have enrolled at the undergraduate level, only several thousand undergraduate degrees have been conferred. While these figures could imply poor student satisfaction with the courses, biennial Government of Alberta Graduate Satisfaction and Labour Market Experience surveys consistently report high perceived quality ratings from Athabasca University students. Most students attending courses at Athabasca do not intend to System Analysis 5 complete a degree at Athabasca, but are interested in taking courses to fulfill requirements within other degree programs (about half the enrollment) or for other personal and professional reasons. Tuition Tuition rates for the degree programs are reasonable compared to most public or private universities in North America. Based on current tuition rates, the approximate tuition range for an undergraduate degree is $5,900 - $6,890 CAD for Canadian residents and $10,500 - $11,970 CAD for those residing outside of Canada. Tuition for graduate degree programs currently range from $10,250 – 13,000 CAD for Canadian residents and $12,250 – $15,500 CAD for those residing outside of Canada. During the 2006 – 2007 fiscal year, Athabasca collected $33,485,000 CAD in undergraduate tuition and $12,282,000 CAD in graduate tuition, representing 35% and 13% of revenue, respectively. Instruction Method of Study. Athabasca offers instruction in either grouped or individual study. Grouped study typically begins at a set date, either in September and January, and continues for either 13 weeks, for a 3 credit course, or 26 weeks, for a 6 credit course. Grouped study courses represent only about 20% of total course enrollments (Davis, 2001). The courses are generally facilitated by an instructor and instruction is delivered in either a print based or online format. However, some group study is offered in traditional classrooms at designated Athabasca learning sites or at partner institutions with collaboration agreements with Athabasca. Individual study, the far more common method of study, begins on the first day of any month. Students must simply register by the 10th day of the preceding month. Instructional materials are delivered in either a print based or online format. The course term, known as the “contract period”, lasts 6 months for a 4 credit or less course and 12 months for a 6 credit course. System Analysis 6 While the individualized study courses are self-paced, the learners are offered a tutor. Upon registration in a course, the student is introduced to the assigned tutor via letter or e-mail. The tutor’s role is to provide subject matter assistance, feedback on assignments, exam preparation, and grade assignment. Tutors generally provide assistance via e-mail or phone. Within the School of Business, Athabasca has also begun the use of tutor support call centers as an alternative to traditional tutors. Learners call a toll free number and work with the tutor on duty at the time the call is placed to the call center. Faculty. As of March 31, 2007, Athabasca employed 1,226 people, including 152 in academic full time positions, 168 in academic part time positions, and 322 tutors. The balance comprised management, professional, and support staff. Over the three year period from 2004 – 2006, the annual average number of referred articles, books, and conference presentations by faculty was 146, 53 and 281, respectively. Design and Delivery of Instruction. Athabasca employs a team approach to course design. A typical course design team includes a subject matter expert, visual designer, digital media technologist, copyright officer, and editor. Course materials are delivered via fax, regular mail, or the Internet. Print and digital course materials are delivered to students as part of a course package of resources distributed from the Materials Management Office which, depending upon the course, may include student manuals, study guides, and text books. While print and digital media, including CDs and DVDs, have historically been mailed to students, instruction via the Internet is rapidly becoming a primary means of instructional delivery. As outlined in the 2006 – 2007 Annual Report, Athabasca has budgeted $21 million for information technology hardware and software upgrades between 2006 and 2011 in order to accommodate this shift in instructional delivery. Along with e-mail delivery of content, other System Analysis 7 forms of Internet based instructional delivery are also employed. In 2005, Athabasca began facilitating online courses using Moodle, the open source learning management system. In addition, live instruction is often delivered via streaming audio and video and some live class sessions are being held using web conferencing tools, such as Elluminate. Other technology. In addition to the learning technologies noted above, Athabasca maintains an online student web portal, myAU, based on the open source uPortal software (Guohua & Bonk, 2007). This online web portal offers students and faculty a single sign-on to university services, including the campus administrative systems, the learning management system, as well as the library information systems. Student Services Financial Assistance. Students at Athabasca are eligible for financial assistance. Full time students may apply for grants, loans, and scholarships while part time students (those enrolled in less than 9 credits in a 4 month period) are only eligible for grants and loans. Learning Services. Athabasca offers students a host of learning services, including academic advising, access for students with disabilities, admission and registration services, and exam supervision. In addition, all actively registered students have library borrowing privileges. The library information desk is manned 24 hours a day via e-mail, fax, mail, or phone to provide instructions on how to access information or to provide research assistance. The Athabasca library website provides online access to the entire library catalogue, thousands of electronic books and reference websites, and over 32,000 journal articles contained within 200 full text subscription journal databases. Athabasca also has inter-loan library agreements through the Alberta library system and the Canadian University Reciprocal Borrowing Agreement (CURBA). System Analysis 8 Student Interaction. Students have the option of participating in established school clubs, peer support groups, online discussion forums, or social groups. Athabasca also publishes a quarterly online magazine (the au.world e-zine) which highlights current information of interest to Athabasca students. Open Access Publishing. Athabasca is committed to providing open access and online dissemination of publications produced by the university. This includes open online access to The International Review of Research in Open and Distance Learning, a refereed journal published by Athabasca. Evaluation and Accreditation Athabasca’s internal review protocols are contained within a comprehensive Program Review Policy document. Under the terms of the review policy, all programs must be reviewed by internal and external assessors at least every six years. The material assessed during the review includes such items as the current course syllabi and related course materials, feedback from partner institutions, program financial statements and budges, surveys of students and graduates, and opinions of tutors and instructors. As in the United States, the Canadian central government does not accredit universities. However, Athabasca was accredited by the United States’ Middle States Commission on Higher Education (MSCHE) in June 2005. In addition, the recently established Campus Alberta Quality Council, formed as a quality assurance agency as part of the 2004 Post-secondary Learning Act and the Approval of Programs of Study Regulation (51/2004), has reviewed and recommended several new programs within the university, including the new Distance Education EdD program. According to the mandates under the Act, all new degree programs must be reviewed and recommended by the Council. System Analysis 9 Strengths and Weaknesses Strengths As a pioneer in the delivery of university level distance education, Athabasca offers distance learners a flexible, affordable, and accredited education with a comprehensive roster of student services. Enrollments are growing and the university has seen an increase in research funding ($2,117,000 CAD for the fiscal year ending March 31, 2007 representing a 14% increase from the prior year). Further, with backing and oversight from the Alberta government, Athabasca is financial secure and is operating at a net profit ($1,446,000 CAD for the fiscal year ending March 31, 2007). Weaknesses Athabasca’s rapid growth over the past decade is straining the university’s infrastructure. As noted in the 2006 – 2007 Annual Report, Athabasca’s infrastructure was designed for 10,000 students and the rapid growth over the past decade has caused a critical need for additional space. While space is not needed to support classrooms, physical space is needed to house curriculum development, learner support services, and research functions. Further, Athabasca is struggling to recruit and retain faculty to accommodate the growing enrollment; a difficult task given the limited pool of doctoral level candidates within the university, the location of the main campus in Athabasca, and the limited research opportunities outside of distance education. However, there are threats to Athabasca’s continued growth. Once one of only a few distance education universities, Athabasca now faces increased competition from both stand alone online universities and distance education arms of traditional universities. In addition, while the Alberta government subsidy to Athabasca covers a substantial portion of the operating System Analysis 10 budget, currently over 30% of annual revenue, Athabasca’s operations would be at risk should the government decide to alter the amount or provisions of the operating grant. Further, graduation rates are low. While this does not point to a problem in overall quality or learner satisfaction, it does suggest that most students are merely pursuing individual courses or taking transfer credits back to a home institution. As such, it becomes difficult for Athabasca to make a mark as a standalone degree granting intuition when the majority of students are taking individual courses for transfer credit to receive a degree from a traditional (bricks and mortar) institution. System Analysis 11 References Athabasca University - about Athabasca University. Retrieved from http://www.athabascau.ca/ Alan Davis. (2001). Athabasca university: conversion from traditional distance education to online courses, programs and services, The International Review of Research in Open and Distance Learning; Vol 1, No 2 (2001). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/19/358. Alberta Government. (2004). Alberta Post Secondary Learning Act. Retrieved from http://www.qp.gov.ab.ca/documents/Acts/P19P5.cfm. Athabasca University, Office of the President. (2003). Athabasca University Policy - Program Review Policy. Retrieved March 13, 2008, from http://www.athabascau.ca/policy/academic/programreviewpolicy.htm. Athabasca University. (2007). Athabasca university annual report 2006 - 2007, 30. Athabasca, Alberta Canada: Athabasca University. Retrieved from http://www.athabascau.ca/report2007/ Athabasca University. (2006). Athabasca university annual report 2005 - 2006, 30. Athabasca, Alberta Canada: Athabasca University. Retrieved from http://www.athabascau.ca/report2006/. Campus Alberta Quality Control Council (CAQC) - Program Assessment Standards, Campus Alberta Quality Council. Retrieved from http://www.caqc.gov.ab.ca/default.asp. Guohua Pan & Curtis J. Bonk. (2007). The emergence of open-source software in north America, The International Review of Research in Open and Distance Learning; Vol 8, No 3 (2007). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/496/938. Powell, R., & Keen, C. (2006). The axiomatic trap: stultifying myths in distance education, Higher Education, 52(2), 283-301. doi: 10.1007/s10734-004-4501-2.
The focus of this report is to review the literature for assessments of the effect of computer-mediated backchannel interaction during live instructional presentation. The goal is to consider the impact on the learner as both a receiver of instructional messages sent from the instructor, as well as an active participant within the learning process.
Backchannel Interactions Running head: THE EFFECT OF BACKCHANNEL INTERACTIONS 1 The Effect of Backchannel Interactions on Cognitive Load Jennifer Maddrell Old Dominion University IDT 895: Message Design June 25, 2008 Backchannel Interactions Backchannel Interactions Computer-mediated communication (CMC) technologies offer instructional designers new ways to design and deliver instructional material to learners. New forms of web conferencing technology allow live audio and visual presentation of instructional material. Beyond a one-way or single channel broadcast of the primary instructional message, these web conferencing technologies also support simultaneous multi-channel communication among all participants and support the learners’ real time interactions with the instructional content, with the instructor, and with peer learners. A by-product of the latest synchronous CMC technologies is the appearance of what are being termed backchannel interactions which occur simultaneously with the primary instructional presentation (Yardi, 2006). No longer a passive recipient of the single channel instructional message, a learner within a web conferencing session has the ability to interact directly with the content, with peer learners, and with the instructor during the instructional presentation. The interactive features included with the newest forms of web conferencing technologies allow learners to annotate directly on the presentation slides while the presenter is speaking, to route or receive files, to send and get links to web sites during the presentation, to type viewable notes to the class in the margins of the presentation window, or to conduct text based conversations while the live instructional presentation is being delivered. 2 While the term backchannel is used in other contexts and is spelled as backchannel, backchannel, or back channel, a consistent definition within the context of synchronous CMC does not exist. Cogdill, Fanderclai, Kilborn, and Williams (2001) suggest that backchannel interactions tend to fall into the following five categories: 1) process-oriented interactions which steer the main channel discourse, 2) content-oriented interactions which respond to the content in Backchannel Interactions 3 the main channel, 3) participation-enabling interactions which include assistance to participants, 4) tangential interactions which branch from or continue a completed main channel discussion , and 5) independent interactions which are private and unrelated to the main channel. These backchannel interactions are well outside the norm of learner behavior in traditional face to face lecture settings. However, the availability of this form of computermediated interaction, as well as ongoing discussion about the classroom role of the learner as either an active participant or passive recipient, is sparking debate among practitioners regarding what interactions learners should engage in during both face to face and computer-mediated instructional presentation (Fried, 2008). As written in an April 2008 article entitled Hey, You! Pay Attention! at InsideHigherEd.com and in the approximately 50 ensuing comment posts to the article, learners computer-mediated interactions during lecture are viewed by educators as both a bold step forward in instruction and a tremendous distraction to the learning task at hand (Guess, 2008). In what one commenter to the article called a “ridiculous debate”, one side views computer-mediated interactions during instructional presentation as nothing more than virtual note passing which is a distraction to the learning task at hand and a symbol of the growing lack of respect for teachers during lecture. In contrast, the other side views these backchannel features as a powerful opportunity to facilitate increased content and human interaction. The focus of this report is to review the literature for assessments of the effect of computer-mediated backchannel interaction during live instructional presentation. The goal is to consider the impact on the learner as both a receiver of instructional messages sent from the instructor, as well as an active participant within the learning process. Unfortunately, there has been little research conducted to specifically assess this relatively new phenomenon. In reviewing prior literature on computer-mediated backchannel Backchannel Interactions interactions in the classroom, Yardi (2006) found nothing beyond qualitative reviews assessing the use of text chat during live conference proceedings or essays considering the potential advantages and disadvantages of utilizing synchronous CMC to facilitate classroom discussion. While a body of theory and research exists assessing the effects of interacting in computermediated learning environments, the focus is largely on asynchronous computer supported interactions rather than synchronous interactions (Paulus, 2007). Further, as noted by Moore, Burton, and Myers (2004, p. 998) within their extensive review of multiple-channel communication research, “We feel that instructional designers, looking for simple rationale, methods, or guidelines for effective multimedia (multiple-channel) presentation will be disappointed in the relevant research” which they feel is “confusing at best.” While there is little research to report from direct studies on computer-mediated backchannel interactions in the classroom, the objective here is to glean information from other areas of research regarding learner interaction with instructional content, other learners, and the instructor to begin to assess how this new form of backchannel interaction could impact the student’s ability to learn from the instruction. The following presents a review of literature in both traditional and computer-mediated instructional settings. While multiple theories of learning, communication, and instruction are presented in the review of literature, this review is presented within the context of cognitive load theory (CLT). Cognitive Load Theory CLT Described CLT suggests that working memory faces important processing limitations which ultimately impact a learner’s ability to process, encode, and retrieve information (Sweller & Chandler, 1994). CLT has evolved over the past two decades and is concerned with a learner’s 4 Backchannel Interactions limited working memory processing capacity and the combined effect of intrinsic, extraneous, and germane cognitive load (Pociask & Morrison, 2004). Intrinsic cognitive load is imposed by the inherent nature of the to-be-learned information (van Merrienboer & Sweller, 2005). Extraneous cognitive load is imposed by inappropriate instructional design choices. This includes the instructional message design, the instructional presentation, and interface choices related to the delivery mode (visual or verbal), modality (text or narration), and spatial arrangements on the page or screen (Lee, Plass, & Homer, 2006). Germane cognitive load is associated with processes to assist in learning, including processes to facilitate schema acquisition and automation (van Merrienboer & Sweller). Fundamental to CLT is the notion, as summarized by Sweller and Chandler (1994, p. 5 192), that the learning environment should eliminate “irrelevant cognitive activities”, defined by them as “any activity not directed to schema acquisition and automation” which they deem to be significant learning mechanisms. They note that such irrelevant activities may unnecessarily increase cognitive load and hamper the processing of to-be-learned material. Kester, Kirschner, and van Merrienboer (2005, p. 168) suggest that the instructional design of the learning environment should “properly manage intrinsic load, minimize extraneous load, and optimize germane load within the boundaries of working memory capacity.” Framework for Review Given that cognitive load is a central consideration in multimedia learning (Mayer & Moreno, 2003), the primary task of this review is to assess where computer-mediated backchannel interactions during instructional presentation fall within the cognitive load equation. While the review is presented within the context of CLT, theories and findings from research in areas both inside and outside of learning and instruction are also considered. By taking into Backchannel Interactions account research in a range of areas, this review attempts to highlight prior findings which may shed light on the following questions. Extraneous cognitive load. Do backchannel interactions increase extraneous cognitive 6 load? Are these interactions unnecessary instructional activities imposed by poor instructional or message design that could and should be eliminated to reduce extraneous load? Germane cognitive load. Are backchannel interactions germane to the learning process as part of effective presentation, communication, and dialogue to support the learner? Do these interactions help learners reflect upon the material, create meaning from the presented content, and process the to-be-learning material within memory? Intrinsic cognitive load. Does interaction within the backchannel help presenters to more effectively sequence and segment instruction based on the cues of learner understanding found in the backchannel responses? In turn, could these interactions be used to manage intrinsic cognitive load? Extraneous Cognitive Load The focus of this section is to assess whether the incorporation of backchannel interactions during instructional presentation is a poor design choice that takes away from the processing of to-be-learned information, creates unnecessary interactivity, and results in high extraneous cognitive load. Given the similarity between backchannel interactions and practices which are outside acceptable norms within a traditional face to face classroom, such as note passing, whispering to peers in class, or talking while the presenter is speaking, it is understandable why some would predict that backchannel interactions are distraction to the learning task at hand. Beyond seemingly obvious violations of traditional classroom norms, there is evidence from research on interactivity, split attention and redundancy effects, and laptop use Backchannel Interactions in the classroom which may suggest how backchannel interactions impact extraneous cognitive load. Interactivity in Learning Environment Sweller and Chandler (1994) suggest that high cognitive load is directly related to 7 interactivity caused by either the nature of the to-be-learned material (intrinsic cognitive load) or by the presentation (extraneous cognitive load). The to-be-learned material is considered to have high interactivity if there are numerous elements which must be processed simultaneously (van Merrienboer & Sweller, 2003). If the element interactivity is low (hence the intrinsic cognitive load is low), then extraneous load may not be a concern; but in complex learning situations where the intrinsic element interactivity is high, it is necessary to carefully manage the learning environment to avoid unnecessary instructional interactivity in order to reduce extraneous cognitive load (Sweller & Chandler). Moreno and Mayer (2007) examined interactivity as a characteristic of the learning environment in which the interactivity results in a variation in the instruction based on the learners’ actions. They suggest five types of learner interactivity, including 1) dialoguing in which the learner asks questions and receives feedback, 2) controlling in which the learner establishes the pace or order of presentation, 3) manipulating in which the learner sets aspects of the presentation 4) searching in which the learner seeks new information, and 5) navigating in which the learner selects from among content choices. They suggest the interactivity can be considered a continuum of no interactivity to high interactivity. Moreno and Mayer note that the challenge for designers working in interactive multimodal learning environments with ever increasing opportunities for interactivity is to reduce extraneous cognitive load imposed by the interactivity while at the same time using the interactivity to increase generative cognitive Backchannel Interactions processing, as discussed below in the section on germane cognitive load. Therefore, the unanswered question becomes whether the interactivity involved with backchannel interactions is extraneous load within the learning environment or germane to the process of learning? Cognitive Load Effects 8 Decades of research have provided findings that suggest a number of instructional effects which increase extraneous cognitive load. Three effects which may be most applicable to backchannel interactions found within the synchronous computer-mediated learning environment are 1) split attention effects, 2) redundancy effects, and 3) expert reversal effects. Split attention effect. The learners’ text based backchannel interactions which occur concurrently with the instructor’s audio and visual presentation may result in a split attention effect. While research suggests that dual presentation from both auditory and visual sources may distribute the processing of information and increase working memory capacity, other research suggest that instruction requiring learners to devote their attention to multiple sources of information may unnecessarily cause extraneous cognitive load (Sweller & Chandler, 1994). Redundancy effect. The discussion about the instruction content within the backchannel occurring concurrently with the instructional presentation may result in a redundancy effect. Research suggests that as learners must attend to and integrate sources of overlapping or redundant information, unnecessarily extraneous load is imposed (Sweller & Chandler, 1994). Expertise reversal effect. Cognitive load research findings also that suggests an expertise reversal effect in which the conditions which are appropriate for a novice learner may not be appropriate for a more experienced learner (Kalyuga, Ayres, Chandler, & Sweller, 2003). Kalyuga et al. note that many CLT effects, including split attention and redundancy effects, are not applicable to experts who possess schemas in the domain being presented. These learners Backchannel Interactions may find the instruction to be redundant with their existing schemas and the integration of the redundant information is seen to be a source of extraneous cognitive load. 9 Research on split attention, redundancy, and reversal effects suggest that instructional design choices should be appropriate for the expertise level of the intended learners and that methods must change as learners’ expertise increases (van Merrienboer & Sweller, 2005). While backchannel interactions could increase extraneous load for some learners, could backchannel interactions be used as a means for the presenter to gauge the level of expertise of the learners during instructional presentation? If so, is it possible that the same backchannel interactions which may cause extraneous cognitive load effects in novice learners could act to reduce extraneous cognitive load as the learners advance by providing signals to the presenter of the learners’ level of expertise? Laptops in the Classroom Research on laptop use in the classroom lecture setting may provide one of the closest bodies of research to backchannel interactions. It may be possible to draw a parallel between backchannel interactions in a synchronous computer-mediate instructional presentation with laptop use in face to face classroom lecture settings. As noted in a recent review of classroom laptop literature by Fried (2008), contradictory research results abound. Yet, there is a body of research which suggests that laptop use in the classroom lecture setting is a source of overload and distraction. Included in Fried’s review are her own research findings which suggest that students using laptops during classroom lectures regularly use the laptop for things other than taking notes. Further, the students’ laptop use was negatively related to several measures of learning and was reported to be a distraction from fellow students. Backchannel Interactions 10 Fried (2005) viewed the results as clear support for prior research that suggests classroom performance is negatively related to the learners’ laptop use within the classroom. However, it is important to note that in Fried’s research the students’ laptop use was in no way integrated into the classroom lecture. The students were informed that the laptops would not be needed during the semester in lectures, the presenter made no attempts to guide the learners’ use of the laptops, nor were their laptops used to display instructional presentation. Apparently, the assumed sole purpose of the laptops in the studied classroom was as a note taking device given that any other activity outside of note taking, including the 45% of the students in the study who reported regularly using instant messaging, was dismissed within the study as an unnecessary and distracting activity. However, Fried did acknowledge that the primary limitation to the generalization of the findings was that laptop use was not integrated into the lecture. Further, as Fried suggests, the findings associated with learning measures may also reflect that struggling students are more likely to be diverted from the lecture. Germane Cognitive Load Instructional activities that encourage mental effort in schema construction and automation are viewed as processes that increase germane cognitive load (van Merrienboer & Sweller, 2005). As suggested by Winn (2004), advances in computer-mediated technologies make it possible to do more than direct teaching and to use the technology to assist learners as they actively select, organize, and integrate new information. Some suggest that synchronous computer-mediated discussion helps learners to move from surface understanding to more deep learning as they reflect and respond to questions from peers and the instructor (Havard, Jianxia & Olinzock, 2005). Moreno and Mayer (2007) view this as a difference between facilitating information acquisition and supporting knowledge construction. Could backchannel interactions Backchannel Interactions optimize germane cognitive load by aiding the learners’ understanding of the instructional 11 content, adding context to the presentation narrative, providing opportunities for reflection, and promoting engagement? Laptops in the Classroom - Revisited In contrast to the laptops in the classroom research cited above, other research suggests that computer use during classroom presentation can facilitate classroom interactions and class participation which, in turn, increases engagement, motivation, and active learning (Fitch, Partee, Stephens, & Driver, as cited in Fried, 2008). In summarizing their research on laptop use during classroom instruction, Barak, Lipson, and Lerman (2006) suggest that computer use by students during class facilitates construction of understanding of the learning material, immediate feedback and help, multiple interactions among learners and instructors, and the ability to share work, ideas, and learner interpretations. Computer Mediated Communication In a review of literature related to CMC processes, Marshall and Novick (1995) suggest that CMC differs from face to face communication and is generally characterized by longer turns, fewer interruptions, less overlaps, and increased formality in switching among speakers. DeSanctis and Monge (1998) report research that suggests electronic communication tends to decrease levels of communication as compared to face to face communication. They note that this may be the result of reduced use of speech acknowledgements, such as “Uh-hmm”, or typical social greetings. DeSanctis and Monge also cite findings that suggest participants engaging in CMC conversation may experience difficulty in establishing meaning of information and managing feedback in conversation which may negatively affect message understanding, but that attention to maintaining mutual understanding across the group can help to ensure effective Backchannel Interactions communication. Could synchronous backchannel interactions help to overcome some of these obstacles associated with CMC and foster mutual understanding across the group? Marshall and Novick (1995) also note that the characteristics of CMC may affect conversational effectiveness which they describe as the degree to which the mutual conversational goals are achieved. They cite a large body of research which supports a collaborative theory of conversation which focuses on the joint construction of conversation in 12 which interactive and collaborative aspects of the conversation help to support full understanding and to achieve the overall expectations for the conversation. In summarizing their own research findings, Marshall and Novick (p. 75) suggest CMC is “enhanced by the addition of a channel which allows conversant to share relevant visual context, particularly where visual context is relevant to the task” thereby allowing users more control over the social distance or presence. Do backchannel interactions offer learners more control over social distance and help to improve CMC effectiveness? Pelowski, Frissell, Cabral, and Yu (2005) conducted research to identify various immediacy behaviors within synchronous text chat logs with the hope of shedding light on learners’ feelings of social presence and the impact on learning. The authors indicate that while a positive relationship has been found between perceptions of immediacy and performance in face to face environments, little immediacy research within synchronous computer-mediated instruction has been studied. Citing various research findings from traditional face to face classrooms, Pelowski et al. note that immediacy behaviors, such as calling others by name, smiling or engaging in eye content, have been shown to enhance perceptions of closeness or immediacy to others. Backchannel Interactions 13 Pelowski, et al (2005) found significant variation in overall chat participation, as well as in immediacy behaviors. Acknowledgement, salutations, and questions were observed in nearly all students at least once. Agreement or disagreement was shown at least once by over 80% of the students. Humor, self-discloser, and value statements appeared less frequently, but at least once by over 60% of students. However, no significant correlation was found between immediacy behaviors in the text chat environment and learner performance. Could this be an indication that while backchannel interactions may facilitate more effective communication, the communication may mean nothing in terms of learner performance? Constructing Meaning Do learners gain context and insight from the commentary of students within the backchannel interactions? Some social constructivists view synchronous CMC technologies as vehicles to support student to student co-creation of meaning and understanding, including Gunawardena, Lowe, and Anderson (as cited in Paulus, 2007) who suggest knowledge construction via CMC consisting of five phases: 1) sharing and comparing of information, 2) discovery and exploration of cognitive dissonance, 3) negotiation of meaning and coconstruction of knowledge, 4) testing and modification of proposed co-construction, and 5) agreement and applications of newly constructed meaning. Yet, as Paulus notes, the research literature suggests that learners in most CMC supported environments, which tend to rely heavily on asynchronous interactions, rarely move beyond sharing and comparing of information. Could the synchronous backchannel provide support for more immediate meaningful reflection and cocreation of knowledge? In noting the small, but growing body of research on synchronous CMC tools in learning, Stein, Wanstreet, Glazer, Engle, Harris, and Johnston (2007) investigated if and how shared Backchannel Interactions meaning is achieved in a synchronous text based chat. Their review of formal small group synchronous text chat logs suggest that text chat can lead to a shared understanding from a pattern of interaction which establishes a) social presence, characterized by group inquiry and integration, b) teaching presence, characterized by the teacher’s efforts to focus and organize, 14 and c) cognitive presence characterized by the learners’ patterns of reflection and revision. Stein et al. observed: In a more casual, immediate environment than asynchronous discussion boards, chats give learners the opportunity to transform their personal meaning into shared solutions through a nonlinear process of asking questions, exchanging information, connecting ideas, and defending solutions … In addition, the group as a whole has the ability to see the progression of logic and higher-order thinking as the text unfolds on the members' computer screens and is revised, amplified, and integrated into shared understanding through feedback. (p. 113) Would synchronous text based backchannel interactions during instructional presentation result in similar findings? Or would the larger group size and split attention from the presentation to the screen alter the results? Learners as Presentation Co-narrators Research in dialogue and communication suggests a joint role for learners as co-narrators in the instructional presentation. Could learner responses in the backchannel enhance the main channel message of the presentation? Does the backchannel provide on-the-fly reflection which the instructor can monitor to check for learners’ understanding and adjust the presentation based on the learners’ responses? Backchannel Interactions Bavelas, Coates and Johnson (2000) explore the various conceptions of information 15 communication models beginning with the classic Shannon and Weaver model which focuses in on a single channel from sender to receiver. They cite Schober and Clark who referred to this conception as an autonomous view of conversation in which the listener passively receives information delivered from the speaker. In contrast to this view, Bavelas et al. note other conceptions and research which focus on dialogue as a joint activity, including Yngve’s focus on a reciprocal effect of a backchannel which recognizes the impact of listener responses. In this view, communication is not just for information transmission, but also for co-construction of the message wherein dialogue is considered collaborative and evolves from the reciprocal influence between narrators and listeners (Bavelas et al). Generic and specific listener responses within in the reciprocal dialogue was the focus of research for Bavelas et al., (2000) in which they studied what listeners do (backchannel) during narration to form an integrated message with the narrator. Table 1 compares the generic responses to specific responses that were observed. Table 1. Generic versus Specific Listener Response. (Bavelas et al., 2000) Generic Responses Listening Made to or at the story or narrator Generally related to narrative External to the narrative Respond to the narrative Communicate general understanding Indicate understanding of the Specific Responses Co-telling Made with the story or narrator Specific to the narrative Internal to the narrative Add to the narrative Communicate specific understanding Indicate understanding of the words implications of the words As represented within Table 1, generic listener responses (discussed in the context of project markers below), do not convey narrative content, but allow the speaker to track the listener’s comprehension. In contrast, specific listener responses closely relate to the speaker’s Backchannel Interactions content and allow the listeners to become co-narrators who add to the narrative as they 16 communicate comments regarding their understanding. If learners were encouraged to contribute responses of confusion or understanding in the backchannel, could their responses help to conarrate the message being delivered by allowing the presenter to tailor the message to the audience, as is suggested by this research? Task Engagement Given the lack of research on backchannel interactions, it is unclear whether learners chatting in the backchannel are engaged in the presentation at hand. Do their comments reflect that they are receiving the intended message or that they are heading off in another direction? Zimbardo (as cited by Coleman, Paternite, & Sherman, 1999) suggests that due to factors such as increased anonymity, a sense of altered responsibility, and novel or unstructured situations, participants in synchronous CMC tend to become more engaged in the task at hand and less concerned with self-monitoring. Coleman et al. report similar findings from their research in which some (but not all) participants in synchronous CMC were more group focused, more selfdisclosing, and reported feeling that the physical separation provided a freedom from distraction! Do these findings suggest the text based backchannel may lead to greater task engagement? Intrinsic Cognitive Load The focus of this section is to assess whether the dialogue and interaction within backchannel interactions could help to manage intrinsic cognitive load. Research suggests intrinsic cognitive load can be more effectively managed if content is presented in segments (Mayer & Moreno, 2003). Further, presenting declarative information separately, either during presentation or during practice, from procedural information has been shown to increases the effectiveness and efficiency of learning (Kester et al., 2006). Research findings also indicate that Backchannel Interactions 17 content sequencing should be based on the learners’ level of expertise and that the preplanning of content sequencing becomes less important if the sequencing can be continuously adapted during the instructional presentation based upon observation of the learners’ expertise (van Merrienboer & Sweller, 2005). Therefore, could cues from the learners’ dialogue in the backchannel help the presenter to segment and sequence the presentation of content based on the learners’ responses of either understanding or confusion thereby helping to manage intrinsic cognitive load? Also, could the backchannel be used to segment presentation and practice opportunities though the use of formal problem statements, learner responses, and feedback? Based on dialogue analysis research it seems feasible that the backchannel can provide presenters with signals or markers from the learner to gauge their level of understanding which would allow an adjustment to the presentation based on the cues from the learners. Tied to the research noted above, research on self and other monitoring during dialogue suggests that speakers monitor their own speech and adjust their presentation based on their assessment of the listener’s level of understanding (Clark & Krych, 2004). As such, dialogue includes two activities, including support for the primary presentation of information and management of the dialogue itself. As described by Bangerter and Clark (2003), dialogue exists in both the front (or main) channel which includes the primary speaker and in the backchannel which includes the speech and signals from others occurring at same time as primary speaker’s turn. As discussed above, these backchannel responses, also known as project markers, play a role in shaping the presentation and include a) acknowledgement tokens in which the listener acknowledges the presentation through utterances, such as “uh-huh”, b) agreement tokens in which the listener Backchannel Interactions 18 agrees with the presenter’s position, such as “right”, and c) consent tokens in which the listener approves of the presenter’s comments, such as “okay”. Bangerter and Clark suggest that these project markers provide the primary speaker with marks to chart progress and signal to the presenter that the listener is ready to transition with the presentation. For example, the listener can offer the speaker a) continuers, such as “yes”, which signal the listener is ready to hear more, b) assessments, such as reactions of “wow” or “gosh”, which signal comprehension and evaluation of what has been said, or c) recipiency markers which signal the listener wants to speak. If backchannel interactions are considered signals from the learner as listener, it is conceivable that the presenter could use the responses as project markers to gauge how to segment and sequence the presentation. By monitoring the learners’ backchannel conversations and by assessing when the learners are ready to make transitions within the presentation, the presenter may be able to use the backchannel interactions to manage intrinsic cognitive load. Summary While no research was found that specifically evaluates backchannel interactions in the computer-mediated classroom, findings in areas that share key features with this relatively new instructional phenomenon may shed light on the effects the backchannel has on cognitive load. In evaluating this prior research from synchronous text based discussions, laptop use during live face to face classes, CMC, and dialogue analysis, the findings seem to suggest support for both negative and positive effects on cognitive load. The potential for distraction, split attention, and redundancy effects may indicate backchannel interactions place unnecessary extraneous cognitive load on learners. However, findings may also suggest that the backchannel interactions directly facilitate learning through more effective and efficient processing of the to-be-learned Backchannel Interactions material. Further, the signals and cues within the dialogue may help presenters to more 19 effectively and efficiently sequence and transition within the presentation of content which may help to manage intrinsic cognitive load. These findings have implications for both instructional designers and researchers. Clearly, the new features of synchronous computer-mediated classrooms necessitate a closer review of these new types of interactions. The noted findings offer many stepping stones for future research and suggest a host of research questions. Do backchannel interactions distract the learner from the task at hand and interfere with their receipt of the instructional message? Could backchannel interactions be used as a means for the presenter to gauge the level of expertise of the learners during instructional presentation? Could the synchronous backchannel provide support for immediate meaningful reflection? Do backchannel interactions help to foster mutual understanding and co-creation of knowledge across the group? Does this mutual understanding ultimately lead to better individual performance? Do backchannel interactions offer learners more control over social distance and help to improve CMC effectiveness? Do backchannel responses help to co-narrate the message being delivered and allow the presenter to tailor the message to the audience? Do these findings suggest the text based backchannel may lead to greater task engagement? Answers to all of these questions may help us to one day address the primary question raised at the beginning of this report. Is the interactivity involved with backchannel interactions extraneous load within the learning environment, germane to the process of learning, or helpful in managing intrinsic cognitive load? As we gain more insight into these backchannel interactions, a new set of heuristics and online classroom norms (netiquette) will evolve. Teachers in traditional classes are already trying Backchannel Interactions 20 laptop up / laptop down procedures where the instructor asks for uninterrupted attention during presentation of material and then invite increased computer-mediated interaction and dialogue from learners during breaks in the formal presentation (Levine, as cited in Fried, 2008). In the same way learners in traditional classrooms know when it is time to speak in class and when it is time to listen, learners and instructors will one day know when it is appropriate to backchannel in class. Backchannel Interactions 21 Bangerter, A., & Clark, H. H. (2003). Navigating joint projects with dialogue. Cognitive Science, 27(2), 195. doi: 10.1016/S0364-0213(02)00118-0. Barak, M., Lipson, A., & Lerman, S. (2006). Wireless Laptops as Means For Promoting Active Learning In Large Lecture Halls. Journal of Research on Technology in Education, 38(3), 245-263. Bavelas, J. B., Coates, L., & Johnson, T. (2000). Listeners as co-narrators. Journal of Personality and Social Psychology, 79(6), 941-952. Clark, H. H., & Krych, M. A. (2004). Speaking while monitoring addressees for understanding. Journal of Memory and Language, 50(1), 62-81. Cogdill, S., Fanderclai, T., Kilborn, J., & Williams, M. (2001). Backchannel: whispering in digital conversation. In System Sciences, 2001. Proceedings of the 34th Annual Hawaii International Conference on System Sciences. Coleman, L. H., Paternite, C. E., & Sherman, R. C. (1999). A reexamination of deindividuation in synchronous computer-mediated communication. Computers in Human Behavior, 15, 51-65. DeSanctis, G., & Monge, P. (1998). Communication Processes for Virtual Organizations. Journal of Computer-Mediated Communication, 3(4), 0-0. Fried, C. B. (2008). In-Class Laptop Use and Its Effects on Student Learning. Computers & Education, 50(3), 906. Guess, A. (2008, April 18). Hey, You! Pay Attention! :: Inside Higher Ed: Higher Education's Source for News, Views and Jobs. Inside Higher Ed. Retrieved June 10, 2008, from http://insidehighered.com/news/2008/04/18/laptops. Backchannel Interactions Havard, B., Jianxia Du, & Olinzock, A. (2005). DEEP LEARNING The Knowledge, Methods, 22 and Cognition Process in Instructor-led Online Discussion. Quarterly Review of Distance Education, 6(2), 125-135. Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). The Expertise Reversal Effect. Educational Psychologist, 38(1), 23-31. Kester, L.; Kirschner, P. A.; van Merrienboer, J. J. G. (2006). Just-in-Time Information Presentation: Improving Learning a Troubleshooting Skill. Contemporary Educational Psychology, 31 (2), 167-185. Lee, H., Plass, J., & Homer, B. (2006). Optimizing Cognitive Load for Learning from ComputerBased Science Simulations. Journal of Educational Psychology, 98(4), 902-913. Marshall, C. R., & Novick, D. G. (1995). Conversational effectiveness in multimedia communications. Information Technology & People, 8(1), 54 - 79. doi: 10.1108/09593849510081602. Mayer, R. E., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38, 43–52. Moore, D. M., Burton, J. K., & Myers, R. J. (2004). Multiple channel communication: The theoretical and research foundations of multimedia. In D. Jonassen (Ed.), Handbook of Research on Educational Communications and Technology, 2nd Ed. Chapter 36, pp. 9791005. Moreno, R., & Mayer, R. (2007). Interactive multimodal learning environments. EDUCATIONAL PSYCHOLOGY REVIEW, 19(3), 309-326. Paulus, T. M. (2007). CMC Modes for Learning Tasks at a Distance. Journal of ComputerMediated Communication, 12(4), 1322-1345. doi: 10.1111/j.1083-6101.2007.00375.x. Backchannel Interactions Pelowski, S., Frissell, L., Cabral, K., & Yu, T. (2005). So Far But Yet So Close: Student Chat Room Immediacy, Learning, and Performance in an Online Course. Journal of Interactive Learning Research, 16(4), 395-407. 23 Pociask, F. D., Morrison, G. (2004). The Effects of Split-Attention and Redundancy on Cognitive Load When Learning Cognitive and Psychomotor Tasks. Association for Educational Communications and Technology. Retrieved June 17, 2008, from http://www.aect.org. Stein, D. S., Wanstreet, C. E., Glazer, H. R., Engle, C. L., Harris, R. A., Johnston, S. M., et al. (2007). Creating shared understanding through chats in a community of inquiry. The Internet and Higher Education, 10(2), 103-115. Sweller, J. & Chandler, P. (1994). Why some material is difficult to learn. Cognition and Instruction, 12(30, 184-233. van Merrienboer, J., & Sweller, J. (2005). Cognitive Load Theory and Complex Learning: Recent Developments and Future Directions. Educational Psychology Review, 17(2), 147. Winn, W. (2004). Cognitive perspectives in psychology. In D. Jonassen (Ed.), Handbook of Research on Educational Communications and Technology, 2nd Ed. Chapter 4, pp. 179112. Yardi, S. (2006). The role of the backchannel in collaborative learning environments. In Proceedings of the 7th international conference on Learning sciences (pp. 852-858). Bloomington, Indiana: International Society of the Learning Sciences. Retrieved June 11, 2008, from http://crlt.indiana.edu/iclsvideo/index.html and http://dream.sims.berkeley.edu/groups/classchat/papers/SaritaYardi_ISLS2....
Cognitive Task Analysis Running head: COGNITIVE TASK ANALYSIS 1 Cognitive Task Analysis Jennifer Maddrell Old Dominion University IDT 873 Advanced Instructional Design Techniques Dr. Gary Morrison October 15, 2008 Cognitive Task Analysis Traditional Task Analysis A traditional procedural task analysis describes a task as a series of discrete actions (Jonassen, Tessmer, & Hannum, 1999). Figure 1 diagrams a procedural task analysis for the insurance underwriting submission review task. Within this triage task, the underwriter must evaluate various aspects of the new submission and decide whether to quote or decline the submission. Figure 1. Procedural Analysis of Insurance Underwriting Submission Review Task. 2 Guide to symbols: = Input and exit point; = Mental operation; = Decision Point; = Direction in Step Cognitive Task Analysis 3 As depicted in Figure 1, in completing the submission review task, the underwriter must make a series of mental operations and decisions in route to a conclusion to either a) decline the submission or b) quote the submission. These mental operations and subsequent decisions include the following: Assessing the viability of the opportunity. Upon receipt of the submission, the underwriter must make a quick review of the information provided to assess the viability of the opportunity. Given the information presented within the submission and discussions with the broker, the underwriter must judge the likelihood the account will actually leave the incumbent carrier. Critical cues to consider include prior service and claims handling problems with the incumbent carrier, time to transition the account, and completeness of the submission. If the relationship with the prior carrier has been good, there is little time to transition the account, or the broker only provided enough information to provide a price (not service) quote, it is likely the insured is not serious about moving from the incumbent carrier and the broker is just seeking comparative price quotes. However, if the insured is dissatisfied with the incumbent carrier’s service, there is ample time to transition the servicing of the account, or the submission provides a comprehensive overview of both price and service requirements, it is likely the opportunity is viable. If the assessment of the information leads to a conclusion that the chances are slim the account will move, the underwriter makes the decision to decline the account. However, if the assessments leads to a conclusion that there is a good chance of writing the account, the underwriter makes the decision to continue working on the account. Examining the employee concentrations. Given the potentially catastrophic exposure of providing casualty insurance at locations with high employee concentrations, the underwriter’s triage of the submission includes an examination of employee concentrations. If the insured has employee concentrations at any one location above company guidelines, the underwriter makes the decision to decline the account. Otherwise, the underwriter makes the decision to continue working on the account. Comparing the account’s exposures to the company’s underwriting guidelines. Upon receipt of the submission, the underwriter must compare the prospective account’s exposures to the insurance company’s underwriting guidelines. Critical to this comparison is a review of the insured’s current and prior operations. If the company is involved in any operations which result in exposures that are against the underwriting guidelines, the underwriter makes the decision to decline the account. Otherwise, the underwriter makes the decision to move forward with the quotation task (beyond the scope of this submission triage task analysis). Cognitive Task Analysis A cognitive task analysis (CTA) offers an alternative means of describing the cognitive elements of the evaluation and decision making processes involved in the task. The following provides the results of an Applied Cognitive Task Analysis (ACTA) based on interviews conducted with an underwriting subject matter expert (SME) to gain information about cognitive strategies used to complete the submission triage task (Militello & Hutton, 1998). The ACTA includes a task diagram, knowledge audit table, simulation interview, and cognitive demands table. Cognitive Task Analysis 4 Task diagram Figure 2 is the task diagram generated after an initial interview with the underwriting SME. The task diagram offers a high level overview of the submission triage task which focuses on the most difficult cognitive aspects. The SME was asked, “Think about what you do when you triage a new prospect. Can you break this task down into less than six, but more than three steps?” The SME mentioned five steps, but one was eliminated (financial approval) as it is not task performed by underwriter. Figure 2. Task Diagram for New Account Prospect Triage. Knowledge Audit Table During interviews with the SME, the interviewer probed for concrete examples, cues and strategies, and reasons why the task is often difficult for novices. The interviewer asked the SME to focus on specific examples for each aspect of expertise. Table 1 summarizes the results of the knowledge audit for the submission triage task. Simulation Interview During a simulation interview with the SME, the interviewer asked the SME to focus on the challenging aspects of a specific representative scenario associated with new submission triage. Table 2 summarizes the results of the simulation interview, including the actions, assessments, cues, and potential errors identified for each central event. Cognitive Demands Table Table 3 consolidates and synthesizes the data collected during the interview process. The cognitive demands table centers on the common themes that came from the interviews and identifies the difficult cognitive elements, common errors, and cues or strategies used by experts to overcome these challenges. Cognitive Task Analysis Table 1. Knowledge Audit Table. Aspect of expertise Past and future Example: Call from broker about account where incumbent carrier messed up on claim and insured’s legal department insisting the account must move. 5 • • Cues and strategies High level nature of incumbent mess up Level of people involved in decision (low level versus high level) • • Why Difficult? Novice may not recognize significance of messed up claim handling Novice may not link level of insured to severity of problem Novice may not link severity of problem to increased chance of writing account. Novices may not consider other issues beyond price that influence buying decision Novices do not have relationship with broker to know when you are getting the “straight” facts versus a “sales pitch” Novices may get into the minutia of the account specifics and not step back and realize the timeframe is not feasible to actually move the account Novices are focused on details within submission Novices are familiar with “outside” considerations that affect the likelihood of writing the account • • • • Big picture Example: Steps back from all the facts about the account presented by the broker to consider what is the “real” motivation behind looking for a quote? Is this prospect a true opportunity or does the broker just need a competing price quote? If it is only a need to get competing price quotes, highly unlikely the account will move. Noticing Example: Broker not soliciting TPA quotes for claim handling which would be a #1 condition of actually moving the account. Job Smarts Example: Focus on what broker said in conversation versus purely what is presented in the quote. Opportunities Example: Our unit can’t work on this account, but other units in company can. Anomalies Example: Broker doesn’t return phone calls. Shows a lack of interest. • • • • Beyond price, there service issues with prior carrier Your personal history with that broker. Time frame to release quote What other carriers are quoting • • • • • Going beyond underwriting information presented in the submission Considering conditional things that impact your quote Timeframes Others carriers being asked to quote. Reasons for leaving Understanding of underwriting appetite of other units Knowing how to access those people Timing of returned phone calls Extent of response to questions • • • • • • • • • Novices tend to be preoccupied with verifying details within submission Novices not aware of situational issues that can be “deal breakers” or “deal makers” Novices don’t know underwriting appetite of other units Novices don’t know people outside of the unit Novices may not recognize they are “getting blown off” and they continue working on submission • • Cognitive Task Analysis (either lacking or detailed) 6 • Novices don’t recognize significance of “out of sight / out of mind” which is signal if you are alive or dead Table 2. Simulation Interview. Events Discussion about prospect with broker Actions Ask probing questions about opportunity Sensing tone from broker of urgency and desire to have you quote. Assessment Answers to question make sense or not with what is in the submission Broker wants to work with you or just wants a quote for comparison purposes How much time is there between now and effective date? Are the exposures inherent in risk acceptable under our underwriting guidelines? Critical Cues Can you meet the issued There is disaffection with incumbent Openness of the broker Willingness to provide additional information Too much time signals the broker is “shopping” for an early quote. Too little time signals that broker just wants to keep current carrier “honest” “Red flag” exposures that we cannot write “Go” classes of business that we are targeting Potential Errors Being overly optimistic about any opportunity Not probing deeply for hidden facts about situation Not reading the verbal and nonverbal cues the broker is giving you. • • • • • • • • • • • Deciding whether to quote • • Evaluating time frame between quote deadline and effective date Assessing if account meets underwriting guidelines • • • • • • • • • Being so excited about the opportunity that you rush to judgment Spin wheels on accounts where there isn’t a true opportunity Don’t dig deeply enough into what the account really does or did in the past that could represent “hidden” exposures Cognitive Task Analysis 7 Cognitive Task Analysis 8 Table 3. Cognitive Demands Table. Difficult cognitive elements Assessing whether broker’s answers make sense or not with what is in the submission Considering the “real” opportunity and exposures beyond the obvious information given in the submission Comparing account’s exposure information with underwriting guidelines Why difficult Common errors Cues and strategies used • • Consider if you really know the story behind the story Get and keep the broker talking to elicit information beyond the submission Ask about reasons why account would move Consider whether timeframe to move account is realistic • • • • • • • Considering and suggesting alternatives • • Novice underwriters tend to focus on basic facts in the submission versus what the broker is telling them Brokers reluctant to voluntarily air dirty laundry about account Novices underwriters tend to focus on information given versus information needed to make decision Can be uncomfortable situation for novice underwriters to probe for answers Companies often have many types of operations which cross several classes of business Novice underwriters tend to focus on the primary business operations Novice underwriters often have difficult assigning an account to the appropriate business classification within the guidelines. Novice underwriters tend to focus on what broker is asking you to do Novice underwriters often fail to identify ways to adjust quotation options to meet guidelines • • Don’t recognize or probe for hidden “red flags” Focus exclusively on information in submission Taking the submission at “face value” Failing to engage in uncomfortable probing conversations with the broker Failing to fully capture exposures Getting lost in the details Misinterpreting underwriting data Misinterpreting the underwriting guidelines • • • • • • • • • Review account with senior underwriter Check multiple sources to evaluate exposures • • • Quote only what is asked by broker Failing to probe for alternate opportunities with the broker • • Consider ways to adjust quotation options to fit within underwriting guidelines. Consider other coverages and limits that you or other departments could quote Cognitive Task Analysis Comparison of Approaches 9 Analysis Comparison In comparing the results of the traditional task analysis with the cognitive task analysis, significant differences emerge in following areas: a) the identification and analysis of hidden cognitive processes, b) the relative level of elaboration regarding the central task elements, c) the focus on expert and novice differences. Overt behaviors versus cognitive processes. The key strength of the traditional task analysis is the ability to examine overt behaviors required to complete a task. However, as seen in this example, additional critical cognitive processes and actions were uncovered within the ACTA. Further, the ACTA offered a means of analyzing the relative significance and difficulty of the required task elements. Level of elaboration. The traditional task analysis identified the relevant processes and decision points in the submission triage task. However, by focusing on the difficult cognitive aspects of the task, the ACTA provided greater elaboration with regard to the knowledge and cognitive processes required to perform the task. As the cognitive demands table highlights, the ACTA focused attention on the difficult cognitive elements, common errors, and strategies to overcome those difficulties and errors. Unfortunately, these elements were not unearthed within the traditional task analysis. Focus on expert and novice differences. Unlike the traditional task analysis, the ACTA analysis focused on the central differences between how an expert and a novice perform the submission triage task. The result is a comparison of current state (novices) and desired state (experts), as well as strategies to take the novice to an expert level. Implications for Practice Traditional task analysis allows practitioners to target the inputs, central operations, and decision points involved in carrying out a task. While this provides a good overview of what happens as the task is carried out, it does not provide the designer with an understanding of the nature of the cognitive processes required to complete the task. Further, following a traditional task analysis, the practitioner cannot gage the relative importance of the various tasks elements or which aspect(s) of the task are harder for the novice. As seen in the results between the two analyses, the cognitive task analysis provides practitioners with a better understanding of the difficult and critical cognitive processes, as well as the and cues and strategies, which are central to successful completion of the task. When to use Traditional Task Analysis versus Cognitive Task Analysis Both a traditional task analysis and cognitive task analysis highlight key aspects of the task. However, as seen in the two analyses above, each produces different results. As noted, the cognitive task analysis offers a better analysis of the central knowledge and decision making cognitive processes. Given that each task is different, the following provides a comparison of which analysis is more appropriate based on the degree of observable behaviors, the degree of required expertise, and the relative cognitive difficulty of the task. Cognitive Task Analysis 10 Degree of observable behaviors. The difference in outcomes between the two approaches is likely less significant when the task involves primarily observable behaviors. However, if the task involves primarily mental actions that result in less observable behaviors, a cognitive task analysis is the more appropriate option. Expert versus novice differences. When little task related expertise is required to perform the task, the results of both analyses would likely be similar. However, if successful completion of the task requires knowledge that a novice would not possess, a cognitive task analysis allows the practitioner to uncover or drill down on the difficult cognitive elements. As noted, these cognitive elements are less likely to be adequately analyzed in a traditional task analysis. Relative cognitive difficulty. While a traditional task analysis provides a comprehensive outline of the steps in the task, it does not offer a relative assessment of which steps are harder or more critical to successful completion. Instead, each step in the task is considered equally. However, as seen in the cognitive demands table, some tasks hinge on a smaller number of critical or difficult elements. Therefore, the ACTA is more appropriate when successful task outcomes depend upon cognitively difficult judgments or decision. Cognitive Task Analysis 11 References Jonassen, D. H., Tessmer, M., & Hannum, W. H. (1999). Task analysis methods for instructional design. Mahwah, N.J.: L. Erlbaum Associates. Militello, L. G., & Hutton, R. J. B. (1998). Applied cognitive task analysis (ACTA): a practitioner’s toolkit for understanding cognitive task demands. Ergonomics, 41(11), 1618-1641.
Critique of OECD Innovation in the Knowledge Economy
The following is a critique of the 2004 publication Innovation in the knowledge economy: implications for education and learning from the Organisation for Economic Co-operation and Development (OECD). The following assesses the strengths and weaknesses of the publication and concludes with a proposed outline for a similar report for instructional designers.
Design Critique Running head: Personal Critique 1 Personal Critique Jennifer Maddrell Old Dominion University IDT 895: Knowledge Management June 1, 2008 Design Critique Innovation in the Knowledge Economy The following is a critique of the 2004 publication Innovation in the knowledge economy: implications for education and learning from the Organisation for Economic Co- 2 operation and Development (OECD). The following assesses the strengths and weaknesses of the publication and concludes with a proposed outline for a similar report for instructional designers. Strengths of Report Assessment of knowledge-based communities. A key strength of the report is the detailed assessment of new forms of knowledge-based communities in sectors outside of education. The report highlights the possibility for innovation that is created when unrestricted access and a free flow of information exists within knowledge-based communities. The overriding premise is that through open access to people, technologies, and information, new and exciting avenues for knowledge generation, innovation, and sharing are possible. The report provides an effective argument that these interactions and connections are far different from knowledge sharing and collaborations of the past. Through the use of open information and communication technologies, there has been a democratization of the knowledge generation and sharing process. All users, not just “experts”, are able to participate, share, and debate issues. In addition, these same information and communication technologies offer the ability to rapidly and inexpensively codify and transmit information which further energizes innovation. No longer must participants rely on formal networks and commercial publishers to produce and disseminate information. The report also highlights an important characteristic of these new forms of knowledgebased communities which has been shown to rapidly stimulate innovation and foster a sense of community around a topic. Participation is based on what is described as “general reciprocity Design Critique obligations”. An assumed condition of membership is the sharing of knowledge. In other words, the cost of participation is participation! These assumed reciprocal obligations fuel the network and, in turn, discourage lurkers (takers, but not contributors). Coverage of drivers of innovation. The report also effectively highlights the drivers that are likely to propel innovation in education. At the forefront are advances in information and communication technologies which allow ever increasing access to information and people. Through examples in other sectors, the report effectively demonstrates how technology is helping to overcome physical, social, and cultural barriers to reach previously isolated or excluded participants. With expanded access, more individual can connect and contribute. In addition, the report effectively argues that the ability to “learn by doing” spurs innovation. No longer do users need to wait for producers to create and deliver a product. With greater access to tools, information, people, and resources, users are empowered to innovate to solve their own, but likely shared, problems. In turn, the innovations they create can be contributed back to the community to continue the cycle of “general reciprocity obligations”, described above. Coverage of barriers to innovation. The report also highlights important barriers that must be overcome. At the forefront is unequal access. While there has been a noted democratization of the knowledge generation and sharing process, only a very small percentage of the world’s population is able to participate due to limited access to the information and communication technologies highlighted in the report. In addition, the rapid innovation described in the report has occurred only in pockets. While examples of computer technology innovation abound, many sectors operate in much the same way they have for generations. Therefore, not all sectors will be as willing or able to 3 Design Critique embrace change. The field of education is described as one such sector which tends to rely on tradition and generally resists change. The report effectively argues that educational institutions 4 and practices that exist today are not significantly different than they were hundreds of years ago and that many prevailing “best practices” have been developed and passed down from one practitioner to the next. While the report may be criticized for discrediting the vast body of existing educational research (discussed below) and the experience of practitioners, the report effectively describes the struggle within the field of education between what is termed “scientific” and “humanistic” approaches. Coming to terms with this issue will be central to future education reform. This is particularly the case within the United States where No Child Left Behind proponents, who favor educational standards based on what the report terms “scientific based research”, are facing off with opponents who favor empowering practitioners to rely on time tested “best practices” based on their judgment and experience. While the report clearly advocates a “scientific” approach, it is likely that the debate between scientific and humanistic approaches will rage on for some time to come. The report also addresses the fact that effective knowledge-based networks require a high degree of open access to information and people. Copyright to protect intellectual property rights or access fees in the form of tuition, membership fees, or journal subscriptions can all be barriers to this free flow of open access to information and people. However, there is no easy solution to remove these barriers. While revenue generation can be a barrier to access, it is also an incentive for producers to innovate. Likewise, while copyright limits access, it also affords important protections to producers who have invested their time and recourses into creating the material. Design Critique Weaknesses of Report As noted, the report provides a very compelling vision for the future; one in which expanded access, free flows of information, and knowledge-based networks propel innovation. However, in tackling the very complex topic of generating and sharing knowledge and innovation in the field of education, the report leaves many unanswered questions. Why the disregard for previous educational research and practice? A clear message in the report is that the field of education lacks a sufficient body of existing scientific research to 5 propel innovation in education and learning. Is this a valid assertion? It is likely that many of the thousands of members of the American Educational Research Association and other professional research organizations would welcome the opportunity to pose an argument against this position. While there are many avenues of new research to pursue and the field of education struggles to transfer prior research into effective practices, it seems unwarranted to disregard (or at best to discredit) decades of existing educational and instructional research and practice. How will knowledge-based networks support innovation in education? The report focused heavily on the transformative power of knowledge-based networks. Yet, is unclear how the authors propose knowledge-based networks should support the field of education. Is the intent to bring learners into the networks (as a means of providing instruction and education) or is the intent to use knowledge-based networks to support the knowledge generation and sharing among “thought leaders” as a means of driving innovative practices in education? If the intent is to use knowledge-based networks to support educators within a globally connected professional development community, then many of the examples in the book are applicable, including using the open forms of connection to generate and share new innovative practices. However, if the Design Critique 6 intent is to use knowledge-based networks as means of providing and delivering education, then there are numerous barriers to consider that are not addressed in this report, as discussed below. What about other factors and barriers in the education sector? As noted, the report effectively summarizes examples of innovation in other sectors and attempts to argue that these same opportunities and practices will be drivers for education. However, the education sector faces many barriers that were either not addressed within the report or only given passing mention. Yet, these barriers make transfer of many of the innovation drivers very difficult for the field of education. Most importantly, education in many countries is compulsory. Therefore, access cannot be limited to pockets of a fortunate few. A condition of educational delivery in most countries is that access must be universal. As such, knowledge generation and sharing strategies which can only be supported by advanced forms of information and communication technologies are not feasible for the vast majority of the world’s population, including teachers and learners in many developed countries. What are the suggestions to reach this “unconnected” majority? In addition, beyond access, education must provide guidance and context around information through effective instructional practices for all learners; not just some learners or those intrinsically motivated to learn-by-doing. Are the hypothetical participants in the cited knowledge-based communities assumed to be high achieving early adopters? If so, what is the prescription to reach and engage teachers and learners of all levels of ability and motivation? In this new world of open access, who will be responsible for providing the required universal access, instructional guidance and context, assessment of performance, and credit for the experience? Formal educational institutions? Informal knowledge networks? Who will pay for this access? Individual learners? Governments? Private foundations? While the report Design Critique effectively argues that much of the innovation in other sectors has come from free and open informal knowledge networks, it is far less clear how free and open informal networks can provide the necessary universal access, context, assessment, and credit for the experience. Is the take-away message that existing formal educational institutions should become more agile and nimble like informal knowledge networks? If so, the report falls short of describing how that major miracle can happen. Does access lead to knowledge generation or transfer? The report provides numerous examples of how information and communication technology can expand access to people and 7 information. Unfortunately, the report leaves to future research any prescriptions for instructional practices and processes to facilitate learning. As such, the focus of the report becomes the transformative power of the delivery medium. Most instructional designers would argue that content and interaction are important, but not sufficient conditions for instruction. Effective presentation, practice, and guidance strategies are also needed to facilitate learning. While the writers of this report forcefully stressed the need for research into effective instructional practices, many well intentioned open access programs, such as the One Laptop per Child project and numerous open educational resource (OER) projects, are founded on an “if you build it / share it, they will come” premise. By the report’s focus on the transformative power of the delivery medium, readers may be left with the notion that free flows of information and access to communication technologies will naturally lead to learning. However, is offering learners access to an information technology and an open educational resource significantly different than offering learners a bus pass and a library card? While both options would likely benefit the learners, few would consider either to be instruction or models for the future of education. Design Critique Innovation through Effective Instruction: From Information Transfer to Knowledge Generation and Management In the section that follows, an outline is provided which proposes instructional design practices to support and foster innovation. The presented instructional design considerations go beyond basic information transfer to practices which foster effective knowledge generation and management. Move from Problems to be Corrected to Opportunity Identification Problem to be corrected. Most instructional design plans begin with an identification of instructional problems. The identified problems form the reason for undertaking instruction. Unfortunately, when viewed as problem correction, instruction becomes an intervention to correct deficiencies. Opportunity identification. In contrast, instruction conceived of as opportunity identification focuses on answering three important questions which are central to any planning activity, including: 1)”Where we are today?” 2) “Where we want to be in the future?” and 3) “What do we need to do to get there?” While there is a subtle difference between the identification of problems to be corrected and opportunity identification, the impact on the instructional objectives can be profound. The focus of the instructional objectives and, in turn, 8 the design plan shifts from making the learner or organization whole today (by filling in existing skill and knowledge gaps) to creating learning experiences tied directly to long term goals for the future. Leverage Learner Experiences and Knowledge through Co-creation of Knowledge Build upon what learners know. Learner analysis is one of the primary steps in most instructional design plans. By ensuring that the learner analysis includes the learners’ entry Design Critique 9 competencies, their level of expertise, and their background knowledge an instructional designer is able to create and facilitate instructional strategies which are authentic and build upon the existing knowledge of the learners. In turn, learners will be more engaged in the instruction and contribute to the knowledge creation and sharing process. Ground in authentic tasks. Tied to above, it is important to consider the context in which the learners will use the to-be-learned knowledge. The design plan should incorporate instructional strategies which are grounded in authentic tasks and situations that the learner faces now and will face in the future. By doing so, it is more likely that the learners will integrate the instruction and contribute to future innovation after the instructional event is over. Re-think Traditional Instructional Roles Instructors as facilitators versus transmitters. The instructor role is often conceived of as a transmitter of information. However, in order to foster knowledge generation among learner participants, it is necessary to re-think the traditional role of the instructor from transmitter of information to facilitator of knowledge co-creation. Learners as Co-creators. In order to foster knowledge generation among participants, it is important to go beyond a notion of learners as sponges who absorb information transmitted by the instructor. Instead, learners must take on active roles as co-creators of knowledge. As noted, learners have experiences to build upon which can not only add to given instructional sessions, but also to knowledge generation and sharing beyond the classroom. Re-assess Instructional Time Horizon The instructional event. When instruction or training is considered as an intervention to fix or address a specific problem or need, instructional design plans tend to be limited to the Design Critique instructional event. However, it is shortsighted for a designer to not contemplate knowledge creation and sharing opportunities after the instructional event is over. Beyond the instructional event. An overriding goal of instruction should be integration. The benefit of gaining knowledge is the ability to use it. If learners do not integrate the newly formed knowledge into their lives, there is not much benefit to be derived from conducting the 10 instruction in the first place. Therefore, instruction should contemplate what the learners will do with the instruction and attempt to extend the knowledge generation and sharing beyond the instructional event. One way to accomplish this is to effectively support the learners’ integration into formal and information networks, as discussed below. Support Knowledge Networks Formal Networks. Formal networks are formed to support a shared goal or vision. Instruction is typically undertaken to support the needs of formal networks; to train employees or educate members to be more productive within the greater formal network. Therefore, it is a traditional practice within instructional design to fully consider the needs, characteristics, and context of the formal network. Who are the players? What are the roles? How will the instruction support the knowledge generation and management needs of the formal network? Informal Networks. Beyond support of formal networks, instruction should also recognize and foster knowledge creation and sharing within informal networks. In contrast to formal networks, informal networks are formed by individuals who hold common, but not necessarily shared, needs and goals. Informal networks tend to be loosely joined, but are often very influential and powerful innovators. While informal networks are often hard to identify, it is important to consider ways to contemplate existing and potential informal networks within the design of instruction and to help the network members recognize their common needs and goals. Design Critique Open Lines of Communication 11 For any formal or informal knowledge network to function, the members must be able to communicate. An important step in the instructional design process is ensuring all members of the learning community have access to communication channels and are versed in the shared language. However, beyond access and knowledge of important terminology, it is important for learners to know the people who hold expertise in key areas and how to connect with them. Address Barriers to Knowledge Generation and Sharing Barriers to Accessing People. As discussed above, participation among individuals is central to the notion of knowledge co-creation. However, barriers among participants can impede the knowledge generation and sharing among participants. The barriers may be intentional or unintentional. Some participants may not want to engage with others, due a lack of incentive or to conceal information and knowledge, while others may simply not be aware of the existence of others with a particular expertise. Instructional designers can help learners form connections and, as noted above, establish a shared language and platform from which to communicate. Barriers to Accessing Information. Information is fuel for a knowledge network. Barriers to a free flow of information can mean that participants will not be able to leverage the information during knowledge generation. However, the free flow of information can be impeded, again either intentionally or unintentionally. Unintentional barriers are often created when someone holds information without knowing others would benefit from it or when there is too much information to sift through (the “firehouse” effect), validate, or synthesize. In contrast, intentional barriers are created when individuals or groups attempt to protect the value of the information they possess through copyright or other access fees. It is complicated balance Design Critique between protecting intellectual property and trade secrets while also ensuring that participants are benefiting from a free flow of information. Instructional designers can help learners overcome these barriers by identifying or providing vetted information sources. Barriers to Accessing Tools. Tools are the medium with which information and knowledge are codified and disseminated. Without access to and knowledge of how to use the 12 tools, a barrier exists. Again, this barrier can be intentional, such as in the form of access fees or patents, or unintentional, such as in a lack of awareness that the tool exists or knowledge of how to use the tool. Instructional designers should contemplate the tools available to learners and ensure that learners have the necessary access and skills to use the tools. Design Critique References 13 Centre for Educational Research and Innovation, & Organisation for Economic Co-operation and Development. (2004). Innovation in the knowledge economy: implications for education and learning. Knowledge management. Paris, France: OECD.
Drupal Presentation from the Indiana University IST Conference
Attached is the presentation material from the Drupal workshop I hosted today during the 2007 Indiana University IST Conference. Please leave any comments below or use the contact form (on the menu bar above) if you have any questions ... or ... better, yet ... join us on Monday nights at 9:00 p.m. ET for live web casts at the Drupal CMS Academy!
Here is a recap of a Framegame (by Thiagi) that I conducted toward the end of last semester. The book provides great insights into facilitating an educational game ... I amended the core approaches to work within an online setting.
Audience: The audience for the session included a group of educators who meet
online Thursday nights to discuss and share ideas about various topics in the
field of education during an informal peer learning session. On the night of
the Framegame, 7 participants played in the game and a few others lurked in the
virtual "corner". While many of the weekly attendees are k-12 teachers, some
are educational technology specialists in college or corporate settings. Nearly
all of the participants are using Internet based technologies to support
learning activities. A growing concern is how to foster appropriate participant
interactions within the online learning environment.
Topic - Tactics to promote positive online
interactions: While most
would agree that the goal is to create and support positive interactions (free
of "flame wars" or "cyber bullying"), many are unsure how to go about
it. Is it a matter of just setting strict online codes of conduct? Is it about
modeling good behavior? Is it about discussion moderation and tossing out the
bad apples?
Framegame - Take Five: I selected the Take Five Framegame to
generate possible tactics which educators could use to create and support
positive interactions in their online environments.
Online Communication: This group is made up of technology geeks
who love new online learning challenges, so I was not too concerned about
running this online versus f2f. While I ran into a few glitches, the following
online communications tools provided and effective and efficient means of facilitating
the game:
Skype:http://skype.com was used to facilitate
the voice discussions.
Whiteboard:http://www.skrbl.com/
- Skrble is a free VERY user friendly web based whiteboard used to document the
brainstorming topics and to facilitate the "voting" (see below). See actual skrble from the game here http://www.skrbl.com/badgerjen01Note:
if you scribble, you will be amending the document - so skrble with care J
Audio Recording:http://edtechtalk.com/
The audio from the entire session was recorded which will soon be uploaded and stored
at the EdTechTalk web site.
Game Play: The following outlines the planned script (talking points and actions)
for the session. The "Notes" sections
in italics below highlight deviations from either the basic Framegame or the
script.
Brief players: The topic for this brainstorm is tactics to
use as educators to foster positive interactions within online learning
environments. In the next half hour, we will explore this topic by coming up
with a list of effective tactics to take back to the classroom to both foster
appropriate participant interactions and address inappropriate conduct should
it arise.
Individual brainstorm: For the next few minutes, please reflect on
the topic and come up with a short list of effective techniques that you would
use to foster appropriate and positive participant interactions in an online
learning environment. Try to keep your list of solutions short and to the point
as we will be discussing them in more detail in the next step.
[Note: Given 7 attendees joined the session,
I ran this version of Take Five as an individual version (per the suggestion on
p. 261 in Framegames). However, if I had been faced with a group of 10 or more,
my plan was to ask for volunteers to pair up and use the private text chat
function to communicate with a partner.]
Group Brainstorm on Whiteboard: Let us now move over to the whiteboard and
begin forming a combined list of possible techniques. Using the text tool, take
turns typing in your list of suggested techniques in the left hand column. Try
to keep an eye on the list to avoid adding an item already offered by another
player. Duplicate entries will be combined before we move to the next step.
Let's try to get a combined list of 10 to 12 good possible tactics on the whiteboard.
Also, please speak up if you would like clarification about a possible
solution.
[Note: This process prompted participants to
share lengthy descriptions about each of the tactics ("Here is what I do with
my students ..." , which ate into the time previously allocated for the final
debrief. However, it didn't seem to make any sense to stifle the brainstorming
conversation (the point of the game) just to stick to the game script.]
Voting - Round 1: We
will now begin the process of voting for the most effective approach. Using the
pen tool, select the blue color and place a check mark next to the recommendation
that you feel is the most effective tactic.
Scoring - Round 1: Now that we have all made our top choices,
we will score this round. The most popular response for this round was
"**". Therefore, those who selected this response should score 1
points. As "**" suggested this tactic, s/he should score 5 points. As a
friendly reminder, all scoring is on the honor system. Therefore, as I read out
the score for each round, please keep a running tally of your points.
Voting - Round 2 - 5: Consider this list again (minus any item
that is crossed out) and use the pen tool (with the red color) and place a
check mark next to the recommendation that you feel is the most effective
tactic. It can be the same item you selected in a previous round, as long as
that item is not already in our top choices. [Repeat through round 5]
Scoring - Round 2 - 5: Now that we have all made our top choices,
we will score this round. The most popular response for this round was "**".
Therefore, those who selected this response should score 1 points and "**" who
suggested the tactic should score 5 points.
[Note regarding scoring: For ease of facilitation
in this online environment, I made slight adjustments to the scoring scheme from
the original Framegame model.]
Conclusion of the Game: We now have a list of the top 5 techniques.
Please add up your personal score and share it with the group. Congratulations
to our winner!
Debrief: Let's more fully consider the list and discuss the merits of not only
our top 5 choices, but how we arrived at this list. Do you think through this
method we arrived at a good working list of 5 effective techniques? What
surprises you about the list? Would you like to make a case for or against
items that were selected or any that were not selected in the top 5?
[Note: Given the time we spent discussing
the recommended tactics during the brainstorm, we did not have time for a full
debrief. Instead, I offered to compile the list form the session so that the
group can pick up their discussion next time.]
Critique / Reflection:
Time: We spent about 40 minutes playing the game. As noted above, we spent
more time than I had originally allocated for the brainstorm portion, which
left us out of time for a full debrief. If I had more time, I would like to
have heard more from the group about what they thought about the process
(effectiveness, efficiency, etc).
Scoring / Voting: The participants were not that interested
in the scoring and voting aspects. They were far more interested in discussing
the topic than coming up with the most "popular" vote. Therefore, I'm not sure
if the scoring or voting is a crucial element when you have highly motivated
participants. However, I can see how it could add interest with less motivated
participants and when you have enough participants to make it a team based
game.
Quality of Participant Recommendations: The list of participant recommendations, as
well as the ensuing discussion was great. I think this was due to the participants'
interest in the topic, as well as the participatory nature of the brainstorm on
the whiteboard. Each recommendation seemed to build on another until it seemed
the group fleshed out the key tactics.
Interest / Engagement: Participant interest level and engagement
was great! However, I purposely chose a topic that I knew the group would find interesting.
Coincidently, this group had been criticized in a discussion board comment
about some dialog that a listener found less than positive the week before. We
were able to kick off our discussion by using that as an example of how
difficult it can be to keep online interactions positive for all participants.
This clearly helped to gain participant interest. In addition, the process does
encourage participation from all players, both in posting recommendations as
well as in voting. The participants indicated that they liked the game and felt
it helped to bring out a good discussion and a list of tactics to use in the
future. However, as usual, I INTERJECTED TOO MUCH!
Online Facilitation: The communication tools made this an easy
game to facilitate online. See the screen captures from the session below.
However, that being said, I had a group of self-described "tech geeks" who love
to play with new technology. I can see how the whiteboard could become a
free-for-all disaster in the hands of kids (but they would probably have a
blast in the process!) However, I also see how this could be facilitated
asynchronously using a discussion board and an online polling / voting
mechanism, such as in a content management system.
My Key Take Always: I now appreciate how games can raise
interest level and engage a group. In addition, the planning process and
structure of the game helped me as the facilitator organize and prepare my
thoughts. However, I can also guess that some audiences would not be as
interested in the "game play" (competitive scoring) aspect - especially when
discussing serious or heavy topics. I think if I would have pushed to stick to
the voting / scoring scheme I would have definitely taken away from the flow of
the discussion. In the end, I found Take Five to be a very effective format to
get the group brainstorming to come up with recommended tactics. I will definitely attempt this format again
in the future!
Article Reviews Running head: ARTICLE REVIEWS 1 Article Reviews Jennifer Maddrell Old Dominion University IDT 848 Evaluation Study Abstracts Dr. Adcock October 22, 2008 Article Reviews Reference Burton, J., & Aversa, F. (1979). Formative evaluation information from scripts, scratch tracks, and rough cuts: A comparison. Educational Communication and Technology Journal, 27(3), 191-194. Summary 2 Given the significant time and expense outlay involved with television course production, Burton and Aversa (1979) sought to understand how early in the televised course development process the learner content review should occur. While prior research on formative evaluation suggested that review should begin when the instructional product is still “fluid”, Burton and Aversa questioned how useful learner script review is at the early production stage and predicted early stage scripts would be too incomplete for the learner to discern the instructional message. Design Sample and Studied Variables 82 adult learner reviewers were selected from a group of potential students who fit the learner profile for the course. The students were randomly assigned to one of three treatment groups, including those who reviewed (a) the written script alone, (b) the written script and an audio scratch track, and (c) the first rough cut version of the video. The three groups were compared based on both learning outcome, as well as on learner responses to the course material categorized into three areas, including the overall appeal of the program, the learner’s affective responses to the subject matter, and the design of the structural elements of the program. Treatment Members of all three groups provided basic demographic information, including age, education level, subject background, and received the same introduction to the course entitled, Japan II: The Changing Tradition. Those in the script group read through the written script once. Article Reviews The learners in the scratch track listened to an audio recording of a single voice reading all narrations while following along with the written script. The rough cut group viewed the initial version of the video without visual effects or music. After reviewing the materials, the learners completed a 5-point Likert scale opinion questionnaire about the instructional product followed by a short answer test to assess their understanding of the presented material. Analysis and Results 3 The collected demographic information confirmed the groups did not differ significantly. Further, the differences in the mean scores across the three groups for the short answer test were not statistically significant. However, in terms of learner responses to the questionnaire, the mean differences across the three groups were statistically significant. For each learner response measures, the mean scores for the scratch test group were greater than for script group which were greater than for the rough cut group. Critical Summary This study provides support for the use of early scripts and audio scratch tests in high production courses. However, as was most striking to the researchers, the relatively harsh response to the rough cut video appears to contradict prior research. As a possible explanation for the poor learner responses in this study, the rough cut used in this study may have been too rough and too far from the finished representations to allow a viable comparison. Application This study offers support for early evaluation, especially when production time and expense is high and late term revisions would be costly. As this study suggests, learner reviewers are able to discern the instructional message in very early drafts within the development process. Reference Article Reviews Jones, T., & Richey, R. (2000). Rapid prototyping methodology in action: A developmental study. Educational Technology Research and Development, 48(2), 63-80. Summary Citing mixed findings in research literature, Jones and Richey (2000) questioned the 4 effect of rapid prototyping (RP) on instructional design development cycle time, product quality, and customer satisfaction. The purpose of their qualitative study was to gain a better understanding of how RP methods are applied, what the customer’s role is within the RP process, how (if any) concurrent completion of design tasks occur, what (if any) instructional systems process and quality enhancements result, and how customer satisfaction is impacted. Design Qualitative Details and Methodology This qualitative study was conducted at an instructional design firm with 14 employees. Several years prior to the study, the firm adopted a RP process for their custom designs that focused on three milestones, including (a) kickoff involving a customer meeting where roles, responsibilities, and schedules are determined, (b) design freeze when full agreement on product format, content, and instructional strategies is reached between the designers and customer and rapid development occurs, and (c) pilot ready when the product ready for learner pilot testing. The activities of two senior instructional designers on two separate projects, as well as one client contact per project, were examined. Both projects were one-day instructor led classes, but were delivered using different media. Data collection included reviews of designer task logs and other project data, as well as personal interviews. Data analysis focused on the nature of the RP process, attitudes about RP and the product, cycle time, and overall customer involvement. Analysis and Results While the projects were completed fairly linearly, especially in the final stages, the data Article Reviews analysis revealed that the 14 key tasks prescribed by the firm’s RP model were performed for each project with concurrent processing occurring in the completion of 10 out of the 14 key tasks. Work time varied between the two projects at the task level, but total work time was similar at 79.25 hours for Project 1 and 74.0 hours for Project 2. Both the designers and customers perceived reduced cycle times as compared with traditional instructional design. The researchers noted the relatively high degree of customer interaction in both projects. Customers were actively involved in (a) analyzing the training needs, (b) providing input, feedback and approval of content, learning activities, and the prototype, and (c) participating in the pilot. Given that learner achievement data was not collected, the researchers focused on satisfaction (of the designer and customer) and usability to the end customer. Satisfaction with 5 the project was high for both the customer and designers. Further, both projects were put into use immediately after delivery to the customer and were in use one year after which the researches deemed to be a measure of product quality from an external consultant’s perspective. Critical Summary Given the results of this limited qualitative review, the chosen RP design process resulted in acceptable production cycle time, a usable instructional product, and a satisfied customer; all good outcomes for an instructional design consultant. However, without a measure of learning outcome, effectiveness was not fully evaluated. Further, it is possible that the relatively high degree of stakeholder involvement during the entire instructional design process, not simply RP process changes, was the key factor in the project’s success. Article Reviews Application The most significant outcome of this study is the reinforcement of the need for frequent communication and buy-in from the stakeholder. As suggested in this study, a project will run 6 efficiently and result in a more satisfactory outcome if there is open communication, stakeholder input, and agreement on key decisions throughout the design process. Reference Brown, K. G. (2005). An Examination of the Structure and Nomological Network of Trainee Reactions: A Closer Look at 'Smile Sheets'. Journal of Applied Psychology, 90(5), 9911001. Summary Brown (2005) studied training reaction from an affect-based theoretical framework examining affect as a subjective state that is either positive or negative. Based on cited prior theory and research, Brown predicted: (a) affective training experiences create an overall evaluation of satisfaction which in turn influences specific reactions to the training; (b) content interest is positively related to reactions; (c) learner personality traits and orientations are related to reactions; (d) media aesthetic appeal influence reactions, and (e) reactions and learning are related. Design Sample and Studied Variables Two studies were held to examine these predictions. In the first study, 178 undergraduate business students and 101 graduate business students volunteered to answer a survey regarding a pre-recorded videotaped lecture with 64% and 58% response rates, respectively. The second study included 97 undergraduate business students who were randomly assigned to one of three groups who viewed the same lecture presented via three different technologies, including (a) a Article Reviews 7 computer-delivered presentation, (b) audio and print, and (c) video, audio, and print. Between the two studies, a host of measures were evaluated and compared, including learners’ computer experience, personality traits (extraversion, neuroticism, and openness), mastery and performance goal orientation, content interest, technology satisfaction, perception of relevance, enjoyment, overall satisfaction, engagement, and learning. Treatment In the first study, learners viewed a videotaped lecture and took a survey assessing content interest and reaction. In the second study, participants viewed the identical instruction, but via the noted technologies. At the lecture’s midpoint, a brief engagement survey was conducted. After the lecture, learners completed a reaction survey, an intention questionnaire related to future use of the technology, and a 25 item multiple choice knowledge test. Analysis and Results From the first study, a factor analysis suggested that (a) reactions are related, (b) overall satisfaction is a good predictor of other reaction measures, and (c) attitude (interest) and disposition (master goal orientation) predict reactions. Within the second study, a multivariate analysis of covariance with ACT score as the covariate showed statistically significant differences in reactions across delivery technologies with audio conditions having statistically lower satisfaction measures. In addition, regression analysis suggests reactions can predict engagement, intentions, and learning. Critical Summary While Brown’s paper presents an intriguing affect-based theoretical framework for the study of trainee reaction, he acknowledged the conflict between his research findings and prior research, especially within the suggested relationship between reaction and learning. It is Article Reviews 8 troubling that Brown is satisfied that his findings from these very short and limited single session interventions can challenge this extensive body of prior research. Application Brown’s affect-based theoretical framework is an intriguing basis for future research on trainee reactions. If his findings are correct that an overall satisfaction measure is a predictor of other reaction measures and that reaction can predict engagement, intentions, and learning, reaction surveys could be streamline to just a few items addressing overall satisfaction with the experience. Reference Kandaswamy, S., Stolovitch, H., & Thiagarajan, S. (1976). Learner verification and revision: An experimental comparison of two methods. Audio-visual Communication Review, 24(3), 316-328. Summary Kandaswamy, Stolovitch, and Thiagarajan (1976) report on one in a series of studies on learner verification and revision (LVR). Noting increased advocacy and use of learner feedback during formative evaluation, the researchers assess the generalizability of prior studies which support LVR and compare the effectiveness of tutorial LVR and group-based LVR methods. Design Sample and Studied Variables 140 eighth grade girls were randomly selected from two different schools in India. 60 girls were randomly assigned to the LVR group while the remaining 80 were included in a final summative comparison. Four teachers from the schools were randomly selected as evaluator / revisers. The studied LVR methods included (a) tutorial LVR in which the evaluator / reviser probes and monitors the learner’s nonverbal and verbal feedback while the learner completes the Article Reviews 9 material and (b) group based LVR in which the evaluator / reviser analyzes patterns of errors and predicts causes after the learner completes the material. The 60 assigned to the LVR group were stratified based on prior math achievement and one top, average, and poor student was randomly assigned to each of the four evaluator / revisers for the tutorial LVR treatment. The remaining 48 were randomly assigned to the four evaluator / revisers for the group-based LVR treatment. Treatment For the 48 learners in the group-based LVR treatment, a proctor administered a printed self-study instruction booklet which contained a pretest, instruction, and posttest. Upon completion, each of the two group-based evaluator / revisers took 12 booklets each and made independent group-based revisions. The other two evaluator / revisers conducted separate tutorial LVR sessions with the 3 students assigned to them and made independent revisions from the evaluation. In a second phase, the review and revision process was reversed. These two phases resulted in a total of eight revised versions and allowed an evaluation of the order in which the methods were used. The 80 students in summative comparison group were randomly assigned to one of eight groups and completed one of the revised pretest, instruction, and posttest materials. Analysis and Results A one-way analysis of variance of the posttests of the original and eight revised versions show statistically significant differences between each of the eight revisions and the original which supports the research prediction that learner review and revision improves the instructional material. However, there was no significant difference in outcomes between the tutorial and group methods of LVR or from the order in which the methods were used. Yet, the revisions by different evaluators and revisers did have different degrees of effectiveness supporting the prediction that not all revisions by evaluators are of equal value. Article Reviews Critical Summary This study is significant in that it suggests support for conducting learner based review. 10 Further, the findings suggest that the evaluation method and order of use of different methods are less important than the person chosen as the evaluator and reviser. Application These findings suggest that not all evaluators are equally effective at evaluation and revision. Therefore, an evaluator quality control process should be contemplated which includes a cross-comparison of revision recommendations across multiple reviewers and evaluations of the evaluators. Reference Medley-Mark, V., & Weston, C. B. (1988). A comparison of student feedback obtained from three methods of formative evaluation of instructional materials. Instructional Science, 17(1), 3-27. Summary Twelve years after the 1976 study by Kandaswamy, Stolovitch, and Thigarajan discussed above, Medley-Mark and Weston (1988) sought to quantitatively and qualitatively compare the data collected from one-to-one and small groups during learner verification and review (LVR). Given the lack of research on the characteristics of the data collected, the stated purpose of the study was to examine the identified student problems across various LVR conditions. Article Reviews Design Mixed Method Details and Methodology 24 students volunteered to participate in the evaluation of two modules in their undergraduate educational media course. The volunteers were stratified based on grade point average. From this, six students were selected based on availability and assigned into one of 11 three groups, including a one-to-one group with the student with the highest grade point average (GPA) (1-1), a small group of two students with comparable mid-range GPA’s (1-2), and a small group of three students from each of the three GPA levels (1-3). All six students participated in two sessions. In the first, the unmodified version of the prior year’s one hour lecture was given by the instructor followed by Assignment 1 which the learners completed based on their assigned condition, as discussed below. In the second session, learners viewed a video-taped lecture followed by Assignment 2, again completed based on their assigned condition. The two print based assignments included short-answer and essay exercises which were graded and evaluated. In the 1-1 session, the evaluator assumed a passive role while the learner was encouraged to think aloud during assignment completion. Students in the 1-2 session were encouraged to discuss encountered problems as they individually completed the assignments. Those in the 1-3 group were instructed to work independently and passively. After the learners completed the assignments, the evaluator in each group conducted a debriefing session with prepared questions. Analysis and Results Overall, the 1-1 condition identified the most number of problems with the greatest detail, but the evaluation process involved the most time and effort on the part of both the learner and evaluator. The 1-2 group identified the second highest number of problems, but evaluated the Article Reviews product in conditions that were most similar to actual use. The 1-3 group identified the fewest 12 number of issues, but their time to complete the assignment was closest to actual use. In addition, the groups differed based on the types of problems identified. The 1-1 and 1-2 conditions focused heavily on problems associated with the situation statements while the 1-3 group focused on the problem or the problem choice within the assignment. Critical Summary This study is valuable for its qualitative comparison of the one-to-one and small group learner evaluation. While no clear cut winner is established, that was not the point of the study. Rather, the study suggests a comparison and tradeoff between efficiency and effectiveness. While one-to-one evaluation may provide the greater efficacy in terms of problem identification, it will come at a higher cost in terms of time and energy on the part of the learner and evaluator. In contrast, the small group learner evaluation may offer a more efficient and practical evaluation method, but may not offer the most breadth and depth of problem identification. Application This study suggests that evaluators need to consider the practical efficacy and efficiency tradeoffs and implications when creating an evaluation plan. While one-to-one evaluation may lack efficiency, greater efficacy may result in terms of breadth and depth of problem identification. In contrast, small group may offer a more efficiency when the evaluator does not have the luxury of time and budget to run a series of one-on-one learner reviews, but fewer issues may be identified.
IDT 873 Abstract: Cognitive Task Analysis Jennifer Maddrell van Merrienboer, J. J. G., Kirschner, P. A., & Kester, L. (2003). Taking the Load Off a Learner's Mind: Instructional Design for Complex Learning. Educational Psychologist, 38(1), 513. Overview Citing decades of prior cognitive load theory and research, van Merrienboer, Kirschner, and Kester (2003) offer a theoretical framework and instruction design model for complex learning. Noting a recent emphasis on authentic learning tasks (such as project and problembased learning approaches) to support complex learning, they consider the implications on cognitive load and offer a model designed to manage both intrinsic and extraneous cognitive load. Theory While the theories underlying the use of authentic learning tasks may vary, a common assumption is that authentic tasks help learners to integrate the knowledge and skills necessary for complex task performance (van Merrienboer et al., 2003). However, given the novice learner’s weak problem-solving methods, they face high extraneous cognitive load when confronted with authentic tasks. In addition, the complexity inherent in the authentic task presents high intrinsic cognitive load. Therefore, based on cognitive load theory, engaging in highly complex authentic learning tasks may strain the novice learner’s limited working memory and subject the learner to excessive cognitive load. Proposal van Merrienboer et al. focus their attention on both the nature and the delivery timing of the presented information. They suggest that supportive information (knowledge necessary for problem solving and reasoning) is best presented before the learner engages in the learning task. Such supportive task specific information is inherently complex and needed in order to know how to approach the learning task. Presenting the supportive information first helps learners construct schemas to be used as they begin task performance. In contrast, van Merrienboer et al. suggest that procedural information (the how to instructions for rule application) is best presented when needed during task performance. They argue that such just-in-time presentation of procedural information reduces the potential for splitattention effects that may occur when the learner attempts to integrate procedural information learned previously with actions he or she is taking now. Heuristics From these suggested practices, van Merrienboer et al. offer an instructional design model (the 4C / ID model) for complex learning that focuses on four components: 1) learning tasks, 2) supportive information, 3) procedural information, and 4) part-task practice. The heuristics for designers within the 4C / ID model is to sequence from simple versions of the whole task beginning with a high level of support and ending with a complex version without support. In addition, as discussed above, supportive information is to be presented in advance of performance while procedural information required to perform the task is to be presented as the task is being performed. Finally, to encourage automaticity, additional repetitive practice should be incorporated for parts of the task. Critique The focus of the article is not an examination of the effects of authentic learning tasks on learning, but rather the implications of incorporating such tasks on the learner’s cognitive load. As such, the article offers a bridge across theory, research, and practice. A key strength of the article is the authors’ focus on the reality of limited working memory and the high cognitive load IDT 873 Abstract: Cognitive Task Analysis Jennifer Maddrell imposed by authentic learning tasks. The 4C / ID model offers designers a way of incorporating authentic tasks while at the same time better managing cognitive load. However, as a theoretical article, it does not offer results from a study of the model in practice. Do the heuristics within the 4C / ID model help to manage cognitive load? Further, do authentic learning tasks designed within the framework of the 4C / ID model effectively and efficiently support learning? These questions are left to future research.
IDT 873 Abstract: Cognitive Task Analysis Jennifer Maddrell Schaafstal, A., Schraagen, J. M., & van Berlo, M. (2000). Cognitive task analysis and innovation of training: The case of the structured troubleshooting. Human Factors, 42(1), 75–86. Research Overview. Following an instructional design evaluation of an existing Royal Netherlands Navy maintenance training course, Schaafstal, Schraagen, and van Berlo (2000) observed a gap between the instruction and the practice of troubleshooting the subject system. They observed that the existing instruction was based largely on the technical equipment documentation from engineers which focused exclusively on the system’s components. Following a comprehensive cognitive task analysis (CTA), Schaafstal et al. revised the instruction under the assumption that maintenance system troubleshooting is a complex cognitive task requiring not only knowledge about the system’s components, but also knowledge about how the system functions and how to consider possible causes and solutions to maintenance problems. The CTA consisted of several observational studies of troubleshooting with technicians of varying expertise levels. Based on information from the CTA, a modified course was prepared which focused on a functional understanding of the system versus the component orientation of the prior course. In addition, general troubleshooting strategies were incorporated which gave learners instruction on how to a) describe the problem, b) generate causes, c) test causes, d) repair, and e) evaluate solutions. Purpose. The purpose of the presented research was to evaluate the modified structured troubleshooting training course and to compare it with the exiting maintenance training course. Schaafstal et al. predicted superior outcomes from the revised course. Methodology. A series of experimental studies compared the learning outcomes of maintenance trainees taking the new structured troubleshooting training course with groups of maintenance trainees taking the existing training course. Outcome measures included malfunction identification, reasoning, and functional understanding of the system. Conclusions. The modifications in the course reduced the course duration by 33% (from six to four weeks). Even at the shortened length, those participating in the new course achieved statistically superior results as compared to those in the original course. Based on the results of the study, Schaafstal et al. suggest that novice technicians lack both a systematic approach to troubleshooting, as well as a functional understanding of the equipment. As seen in prior research, they observed that novices face information overload (lose the forest for the trees), lack hierarchically organized cognitive frameworks, lack functional understanding, possess inadequate mental models of underlying system, and lack the ability to isolate causes of the problem. Therefore, based on the results of their evaluation, they suggest that training in troubleshooting should focus on three areas: 1) system independent troubleshooting strategies to be used across systems, 2) system specific functional models, and 3) system specific domain knowledge. Heuristics Results of this research suggest the importance of moving away from a purely component oriented analysis to what the researchers term a functional decomposition when designing troubleshooting skills instruction. While analysis and instruction on the components is necessary, it is not sufficient. Analysis and instruction should also focus on the functional processes, including likely causes of potential problems and paths to solutions, in order for learners to know what to do when troubleshooting. Further, the results indicate that training in system independent troubleshooting skills can further augment the troubleshooting skills instruction. IDT 873 Abstract: Cognitive Task Analysis Jennifer Maddrell Critique The presented research is important for two reasons. The research suggests a positive influence of CTA on outcomes in troubleshooting training. By revising the instruction to focus on a functional understanding of the system from information gleaned in the CTA, the instruction appears to have been significantly improved. In addition, the findings suggest a positive impact from teaching system independent troubleshooting skills. Also, the paper is valuable for the information provided about the evolution of the authors’ CTA process. This information will be helpful to future designers and researchers. Unfortunately, the written presentation of this paper is horribly disjointed. It is doubtful that most readers will devote the time necessary to weave a coherent narrative out of the broken threads of theory, prior research, CTA processes, instructional design considerations, research methodologies, and conclusions. There is a wealth of information included in the paper, but unfortunately the reader must devote an unnecessary amount of effort to piece it all together.
IDT 873 Abstracts: Concepts Jennifer Maddrell Klausmeier, H. J., & Feldman, K. V. (1975). Effects of a definition and a varying number of examples and nonexamples on concept attainment. Journal of Educational Psychology, 67(2), 174-178. Research Purpose and focus. Klausmeier and Feldman (1975) focused their research on concept attainment which they defined within their study as the ability to a) discriminate defining attributes, b) name the concept and each defining attribute, c) evaluate examples and nonexamples, and d) define the word representing the concept. In reviewing prior literature on concept attainment, they highlighted four categories of variables generally studied, including 1) a rational set of examples and nonexamples, 2) definitions of a concept (based on the relevant attributes of the concept), 3) emphasizers to facilitate discrimination, and 4) feedback. The purpose of this study was to evaluate the effect of presenting various combinations of concept definitions and rational sets. They predicted better attainment from those presented with both a rational set and a definition than those presented with either one or the other. Further, they predicted better attainment from those presented with the definition and additional different rational sets. Methodology. 134 fourth-grade students from two Wisconsin (Go Badgers!) elementary schools participated in the study. The students were stratified into high, medium and low levels based on their performance on the most recent Iowa Tests of Basic Skills test. The subject matter concept was the equilateral triangle. Students within each stratification level were randomly assigned to one of four treatment groups which included those presented with 1) a definition of the concept without examples or nonexamples, 2) a rational set of three examples and five nonexamples, 3) a combination of the same definition and rational set, and 4) a combination of the same definition and three different rational sets of three examples and five nonexamples. The treatment lesson was presented in four printed lesson booklets. Following instruction, students were given 1 minute to read each lesson page and then were instructed to turn to the next page allowing 5 minutes per lesson booklet. Immediately following the last lesson, a classification task within a printed booklet measured concept attainment. Without time limit, students viewed 38 instances and were asked to identify whether the instance was an example (by circling yes) or nonexample (by circling no) of an equilateral triangle. Results and conclusions. Means for the stratified groups reflected the initial levels with means for high > medium > low. As predicted, no significant difference in concept attainment was found between those who were presented with either a definition or a rational set. Contrary to the researchers’ prediction, there was also no significant difference from a combination of a definition and the single rational set. However, there was a significant difference between those presented with a definition and those who also received three rational sets. These findings are important as they suggest an advantage for presenting additional rational sets of examples and non-examples. Heuristics The results of these experiments suggest that designers should augment the presentation of the concept definition with multiple rational sets of examples and non-examples when teaching concepts. As seen in this experiment, providing learners with additional rational sets to consider may increase their attainment of the concept. Critique Page | 1 Submitted 20081008 IDT 873 Abstracts: Concepts Jennifer Maddrell The results of this study are important as they provide support for the hypothesis that presenting learners with more examples and non-examples is better. However, if three sets of examples and non-examples are better than one, is more than three even better? A criticism of this study is the short intervention and the focus on a single math related concept. Would these results be replicated over a longer period of time with other types of concepts and with different age groups of learners? Tennyson, R. D., & Rothen, W. (1977). Pretask and on-task adaptive design strategies for selecting number of instances in concept acquisition. Journal of Educational Psychology, 69(5), 586-592. Research Purpose and focus. Tennyson and Rothen (1977) sought to expand the previously reviewed work of Klausmeier and Feldman (1975) by evaluating the effect on concept attainment of adapting the number of examples and nonexamples based on individual need. They predicted that an adaptive design strategy that varied the presentation of examples and nonexamples based on student need would improve concept attainment over a nonadaptive strategy. Methodology. 67 undergraduate students participated in the study. The students were randomly assigned to one of three treatment groups, including 1) full adaptive, 2) partial adaptive, and 3) nonadaptive. The adaptive designs were modified using a computer-based Bayesian adaptive strategy which altered the number of examples learners viewed based on a) pretreatment measures of aptitude, b), pretests of prior achievement, and c) task performance. A pretest, treatment lesson, and posttest were administered individually via computer. The untimed lesson focused on two legal concepts, including best evidence rule and hearsay. For all groups, the learning task defined the concept based on the critical attributes of the concepts. The number of instances presented to students varied based on their assigned treatment group. The nonadaptive group received the same number of instances. The number of instances in the partial adaptive model was based on pretest data while the number presented within the full adaptive model was modified based on both pretest data and on-task responses. The study also evaluated the time on task which did not include pre- or post-test time. Results and conclusions. While no significant mean differences were found in pretest measures, significant mean differences were reported regarding time on task and posttest score measures. As predicted by the researchers, the results suggest that full adaptive strategies were more effective than partial adaptive strategies and that the two adaptive strategies were more effective than nonadaptive conditions. In addition, the full adaptive group finished the program significantly faster than the partial group which in turn finished faster than the nonadaptive groups. In attempting to explain the results, the researchers suggest that learning tasks where instance presentation is not modified based adaptive strategies may not keep learners’ interest in the task. Heuristics The results of these experiments suggest modifying instructional concept presentation based on learner mastery. Based on the findings of this study, presentation of examples and nonexamples after the learner has achieved mastery may result in learners losing interest in the learning task. Critique Page | 2 Submitted 20081008 IDT 873 Abstracts: Concepts Jennifer Maddrell The results of this study are important as they suggest that optimal presentation varies based on the each individual learner’s level of mastery. In this controlled experiment, using a computer based model, the researchers were able to alter the individual presentation based on each learner’s level of mastery which resulted in more effective instruction. However, altering presentation to an individual learner in real-world instructional settings is difficult, especially in group face to face settings. Therefore, while the results suggest an important finding with regard to tailoring instruction to meet the individual learner, such modifications may not be feasible in practice. Page | 3 Submitted 20081008
Concept Learning and Instruction Running head: CONCEPT LEARNING AND INSTRUCTION 1 Concept Learning and Instruction Jennifer Maddrell Old Dominion University IDT 873 Advanced Instructional Design Techniques Dr. Gary Morrison November 12, 2008 Concept Learning and Instruction 2 Concept Learning and Instruction While concept learning has been considered across a broad spectrum of theoretical foundations, the prescriptions for instruction are strikingly similar. A rich history of research in concept learning and instruction has led to empirically based instructional design strategies which focus on (a) defining and presenting a concept’s attributes, (b) creating and presenting instances of examples and non-examples of the concept, and (c) fostering guided learner practice in attribute isolation, instance discrimination and generalization, and concept use. This paper offers a summary the central views on the nature of concepts, concept learning measurement, and concept instruction based on a survey of concept learning and instruction theory and research. The Nature of Concepts Whether viewed as the object of learning or a building block to more meaningful learning, there is general agreement regarding the concept construct. Markle and Tiemann (1970, p. 52) considered the similarity in the conception of concepts across theorists to be “remarkable.” A concept is generally described as a category (class, group, or set) of objects, events, symbols, or relationships with shared characteristics or properties, often referred to as attributes (Tennyson & Cocchiarella, 1986). A category is often described as a grouping of objects, events, symbols, or relationships while an attribute describes the dimension from which the objects and events differ (Brown, 1958). Further, membership to the category is generally considered based on either the perceived physical attributes (concrete concepts) or solely by definition (abstract concepts) (Smith & Ragan, 1999). Concept Learning Measurement While the theoretical perspectives vary regarding how concept learning occurs, there is noteworthy similarity in how concept learning, also referred to as concept attainment or acquisition, is measured. Concept learning is generally deemed to have occurred when the learner is able to discriminate among attributes of a concept and to evaluate new examples based on membership to the concept category (Klausmeier & Feldman, 1975). Based on the terminal objective of the instruction, concept learning and assessment can focus on both recall and application of the to-be-learned concept (Morrison, Ross, & Kemp, 2001). The following examines the role of attribute isolation, instance discrimination and generalization, and conceptsin-use in measuring concept learning. Attribute Isolation Concept learning assessment typically focuses on a learner’s ability to consider the nature of instances encountered based upon defining attributes belonging to the concept category (Bruner, Goodnow, & Austin, 1956). A common focus in concept learning assessment is the learner’s ability to discern the relevant criteria by which attributes are grouped into the concept categories (Joyce & Weil, 1972). In measuring concept attainment, two types of attributes are of concern, including (a) defining attributes and (b) criterial attributes isolated by the learner (Bruner et al., 1956). Defining attributes reflect the standard criteria set by appearance or convention. In contrast, criterial attributes are established by the individual to assess and judge membership in the category. Concept learning (or attainment), therefore, is judged based the extent to which the criterial attributes isolated by the learner match the defining attributes. Instance Discrimination and Generalization Concept learning assessment also centers on a learner’s ability to (a) discriminate between what is and what is not a member of the class and (b) generalize new examples by appropriately judging instances based on the degree of membership to the exemplar class (Markle, 1969). Therefore, successful concept learning is assessed based on the learner’s ability Concept Learning and Instruction 3 to place instances in the exemplar class and to respond to members of the exemplar class as a whole (Gagné, 1965). To do so, learners must be able to discriminate non-members from members of the class while not overgeneralizing (incorrectly judging non-examples as examples) or undergeneralizing (incorrectly judging examples as non-examples) (Markle & Tiemann, 1970). Concepts-in-Use In a recent review of theory and research on the role of concepts in learning and instruction, Jonassen (2006) argued that the historical focus of concept learning has been on concept attainment as a discrete and terminal learning outcome without regard to where the concept fits within a larger conceptual framework. In contrast, Jonassen suggested a focus on concepts-in-use in which concept learning centers on concepts as mental model building blocks. As such, Jonassen argues that the instruction and assessment should shift beyond the learner’s ability to identify, discriminate, and generalize membership based on concept attributes and examples to how the learned concepts are organized within the learner’s overall conceptual framework. He asserts that concept learning and assessment should focus on the learner’s ability to describe or represent conceptual patterns and propositions, as in concept maps, word associations, and model building. While Jonassen (2006) may be correct in advocating an expanded instructional focus and a more meaningful terminal objective, it does not follow from his argument that prior concept learning prescriptions do not lead to the learner’s ability to demonstrate application of the concept. Beyond assessing the learner’s ability to correctly identify or categorize concepts, countless other means have been suggested to measure the learner’s ability to use and apply the concept, to make judgments and arguments on the basis of the concept, and to infer membership in superordinate categories (Tessmer, Wilson, & Driscoll, 1990). Therefore, instead of a call for abandonment of past instructional prescriptions, a call for enhanced practice and assessment which forces more meaningful learner application of the to-be-learned concept may be more compelling. Concept Instruction The similarity across theoretical foundations that has been described thus far continues across a review of concept teaching models regarding instructional presentation, learner practice, and guidance. Concept instruction typically includes presentation of a concept definition, presentation of sample instances, and practice in classifying instances of examples and nonexamples (Tennyson & Cocchiarella, 1986). While some differences exist along behavioral, cognitive, or social-cognitive theoretical lines, the prescriptions for presentation, learner practice and guidance cannot be contrasted purely on differences in theoretical foundation. Instead, the prescriptions across theoretical foundations are quite similar with differences occurring in areas such as sequencing, the degree of learner autonomy to discover attributes and instances, and the terminal objective of the lesson. In general, instructional strategy differences can be seen as either expository (direct presentation of attributes and instances) approaches inquiry (learner exportation or discovery of attributes and instances) approaches or (Smith & Ragan, 1999). Setting aside an analysis of the various media and instructional delivery alternatives, the following highlights common presentation, learner practice, and learner guidance techniques stemming from a variety of inquiry and expository approaches. Concept Learning and Instruction 4 Defining Concept Attributes Research suggests that learning is enhanced when a concrete definition is presented and that a definition alone is roughly as effective as a single set of examples and non-examples (Klausmeier & Feldman, 1975). Therefore, concept instruction generally includes providing learners with a stated definition of the domain of the concept based on the properties (attributes) of the concept class (Markle, 1975). Some advocate that the concept definition should identify the name of the concept, the attributes, and how the attributes are combined to determine class membership (Merrill & Tennyson, 1977). When attributes are defined and presented, their characteristics are typically considered based on their function and the degree to which they vary, can be observed, and relate to one another. A critical attribute refers to the necessary characteristics for determining membership while variable attributes are characteristics shared by some members of the class, but are not necessary for class membership (Merrill & Tennyson). Attribute characteristics that are stable across contexts are of constant-dimension while those that vary or change are of variable-dimension (Tennyson & Cocchiarella, 1986). Some suggest further defining attributes based on their (a) intrinsic prosperities, referring to their observable and invariant properties, (b) functional properties, referring to how something functions or is used, or (c) relational properties, referring to the invariant relationship between items. (Klausmeier, 1992). When defined based on their relational properties, concepts fall within three categories, including (a) conjunctive concepts which are defined by one attribute and another, (b) disjunctive concepts which are defined by one attribute or another, and (c) relational concepts which are defined by a relationship between attributes (Fleming & Levie, 1978) Creating Instances Research suggests that factors such as the number, categorization, type, and range of instances presented to learners influence concept learning. Instances refer to examples and nonexamples of the concept being considered and, depending upon whether the concept is physical or abstract, can take the form of (a) a referent or actual object, (b) an isomorphic representation or model of the object, or (c) a symbolic representation including words or other symbols (Merrill & Tennyson, 1977). It is generally suggested that designers augment the presentation of the concept definition with multiple rational sets of examples and non-examples (Markle, 1969). Others suggest that a wide variety of examples be included (Fleming & Levie, 1978) and that the set of example and non-example instances should be matched (Merrill & Tennyson, 1977). Markle (1969 and 1975) offers a standard case concept analysis which focuses on the creation of a rational set of examples and non-examples to be used in both instruction and testing that involves (a) the identification of both critical and variable attributes (b) creation of examples in which all of the critical attributes are present, and (c) creation of non-examples. The ideal non-example is suggested to be one that shares all but one critical property with the concept class and is as concrete as possible (Markle & Tiemann, 1970). In contrast to presentation of sets of examples and non-examples, others suggest presentation of prototypical examples (Tennyson & Cocchiarella, 1986). An alternate view from the previously stated exemplar perspective, the prototype viewpoint suggests that a concept is encoded in memory as a prototypical example of a category member (Klausmeier, 1992). The prototype (or central example) is deemed to be constructed based on the learner’s experiences with examples of the class (Tessmer et al., 1990). Concept Learning and Instruction 5 Presentation, Learner Practice, and Guidance Presenting the concept label and attribute definition. Research suggests that presentation of concept labels and definitions assists learners in concept attainment by establishing the dimensions and boundaries of the learning task (Tennyson & Cocchiarella, 1986). Some argue that a definition of the concept focusing on the critical (defining) attributes should be presented prior to the presentation of instances of examples and non-examples (Tennyson & Park, 1980). Such an approach is often referred to as a RULEG approach in which, rules, principles, generalizations, or definitions (RU) are presented prior to examples (EG) (Markle, 1969). Others suggest beginning with presentation of the definition followed quickly by a recall or recognition activity (Merrill & Tennyson, 1977). However, others advocate an EGRUL presentation sequence in which the example is first presented followed by the rule, especially in cases where the concept is difficult or abstract (Fleming & Levie, 1978). As part of an inquiry approach, others suggest beginning with instructional activities that encourage learners to speculate about the defining attributes based on presentation of examples and non-examples (Joyce & Weil, 1972). Overall, while some variations in approach exist across the reviewed models, early initial presentation or discovery of the concept label and attribute definition is generally advocated. Presenting instances. While the emphasis and sequencing of instance presentation varies across models, the presentation of some form of instance was present in every reviewed concept teaching strategy. As noted, rarely in the reviewed models was an EGRUL approach described in which presentation of examples preceded the definition. Again, an EGRUL approach is generally suggested as a strategy when the concept is difficult or abstract and the learner may not have experience with examples of the concept. Presentation of instances tends to fall along two lines, either (a) presentation of sets of examples and non-examples or (c) presentation of prototypical examples. Some deemphasize the prescription for expository presentation of instance examples and non-examples, but advocate an inquiry approach in which data or information about the concept is presented to the learner from which the learner is able to draw inferences about the concept’s attributes (Joyce & Weil, 1972) In contrast, some advocate incorporating presentation of carefully selected example and non-example instances within either expository or inquiry presentation (Merrill & Tennyson, 1977). In expository presentation, the instance is presented, often highlighting or isolating critical attributes, which does not require a response from the learner. This is in contrast to inquiry approach where learners are presented with either an example or non-example and then asked to immediately identify whether it is a member of the concept category. Others favor the use of prototypical example presentation and suggest presenting learners with the best example of the typical class, followed by expository and interrogatory examples and non-examples (Tennyson & Cocchiarella, 1986). Research suggests that presentation of a best example forms a prototype for the learner and the additional examples provide elaboration of key dimensions of the prototype (Park, 1984). Some advocate a modified approach in which learners are offered various transformations of the instance which lead to a best example (Jacob, Deming, & Walbesser, 1976). However, most suggest that a single example is not sufficient and that a range of examples is needed (Markle, 1969). This view is supported by research that suggests an advantage for presenting additional rational sets of examples and non-examples over a single set (Klausmeier & Feldman, 1975). Yet, research also indicates the importance of modifying Concept Learning and Instruction 6 instance presentation based on learner mastery as instance presentation after the learner has achieved mastery may result in the learner losing interest (Tennyson & Rothen, 1977). Guiding Learner Practice. As discussed previously, instructional can lead to either learner recall or application of the to-be-learned concept. A common approach geared toward recall is to offer learners various practice opportunities to classify new instance as members or nonmembers of the class followed by corrective guidance. As noted, this typically takes the form of rule presentation and example presentation which is followed by learner practice and instructional guidance indicating either a correct or incorrect learner classification of the concept (Merrill & Tennyson, 1977). In addition, a host of inquiry and generative approaches are recommended which are often geared toward learner application of the concept, including the previously mentioned model building exercises. Concept mapping, as a form of model building, can assist learners to not only organize definitions and examples, but also to infer relationships within a larger conceptual framework (Tessmer et al., 1990). Some view model building as an ideal practice and guidance strategy for concept learning as models require learners to externalize their understanding of not only the concept, but also conceptual relationships (Jonassen, Strobel, & Gottdenker, 2005). Summary Heuristics for Designers As discussed, research in concept learning and instruction across a spectrum of theoretical foundations has led to empirically based instructional design heuristics which focus on (a) defining and presenting a concept’s attributes, (b) creating and presenting instances of examples and non-examples of the concept, and (c) fostering guided learner practice in attribute isolation, instance discrimination and generalization, and concept use. Within this common framework, differences in strategy can be viewed as either expository approaches where the instruction offers direct presentation of attributes and instances or inquiry approaches where the learner is offered opportunities to explore or discover relevant attributes and instances. As a summary of heuristics for designers, the following highlights common presentation, learner practice, and learner guidance techniques stemming from a variety of inquiry and expository approaches: 1. Define the concept. Prepare a concept definition which focuses on attributes of the concept. In doing so, consider the critical attributes that are necessary characteristics for determining membership, as well as the variable attributes which are shared by only some in the concept category. When defining the concept, it is helpful to consider whether it is a conjunctive concept that can be defined by one attribute and another, a disjunctive concept which is defined by one attribute or another, or a relational concept which is defined by a relationship between attributes. 2. Create instances. Create instances for presentation to the learner including examples in which all of the critical attributes are present and non-examples in which all but one critical property is present. Consider also the prototypical example. Depending upon the type of concept, the instances may be a referent or actual object, an isomorphic representation or model of the object, or a symbolic representation including words or other symbols. 3. Design presentation and guided practice opportunities. Incorporate presentation and guided learner practice opportunities which lead to not only recall, but also application of the concept within a larger conceptual framework. Consider using a RULEG approach in which the definitions (RU) are presented prior to examples (EG), unless the concept is Concept Learning and Instruction 7 difficult or abstract in which the EGRUL approach may be more appropriate. When assessing at a recall level, offer learners the opportunity to classify new instance as members or nonmembers of the class followed by corrective guidance. When assessing at an application level, include practice and guidance approaches which require the learner to use the concept. Such exercises might include asking the learner to make arguments or judgments on the basis of the concept or to infer relationship or membership by creating a concept map. Concept Learning and Instruction 8 References Brown, R. (1958). Words and things. Glencoe, Ill.: The Free Press. Bruner, J. S., Goodnow, J., & Austin, G. A. (1956). A Study of Thinking, A Wiley publication in psychology. (p. 330). New York, Wiley. Fleming, M. L., & Levie, W. H. (1978). Instructional message design : principles from the behavioral sciences. Englewood Cliffs, N.J.: Educational Technology Publications. Gagné, R. M. (1965). The Conditions of Learning. New York, Holt, Rinehart and Winston. Jacob, S. H., Deming, B. S., & Walbesser, H. H. (1976). They Too Teach Concepts. Educational Researcher, 5(1), 15-16. Jonassen, D. (2006). On the Role of Concepts in Learning and Instructional Design. Educational Technology Research and Development, 54(2), 177-196. Jonassen, D., Strobel, J., & Gottdenker, J. (2005). Model building for conceptual change. Interactive Learning Environments, 13(1/2), 15-37. Joyce, B. R., & Weil, M. (1972). Models of Teaching (p. 402). Englewood Cliffs, N.J., PrenticeHall. Klausmeier, H. J. (1992). Concept Learning and Concept Teaching. Educational Psychologist, 27(3), 267. Klausmeier, H. J., & Feldman, K. V. (1975). Effects of a definition and a varying number of examples and nonexamples on concept attainment. Journal of Educational Psychology, 67(2), 174-178. Markle, S. M. (1969). Good Frames and Bad; a Grammar of Frame Writing. New York, Wiley. Markle, S. M. (1975). They Teach Concepts, Don't They? Educational Researcher, 4(6), 3-9. Markle, S. M., & Tiemann, P. W. (1970). Problems of Conceptual Learning. Journal of Educational Technology, 1(1). Merrill, M. D., & Tennyson, R. D. (1977). Teaching Concepts: An Instructional Design Guide (p. 213). Englewood Cliffs, N.J.: Educational Technology Publications. Morrison, G. R., Ross, S. M., & Kemp, J. E. (2001). Designing effective instruction (3rd ed.). New York: John Wiley & Sons, Inc. Park, O. (1984). Example Comparison Strategy versus Attribute Identification Strategy in Concept Learning. American Educational Research Journal, 21(1), 145-162. Smith, P. L., & Ragan, T. J. (1999). Instructional design. New York: Wiley. Tennyson, R. D., & Cocchiarella, M. J. (1986). An Empirically Based Instructional Design Theory for Teaching Concepts. Review of Educational Research, 56(1), 40-71. Tennyson, R. D., & Park, O. (1980). The Teaching of Concepts: A Review of Instructional Design Research Literature. Review of Educational Research, 50(1), 55-70. Tennyson, R. D., & Rothen, W. (1977). Pretask and on-task adaptive design strategies for selecting number of instances in concept acquisition. Journal of Educational Psychology, 69(5), 586-592. Tessmer, M., Wilson, B., & Driscoll, M. (1990). A new model of concept teaching and learning. Educational Technology Research and Development, 38(1), 45-53.
IDT 873 Abstract: Design Based Research Jennifer Maddrell Wang, F., & Hannafin, M. (2005). Design-based research and technology-enhanced learning environments. Educational Technology Research and Development, 53(4), 5-23. doi: 10.1007/BF02504682. Overview Wang and Hannafin (2005) review theory and practice of design-based research (DBR), which fosters concurrent design, research, and practice, as a methodology for technologyenhanced learning environments (TELEs). Following a review of proposed models, Wang and Hannafin suggest DBR’s iterative, participative and situated processes as a means of forwarding instructional design research, theory, and practice. Definition and Characteristics Wang and Hannafin assess the various conceptions of DBR, including design experiments, design research, development research, and formative research. Given the varied emphasis across these conceptions, Wang and Hannifin offer a working definition of DBR within their paper, including “a systematic but flexible methodology aimed to improve educational practices through iterative analysis, design, development and implementation, based on collaboration among researchers and practitioners in real-world settings, and leading to a contextually-sensitive design principles and theories.” Their conception of DBR includes five key characteristics, including being (a) pragmatic (linking theory, research, and practice), (b) grounded (anchored in theories of learning and instruction, as well as real-world contexts) (c) interactive / iterative / flexible between researchers, designers, and practitioners, (d) integrative (blending a host of analysis and evaluation methods), and (e) contextual (localized results linked to prior observations and articulated in the form of heuristics to advance both theory and practice). Heuristics for Designers and Researchers Wang and Hannifin offer DBR principles which parallel many traditional ID activities, including formative and summative evaluation practices. A key distinction between ID evaluation and DBR is the eye toward theory development and the forwarding of generalizable, yet contextually influenced, design principles. However, the incorporation of theory development goals shifts the focus of the traditional ID evaluation approach. Instructional design plans do not have theory development as a central goal which may significantly alter the scope and methodology of the instructional design and evaluation process. The added time to collect and analyze data may interfere with the goal of ID efficiency. Critique An intriguing element of DBR is the synergy of research and instructional design practice within real-world instructional settings. DBR offers a means of placing the instructional design evaluation of a single intervention within a broader context of prior similar evaluations. However, a risk of a DBR approach is the “look what I did last summer” report of localized findings. Therefore, the ability to ground the evaluated instruction in prior theory and research and to offer valid findings which are generalizable beyond the specific instructional setting seems central to the DBR versus traditional research debate. Wang and Hannafin frame this within a discussion of meta-design knowledge and context-based knowledge which they note must transcend the specific design. Submitted: November 20, 2008
IDT 873 Abstracts: Facts Jennifer Maddrell Woloshyn, V. E., Willoughby, T., Wood, E., & Pressley, M. (1990). Elaborative interrogation facilitates adult learning of factual paragraphs. Journal of Educational Psychology, 82(3), 513-524. Research Purpose and focus. Woloshyn, Willoughby,Wood, and Pressley (1990) evaluate elaborative interrogation as a means of teaching facts. Two separate experiments were conducted. While the second was performed after the results from the first were known, the purpose was the same, namely to evaluate whether encouraging learners to activate and elaborate upon relevant prior knowledge facilitates fact acquisition. Methodology. In the first experiment, 80 college students participated in a single session study at a Canadian university. The students were randomly assigned to one of four experimental conditions, including 1) elaborative interrogation, 2) imagery, 3) self-reference, and 4) reading control groups. Those in the elaborative interrogation condition received accompanying “why” prompt questions on the slide and were asked to respond to the question. Those in the imagery condition were asked to “create an image” of the fact and associate it with the university. Selfreference subjects were asked to consider whether the shown fact would influence a decision to attend the school. Reading-control subjects were prompted to “read the sentence out loud at a rate that allows you to understand the fact.” The initial study consisted of four phases including 1) the instructions, 2) presentation of the study material, 3) tests which included both fact recall for each school and associative matching of the facts to the school, and 4) a post test interview which addressed the students’ attention to the instructions, the difficulty of the task, and prior knowledge about the school. The 20 students in each group viewed 43 slides which contained facts about the Canadian universities not well known to those not attending the schools. However, the instructions to each group differed based on the previously noted experimental condition. The second study included 240 college students from the same school, using the same materials, and incorporating the same four phases. However, the focus was on longer paragraphs beyond the single sentences in the first experiment. In addition, a free-recall memory test was included. Results and conclusions. While there were no significant differences between elaborative interrogation and imagery conditions, both groups performed significantly better than the reading control condition in the test measures. Further, in the first experiment, the self-reference subjects performed significantly better on the associative matching test than the reading control group, but the difference was not deemed significant on the fact recall. In post test interviews, subjects deemed elaborative interrogation and imagery as more “difficult” and requiring “extra efforts” than the reading control group. These findings are important as they suggest that elaborative interrogation yields superior results to the reading alone. Further, elaborative interrogation was as effective as the other elaborative procedures studied. Heuristics The results of these experiments suggest that designers should incorporate elaboration strategies when teaching facts. As seen in this experiment, prompting learners to elaborate beyond the presented fact (as in considering specific questions, creating mental images, or selfreflecting on the presented facts) provides greater support than simply presenting the fact alone. Page | 1 Submitted 20081001 IDT 873 Abstracts: Facts Jennifer Maddrell Critique The greatest strength of this research is the contribution of prescriptions for the teaching of facts. As noted, the results suggest instruction should encourage and foster elaborative strategies to support the learning of facts. In addition, the research sets the stage for further study on whether these strategies can be taught. Can learners be taught to use these strategies in a selfregulated manner? However, while the article presents a review of literature suggesting that elaborative interrogation strategies may involve more conscious and effortful memory processes, the articles falls short in explicitly evaluating the reported experimental results in the context of specific theory within either the discussion or conclusions. In other words, how do the results support or refute a specific theory being tested? Kuo, M., & Hooper, S. (2004). The effects of visual and verbal coding mnemonics on learning Chinese characters in computer-based instruction. Educational Technology Research and Development, 52(3), 23-34. Research Purpose and focus. The purpose of the research conducted by Kuo and Hooper (2004) was to evaluate the effects of visual and verbal mnemonics on memorization, as well as any differences in outcomes between self-generated versus supplied mnemonics. The study focused on the learning and recall of Chinese characters. Kuo and Hooper noted that Chinese characters contain both visual and symbolic meanings, yet many traditional instructional methods tend to ignore the underlying meaning of the character and instead focus on repeated copying of the character to improve recall. In the reported study, Kuo and Hooper questioned whether or not visual and verbal mnemonic strategies could help the learner to generate meaning based on the visual and semantic information of the characters and if those strategies could improve recall. They examined whether verbal or visual mnemonics for learning Chinese characters would benefit both immediate and delayed recall, as well as the relative efficiency and efficacy of self-generated versus experimenter-supplied mnemonics. Methodology. 92 English speaking high school students with no previous Chinese language knowledge volunteered to participate in the study. The students were randomly assigned to one of five treatment groups, including 1) translation in which students were presented with the Chinese character and the English translation and told to memorize the character’s meaning, 2) verbal mnemonics (experimenter-supplied), 3) visual mnemonics (experimenter-supplied), 4) dual coding which included both verbal and visual experimentersupplied mnemonics, or 5) self-generated mnemonics in which students were presented with the Chinese character and the English translation and encouraged to create memory aid such as a picture, written sentence, or an associating story. The instruction to all of the groups was done through a self-paced computer-based tutorial conducted during a regularly scheduled single class period. The tutorial for all groups included the same 30 characters, divided between concrete words (representing physical objects) and abstract words (without referents). Two posttests were also administered via computer and included the same, but shuffled multiple-choice questions. The first test was administered immediately following the tutorial and the second one week later. The study also evaluated the time on task during the tutorial to learn Page | 2 Submitted 20081001 IDT 873 Abstracts: Facts Jennifer Maddrell the 30 characters. The self-generated mnemonics were also collected, sorted by character, and analyzed. In addition, a short survey was conducted from a subsample of students reflecting a representative cross-section of ability to examine the students’ learning strategies. Results and conclusions. While participants scored higher on the immediate versus the delayed posttest, students who self-generated their mnemonics performed significantly better in the recall posttests and spent more time on task than those in the translation, verbal, visual, and dual coding conditions. Further, the survey responses suggested that the learners used the presented or self-generated mnemonics techniques and called upon the mental images to link the character with the English word. The analysis of the self-generated mnemonics indicated that fewer and less descriptive mnemonics were created for abstract words than for concrete words. In addition, the descriptions reflected the students’ Western cultural backgrounds and experiences. These results suggest that the learners linked the to-be-leaned information with prior knowledge. Heuristics The researchers suggest that the findings support the use of generative learning strategies that extend beyond just efficient information presentation. Strategies should encourage learners to self-generate meaningful linkages between the content and prior knowledge. However, while the findings suggest that self-generated relationships are more effective, they may be less efficient than provided relationships. Critique This experiment and article describing the research were refreshingly straightforward with no doubts as to the researchers’ questions, methodology, and suggested findings. The study furthers prior theory and research and offers suggestions for future research. Nice package. Tight bow. Page | 3 Submitted 20081001
Generative Strategy Abstract Running head: GENERATIVE STRATEGY ABSTRACT 1 Note Taking as a Generative Strategy Abstract Jennifer Maddrell Old Dominion University IDT 873 Advanced Instructional Design Techniques Dr. Morrison September 2, 2008 Generative Strategy Abstract Note Taking as a Generative Strategy 2 Overview Citing a large and conflicting body of prior research, Peper and Mayer (1986) suggest that three main hypotheses are forwarded by prior research on the effect of note taking on a learner’s cognitive processing, including 1) the attention hypothesis (note takers pay closer attention to the to-be-learned material), 2) the distraction hypothesis (note takers concentrate on the act of writing instead of listening), and 3) the generative hypothesis (note taking enables learners to actively relate material to existing knowledge). Peper and Mayer suggest that evaluations of both attention and distraction hypothesis have tended to focus on how much is recalled. In contrast, by focusing on the generative hypothesis within their reported experiments, the goal is to evaluate the difference in what is learned between note takers and non-note takers. Research Focus. Perry and Mayer (1986) focus on three generative hypothesis predictions. The first prediction is that note takers will perform better on far-transfer test measures (problemsolving) and worse on near-transfer test measures (verbatim recognition and fact recall). This is based on the assumption that note taking offers an opportunity for integration with existing knowledge, but the process of reorganizing the new information interferes with near-transfer verbatim recall of specific facts. Secondly, these results will be stronger for those unfamiliar with the material given the processing required to integrate and organize new information. Finally, the results associated with the note taking generative activity will be similar to those for other types of generative activities. Methodology. Two separate experiments were conducted to test these predictions. The first experiment involved a group of high school students while the second included college students at the University of California at Santa Barbara. To test the first hypothesis, Experiment 1 included only subjects unfamiliar with the to-be-learned topic. The students were divided equally between either a “notes” and “no-notes” group. The same video lecture was shown to each group. Afterward, the notes were collected from the “notes” group and the same posttest was administered to both groups. Recognition questions asking the students to identify sentences that occurred verbatim in the lecture were followed by fact retention and problem solving questions. To assess the second and third hypothesis, Experiment 2 included some subjects who were familiar with the topic and added a question-answering treatment group. The same materials and posttests were used for both experiments. Conclusions. In contrast to the attention hypothesis, the superior results of the “no-note” group to verbatim recognition measures does not support the prediction that note taking results in better total recall. Further, in contrast to the distraction hypothesis, the “notes” group performed better than the “no-note” group in some measures. However, significant differences existed between the measures of what was learned (far-transfer versus near-transfer measures) supporting the generative hypothesis. Note takers excelled on the far-transfer (problem solving) test measures. In contrast, “no-note” takers were more successful in near transfer verbatim and fact recall of information. Supporting the second prediction, the results in Experiment 2 were strong for learners unfamiliar with the topic, but not for familiar learners. Further, in support of Generative Strategy Abstract 3 the third prediction, the other tested generative activity (within the questioning-answering treatment) had similar results as note taking. Perry and Mayer (1986) viewed these results as support for generative theory. They concluded that the process of note taking (especially for those unfamiliar with the material) encourages the note takers to assimilate new information with past experience and make interconnections among pieces of information. Heuristics Based on the results of these experiments, learners should be offered the opportunity to take notes as a means of supporting the long term encoding of new information. This research suggests that the note taking process offers learners the opportunity for integration and organization of the new information with existing knowledge. However, this research also suggests that these results are more likely when the to-be-learned information is unfamiliar to the learner. Further, the process of re-organization and integration with prior knowledge involved in note taking may interfere with verbatim encoding of information and facts. Critique of Article A key strength of this research is the evaluation of note taking across three separate hypotheses, including attention, distraction, and generative theories. Further, the research highlights the advantages, as well as potential limitations, of note taking on encoding. However, it is important to note that the test measures were based on cued recall versus free recall. A possible source of future research would be to replicate the experiments with free recall test measures. In addition, the research analysis did not provide a qualitative analysis of the notes taken by students. An analysis of the qualitative features of the notes, such as the use of diagrams, would have helped to augment the findings. Also, as noted by the authors, this research provides an incomplete analysis of the relationship between note content and problemsolving performance. Generative Strategy Abstract References 4 Peper, R. J., & Mayer, R. E. (1986). Generative Effects of Note-Taking during Science Lectures. Journal of Educational Psychology, 78(1), 34.
IDT 873 Abstracts: Problem Solving Jennifer Maddrell Atkinson, R. K., Renkl, A., & Merrill, M. M. (2003). Transitioning From Studying Examples to Solving Problems: Effects of Self-Explanation Prompts and Fading Worked-Out Steps. Journal of Educational Psychology, 95(4), 774-783. Research Purpose and focus. Atkinson, Renkl, and Merrill (2003) examined the impact of fading and selfexplanation prompts on problem solving. Two separate experiments were conducted. While the second was performed after the results from the first were known, the purpose was to replicate the experiment in a more authentic setting. The three primary research questions focused on whether or not (a) backward fading (BF) results in better learning outcomes than exampleproblem (EP) pairs, (b) self-explanation prompts lead to better outcomes, and (c) an advantage exists when using both fading and self-explanation prompts. Methodology. In the first study, 78 undergraduate students took part in the 90 minute lab based study. In the second study, 40 highs school students volunteered to participate within their high school. In the first study, the students were randomly assigned to one of four treatment groups, including (a) BF only (worked examples in which all problem solutions are sequentially provided to the learner, but solution steps are gradually eliminated on each subsequent worked examples), (b) EP pairs only (worked examples followed by a problem-solving task), (c) BF plus prompting (BF condition with self-explanation prompts), and (d) EP pairs plus prompting (EP condition with self-explanation prompts). The second study did not include the EP pairs only or EP pairs plus prompting conditions. Otherwise, the instruments, scoring, and procedures for both studies were identical. The instructional treatment for all of the groups was conducted via a computer-based learning tool and involved probability word problems. The instruction included sets of worked examples and problems which ran in one of four modes; one mode for each noted condition, as described above. A demographic questionnaire and pretest to assess prior knowledge were conducted before the treatment. A posttest was administered at the conclusion of the session. In addition, time on task during the lesson was recorded. Results and conclusions. With regard to the original research questions, the research findings from the first experiment suggest that (a) backward fading resulted in better learning outcomes than the example-problem pairs, (b) self-explanation prompts led to better learning outcomes, but (c) there was no apparently advantage from the use of fading and self-explanation prompts. The results of the second study support the finding that self-explanation prompts with a backward fading example sequence supports learning in a time effective manner. Heuristics The results of these experiments suggest that instruction should incorporate a fading technique in which the problem solutions within worked examples are initially fully provided to the learner, but the solution steps are gradually eliminated as the learner’s skill development increases until only independent problem solving practice is utilized in the later stages of instruction. Further, the findings suggest that learning effectiveness and efficiency may be enhanced when this fading approach is combined with self-explanation prompting in which the learner is asked to consider the underlying principle(s) applicable to the problem. Critique In this study, the findings build on prior research on fading (from worked example study to problem solving) that suggests the importance of fading the solution steps in worked examples and progressing to independent problem solving as the learner’s skill level improves. In addition, Page | 1 Submitted 20081030 IDT 873 Abstracts: Problem Solving Jennifer Maddrell the study furthers research on the self-explanation effect which suggests that learning is enhanced when learners are prompted to select the underlying principles during problem solving. The researchers suggest these findings provide support for the importance of feedback following learner self-explanations. However, as noted by the researchers, it would be interesting to further explore the effect of providing feedback to learner self-explanations in less structured domains that the one studied here. In other words, what would be the outcome in domains where the principles involved in problem solving are less clear or less restricted? van Gog, T., Paas, F., & van Merrienboer, J. (2008). Effects of studying sequences of processoriented and product-oriented worked examples on troubleshooting transfer efficiency. Learning and Instruction, 18(3), 211-222. Research Purpose and focus. van Gog, Paas, and van Merrienboer (2008) evaluate the effect of process-oriented worked examples which include the rationale behind the presented solution comparing their use to product-oriented worked examples at both the beginning of training and at later stages. The purpose of the described study was to evaluate whether or not processoriented worked examples are more effective than product-oriented worked examples and whether an expertise-reversal effect occurs when process-oriented worked examples are provided after the learners are familiar with the underlying processes. Methodology. 82 teen aged students with no previous subject matter knowledge volunteered to participate in the study. The students were randomly assigned to one of four treatment groups, including: (a) product-product, (b) process-process, (c) product-process, and (d) process-product training sequences. The study was run in three sessions with participants equally distributed across conditions. The instructional treatment for all of the groups was done through both paper based materials and the TINA Pro software which provided learners with electrical circuit simulations from which either the worked-out solution (product-oriented) or worked-out solution, plus process information (process-oriented) was provided. The process oriented worked examples and product-oriented worked examples contained the same steps, but the process-oriented worked examples contained information about the underlying principles, as well as the systematic problem approach. A prior knowledge questionnaire was conducted first followed by material to familiarize the learners with the TINA system they would be using. Each session included two series of training examples. After learners studied the first series of training examples (either product- or process-oriented examples, based on their assigned condition) they completed test problems which required them to troubleshoot a malfunctioning electrical circuit simulation in the TINA system. The learners were also instructed to write down the faulty components on a pre-printed answer sheet and asked to select the reason for the fault from multiple-choice answers. In addition, the learners were asked to gauge their mental effort in studying and solving the problem giving the researchers a measure of mental effort, as well as the noted performance measures. The learners then studied a second series of examples (either product- or process-oriented examples, based on their assigned condition) followed by the test problems and mental effort rating. The study also evaluated the time on task during the tutorial. Results and conclusions. The results supported the researchers’ hypothesis that processoriented worked examples foster early learning and better efficiency as indicated by the learning and mental effort measures. In addition, in line with predictions, the process information Page | 2 Submitted 20081030 IDT 873 Abstracts: Problem Solving Jennifer Maddrell appeared to become redundant as indicated by lower performance measures on the second series which incorporated process-oriented worked examples. Heuristics The results of these experiments suggest that problem solving support for novices should begin with process-oriented worked examples which fully explain the rationale behind the solution and progress to product-oriented worked examples. As suggested by this and other cited studies, process-oriented worked examples can support learning up and until the process information is familiar to the learners at which point it becomes redundant and impedes learning. Critique The findings of this study provide support for prior research on cognitive load and the expertise-reversal effect. As noted, while process-oriented worked examples may initially support the novice learner, they may impede learning in more advanced learners or as the learner progresses. While this study provides support for sequencing of process- and product-oriented worked problems in simple problem solving exercises, the impact on complex problem solving is less clear. As acknowledged by the researchers, further research is needed to assess worked examples sequencing on more complex cognitive skills. Page | 3 Submitted 20081030
IDT 873 Abstracts: Attitudes Jennifer Maddrell Kardash, C. M., & Scholes, R. J. (1995). Effects of Preexisting Beliefs and Repeated Readings on Belief Change, Comprehension, and Recall of Persuasive Text. Contemporary Educational Psychology, 20(2), 201-221. Research Purpose and focus. Kardash and Scholes (1995) examined how preexisting beliefs interacted with reading of persuasive test to influence information recall and belief change. Citing prior research that suggest preexisting attitudes and beliefs influence how evidence is evaluated, Kardash and Scholes predicted that subjects whose preexisting beliefs and attitudes were consistent with the presented text would recall more causal explanations within the text, as well as more information overall. In addition, they predicted that those who read the text twice would remember more information than those who read the text only once. Finally, they predicted that the persuasive text would influence all subjects, but to a greater extent in those whose preexisting beliefs and attitudes were consistent with the text. Methodology. 61 undergraduate students enrolled in an educational psychology class received credit for their participation in this study. The students were randomly assigned to one of two treatment groups, including (a) a one-read, or (b) a two-read group. Entry beliefs were measured based on 9-point Likert-type scale assessment of the extent to which subjects agreed or disagreed with a variety of offered causes of how AIDS could be transmitted. Post-treatment beliefs were similarly measured. All learners reviewed the same 1,195 word text based passage about causes of AIDS. Both groups returned two days later. Those in the one-read group completed an unrelated exercise while the two-read group read the exact passage a second time. Time spent reading the text was measured in both sessions. One week later, all subject returned for a free recall text and the post-beliefs test. Results and conclusions. Results supported the prediction that beliefs about the controversial topic effect what is recalled about a persuasive text on the topic. Those with beliefs consistent with the text remembered marginally more causal, as well as less central information than those with less consistent entry beliefs. In addition, causal arguments promoted belief change in all subjects, but more so for those with similar preexisting beliefs to the text. Finally, contrary to predictions, the repeated reading did not influence the overall amount or type of information recalled. Heuristics The results of these experiments suggest that a learner’s entry beliefs and attitudes about causal information regarding a controversial topic may influence how the learner recalls and is persuaded by the to-be-learned material. If the information is consistent with entry beliefs, the learner may be more likely to recall or be persuaded by the material than those with entry beliefs that are inconsistent with the presented instructional material. Critique This study provides support for prior research that suggests preexisting beliefs serve as a schema which influences how new persuasive information will impact belief change and recall. Yet, as noted by the authors, this study focused on the subjects’ beliefs about causes of AIDS, not Page | 1 Submitted 20081114 IDT 873 Abstracts: Attitudes Jennifer Maddrell their attitudes towards AIDS and those with AIDS which may or may not influence the reported results. Brannon, L. A., Tagler, M. J., & Eagly, A. H. (2007). The moderating role of attitude strength in selective exposure to information. Journal of Experimental Social Psychology, 43(4), 611-617. Research Purpose and focus. Brannon, Tagler, and Eagly (2007) examined whether attitude strength influences information selection. Citing prior research that suggests people seek out and prefer to receive information that is consistent with their attitudes, Brannon, Tagler, and Eagly predicted that selective exposure would be more pronounced in those with more strongly held attitudes than for those with weakly held attitudes. Methodology. In a series of three studies following roughly the same methodology as the first, their prediction was tested. In the first study, 270 students enrolled in an undergraduate psychology program were recruited to participate. They were randomly assigned to one of two treatment groups, including (a) a one read, or (b) a two-read group. Entry attitudes toward social issues were measured based on a questionnaire that assessed attitude position and strength regarding social issues. Participants’ attitude positions were measured on a 7-point scale. Attitude strength was also measured on a 7-point scale assessing how important the issue was, how sure they were of their position, how central their attitudes were to their self-concepts, how likely they were to change their attitudes, and how much knowledge they possessed on the issue. Several weeks after completing the entry questionnaire, participants engaged in a selective exposure task in which they reviewed a list of ten article titles and abstracts containing two opposite stances toward five difference social issues. For each article, the participants ranked on a 9-point scale how desirable it would be for them to read the article. The participant’s choice of either an attitudinally consistent or inconsistent selection was then measured. Results and conclusions. Results supported the prediction that attitude strength relates to selective exposure. Stronger attitudes were associated with increased preference for attitudinally consistent article titles. Heuristics The results of these experiments suggest that the strength of the learner’s entry attitudes about a topic may influence what information the learner selects on the topic. If the learner has strongly held attitudes on a topic, he or she may seek out information that is consistent with their entry attitudes. Critique This study provides support for prior research that suggests preexisting attitudes influence information selection. These research findings have important implications in instructional settings where learners are free to select instructional content. If learners are less inclined to select material that is in opposition to their entry attitudes, will they select Page | 2 Submitted 20081114 IDT 873 Abstracts: Attitudes Jennifer Maddrell information that gives a balanced perspective on the topic? Given the potential for attitudinally influenced selection, should learning material selection be under the direction of the instructor? Also, how does instructor’s or instructional designer’s entry attitude influence selection of material for a class? Page | 3 Submitted 20081114
IDT 873 Abstracts: Feedback Jennifer Maddrell Olina, Z., & Sullivan, H. (2004). Student self-evaluation, teacher evaluation, and learner performance. Educational Technology Research and Development, 52(3), Research Purpose and focus. Olina and Sullivan (2004) examined the effect of student selfevaluation and teacher feedback on learning. Their research focused on the comparative and combined performance effects of self- and teacher-evaluation, as well as the effect of both on student and teacher attitudes. Olina and Sullivan predicted that teacher evaluation would improve student performance to a greater extent that self-evaluation. Further, they predicted that selfevaluation would result in better performance and engagement than no-evaluation. Methodology. 341 high school students in Latvia took part in the study. Learners came from 16 classes which were selected from eight schools in different Latvian regions, including a diverse mix of rural and urban areas and of socio-economic backgrounds. Each of the eight schools were stratified based on ability (based on ninth grade standardized Latvian exams) and grouped into either the four higher or four lower ability schools. Each of the four schools in each group were then randomly assigned to one of four treatment groups, including (a) no evaluation, (b) self-evaluation, (c) teacher-evaluation, and (d) self-plus-teacher evaluation groups.. Eight teachers taught two classes each. Over the course of a six week term consisting of two 40 minute class periods per week, all students took the same 12 lesson instructional program about experimental research design which included both a student book and teacher guide. Students in all classes conducted the same experiments, produced written reports as required by the instruction, and were introduced to the project rating scale, a descriptive rubric for evaluating the written projects. However, students in the self-evaluation group formally selfevaluated their own work based on the project rating scale. Students in the teacher-evaluation group were provided written feedback from their teachers based on the same project rating scale. Students in the self-plus-teacher group formally self-evaluated their own work and received written teacher feedback. Students in the no-evaluation (control) group received no formal feedback from the teacher and they were not asked to formally evaluate their own work. Performance measures included ratings of the students’ final projects and posttest scores. In addition, student and teacher attitudes were measured in surveys after the course. Results and conclusions. While there were no significant differences between treatment groups on the posttest scores, the teacher-evaluation and self-plus-teacher groups had significantly higher project scores than the no evaluation group and the self-evaluation groups. Further, in both self-evaluation groups, over 90% of students rated their projects higher than the experimenter-based rating in the final projects. Students in both groups with formal selfevaluation reported more positive attitudes toward the program as compared to the other groups, but both students and teachers preferred teacher-evaluation and felt it provided a more valuable evaluation. Heuristics The results of these experiments suggest that incorporating formal self-evaluation may increase a learner’s confidence in his or her future performance. However, teacher-evaluation alone or combined with self-evaluation is more likely to improve learner performance over no evaluation or self-evaluation alone. Page | 1 Submitted 20081104 IDT 873 Abstracts: Feedback Jennifer Maddrell Critique This study, conducted over a six week term, provides support for prior research that suggests teacher feedback improves student performance. While no significant differences were found in posttest measures, the results suggest that teacher feedback may provide superior learning outcomes (based on other than test application measures) as compared to no evaluation feedback or self-evaluation feedback. Yet, the results do indicate value in learner self-evaluation in terms of increased learner self-control and self-confidence. Page | 2 Submitted 20081104
IDT 873 Abstracts: Rules and Principles Jennifer Maddrell Ross, S. M., & Rakow, E. A. (1981). Learner Control versus Program Control as Adaptive Strategies for Selection of Instructional Support on Math Rules. Journal of Educational Psychology, 73(5), 745-53. Research Purpose and focus. Ross and Rakow (1981) sought to extend previous research on the effect of adaptive instructional strategies on math rule attainment. They predicted that an adaptive design strategy which varied the number of practice examples based on the student’s pretest score would improve rule attainment over a nonadaptive strategy. Methodology. 124 undergraduate students volunteered to participate in the study. The students were randomly assigned to one of four treatment groups, including 1) program control (the number of examples were adapted to pretest scores), 2) learner control (the number of examples were selected by subjects), 3) nonadaptive (a constant five examples per rule were presented to learners), and 4) lecture (the nonadaptive program was presented through lecture format) groups. The learning program consisted of presentation of five introductory math rules, including inequalities, factorials, exponents, order of operations, and summation. Presentation for each rule included the rule definition, as well as complete and incomplete examples. Other than the lecture group, the instruction for all of the groups was done through self-study booklets in a lab session with three or less subjects who worked with a single proctor to complete a separate booklet for each rule. Those in the lecture group received the identical content as the nonadaptive self-study group within a presentation given by an instructor. Achievement assessment included a pretest taken a few weeks prior to the instructional phase, a posttest taken after completion of each rule booklet, and a delayed posttest given a few weeks after the instruction. The achievement tests consisted of open ended test questions. The study also incorporated an attitude survey which was taken after the completion of the last rule booklet. Results and conclusions. The results indicated statistically significant differences in posttest achievement based on the treatment strategy. The immediate posttest results indicated the mean score of the program control group was higher than all other treatments while both the lecture and the nonadaptive groups performed better than the learner control group. In the delayed posttest, the differences were the same, but more pronounced. However, the noted results for the learner control group varied based on entrance ability. While low entrance ability students performed well under program control and poorly under learner control, high entrance ability students performed well under both. Further, there was no significant difference among treatment groups on the attitude survey scores. Heuristics The results of this study suggest value in adapting instructional presentation based on learner need. Based on the findings of this study, increasing the presentation of examples for those with low entrance ability and decreasing presentation for those with high entrance ability may improve rule attainment. Page | 1 Submitted 20081015 IDT 873 Abstracts: Rules and Principles Jennifer Maddrell Critique The results of this study support prior research findings which suggest value in modifying the amount of instructional support based on individual need. Further, this study suggests entrance ability assessment may be an effective means of gauging the amount of needed instructional support. In addition, this study suggests that for low entrance ability learners, learner control in gauging an optimal presentation may not be an effective strategy. However, it is important to note that learners in the learner control treatment were required to ask the proctor for additional examples which may have made the learners uncomfortable and less likely to ask for additional examples. Wiley, J., & Voss, J. (1999). Constructing arguments from multiple sources: Tasks that promote understanding and not just memory for text. Journal of Educational Psychology, 91(2), 301-311. Research Purpose and focus. Wiley and Voss (1999) evaluated the effect of student generated arguments on learning historical subject matter. Two separate experiments were conducted. The purpose and methods were similar, namely to evaluate whether argument writing tasks promoted a deeper understanding of the to-be-learned material than other narrative, summary, or explanation writing tasks. Methodology. 24 undergraduate students participated in the second study. The students were randomly assigned to one of four treatment groups, including 1) a narrative group, 2) a summary group, 3) an explanation group, and 4) an argument group. All groups received the same information about Ireland from 1800 to 1850, either in a paper based or computer based newspaper article. After reading the material, students were asked to assume the role of historian and, based on their assigned treatment, develop either a written narrative, summary, explanation, or argument about what produced the significant changes in Ireland’s population between 1846 and 1850. Learners were given approximately 30 minutes to read the material and complete their reflective writing task. After the writing task, participants were assessed based on three learning measures including 1) a sentence verification task (10 true / false questions), 2) an inference verification task (determining if statements were true on the basis of the presented information) and 3) a principle identification task in which students indicated how similar the causes of the Irish Potato Famine were to other historical situations. In addition, the sentences in each student’s writing task were classified based on whether the sentences were a) borrowed from the original source, b) transformed, or c) added information. Results and conclusions. The results indicate little difference between whether students read the newspaper article from the computer or paper. Further, there were no significant differences across treatment groups in the recognition of sentences. However, those in the argument writing treatment demonstrated better identification of inferences and generated essay sentences with more transformed and causal information. In contrast, the other writing tasks resulted in essays with more borrowed and added sentences and less causal information. Heuristics The results of these experiments suggest the nature of the reflective writing task impacts the learner’s attainment of the to-be-learned material. Based on the conclusions of the researchers, writing tasks which require the learners to form and support arguments about causes Page | 2 Submitted 20081015 IDT 873 Abstracts: Rules and Principles Jennifer Maddrell of events may lead to better recognition of the underlying principles in the subject matter than can be achieved in writing tasks that focus on just narrative summaries or explanations. Critique In terms of the teaching of rules and principles, this study offers support for what Wiley and Voss term knowledge-transforming versus knowledge-telling tasks. Wiley and Voss argue that the findings of their study suggest argument writing is a knowledge-transforming task which requires learners to go beyond mere recall of principles to the construction and subsequent defense of arguments to explain the root causes of events. They suggest that through argument writing tasks the subject matter is learned at a deeper level given that learners must retrieve and relate more information in order to justify their positions. While this study was small in scope and scale, it offers an intriguing stepping stone for future research on the effects of various types of reflective writing tasks on rule or principle attainment. Page | 3 Submitted 20081015
IDT 873 Abstracts: Feedback Jennifer Maddrell Olina, Z., & Sullivan, H. (2004). Student self-evaluation, teacher evaluation, and learner performance. Educational Technology Research and Development, 52(3), Research Purpose and focus. Olina and Sullivan (2004) examined the effect of student selfevaluation and teacher feedback on learning. Their research focused on the comparative and combined performance effects of self- and teacher-evaluation, as well as the effect of both on student and teacher attitudes. Olina and Sullivan predicted that teacher evaluation would improve student performance to a greater extent that self-evaluation. Further, they predicted that selfevaluation would result in better performance and engagement than no-evaluation. Methodology. 341 high school students in Latvia took part in the study. Learners came from 16 classes which were selected from eight schools in different Latvian regions, including a diverse mix of rural and urban areas and of socio-economic backgrounds. Each of the eight schools were stratified based on ability (based on ninth grade standardized Latvian exams) and grouped into either the four higher or four lower ability schools. Each of the four schools in each group were then randomly assigned to one of four treatment groups, including (a) no evaluation, (b) self-evaluation, (c) teacher-evaluation, and (d) self-plus-teacher evaluation groups.. Eight teachers taught two classes each. Over the course of a six week term consisting of two 40 minute class periods per week, all students took the same 12 lesson instructional program about experimental research design which included both a student book and teacher guide. Students in all classes conducted the same experiments, produced written reports as required by the instruction, and were introduced to the project rating scale, a descriptive rubric for evaluating the written projects. However, students in the self-evaluation group formally selfevaluated their own work based on the project rating scale. Students in the teacher-evaluation group were provided written feedback from their teachers based on the same project rating scale. Students in the self-plus-teacher group formally self-evaluated their own work and received written teacher feedback. Students in the no-evaluation (control) group received no formal feedback from the teacher and they were not asked to formally evaluate their own work. Performance measures included ratings of the students’ final projects and posttest scores. In addition, student and teacher attitudes were measured in surveys after the course. Results and conclusions. While there were no significant differences between treatment groups on the posttest scores, the teacher-evaluation and self-plus-teacher groups had significantly higher project scores than the no evaluation group and the self-evaluation groups. Further, in both self-evaluation groups, over 90% of students rated their projects higher than the experimenter-based rating in the final projects. Students in both groups with formal selfevaluation reported more positive attitudes toward the program as compared to the other groups, but both students and teachers preferred teacher-evaluation and felt it provided a more valuable evaluation. Heuristics The results of these experiments suggest that incorporating formal self-evaluation may increase a learner’s confidence in his or her future performance. However, teacher-evaluation alone or combined with self-evaluation is more likely to improve learner performance over no evaluation or self-evaluation alone. Page | 1 Submitted 20081104 IDT 873 Abstracts: Feedback Jennifer Maddrell Critique This study, conducted over a six week term, provides support for prior research that suggests teacher feedback improves student performance. While no significant differences were found in posttest measures, the results suggest that teacher feedback may provide superior learning outcomes (based on other than test application measures) as compared to no evaluation feedback or self-evaluation feedback. Yet, the results do indicate value in learner self-evaluation in terms of increased learner self-control and self-confidence. Page | 2 Submitted 20081104
Behavioral Strategy Abstract Running head: BEHAVIORAL STRATEGY ABSTRACT 1 Behavioral Strategy Abstract: Self-Pacing Versus Instructor-Pacing Jennifer Maddrell Old Dominion University IDT 873 Advanced Instructional Design Techniques Dr. Morrison September 8, 2008 Behavioral Strategy Abstract Self-Pacing Versus Instructor-Pacing 2 Overview Morris, Surber and Bijou (1978) report on research conducted to compare achievement, student satisfaction, and retention between self-paced and instructor-paced personalized systems of instruction (PSI). While noting that one of the key features of PSIs is the ability for learners to self-pace, the authors cite prior research that suggests students who are allowed to self-pace may be more likely to procrastinate or withdraw from the course entirely. These finding have led some to incorporate instructor-paced schedules into the PSI. However, what had been less clear in prior research is the impact of self-pacing on learner achievement (both short term and longer term following course completion) and learner satisfaction with the learning experience. Research Purpose. The purpose of the reported study is to compare progress rates, withdrawal rates, achievement, satisfaction, and longer term retention between learners completing selfpaced or instructor-paced PSI. The researchers set out to extend prior research by focusing on the effect of pacing on these measures. Methodology. All 149 students enrolled in an introductory child development class were randomly assigned to either self-paced (S-P) or instructor-paced (I-P) PSI. The syllabi, course materials, and assessments were identical for both groups. Within each of the 15 units of the PSI, all learners were required to either achieve 90% mastery within a 10-item short-answer essay quiz and oral examination at a testing center or take a make-up quiz until 90% mastery was achieved. Learners in the S-P condition were able to complete all 15 required units within the PSI at their own pace within the semester. Semester grades for the S-P group were based solely on the number of units mastered. In contrast, the I-P students were subject to a grading scheme that could result in a one letter grade drop if the student did not complete at least one unit of material each week. To evaluate and compare pacing, the semester was divided into five 15 day increments. For the purpose of measuring student achievement, a 53 item multiple-choice pre and post-test based on a few items from each unit was administered to all learners. In addition, nine months after the semester, students were asked to return (with compensation) for a follow-up test. They were all informed that the pre and post-tests would not impact final grades. A course evaluation questionnaire addressed student satisfaction with the course. Conclusions. As shown in prior research, the completion rates between the S-P and I-P groups were not the same. I-P learners progressed through the material at a more even rate throughout the semester, while S-P learners completed fewer units in the initial time periods as compared to the latter time periods. However, there were no statistically significant differences in course withdrawal rates, final grade distributions, course evaluations, or achievement measures between the two groups. Yet, there were statistically significant differences between the number of repeated quizzes during the semester and the follow up retention scores. S-P students repeated 4.1% of their quizzes, while I-P students repeated 7.2% of theirs. While the S-P learners’ delayed rate of completion may signal cramming or procrastination, self-pacing did not appear to negatively impact course achievement or Behavioral Strategy Abstract 3 withdrawal rates which were two areas of concern in prior PSI practice and research. Further, the S-P learners’ ability to control pacing may have aided in their longer term retention of the material. Heuristics Based on the results of this experiment, lesson pacing by the instructor or designer may reduce cramming and procrastination, but may do nothing to improve learner achievement, overall satisfaction, or course retention. Further, allowing learners to self-pace may improve their longer term retention of the material. However, it is important to note that these results are based on otherwise rigid instructional parameters in which learners were required to complete highly structured lesson units during the single semester. Therefore, while the learners were allowed the ability to complete the units at their own pace during the course of the semester, they otherwise had little control. As such, it is unclear if this heuristic would apply to a more flexible learning environment in which the learners had more choice, such as in the selection or sequencing of instructional content. Critique of Article A key strength of this research is the direct comparison of pacing on achievement, retention, satisfaction, and longer term retention within an otherwise highly structured instructional setting. The research methodology appears effective at comparing the two types of PSI pacing schemes. However, as noted above, these results are based on otherwise rigid instructional parameters. It is unclear if these results would be replicated in situations where more learner choice and control is available. In addition, the research has done little to further an evaluation of the effect of PSIs on a broad range of learning outcomes. In reporting on learning achievement, the authors do not elaborate on what was learned. Did the PSI lead to anything more than basic recall and retention of facts or concepts? Are the learners able to apply the instruction in diverse contexts? Unfortunately, the authors offer the results as a demonstration of learning achievement, but it is unclear from the results what precisely was learned. Behavioral Strategy Abstract References Morris, E. K., Surber, C. F., & Bijou, S. W. (1978). Self- versus instructor-pacing: Achievement, evaluations, and retention. Journal of Educational Psychology, 70(2), 224-230. 4
IDT 873 Abstracts: Procedural Skills Jennifer Maddrell Blakemore, C. L., Hilton, H. G., Harrison, J. M., Pellett, T. L., & Gresh, J. (1992). Comparison of Students Taught Basketball Skills Using Mastery and Nonmastery Learning Methods. Journal of Teaching in Physical Education, 11(3), 235-247. Research Purpose and focus. Blakemore, Hilton, Harrison, Pellett, and Gresh (1992) analyzed mastery learning as a means of teaching psychomotor skills. The study’s purpose was to compare mastery and nonmastery learning methods as means of teaching basketball skills. Methodology. Three physical education classes of seventh-grade boys were randomly selected as treatment and control groups. One treatment group of 39 boys was taught using a mastery model while the other treatment group of 32 boys was taught using nonmastery methods. The control group of 33 boys was taught soccer and hockey. The same instructor taught the three classes. Instruction lasted six weeks for 50 minutes a day five days a week. Students in the mastery treatment were taught using Bloom’s mastery learning model which is based on both individual student need and group mastery. The mastery group’s routine included warm-ups (5 minutes), diagnostic tests (10 minutes), corrective and enrichment practice with feedback (10 minutes), and competitive game play (10 minutes). Each session’s diagnostic test confirmed whether or not 80% or more of the class had achieved mastery which was the trigger to move onto a new skill unit. The tests also served as a means of evaluating and providing feedback about a student’s progress toward skill attainment. Instruction in basketball skills in the nonmastery class included the same skills taught in the same order, but followed a predetermined instructional plan and schedule that included warm-ups, skill instruction, practice, and game play with timing based on the planned schedule. Both isolated skills (dribbling, shooting, and layups) and game play ability (based on shots taken, shots made, and turnover game statistics) were assessed in pretests and posts tests for students in both groups. Results and conclusions. Pretests confirmed that the groups were of equivalent starting skill ability. From pretest to posttest, only the mastery group demonstrated statistically significant improvement in the three assessed skills. The control and nonmastery groups improved only in the ability to dribble, and the nonmastery group decreased in shooting and layups. In terms of game play, there were no significant differences between groups from pretest to posttest. Heuristics The results of this experiment suggest value in incorporating periodic diagnostic evaluation and feedback when teaching skills. As is suggested by Bloom’s mastery learning model and the results of this test, frequent evaluation with corrective and enrichment feedback provides greater instructional support than simply presentation and practice alone. Critique This study is straightforward and clearly outlined in the research report … FINALLY! These findings are important as they suggest individualized feedback with information about learner performance results and suggested corrective strategies delivered immediately following skill testing may enhance skill development beyond presentation and practice alone. However, it is unclear how the results would be impacted if learners of mixed ability were combined within the mastery session. Would higher ability learners become bored waiting for lower ability learners to reach mastery? Would lower ability learners become frustrated and embarrassed about holding back the other members of the group? Page | 1 Submitted 20081023 IDT 873 Abstracts: Procedural Skills Jennifer Maddrell Harrison, J., Preece, L., Blakemore, C., Richards, R., Wilkinson, C., & Fellingham, G. (1999). Effects of two instructional models - Skill teaching and mastery learning - On skill development, knowledge, self-efficacy, and game play in volleyball. Journal of Teaching in Physical Education, 19(1), 34-57. Research Purpose and focus. As a follow up to the 1992 study discussed above, Harrison, Preece, Blakemore, Wilkinson and Fellingham (1999) again analyzed mastery learning as a means of teaching psychomotor skills. However, in this study, volleyball skills were the subject of the instruction and Mastery Learning was compared to a Skill Teaching method. Beyond skill attainment, knowledge and self-efficacy measures were also compared. Methodology. 182 students including both males and females in six college volleyball classes participated in the study. Based on a four skill (set up, passing, serving, and spike) pretest, the students were stratified into high, medium, and low ability groups for analysis only. While all students participated in the classes as enrolled, only the high and low skilled learners (147 in total) were included in the analysis. Given prior studies in which the no instruction intervention control group showed no improvement, a control group was not included. Instruction was taught by three graduate assistants who each taught two classes, one under a Mastery Learning model and one under a Skill Teaching model. Each of the six volleyball classes was randomly assigned to one of the six courses and was taught under either the Mastery Learning model or Skill Teaching model. Instruction lasted 16 weeks with class sessions held two days a week. Students in the mastery treatment were taught using Bloom’s mastery model which is based on individual student need and group mastery. Each mastery group’s class sessions included formative testing, corrective and enrichment practice with feedback, and competitive modified game play. The formative tests confirmed whether or not 80% or more of the class had achieved mastery which was the trigger to move from Subunit I (forearm pass, set, overhand serve, spike, and mini-volley games) to Subunit II (full court games, 4-2 offense, player up defense, serve / receive, block, spike, and dive). While the focus of the Mastery Learning model was frequent diagnostic tests followed by feedback, the focus of the Skills Teaching model was hands on practice in modified game play. Instruction using the Skill Teaching model was sequenced using Rink’s system in which skills were taught using modified equipment (lower nets, non-standard balls) in classes that included warm-up, skill practice, and modified games (simplified rules and extra points for targeted skill attainment). The same skills were taught, but followed a predetermined instructional plan and schedule. Pretest, midterm, and posttest measures of isolated skills (set-up, passing, serving, and spike) were assessed. In addition, self-efficacy was assessed following the pretest, midterm, and post test assessments. A 63 item objective knowledge test on techniques, rules, and strategies was conducted at the end of the term with mastery level set at 80%. Results and conclusions. While significant pretest and posttest differences were found in both groups, the major finding was that Skill Teaching and Mastery Learning produced similar levels of improvement in the measures of isolated skill attainment, game play, self-efficacy, and knowledge. Given the few significant differences between outcomes following the use of the two Page | 2 Submitted 20081023 IDT 873 Abstracts: Procedural Skills Jennifer Maddrell models, the researchers concluded that there was no compelling finding to suggest one model over the other. Heuristics The results of these experiments suggest that designers should incorporate diagnostic evaluation followed by feedback when teaching skills. In addition, including frequent hands on practice using modified equipment and rules can help learners to build skills over time. Critique As with the first reported study, this study is straightforward and clearly outlined … yeah! These findings are important as they add to the understanding of mastery learning as a method to teach skills from the 1992 study. When considered as an extension of the 1992 study, this study suggests that both options are preferable to only presentation and practice alone. However, it is unfortunate that the researchers did not directly discuss and compare these methods and results with the 1992 study’s methods and findings. Suggestions for the similarities and differences between the two studies were not explored. Further, the researchers did not fully explore the connection between gender and skill difference and the effect that may have had on the participants in the study. The researchers noted that the majority of the high skilled group members were male and the majority of the low skilled group members were female and both groups participated together in the classes. Could gender and the comingling of skill levels within the treatment groups have played a more significant role than the researchers discussed? Did the girls (of lesser skill ability) feel intimidated playing with boys (of higher skill ability)? Page | 3 Submitted 20081023
Assignment Summary:The Blog Journal is prepared as a subset of articles reviewed for P540 within a personal blog regularly maintained to collect information and reflect on themes related to Instructional Systems Technology. Articles reviewed for the Blog Journal project are referenced in the last section of this report, along with the Blog Journal article summaries.The following sections highlight the learning outcome of this assignment, including the opportunity to:
Explore education information resources and learning themes during the article selection process,
Prepare personal reflections on the learning theories and educational design principles proposed in the articles, and
Provide and receive peer feedback.
Article Selection:
The article selection process provided an opportunity to explore both the education information resources available to distance students, as well as diverse learning themes.
Resources:The article selection process led to various learning and education resources for distance students, including.
<!--[if !supportLists]-->1.<!--[endif]-->Online Library Resources:The Indiana University Bloomington (IUB) Library offers a wealth of online resources for distance students who are unable to physically access the IU Library system.Several articles in the Blog Journal project were retrieved from the databases at IUB Libraries, including the Education Resources Information Center (ERIC) and the Educational Full Text (Wilson Web) database in which several full journal articles were used.
<!--[if !supportLists]-->2.<!--[endif]-->Industry and Trade Associations (Web Sites and Journals):Publications from Association for Educational Communications and Technology (AECT), Educause, ASTD, and The Sloan Consortium were used.
<!--[if !supportLists]-->3.<!--[endif]-->Internet Searches:Internet searches led to many article review options available on the Internet (including those bookmarked here). Many publications on college and university School of Education web sites provided good options, as did the full text journal articles available on Google Scholar.
<!--[if !supportLists]-->·<!--[endif]-->Themes:Three major themes emerged within the chosen articles.This pattern was noted early on inpeer feedback from Kenneth K. Taken in whole, the themes within the articles center on:
<!--[if !supportLists]-->3.<!--[endif]-->Learning Across Generations
Reflections:
Based on the major themes of 1) constructivism, 2) online learning and 3) learning across generations, the following highlight important reflections regarding each theme.
(1) Constructivism:While articles were not selected for their assessment of specific learning theories or strategies, constructivism became a pervasive theme.Many of the articles provide support for constructivism in both online and adult learning and focus on the importance of creating and supporting:
personal ownership and reflection throughout the learning process,
social interactions,
flexibility and options,
communities of practice, and
learning communities.
(2) Online Learning: Constructivist approaches are presented in the articles that address (a) design and support strategies for online learning and (b) media options to facilitate online learning.
Design and Support:Goa, Baylor and Shen (2005) propose improved design to provide: (1) social structure (shared context) and (2) collaboration (knowledge construction) in an online learning environment.Designer and teacher support for learning in an online learning environment focuses on the importance of establishing and encouraging interaction with peers, teachers, the institution and the course interface as noted by both Swan (2004) and Bray (2004).
Media:Technical innovations have created numerous tools to support online learning.Alexander (2006) provides an excellent media review of many Internet based tools which allow discovery learning, collaboration, learner reflection and practice, as well as peer and teacher feedback.
(3) Learning Across Generations: Several articles raise questions about learning across generations, including: Is there a difference in how students within different generations learn, especially when media and technology facilitate learning?What are the key strategies to motivate learners in different generations? How can knowledge from one generation be shared and transferred to other generations (shown to be an important concern in corporate training)?The articles and blog reflections highlight aspects of the Net Generation, Adult Learners Intergenerational Knowledge Transfer that address these questions.
The Net Generation:There is much discourse about engaging or motivating learners in the so-called “Net Generation” or those who were raised in an age of the Internet and other interactive media.McNeely (2005) suggests the need to “edu-tain” these learners that are used to fast paced and highly interactive entertainment. Prenske (2005) notes in his article that education (in this era of the Xbox and the iPod) needs a complete overhaul in order to engage students so that they are no longer "bored" with school.
However, there are problems with these arguments: 1) members of all generations can become “bored” when learning, 2) learners need to work very hard to stay engaged and challenged, regardless of the learning environment, and 3) education should never be confused with entertainment as the goals are completely different.
Adult Learners:Dobrovolny (2003) and Thoms (2001) support constructivist approaches for adult learners and suggest that adult learners benefit from a learning environment that:
<!--[if !supportLists]-->·<!--[endif]-->encourages past experiences
<!--[if !supportLists]-->·<!--[endif]-->is collaborative between instructor-student and student-student,
<!--[if !supportLists]-->·<!--[endif]-->provides options to explore the material, and
<!--[if !supportLists]-->·<!--[endif]-->incorporates “experiential activities” that allow reflection and are based on authentic experiences.
Drawing on adult learners’ relatively greater level of “experience” is shown by both authors to be an effective adult learning strategy. Adults are motivated to participate when learning activities support their prior experiences and are relevant to their current job or future objectives.Adults draw on those experiences, compare and contrast, and tie the pieces together to create context when constructing knowledge.
Intergenerational Knowledge Transfer:Harris (2006) highlights ways in which one generation of workers can assist in the training and development of the generations of workers that follow.Suggestions include constructivist approaches, such as embracing communities of practice and mentorship programs.Similar recommendations are made by Choi (2006) as part of an alternative learning model for corporate training.
Partner Feedback:
Partner feedback was a valuable part of the Blog Journal project.Kenneth K. made interesting article selections and created insightful commentary in his blog, while also providing thoughtful peer feedback. Kenneth’s article selections focused on entirely different themes providing an opportunity for personal reflection and feedback on topics such as Bandura, Skinner and English as a Foreign Language (EFL).
Kenneth’s peer feedback provided insight to the adult and online learning themes via his experience as an adult online learner in the Indiana University distance Masters program, as well as his experiences as an English as a Foreign Language (EFL) teacher. Within his comments as a reviewer, he:
<!--[if !supportLists]-->·<!--[endif]-->challenged the idea of being able to create a “community” within an online learning environment, as well as the necessity to create one,
<!--[if !supportLists]-->·<!--[endif]-->shared that, as an adult leaner, he draws on life experiences in his process of learning and sees the same when he teaches adults,
<!--[if !supportLists]-->·<!--[endif]-->supported the “constructivist” approaches regarding communities of practice as a learning model for corporate training, but notes elsewhere that some adults (for example, Japanese adults in EFL) may prefer a more teacher-centered approach,
<!--[if !supportLists]-->·<!--[endif]-->questioned if a model for younger learners would be similar to adult learners, and
<!--[if !supportLists]-->·<!--[endif]-->disputed the argument that education should “entertain” in order to engage learners.
Bray, M., Carter-Wells, J., Glaeser, B, Ivers, K, Lee, J., Street, C. (2004). Discovering the Meaning of Community In An Online Master's Degree Program. Association for Educational Communications and Technology, Washington, DC.Presented at Association for Educational Communications and Technology, 27th, Chicago, IL.October 19-23, 2004. Retrieved from: http://www.eric.ed.gov/ERICDocs/data/ericdocs2/content_storage_01/0000000b/80/2b/c9/6b.pdf
Gao, H., Baylor, A. L., & Shen, E. (2005). Designer Support for Online Collaboration and Knowledge Construction. Educational Technology & Society, 8(1), 69-79.Retrieved from http://ritl.fsu.edu/papers/gao_baylor_shen.pdf
Thoms, Karen. (2001) They're Not Just Big Kids: Motivating Adult Learners. Presented at the Annual Mid-South Instructional Technology Conference (Murfreesboro, TN, April 8-1-, 2001). 11. Retrieved from http://www.mtsu.edu/~itconf/proceed01/22.pdf
The movies reviewed for this project are To Sir, with Love (TSWL) and Good Will Hunting (GWH).On the surface, these movies appear to be very different.The time period (1960’s versus 1990’s), setting (high school versus therapist’s office) and relationship between the central characters (student /teacher versus therapist / patient) are all different.However, the movies share many common themes.
At the beginning of both movies, the central characters are struggling with their development into adulthood. Both the high school students (in TSWL) and Will (in GWH) are rebellious, they lack motivation, and they are not interested in participating in the learning and growth process that will help them to achieve their full potential.However, in both films, the characters experience significant development and personal growth.In the process, they learn about themselves and the world around them.“Learning” within both of these movies includes the personal growth that is achieved by the central characters at the end of each movie.
This growth is facilitated by caring adults, including a wonderful teacher (Mr. Thackeray in TSWL) and a caring therapist (Sean in GWH).Both Mr. Thackeray and Sean work closely with the young people to address the factors that are impeding their growth.Mr. Thackeray attacks the learning setting within the high school classroom.He makes changes in both his teaching style and within the classroom that foster increased motivation in the students.In contrast, Sean attacks the impact of a long history of abuse in Will’s past.Through the process, both the students and Will develop confidence in their abilities.By the end of each film, the students and Will head into their lives with the capacity to take on new challenges.
The following sections compare important aspects of both films with learning theories and principles from Marcy Driscoll’s Psychology of Learning for Instruction (Third Edition).Aspects of motivation, the impact of punishment (taken to abuse) and the role of a coach / instructor / mentor within the learning process are assessed.
To Sir, with Love: As noted, the students initially lack motivation in TSWL.Many internal and external factors affect the students’ motivation.These factors are compared within the framework of (a) Bandura’s Self-efficacy Beliefs and (b) Keller’s ARCS Model of Motivational Design
(a) Bandura – Self-Efficacy Beliefs:As cited in Driscoll (p. 316), Bandura proposes that a person’s beliefs about the task at hand, as well as the person’s beliefs about his / her ability to succeed in the task, greatly impact motivation and ultimately the likelihood of success or failure at the task. As we see in the beginning of TSWL, the students had poor self-efficacy beliefs which likely contributed to their poor motivation and lack of success in school.However, by the end of the film, there is a shift to more positive self-efficacy beliefs => increased motivation => greater likelihood of success.The following compares aspects of the film with what Bandura considers the four principal sources that affect a person’s self-efficacy beliefs:
<!--[if !supportLists]-->·<!--[endif]-->Enactive Mastery Experiences:Self-efficacy beliefs are enhanced when a learner has experienced past success.The students were unable to envision their own success, as they had never seen themselves succeed at anything in school.As one teacher stated in the beginning of the film, “Most students are rejects from other schools.”However, as the students tried new experiences they found they could succeed.In turn, their confidence grew and they were soon open to new experiences.
<!--[if !supportLists]-->·<!--[endif]-->Vicarious Experiences: A turning point in TSWL came as students realized that they could have the same success as their teacher.As he explained his background and what steps he took to succeed in life, they began to see him as a role model.In one scene, a student notes, “You’re like us . . but, not.”This demonstrates that through him, they saw their potential and the ability to rise from their current situation.
<!--[if !supportLists]-->·<!--[endif]-->Verbal Persuasion: Verbal persuasion (hearing that others know you can do it) has a powerful influence on self-efficacy beliefs as seen in several scenes the movies.For example, Mr. Thackeray instilled in the students that they had the power to change the world.While other teachers had talked down to them and considered them“morons”, this teacher convinced them that they possessed the ability to better their position in life.He told the students, “The whole world is waiting for you . . . you are a smash hit.”His positive words encouraged the students to strive to be their best.
<!--[if !supportLists]-->·<!--[endif]-->Physiological States: An example of a poor physiological state hampering self-efficacy beliefs (and ultimately failure in a task) is the gym scene when the boy is prodded to attempt the vault.Throughout the term, the gym teacher labeled the boy the “fat kid” and berated him for being unable to perform well in class.By the time it was his turn to try the vault, he had become convinced of his inability to perform.He was visibly shaken and too scared to try.He was sure he could not clear the vault and ultimately failed in his attempt.The conditions that led to his poor physiological state are an example of what Bandura refers to as “social labeling coordinated with experienced events” (Driscoll, p. 322).
(b) Keller’s ARCS Model of Motivational Design: As noted, a turning point in TSWL occurs when Mr. Thackeray makes fundamental changes to both his teaching style and the learning environment. These changes created far more motivated students.His approaches can be compared to the motivational conditions within Keller’s ARCS Model.Keller proposes that the following conditions must be met in order to have a “motivated learner” (Driscoll, p.332).Examples of the changes Mr. Thackeray made to the learning setting are presented within the framework of Keller’s four conditions:
<!--[if !supportLists]-->·<!--[endif]-->A – attention:A pivotal moment in the movie occurred when Mr. Thackeray gained the students’ attention and piqued their curiosity by discarding the text in the trash can.It symbolized “out with the old / in with the new” and signaled that a complete change was coming.In addition, he sustained the students’ attention by continuously engaging the students in whole class debate.
<!--[if !supportLists]-->·<!--[endif]-->R – relevance:At the beginning of the film, the students could not see how the subject matter had any relevance in their lives, hence their complete lack of interest in Mr. Thackeray’s lectures.Mr. Thackeray increased the relevance to the students by opening up the class lectures to debate by asking them what they wanted to talk about, “Whatever you want. Talk about life, survival, death, sex, rebellion, change . . . anything you want.”
<!--[if !supportLists]-->·<!--[endif]-->C – confidence:Mr. Thackeray realized he could increase the students’ confidence by both showing them respect and demanding the same of them.He began to build their confidence by instilling in them his high expectations for their behavior and demeanor.He told them, “You must present yourself well … and treat each person with respect”.As the students saw that they could not only meet his expectations, but also set and live up to their own expectations, their confidence grew and they became far more engaged in school.
<!--[if !supportLists]-->·<!--[endif]-->S – satisfaction:Tied to above, the students experienced tremendous satisfaction in experiencing (for the first time) success in the eyes of their teacher. He challenged them to try their new skills and praised them when they achieved success.As their sense of satisfaction increased, so did their motivation to come to class and participate.
Good Will Hunting: In GWH, nature and nurture collide.Will has clearly been blessed by nature with incredible math and science ability.However, after being orphaned, he experienced extreme physical and mental abuse within foster care.When he is discovered by Dr. Lambeau as the “mystery mathematician” who can solve problems, Will must address the adverse affects of the abuse he endured in order to utilize the incredible gifts he possesses.Using several principles within Driscoll, the following examines the effects of the abuse on Will’s behavior, how his sessions with Sean helped him overcome these effects, and why his apprentice relationship with Dr. Lambeau was not successful:
(a) Adverse Effects of Punishment: Will exhibited numerous physical and psychological side effects as a result of the punishment and abuse he endured.As Driscoll notes (p. 40), if punishment involves pain, “it can lead to undesirable emotional responses being conditioned” including several of the side effects Will experienced, such as:
<!--[if !supportLists]-->·<!--[endif]-->Aggression and anger:Will’s police record included significant violent acts, including his latest offense, striking an officer, which resulted in jail time.In addition, Will became easily angered when prompted to discuss his past, as he did in scenes with both Skylar and Sean.
