My role is not to be silent...

Shor & Freire in A Pedagogy for Liberation (1987):

On the one hand, I cannot manipulate. On the other hand, I cannot leave the students by themselves. The opposite of these two possibilities is being radically democratic. That means accepting the directive nature of education. There is a directiveness in education which never allows it to be neutral. My role is not to be silent.

That's So Meta

Meta-cognition is an important tool in the education world. Adding the prefix meta- to a term refers to additional information about that item.

Meta-data means data about data. If you have data stored in a spreadsheet or database, the meta-data could be things like what types of characters are allowed in each field, which items are required or optional, how many records are currently stored, and so on.

A meta-analysis is an analysis of other analyses. When a scientist writes a research paper, they collect data and do some type of analysis. The meta-analysis is then analyzing the data presented in many published research papers to see if they are consistent or measuring different things.

Cognition is understanding. So meta-cognition is understanding how to gain or measure understanding.

There are different types of meta-cognition discussed by Anderson & Krathwohl (2001), including strategic knowledge, application of cognitive tasks, and self-knowledge.

Strategic knowledge is simply being aware of various tools that can be used to learn - flash cards, mnemonics, mind maps, note taking, watching videos, physical practice, and so on.

The appropriate contextual application of the items listed above is the next one. If you're learning multiplication tables, flash cards serve as a useful tool to help learn those math facts to automaticity. If you're learning how to calculate an integral, a flash card would not be so helpful, since it's more about learning to apply the concepts to whatever numbers you are given. Don't use the wrong tool for the wrong type of learning. Usually the higher you go up Bloom's Taxonomy (remember, understand, apply, analyze, evaluate, create), you need more advanced strategic tools, or you may even create your own new method of learning.

The last one is understanding one's own understanding. This can include a couple levels - what methods of learning tend to be most effective for that particular individual and what the individual knows or doesn't know. A learner who is bad at math and is aware of that fact will take a more structured approach to find and utilize the best available tools for learning math or may choose a program of study that avoids the need to learn math. Someone who is a bad singer but doesn't realize it, may not ever take voice lessons, because they don't realize they need it. It's one thing to know what each note on the keyboard is by name and know the corresponding notes on the staff. It's something else to be able to physically play the right notes on the keyboard when looking at the staff. A piano student who continues to practice with flash cards even though they are all memorized isn't self-aware to the point that they know they need to move on to a different type of practice.

Make sure you take some time in whatever instruction you are doing to build the learner's meta-cognition at all levels. They should know what tools are available to them to learn. They should understand what each tool is used for. They should be able to reflect on their own application of those tools and how they work with their existing understanding in order to be willing to use the right tool at the right time for them.

Anderson, L.W. & Krathwohl, D.R. (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. New York: Longman.

Simplified, but not too simple

In a couple of classes, we teach a basic project management concept - the Work Breakdown Structure. It's an outline of the tasks that need to be performed, with related tasks grouped into phases. It can get complicated, depending on how many levels deep you want to create your outline, but once you can go two levels deep, you can go further if you choose.

A student was for some reason offended that as part of the discussion of the WBS, my colleague used a simple example of making a peanut butter sandwich. For the case study the students complete as part of the course, it's much more complex than that, obviously. But before being able to apply the concept of the WBS to an advanced situation, you start with applying it to a basic situation. It's a common technique that many of us use. You take a very simple task that many of us do all the time, such as making a sandwich and break it down so that you can focus attention on what matters - in this case, the WBS. I will sometimes do an example with cooking a steak and making a salad. Nothing big and fancy, but people understand the basics of cooking and realize that you want to time things right so that both parts of the meal are ready at the same time. Even people who can't cook understand the annoyance when the waiter at a restaurant brings some people their food but some of the plates aren't ready yet. I usually let the students choose what example they want to use.

Instead of splitting their attention between two topics - the WBS and programming an ERP system (or doing construction on a skyscraper or whatever other more complex example you might want to come up with), you just focus on the WBS. If someone doesn't know what an ERP system is, you wouldn't want to spend a lot of time dealing with that when you should be focusing on the topic at hand.

The funny thing about making a sandwich is that it's simple on purpose, but when you're done with breaking it down, you realize it's not as simple as it seems (although maybe this student didn't catch this point). The idea is you take something everyone knows and you help them break it down to a level of detail that goes a little beyond what they normally would write in order to establish what assumptions they may be making without realizing it. This transfers to other more complex projects very well.

Get out a piece of bread, but what if you're out of bread? Are you baking your own bread, too? You have to start with a build vs. buy analysis first to compare the quality of and time to bake the homemade bread to your options to get to the store and buy some.

Is the bread already sliced? How thick?

Put the end of the knife in the jar of peanut butter, oh, but did you open the jar first?

Are the stakeholders expecting bananas, sprinkles, toasted bread, seedless jam? Anyone who has made a sandwich for a toddler knows the struggle.

If you have an example that is too complicated, then the students can get distracted by something that is too hard and they don't understand the example well enough to be able to challenge assumptions and pick out what might be missing. Using a simple example lets them focus on the concepts being taught and how to break down the work instead of on trying to figure out the example itself.

Properly Balancing Formative and Summative Assessment

Formative assessment is one of the most powerful tools for promoting student achievement by providing an understanding about how each student is progressing in their learning continuously, as it happens (Stiggins & DuFour, 2009). Summative assessment is focused on determining the outputs of the learning process in order to assign grades or for evaluation of the course, keeping in mind that it can be used for both evaluating output and evaluating the process used to create the output (Anderson & Krathwohl, 2001).

Summative assessment may be seen by some learners, especially those with a performance-focused orientation, as a way of comparing oneself to others, which can lead to less effective learning since the focus is on the display of mastery rather than on the actual learning process (Roll et al., 2011; Azevedo et al., 2008; Aleven et al., 2003; Butler, 1998). On the other hand, when formative assessments are used appropriately, they provide an opportunity to promote learning, self-improvement, and progress (Butler, 1998; Roll et al., 2011).

In order to allow learners to progress at their own pace, a combination of formative and summative assessment can be effective in helping learners understand what they need to focus on while learning and how to show they have attained competency in order to complete the course. This is especially important in an adult education environment, where each learner will come in with different levels of knowledge and experience, so effective assessment can help keep individuals progressing in those areas where they need assistance and complete the course when they can show they can perform at the appropriate level.

I'm always surprised, although I suppose I shouldn't be, how many students (yes, I'm purposely changing what I was calling learners previously to students, who may or may not be true learners) want to rush to the summative assessment and just "get it over with" instead of taking the time to use formative tools that are built into a course and provided in a very obvious way to guide their learning toward success. In some classes, a summative paper or objective exam may be actually used in a formative way as it is graded, feedback is given, and if the student is not happy with their result may revise and resubmit. It's a great way to approach things in theory, taking a bit of the edge off a high stakes summative assessment. But it also can lead to students throwing things at the wall to see what sticks even though they know they are not ready. I frequently make the point that when students say they don't know what they should be studying, then they know exactly what they should be studying, i.e. everything.

That is a key metacognitive error (White & Frederiksen, 2005), to know one is not prepared but not know what they are missing, so instead of taking the time to use provided course materials and formative assessments and guidance from a faculty member to figure out what they are missing, they jump immediately to whatever the summative assessment activity is and try repeatedly to ram it through even though they are not at the right level of cognitive preparedness (Anderson & Krathwohl, 2001) for it. It is thus important not only to provide appropriately designed scaffolding leading up to a summative assessment but also help learners understand why they are built that way so they are willing to use it.

Aleven, V., Stahl, E., Schworm, S., Fischer, F., & Wallace, R. (2003). Help seeking and help design in interactive learning environments. Review of Educational Research, 73(3).

Anderson, L.W. & Krathwohl, D.R. (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. New York: Longman.

Azevedo, R., Moos, D.C., Greene, J.A., Winters, F.I., & Cromley, J.G. (2008). Why is externally-facilitated regulated learning more effective than self-regulated learning with hypermedia? Educational Technology Research and Development, 56(1).

Butler, R. (1998). Determinants of help seeking: Relations between perceived reasons for classroom help-avoidance and help-seeking behaviors in an experimental context. Journal of Educational Psychology 90(4).

Roll, I., Aleven, V., McLaren, B.M., & Koedinger, K.R. (2011). Improving students’ help-seeking skills using metacognitive feedback in an intelligent tutoring system. Learning and Instruction, 21(2).

Stiggins, R. & DuFour, R. (2009). Maximizing the Power of Formative Assessments. Phi Delta Kappan 90(9). Retrieved January 31, 2013 from http://alaskacc.org/sites/alaskacc.org/files/Stiggins%20article.pdf.

White, B., Frederiksen, J. (2005). A theoretical framework and approach for fostering metacognitive development. Educational Psychologist, 40(4).

Andragogy - An Annotated Bibliography

Ausburn, L.J. (2004). Course design elements most valued by adult learners in blended online education environments: An American perspective. Educational Media International 41(4).

Part time adult learners make up over half of the higher education market. The for-profit schools have grown rapidly due to their focus on serving these nontraditional students. Established principles for meeting the needs of adults include understanding the purpose of the learning, how it will be applied, relevance to experience, and self-direction. Studies show that previous experience in non-traditional education environments and gender affect attitude and behavior in distance learning. Interactive course designs and promote communication and bonding in learning communities is desired. The study looked at gender, technology skills, prior experience in distance learning, prior experience with self-directed learning, and the features of the course they thought were most helpful. The ATLAS instrument used in the study identified three main types of learners: navigators (results-oriented learners looking for efficient navigation of the course), problem solvers (critical thinkers who analyze a problem deeply before solving it), and engagers (passionate learners with high involvement). The most important course features to learners dealt with the structure and guidance of the course. Next comes the instructional content. The next was convenience factors that streamlined common tasks. And last was communication features. In terms of content, the most important instructional goals related to individualization, self-directed learning, variety of activity types, and communication/interaction. Some features showed up in different orders for each of the student types.

Bergeven, P. (1967). Concepts to implement the education of adults. In A philosophy for adult education. New York: The Seabury Press.

Adults can learn. Some adults may not participate in lifelong learning due to lack of organization, discipline, or a desire to change. While everyone is limited to some extent by age, mental capacity, or otherwise, everyone can break through these differences and excuses to learn. Adults and children have different educational processes. Some teachers are hired for adult education because they are good at child education, but understanding the differences between adults and children will help teachers of adults be more effective. Adults are more focused on creating/maintaining family relationships, raising children, working, keeping up the home, and social responsibilities. Physical ailments make it more difficult to learn as one gets older, and changing one’s opinions and mindset is also more difficult as one ages and becomes more set in their ways. They do, however, have longer attention spans than children, as well as experiences to draw from. Existing institutions are effective channels. Schools, libraries, churches, hospitals, and other organizations should all take on the responsibility to push for adult learning. Adult education programs should be indigenous. Education should be natural, built around real problems they are experiencing and in a familiar environment. Expectations should be reconciled. Each person has something different they are looking to learn from a class or a lesson, including the teacher. Freedom is important in learning. Learners should be able to choose the content, leaders, time, and place of a program of learning. Too much freedom can be destructive if not equally shared across all participants. Goals are vital. Realistic, specific goals should help shape learning (SMART). Learning how to learn is helpful. Metacognition is one of the most important trainings that can take place. Needs must be considered. Both the learner and the organization sponsoring training have needs that need to be aligned. Problem-centered learning is basic. Adults understand a need to learn something if they can see a problem it helps solve. Resources should be appropriate. Technology, format of content, and other appropriate equipment should be utilized. A programming cycle should be a cooperative experience. Participants should help in the planning, organizing, conducting and evaluating of an educational program.

Blaschke, L.M. (2012). Heutagogy and lifelong learning: A review of heutagogical practice and self-determined learning. The International Review of Research in Open and Distance Learning 13(1).

Self-directed and self-determined learning is becoming more important for our increasingly complex world. Distance education is especially suited to this self-directed learning in the way of heutagogy. The article begins with andragogy as the basis for showing the need to help learners become self-directed. They need to learn to diagnose their own needs, set goals, identify appropriate resources and strategies, and evaluate learning outcomes. The educator is a mentor. With heutagogy, the processes are similar, but learners take an even greater role. Double loop learning is a key strategy, where learners not only determine how their actions solve a problem but also how the problem solving process influences the learner’s beliefs. Learners become both capable and competent. Due to the technologies we have available to us, the experience and maturity of adult students, and the built-in autonomy of distance education environment, heutagogy seems a natural fit. Social media technologies support education going mobile and generating their own content. Heutagogy certainly causes concerns when it comes to accreditation, but some higher educaiotn institutions have found success handing over more control to students. Aspects of a heutagogical learning environment include learner defined learning contracts, flexible curriculum, learner-directed questions, flexible assessment, and a holistic approach to being a lifelong learner.

Christensen, C.M. & Eyring, H.J. (2011). Change and the indispensable university. In The Innovative University. San Francisco, CA: Jossey-Bass.

There is great demand for higher education, but also great need for change. Higher education has grown in the past due to innovation, and it needs that again. Decreasing prices and better service to students are some of the keys, in addition to continuing to support traditional research roles. By growing what it means to be a university, rising tides lift all ships, everyone can win. Schools should be ranked based on their outputs, not their inputs, so traditional ranking systems will (or should) become relics of the past. Schools with different missions shouldn’t compete against each other but for their own definition of success.

Dewey, J. 1998. Experience and education. (60th anniversary ed.) West Lafayette, IN: Kappa Delta Pi.

Different schools of thought exist around what education is – a natural growth of one’s self or a change pushed at someone from the outside. While genuine education may come from experience, that does not mean all experience are educational. Every experience we have modifies us and affects the quality of subsequent experiences (good or bad). Factors of both the internal thoughts of the learner and the objective goals of the teacher need to be considered. Traditional educators create a place for students to learn without considering the background of learners; just because a condition is effective for one group at one time does not mean it will work again. Education can be used to improve desires and impulses, but that should be done by helping the individual reflect and make self-judgment rather than impose judgment from the outside.

Freire, P. (1970). Chapter 2. In Pedagogy of the oppressed. New York: Herder and Herder.

The banking model of education consists of the teacher expounding to and filling up passive students with detached words. As empty receptacles, they lose their natural creativity. The solution to this problematic approach to oppressing learners is to transform the educational structure to allow students to be themselves. Rather than wait and hope that students will break the mold on their own, it is the calling of the revolutionary educator to help students become partners in their education. Problems to be solved liberate the students. Teachers learn and students teach in the new liberated model.

Gagne, R.M., Wager, W.W., Golas, K.C., & Keller, J.M. (2005). The Learner. In Principles of Instructional Design (5th Edition). Belmont, CA: Wadsworth.

Learners have many of the same characteristics, but vary in their abilities for each characteristic. That is, if all the students can read, perhaps one reads well and another poorly. Students can learn intellectual skills or procedures, cognitive strategies or productions, verbal information or facts, attitudes, and motor skills. Learning the above is affected by factors of motivation (ARCS), developmental and social factors, and individual differences. The ability to process concepts to build schemas and store in memory is one that differs from learner to learner.

Gibbings, P., Lidstone, J., & Bruce, C. (2010). How do student attributes influence the way students experience problem-based learning in virtual space? Australasian Journal of Engineering Education, 16(1), 69–80.

Study of engineering students in a PBL engineering course found that students expected the class to help them learn as they started the class, and that after the PBL class, students found that a PBL course really did help them learn better how to learn.

Isenberg, S. (2007). Background. In Applying andragogical princples to internet learning. Youngstown, New York: Cambria Press.

There is a lot of talk about andragogy and about the Internet, but little about both together. The Socratic method promotes probing questions from the learner and self-assessment to help the student construct knowledge. The field of mass communication has shown that diffusion of knowledge needs to happen systematically, with clear scaffolding. The Internet allows for enhanced systematic diffusion of knowledge. The Internet allows for customized curriculum for each learner, both helping the learner with the content and with his or her own metacognition. The whole-part-whole model can be facilitated online, by showing worked examples, helping the learner practice the parts, and then practice all the skills in one immersive case. Preassessments can automatically guide students to areas where they are deficient. Self-learning does not have to be alone learning. Virtual groups can be provided to allow for important interactions with peers while still being self-paced. Internet learning requires learners and teachers to be engaged and needs to be made easier to use.

Kenyon, C., & Hase, S. (2001). Moving from andragogy to heutagogy in vocational education. Retrieved April 2, 2013 from http://www.avetra.org.au/abstracts_and_papers_2001/Hase-Kenyon_full.pdf

Traditional education is pedagogical, where the teacher is in control of everything. Andragogy improves educational methods but still has a student/teacher dichotomy to deal with. Heutagogy provides a movement even further in self-directed learning. The key is a holistic approach to build independent capability. Heutagogy is most effective in a flexible environment, where resources are provided without too much direction as to the specific content that needs to be studied. The ideal learning environment challenges learners to ask deep questions without getting in the way too much.

Knowles, M.S., Holton, E.F., & Swanson, R.A. (2011). A theory of adult learning: Andragogy. In The adult learner: The definitive classic in adult education and human resource development (7th ed.). Burlington, MA: Butterworth-Heinemann.

The most famous teachers in history worked with adults, not children. They used processes of mental inquiry rather than passive reception of transmissions. Learners delved into cases and dialogue. They challenged and confronted and defended. Proponents of traditional pedagogical methods will claim that adults are uninterested in learning; if they were interested they would make it happen. Of course, the key is to engage adults and make them aware of the learning they have available to them. Adults are motivated by satisfying needs, centered around their life, have much experience, work best when self-directed, and have unique needs.

Kopcha, T. J., & Sullivan, H. (2008). Learner preferences and prior knowledge in learner-controlled computer-based instruction. Educational Technology Research and Development, 56(3), 265-286.

Studies have shown that giving students control over their learning motivates them. Computer-based instruction is an effective method for maximizing learner control. In the study, students were given a computer based instruction program for teaching math, with varying levels of control over the program. Students were broken out by preference for control and prior knowledge. They found that when students had high prior knowledge, they preferred having control over their learning, while students with low prior knowledge preferred high teacher control. This may be because lower performing students simply want to get through the course quickly or are not confident in their abilities, but preference for control had no significant interaction.

Merrill, M. D. (2002). First principles of instruction. Educational Technology Research and Development, 50(3), 43-59.

Merrill’s model builds on many existing instructional design models. The key principles are that learning should deal with real-world problems, activate prior knowledge, demonstrate new knowledge, and apply and integrate new learning. Problems should become progressively more complex. Both examples and non-examples should be demonstrated, and relevant media should be used. Coaching should be gradually withdrawn as the learner’s ability ramps up.

Tomei, L.A. (2010). A theoretical model for designing online education in support of lifelong learning. In Online Learning and Adult Education. Hershey, PA: Information Science Reference.

Web-based courses increase the potential for students to collaborate and share information with others, potentially leading to higher order skills. The traditional ADDIE model has stood over time as a solid yet generic process for designing instruction in a straight forward way. Wiggins & McTighe’s Backwards Design model and others have refocused ADDIE to meet the needs of the educational environment. Behaviorists, cognitivists, and humanists each describe different theories about how we learn. Just as learning taxonomies, such as Bloom’s categorize the types of knowledge that can be learned, distance learners need a taxonomy of skills for dealing with technology: literacy, communication, decision-making, infusion, integration, and technology. Learning materials, whether print, video, data, or otherwise provide tools and technologies that may be swapped out with each other to meet learner preferences. Adult learners prefer collaborative and realistic learning experiences. Distance education students use various synchronous and asynchronous tools to help them achieve a similar collaborative environment. Adults may be assessed by having them build higher order responses on Bloom’s taxonomy, while built-in analytics in various forms can help the learning environment assess the student with his or her input.

Weigel, M., James, C., Gardner, H. (2009). Learning: Peering backward and looking forward in the digital era. International Journal of Learning and Media 1(1).

While methods of education have changed greatly over the past decades or even centuries, the need for relevance has never been so important. Current youth use digital technologies extensively, yet understand little about how they work or how to create the technology. The rate of change with new technology means that even once they get a technology down, it won’t stay around that way for very long. Constructivism, with its increased engagement and investment of the learner and decreased control of the teacher, modern learning theories fit the Internet’s potential for deep exploration. But for that to happen, they need to have solid skills to navigate the technology, including the underlying biases and structures that affect its use. The number and vast difference in quality of resources found online serve to make the process difficult for the non-information-literate student. The social aspect, which allows for back channel communication and instant sharing of ideas serve to add to the crowdsourcing culture, which helps construct new knowledge effectively with a risk of drowning out dissent. Moderate changes to schooling have changed little about how students learn, but the technologies now available, in combination with the accepted learning theories such as andragogy are pushing towards a radical shift in education.

Syllabus Crap Detector

As I've been working recently on designing several courses, I've found some weird stuff, as usual. Part of our process is to review syllabi of similar courses being taught at other universities. We perform somewhat of the equivalent of a literature review to determine what is being taught and how, in order to determine what minimum standards need to be covered and where there may be places that we can make our programs better by filling in the holes missed by others.

The following are a few quotes that I have found interesting for various reasons:

• Regular attendance is rewarded because you will be exposed to what I think you need to know.
  
• When I talk, be quiet.
  
• Put newspapers away 20 seconds before class starts.
  
• I will clear the memory of all programmable calculators before each test.
  
• Get yourself a notebook or 3-ring binder, and start a journal to record your impressions/analysis about what you’re learning from your journalism reading. (in a class with large blogging, twitter, and social media components)

I found a particular faculty member, though, whose syllabus really takes the cake. Looking up his profile on ratemyprofessors, student comments seemed to confirm what I observed: unorganized, hard to follow, illogical, makes no sense whatsoever, don't expect straight answers to your questions, writes smart remarks back to the students, and gets off subject very easily. He seems to be the opposite extreme of the faculty who wrote the first four statements in the list above. He may be friends with the person who wrote the inconsistent final statement above.

So what is so illogical or hard to follow? Let's go to the syllabus (which happens to be in comic sans of all things), shall we?

The course has a subtitle of How to build your "Crap Detector" which is only referred to in one other place in the syllabus. I was hoping the other reference would help in understanding what he's talking about. You can probably guess that it doesn't. One of the learning objectives of the course is to "provide personally and socially meaningful responses to what Roland Barthes once called the 'question par excellence': 'Why the world? what is the meaning of things?' (e.g., design and use an effective 'crap detector')". So I looked up Barthes and found that he was a French literary theorist. Good enough, but what of this question par excellence?

I looked that up also. In talking about the difference between the technical and artisanal functions of writers and authors, Barthes discusses a paradox that occurs. "And the miracle, so to speak, is that this narcissistic activity has always provoked an interrogation of the world: by enclosing himself in the how to write, the author ultimately discovers the open question par excellence: why the world? What is the meaning of things? In short, it is precisely when the author's work becomes its own end that it regains a mediating character; the author conceives of literature as an end, the world restores it to him as a means; and it is in this perpetual inconclusiveness that the author rediscovers the world, an alien world moreover, since literature represents it as a question-never, finally, as an answer."

It's pretty deep stuff. Mull it over. Ponder. Enlighten yourself as you discover the hidden meaning of Barthes' paradox. Apply it to your own life and career. Then come back to the question at hand, and ask yourself what this has to do with a crap detector. I'm still not totally sure what a crap detector is, but if I had one, it would be going off right now, and we've got a ways to go.

In terms of inconsistency, in one area of the syllabus regarding how grades are earned, it states that you can receive an F for class abandonment, which in a different part of the syllabus he explains means more than 3 absences. In yet a third location in the syllabus talking about attendance, he seemingly confirms that excessive absences may result in unsuccessful course completion but then immediately proceeds to state that more than 3 absences will result in a reduction of one full letter grade. His syllabus also states that students may fail if they do not demonstrate mastery of writing and numeracy skills. He may fail his own class given his apparent lack of ability to think and express himself effectively in quantitative terms.

According to the syllabus, a student can be considered cheating by communicating in any way with another member of the class. This sounds like something the faculty who wrote the first four statements above might want to adopt in their classes.

There's more, but I'll end with this professor's teaching philosophy from the department's website:

I love mental spelunking. Each semester I get to take students through a dark cave of stalactite and stalagmite ideas, pointing the light at the ceiling-stuck cat or a three-legged donkey. “Over there,” I yell, turning the light in another direction. “Can you see it? Can you see it? . . . the NAKED CHICKEN, caught mid-stride in the next room?” With furrowed brows and searching eyes, students squint at the dark shadows, searching for even a hint of beak or comb. “Where? Where?” they ask. Then at last . . . “I see it! I see it!” one cries.

Then here is the really best part. She takes the light herself and points it far into the cave. “Can you see the lovely apple tree? Don’t you just love apples?” she says. My eyes follow the light, and there it is, right where it has always been, apples and all. But I SEE IT for the first time.

“Is that the tree of knowledge?” one asks. And, “was that the naked chicken of wisdom?” chimes another. “A look under the feathers.”

Then we all laugh together at our possibilities till our joyful tears form new mystery-shapes of our own on the floor, someday to be discovered . . . and I am once again blessed.

I have to say that as strange as it is, the more I read the teaching philosphy, the more I actually like it. I'm not so sure about the naked chicken; I believe the idea is a sound one, even if execution is lacking.

The extremes of the first few items in the list at the beginning of this post and of the who-knows-what-you're-going-to-get class by the naked chicken of wisdom both need to be tempered. If we bring the two components of structure and creativity together, we might come up with this college composition course based on The Simpsons. Now that's a syllabus!

Spreadsheets and Databases

Spreadsheets and databases was a good course. It had the potential to be a great course. It didn't quite make it to greatness, but looking back on it, I can see the possibilities. I know it has potential to be a terrible course as well, because I know several people for whom that was the case. Some of that depends on the person taking the course, as databases just don't make sense to some people, but I think most of it has to do with the person teaching.

We had a seating chart, where we had to sit in the same seats every time. We got to pick where but once we picked, it was set, so that the professor could use his map of names and where people sat. I thought it was nice that he was using some type of system to learn our names. I was lucky to have a professor who was actually savvy in the way of databases. Others I know had someone who didn't understand databases at all teaching the course, so instead of spending 50% of the time each on Excel and Access they would spend 12 weeks on Excel and maybe 3 weeks on Access.

I did learn a lot about Excel. I don't know that I ever really "understood" databases from this class, but I learned a lot of the basics. I could perform the various required tasks but didn't have a solid grasp of why you would actually do what we were doing. A roommate of mine sat next to me. I remember him asking one day for the professor to explain how we might use this in real life. It was just too abstract for him, as it was for me and probably most everyone else in the class. Nobody else had the guts to actually ask why we were learning this stuff. I don't remember his answer other than it wasn't a good one.

This makes me think of one of the key principles of andragogy or adult education, which is that adults will do better if they understand why they are learning something. I contend that children and young adults, for whom a pedagogical or teacher-centered approach is traditionally used, would also do better if they understood the why. The difference is in the power relationship we have when teaching younger students. They still want to know and sometimes are even willing to ask how what they are learning will be useful to them. Because of the imbalance of power, we blow them off instead of taking them seriously. Adults simply hold their ground and require you to come up with a good answer while young students who hold their ground are disciplined.

Benefits of Online Learning

Through the twitterverse came the question of how the list of ten benefits of online learning as presented by students at the Open High School of Utah is different.

Their list:

1. I can work ahead if I'm able to
2. I get nearly instant responses from my teachers
3. I get personalized support when I need it
4. My teachers are just as excited about online learning as I am
5. I can do all my math for the week on one day if I want to
6. I know how I'm doing, my grades are right on the screen
7. My parents can see my work and grades
8. My courses are more challenging
9. I can keep up with my work when my family travels
10. I can work around a busy schedule

Of course, being Twitter, there wasn't much more to the question, specifically how is this different from what? I have to assume it's asking how online schools are different from traditional schools, since good teachers would be doing the stuff in the list anyway. The difference comes in where at a student-centered school, the culture of the entire organization is focused on helping the individual students, where traditional schools are centered on the faculty member so whoever is teaching a class decides whether or not students can work ahead and how challenging the homework is.

There's a big difference between a single teacher running a student-centered classroom and having an entire school be student-centered.

Course Design Rubric

As a follow-up to my last post presenting the idea of a design rubric, here is the first real draft of the course design rubric.

I haven't worried yet about putting the rubric items in any specific order or weighting them in importance at all. That will come later. I will probably group them together somehow so related rubric items are next to each other. They will also be fleshed out with better descriptions of what it takes to get a certain score. There's only so much space in each little box.

The way a rubric like this works is that the rater starts at 0 and moves towards 3, stopping whenever criteria are no longer being met. So you can't meet the criteria for a 3 if the criteria for a 2 were not met. In working to validate the rubric, I'll look at an overall score out of 21 by adding up the 7 individual scores, the correlations between individual rubric items and external criteria, and the correlations/dependencies among rubric items.

There were a few times that I debated which items would score higher on a given criterion when one was not naturally subsumed by the other. For example, what if the recommended learning path does build effectively but is not flexible? Or what if learning activities provide new experiences as a foundation for the content to be presented but do not build at all on student experiences prior to the course? Those are obvious holes that will need to be patched before this is production ready.

As I hinted I might do in my last post, I did decide to combine the Measurable and Clarity items into Clear and Measurable Objectives. That was a more interesting decision to make than it seems. As I wrote last time, the goal was to write a rubric that allowed for rating the design of a course before it was developed. What I've been leaning more towards, however, is a rating of a developed course. Most of the items could still be used in a design-only situation (and really should be if you're not shooting from the hip and developing without designing first), but combining those two items related to writing good objectives in effect serves to weight that portion less than some of the other items that apply more naturally to a developed course.

Of course, the real power of using a rubric is not so much that you can use it to come up with a grade but rather that it can be used during the creation process to improve the quality of the end product.

So what's missing? What's redundant? What's out of order? What can't be measured?

Instructional Design Rubric

In looking around for research opportunities, I made an indirect connection between the research a friend of mine was doing and something I had been thinking about at work. I helped with the research on a pilot study. It was using a rubric (Boote & Beile) to measure the quality of dissertation literature reviews. It turns out literature reviews aren't taught consistently, and (no surprise here) are not consistently performed properly either. I'll just say that after reading a stack of dissertations, my confidence in my own ability to do academic research increased dramatically.

The conversation has been had several times about how strange it was for my friend doing a literature review on doing literature reviews, just as last summer I was working on designing instruction about designing instruction. My thoughts turned to whether it would be possible to do something at a level that could improve more of the higher education world, of which PhD studies are a fairly small part. One interesting idea I came up with deals with an analysis of the gap between expected outcomes from current students in higher education and recognized outcomes from graduates of higher education, with cross sections based on type of institution, level of degree, subject area, etc. My guess is that we'd find that students aren't fully realizing the benefits they expect to receive; that is, there is a misalignment somewhere. Think about what you were asked to do in your first job out of college and whether you used anything that was listed as an objective for any of the courses you took. While I still think that is an interesting topic to pursue at some future date, it's not quite dissertation material at this point.

At work, we follow a fairly well structured design process to come up with what we call the competencies and learning objectives for our courses. Competencies are roughly equivalent to educational objectives in Bloom's Taxonomy, and the more focused learning objectives match up with Bloom's instructional objectives. Going with a more wide angle, we also couch everything for a degree program within a conceptual framework, which Bloom would probably call global objectives.

While there is research on Bloom's Taxonomy, much of what I've seen has to do with validating the order, leading to a recent swap in the last two levels since researchers had identified that evaluation is less cognitively demanding than creation. There are other taxonomies out there, which overlap quite a bit, but I have yet to come across any good research that really supports the most famous of educational taxonomies. I believe it, because it makes sense, but who is to say if something is missing or if all the current components are really necessary? One article talked about how the iconic triangle should be flipped on its point so it opens up as you reach higher cognitive levels. Whatever. I don't care about the job aid. I want to know if the taxonomy is right. So the proposed research is to rate the quality of a course designed using Bloom's Taxonomy and compare that to some measure of the actual quality of the finished course developed according to that design.

I haven't pulled this together as a formal literature review yet (but I will be pulling out my Boote & Beile rubric when I do). Just a quick compilation of features from several models out there, and I come up with an unpolished rubric that I hope may be useful in rating the quality of a course design, with design being the first D in ADDIE, not the kind where you're choosing background colors.

While I have not yet found anything substantive related to measuring the quality of the initial design of a course, Merrill does have his 5 Star and e³ rating forms based on his First Principles of Instruction along with some other theories, which are intended to rate a developed course. At that point, however, it's a little late. The course is already created and perhaps already taught. It would be more helpful to rate the design of a course before it's developed. Fix the problems before developing the instruction on a flawed design.

Again, this is not complete or valid in any way. The models I considered, based on our practice at work and general research I'm familiar with in the area, in no particular order, are as follows:

• Wiggins & McTighe - Backwards Design Model
  
• Merrill - First Principles of Instruction
  
• Gagne - Nine Events of Instruction
  
• Anderson & Krathwohl - Revised Bloom's Taxonomy
  
• Mager - Preparing Instructional Objectives
  
• Knowles - Principles of Andragogy

The principles or characteristics of effective design that align with the above theories, and that serve as a first draft of rubric principles are:

• Problem-based or realistic (publishing work, problem solving, creation)
  
• Alignment between assessment and content (teach what you test and test what you teach)
  
• Appropriate path to complete objectives (flexibility for diverse learners, no orphans, properly scaffolded)
  
• Measurable behavior (specific, discrete, observable)
  
• Level of performance (alignment with content and audience)
  
• Clarity (one verb, understandable, meaningful, maybe combine with measurable)
  
• Metacognition
  
• Bridging prior knowledge and future practice
  
• Any other suggestions as to what's missing?

I'm picturing a four point scale, 0-3 for each item. The 0 is obvious; it simply doesn't meet the requirements. A score of 1 would indicate minimal coverage. A score of 2 would indicate significant alignment but missing higher level components or perhaps most objectives are written correctly and a few incorrectly. The goal is to score a 3, meaning all principles and characteristics are present and appropriately applied.

The rubric itself would take the form of a table, which I'll put together later, but the principles or characteristics would be listed down the left and the points across the top, with a description of what is needed to receive that score in the intersecting boxes.

A graduate-level course, for example, with all multiple choice tests would likely score poorly on problem solving and realism but could score okay on the alignment if you're planning on teaching at the scantron level. One could argue whether completing a multiple choice test is measurable behavior; I'd maybe put that in the 1-2 range, since while the correctness of the student's answers can be easily measured, it's debatable whether it can be considered behavior. The level of performance is obviously wrong if we're talking about giving grad students multiple choice tests, so we're talking maybe a 1 here if the questions get past a superficial level. There's not going to be much metacognition going on.

For an undergraduate-level course, on the other hand, while some of the principles may still be rated low in terms of realism and metacognition, the level of performance may be appropriate. It may even be possible to score decently on bridging prior knowledge and future practice if the scantron content is considered foundational knowledge that will support more in-depth performance in later classes.

Good, bad, too much, not enough? Other principles, theories, or methods that should be included? Has anyone actually done something like this for instructional design? What measures other than inter-rater reliability could be used to validate the score from this type of rubric?

TMI

The acronym TMI is often used as a way of expressing that someone just shared too much information with you, generally something embarrassing or private or that they just don't care about. I believe some of this comes because of an overload of information always flowing around us through computers and mobile devices, so we lose the ability to filter out extraneous or private information from that which should be communicated.

Given the large amounts of information that is put out there, since people do seem to just braindump it all onto various social media sites (or vetted news sites) in a way that's easily accessible by others, those who learn to actually mine the vast data fields will do very well for themselves in our information-based society.

While some people see these vast data fields as a wasteland, like the Abominable Snowman in Monsters, Inc., I would say, "I think you mean wonderland!" We should be able to take the plethora of information created by others and turn it into something useful. There should be no such thing as TMI, because the more data out there, the better we can harness it for the good of ourselves or others.

I'm occasionally made fun of for looking up information on my phone or a computer. Someone asks a question or wants a clarification about a statement made by someone else, and everyone just sits there thinking, yeah, someone should find that out. Then the conversation goes a different direction and everyone forgets there was something they wanted to know. So I look it up before forgetting what the question was. And I get strange looks for providing the answer. I'm the smarty pants because I googled it, when half the room could have also pulled out their iPhone or Droid and looked it up themselves. They're using their phones to text, so it's not like they're put away to be polite to the present company.

Isn't that why we have smartphones? I mean, it's cool that you can use your iPhone as a digital rattle to keep your kids occupied, but isn't access to Wikipedia, Google, Yahoo Answers, Twitter, etc. the best reason to have a smartphone? Nielsen claims that 25% of smartphone users don't even access any data on their phones and that 6% of users consume over half the mobile phone data. The rest of us are on that logarithmic continuum somewhere.

What other information is out there that we aren't using? If you have a blog (or any website for that matter), do you use Google Analytics to see who is coming to your site, from where, and for what? How long do they stay, what do they do while visiting, and do they actually find what they were looking for? Is that just another type of TMI that you don't care about?

Over the past 2-3 years, my two most consistently popular posts are on Pedagogy vs Andragogy and Surf the Channel. Coming in a distant third place is a post related to Cognitive Load Theory.

It's interesting, since I spent a lot more time writing the cognitive load post than I did the other two. Maybe the third place post was too long or too academic or too focused on the specific situation in which I was using it. Something I have been able to figure out, though, is that if you can find something that is an interesting or upcoming topic that not many people are blogging about, you'll get a lot of hits. That may seem obvious, but I only figured it out because I had data that told me. If you think I should have known that already, guess which of your blog posts are the most popular and then turn on Google Analytics and tell me how close you were after a month or two of collecting data.

I've been thinking for awhile now that I need to write another post on Andragogy, because I'm afraid that the one that several hundred people a month find just wasn't all that well written or informative. It was just a quick recap of an experience I had and a few comments on some of the basics of that area. I know a lot more about the topic now and knowing people are looking for that information and having a hard time finding it, I feel it my duty to help others make some sense of it.

They could find academic articles on it or take a class on the subject. They could go to conferences and talk to people using techniques based on these principles. But they don't. At least they're googling it, which many don't even have the motivation to do like I already talked about. So if that many people come asking the question, how many more are out there who don't even ask, because they can't be bothered? There's probably not much I can do to reach out to them, since I can't force my blog down everyone's throats. I should at least try to answer the question people who do find my site are asking.

So what does Google Analytics tell you? And what are you going to do about it?

Staying culturally relevant in today’s global society

Given the speed at which technology is advancing, both in innovation and distribution, groups and individuals seem to be classified as technology haves or have-nots. The haves are seen as smarter or more successful than the have-nots. The haves then suppose it their duty to impart their ways on the poor have-nots, whether or not their help is needed or requested. Looking deep down, however, it does not appear that technology itself really causes anyone to change, rather it facilitates desired changes. Whether those desired changes are for the better or the worse depends, of course, on the situation.

In addition to technological advancements, there are many more aspects by which various cultures may differ, including geography, religion, language, social structures, traditions, available resources, etc. Technology is hailed by some as an equalizer, allowing members of cultures that might not otherwise have a voice to publish their story for the world, but it may come at a price of lost traditions and ways of life (Bowers, Vasquez & Roaf, 2000).

That is not to say that cultures should not communicate and mix to create new traditions and understanding of others. Quite the opposite is true. As we come to learn and know how and why we are different, we will understand our own background better and realize that because of our differences, everyone has something to offer. As the field of Instructional Technology continues to develop and make advances in understanding and dealing with accommodating differences between individuals and groups, practitioners find themselves developing more personalized instruction that must be sensitive to the cultural needs of their constituents.

Thomas, Mitchell & Joseph (2000) point out that the traditional ADDIE process of instructional design addresses culture in the analysis phase, but go on to encourage designers to continue interacting with the learners throughout all phases of development. In fact, they propose a third dimension to ADDIE consisting of Intention, Interaction, and Introspection that are meant to be continually evaluated during each of the steps of the standard model.

These three I’s in their three dimensional ADDIE model serve to ensure that designers pay attention to what cultures they are developing instruction for, communicate and collaborate with stakeholders, and reflect on their own thoughts and actions throughout the entire process. These additional steps may seem redundant, i.e. introspection sprinkled throughout the process just like evaluation; however, the difference between their three I’s and the existing five steps is that the I’s are specifically focused on culture, since that element is often forgotten. By adding the third dimension, designers are reminded that their instruction cannot and should not be culturally neutral, so an extra measure of care should be taken regarding cultural issues.

That said, other than recommending that designers think about culture throughout the process, this three dimensional ADDIE model does little to actually provide any real framework for incorporating cultural methods into instruction. In order to provide a meaningful model to follow, it is likely that a systematic approach is needed. Such an approach provides a scaffold for novices, which they gradually remove as they become expert designers. It is not that experts do not need to perform all the same steps, but they usually perform them intuitively and naturally. The challenge, then, in a new framework is to integrate cultural sensitivity in such a way that experienced designers will be willing to take the time to adjust their current methods and established knowledge to take into account the additional dimension.

In order for an organization to make such a change in their own methods and cultures, a change management program may be required. Jones, Aguirre & Calderone (2004) present a set of 10 tools and techniques for helping a company transform, providing means for individuals to manage their own change and an entire organization to implement a change as well. Without individual change, no collective change can be effected.

Their 10 Principles of Change Management include the following steps:

1. Address the “human side” systematically.
2. Start at the top.
3. Involve every layer.
4. Make the formal case.
5. Create ownership.
6. Communicate the message.
7. Assess the cultural landscape.
8. Address culture explicitly.
9. Prepare for the unexpected.
10. Speak to the individual.

Steps 1, 2, and 3 really state that any change must be planned out before it is implemented, modeled by the leadership of the organization, and implemented everywhere. If it is not, that shows a lack of commitment and the change will fail. Change is embraced by few, especially when it is not carefully thought out before implementation.

In making a case to each level of the organization (Step 4), the need for change must be established and a viable roadmap must be presented. Just as andragogical principles, such as the need for adults to know why they are learning something and that adults learn better with a problem-based approach than with a passive “fill me up with knowledge” approach (Freire, 1970), guide instruction of adults, they must also be utilized in any change management system (which is really just instruction anyway).

This andragogical approach to change management will ensure that all levels of the organization will claim ownership and participate in communicating the message to others (Steps 5 and 6). Steps 7 through 10 are really the core of this approach to change management, since they deal directly with changing the culture of the organization in question. This is especially interesting in terms of the current question of how to ensure the consideration of culture in instructional design.

Of course the new culture must be assessed before it may be addressed. Thorough assessment of the culture will identify values, beliefs, sources of leadership, and sources of resistance. It may be worth a reminder that this model is a change management model, which can be used for any business-related change, not just for instructional projects. A new culture may be desired or two merging companies may be combining their cultures. It also may be that other changes are made within an established culture. This model applies to instructional projects within a different culture, as long as the designers remember their obligation to develop materials that fit within the appropriate cultural context without intentionally or unintentionally disrupting established culture. It may be that an intervention is designed to change culture, such as recent campaigns in China to encourage taxi drivers to shower and brush their teeth in preparation for the Olympic Games held there earlier this year. Even such an intervention that is meant to improve cultural practices needs to be approached correctly in order to obtain buy-in from all stakeholders.

Throughout the change process, all those involved should be prepared to expect the unexpected. No plan goes perfectly, but by being willing to deal with issues as they arrive, everything will go more smoothly. The reason there are differences is because people are involved. The stakeholders should be respected, consulted, and otherwise involved in the process.

When designing instruction for those of other cultures, it may be that it is new instruction from the ground up or it may be a remix of existing instruction. The differences between a designer’s background and that of those who will be participating in the instruction need to be defined and integrated with the appropriate design model. ADDIE principles apply in any situation really, as long as the above-mentioned steps are taken to ensure integration with the new cultural milieu. Hites & Casterline (1986) discuss a few steps in designing instruction for other cultures. Based on a needs analysis, they recommend asking if technology is appropriate to the situation and if the objectives are the same. Next, they remind us to consider motivational factors, the level of English comprehension, entry skills and knowledge, and learning style preferences. At this point, all existing training materials and their delivery methods are up for change as needed. An important observation they make is the need to provide training for the instructor if he or she is not from the same culture as the students. Instructor training should include both content and culture. Following the ADDIE process, formative and summative evaluations should be held and appropriate revisions made.

By utilizing a combination of these models, it is more likely that effective, culturally relevant instruction will result. Designers should stay aware of their own cultural biases and backgrounds, stay focused on providing the best instruction while doing no harm, utilize stakeholders throughout the process of analyzing, designing, and developing instruction to create ownership among those to receive instruction, prepare the instructors as needed, and be flexible.

REFERENCES

Bowers, C.A., Vasquez, M., Roaf, M. (2000). “Native People and the Challenge of Computers: Reservation Schools, Individualism, and Consumerism.” American Indian Quarterly 24(2).

Freire, P. (1970). Pedagogy of the Oppressed. New York: Herder and Herder.

Hites, J.M., Casterline, S. (1986). “Adapting Training for Other Cultures.” Annual Conference of the National Society for Performance and Instruction.

Jones, J., Aguirre, D., & Calderone, M. (2004). “10 Principles of Change Management.” Strategy+Business.

Thomas, M., Mitchell, M., Joseph, R. (2000) “The Third Dimension of ADDIE: A Cultural Embrace.” TechTrends 46(2).

Incremental Andragogy

The principles of andragogy, covered well by the writers over at Open Education dotnet are important to keep in mind, even when running a teacher-centered or content-centered lesson plan.

My experience at work lately has been increased misunderstanding among students of the importance of CIL (Computer & Information Literacy) in their university careers. They don't see the point, so they are immediately disengaged from any learning, treating the tests we give as hoops to jump through rather than a learning opportunity. Thus, they become very frustrated when they can't pass the tests right away, since they're trying to pass the tests without learning anything. A key principle of andragogy is understanding why the things they're learning is relevant to them. Another principle is that you can learn from mistakes. They don't want to make mistakes, and they don't want to learn anything. When I sit down and explain to someone the purpose of the tests and how it can help them, they usually understand and turn around their attitude.

Even though the content we cover is pretty set and they don't have much if any control over what they need to learn, it's still helpful to understand the reasoning behind it.

One of the comments on the above-linked article points out that it may not work to open up the classroom, since almost all kids go to school now instead of just the academically inclined ones, as it used to be when this approach was developed. I think the principles of andragogy are even more important for students that aren't academically inclined. The so-called smart kids will learn no matter what epistemology is practiced by their teachers. Maybe if the students who have a harder time in school were helped to understand why they need to learn things and how what they learn fits in with their existing knowledge, they would be more motivated.

Pedagogy vs. Andragogy

In a class recently, we were discussing a constructivist environment for learning, and I made a contrast between constructivism and pedagogy. The professor looked really confused, and it only took a moment for me to figure out the problem was with my use of the word pedagogy. Defining pedagogy generally as the study or practice of instruction and tools the facilitate such, my statement definitely did not make sense, since constructivism is a theory/tool used in learning.

I modified my statement, substituting something along the lines of "a traditional classroom approach" instead of the word pedagogy, but I'm not quite so sure that was necessary. I think of pedagogy as a contrasting theory with andragogy. Pedagogy is traditionally an approach for teaching children and andragogy is a relatively new approach for teaching adults. Pedagogy is generally teacher-centered, with the teacher deciding who should learn what when and how. Andragogy is learner-centered, allowing learners to direct how learning occurs.

Andragogy brings with it a few principles, popularized by Knowles, which point out that adults, unlike children, have much life experience which allows them to more immediately contribute to a discussion, and because of that experience and the busy lives they lead, adults prefer problem-based learning that is applicable to their lives. Following an andragogical approach, a teacher becomes a facilitator of learning, providing materials and resources and keeping discussions in the right direction, but stays out of the way to let learning happen.

Taking andragogy to an extreme, some have suggested the term heutagogy as completely self-directed learning. I suppose my teaching myself how to put up drywall, texture walls, lay tile, and replace light fixtures as I've been working on renovating our bathroom has been a heutagogical approach, in addition to being very slow. (Of course, my slowness has not necessarily been due to heutagogy itself as much as a lack of time with work, school, scouts, and other family things taking precedence.) I could have taken some classes at the tech school in town or found a mentor to work with, but I've survived by using the internet and reading the little pamphlets at the home improvement stores as well as some trial and error.

These approaches need not be restricted to some set line, so when a person turns 16, he or she moves to a new class that uses andragogy instead of pedagogy. Andragogy can be used with children, and pedagogy with adults, just depending on the preferences and experience of the learners and the type of material being taught. As adults, we like being in control and a pedagogical approach is where we're most comfortable whether we're teaching children or adults, but are we willing to step back and let the lunatics run the asylum to some extent?