My colleagues are awesome teachers. Out entire School’s student evaluations average 4.14 on a 5 point scale. If you take out a couple of problematic part-timers, the average raises to a very impressive 4.48. It may be a flawed, but fundamentally fair process; most students speak their minds. So, we must be doing something right. Some people question validity of these evaluations, mainly because student satisfaction with a class does not always reflect academic achievement. However, we must be doing something right here. The very possibility of identifying a problem speaks to usefulness of the process. Our students expect to be asked for feedback as a matter of course, and many of them provide thoughtful, engaged comments. We have our share of problems, but can feel good about what we do.
However, I have this nagging suspicion that although each course might be good or very good, and most of my colleagues are extremely competent and dedicated teachers, the sum of these courses is less than what it could have been. The very structure of academic courses taught by different instructors may be questioned. It is a very practical, time-tested structure. You have specialists in different subjects who teach their chunks of knowledge and skills. There is an opportunity for each instructor to perfect his or her chunk, and to create a welcoming, supportive environment. However, anyone who has ever tried to either put together, or revise, or evaluate a program, knows that the shortcoming of the subject/course system are as real as its advantages. First, it is extremely difficult to get people to talk to each other and align curriculum. We, university professors, derive both psychic and tangible rewards from success of our individual course, not from the overall success of the program. We tend to be solitary, non-conformist, and fiercely independent people. Certainly, this is one of the main reasons for people to be attracted to the Academe: we want to be in control of our own work. We constantly tinker with our courses, and those drift apart from each other. Inevitably, questions arise: Who is teaching A? Should someone also teach B? Are we all trying to cover C, and waste our students’ time?
Fundamentally, we have very good grasp of individual course, but we do not know what is the totality of curriculum. We know what needs to be taught in a course X, but we have more difficult time understanding what needs to be taught in the whole program. What does an Elementary teacher need to know and be able to do? The State of Colorado, of course, came up with the performance-based standards that try to spell this out; and we pretend to meet them in our various courses. But let’s take a look at them; this one for example:
The teacher has demonstrated the ability to:
5.1 Create a learning environment characterized by acceptable student behavior, efficient use of time, and disciplined acquisition of knowledge, skills, and understanding.
This is an extremely tall order. Our graduates are supposed to know how to make kids behave and to learn. But OK, it’s important. So, how do you even begin to teach them? What specifically, this ability to create a learning environment actually entails? Can anyone give a step-by-step instructions? Are there any exercises we can assign to train these abilities? And if not, if this is still a form of art, how can you demonstrate the ability?
The standards are actually a list of qualities of a superhero; no one living individual can claim to meet all of them. But teaching is a mass profession; we will always have some talented, and some average people in there. Maybe we should begin with a picture of a good-enough teacher, with a much smaller list of very specific, observable skills and specific knowledge? When we are trying to cover all the State standards, we inevitably stretch the truth, and pretend to teach something we do not really know how to teach. We have double-accounting: every one of us is a former classroom teacher, so we know in our guts what’s important, and what cannot be missed. However, explicitly, we proclaim adherence to the super-hero model, and try to cover a lot of ground. We go wide but shallow.
And then, of course, we discover gaps. What is called the classroom management is an 800-pound gorilla; this is where most new teachers’ anxieties are, and for a good reason; the inability to meet the standard 5.1 is probably one of the biggest career-killers in the world of teaching. And yet we treat it as one of many standards. Well, I disagree. I don’t think knowledge of school funding, or school law or even content knowledge are as important as 5.1. It is, of course, related to some other skills: the ability to plan instruction without killing yourself in the process, the ability to relate to children, and to read peer group, etc. One has to have a mental list of what kids of certain age can and cannot do, etc. But please, 7.4:” 7.4 Apply technology to data-driven assessments of learning…” Is it really that important?
All this super-hero stuff keeps us even more entrenched in our classes, because there we at least can prioritize, and ignore the rest. “At least they will have one good class” we think. That is where much of opportunity for learning is lost: we do not explicitly build on each other’s classes, we do not know what no one teaches, and we do not know what is being taught three or four times. We are not consistent in our own teaching practices that our students will inevitably model.
I am just wondering; I do not have an answer. I only know that simple appeals like “let’s talk to each other” are not going to work. I also know that NCATE and State reviews do not achieve their intended goals of forcing people to think in terms of programs, and specific evidence. So, how do we create programs that are less fractured, less territorial, and more focused, more consistent? Perhaps we should start questioning the idea of the subject-based class? Maybe I should have a group of students I will follow through throughout whatever knowledge and skills we want them to have?
Academia as a habitat
I have been writing this blog since 2006. In 2024, I created another blog called "AI in society" . This one will return to postings about life in academia and personal musings.
Jun 15, 2007
Jun 8, 2007
Shift Left
I have spent a significant part of this week working on computer. Last Summer, as soon as I came aboard, a problem presented itself: the teacher education database run on Helix had to be replaced. The company that supports it went bankrupt; the program cannot be run on any PC and on new Mac operating system. The database was developed over some 15 years, and contains thousands of records, dozens of screens, reports, and other features. The dilemma we faced was that a new commercial product that would do the same thing costs about $30,000, plus it would require significant annual maintenance fee. The other option was to develop our own database, but we have neither expertise, nor resources to do so.
The Helix is something faculty do not deal with at all: tracking student admissions to PTEP’s, their compliance with fingerprinting, TB testing, and other State requirements, keeping track of their admission to PTEP status, making sure they took all prerequisite courses before proceeding to the next level, etc. Then we place students for their multiple field experiences, and finally, recommend them for licensure. I have to admit, it took me a couple of months just to figure out what is it we do here in STE office. Again, this is something faculty do not necessarily know or involved in, and in a university, what does not concern faculty directly tends to move to background. Our dedicated staff plugs away at all this ever-more complicated processes, without much attention or appreciation of their work. However, all these tasks are essential for the School’s operations, and I had no choice but to try to figure out a solution.
Eugene, the Dean was actually supportive, although he blinked when he heard the amount we are likely to need to buy the new commercial product. “If you really needed, we can find the money,” he said. Then we explored several competing providers, and met with one of them to discuss a possible deal. That is when I got anxious. First, we would be buying a product that does not exactly fit our needs. Then, in order to modify it, we would have to have endless meetings with providers. You see, with the techies, you have to explain in detail what you want, and they will do it for you. But figuring out what you want is well over half of the solution, so you pay them for the work you did. And finally, the new system they were selling to us would be difficult to modify: we’d have to pay them every time we want some change. And who knows if the company will go bust, just like Helix did?
I called for help on our wonderful staff: Karon, Vicky, Marita and Layne, and we brainstormed a solution. What we came up with is an example of the “shift left” strategy, or improvement through radical simplification. Read about the meaning of the expression here. What many people do not realize is that software industry has a vested interest in selling people ever-more complicated, bloated products. I am playing with Office 2007 right now, and boy it is bloated. It has some nice features, and some features no one will ever use, but it is a memory hog I would never pay my own personal money for. Why do they sell us all those monsters? Part of it is explainable with simple lack of imagination, and lack of attention to consumers. However, a bigger part of it has to do with money – in the absence of cheaper alternatives, consumers have to shell out cash for complicated, cumbersome systems. Anyway, I am proud to say, we resisted, and here is our solution:
1. We invented the checkpoint courses – the fake courses in which students will bring whole packages of paperwork, and we will give them credit. Thus, we will be using the existing Registrar’s data base to perform a function that was not initially intended for it. Thanks to the Registrar people for being such good sports and supporting us on this. This took care of more than half of the old Helix functionality, and is going to save us a lot of time on data entry.
2. We bit the bullet, learned Access and developed our own smaller, simpler database. Of course, as the geekiest person in the office, I had to do most of the developing, but the idea is that at least two or three people here would know enough to tweak it when needed: to add or remove a field, to put together another form, etc. Access is unlikely to discontinue; it is a part of the standard Office, and we can always find an expert if we run into difficulties.
3. And finally, we are going to integrate Blackboard’s test feature to collect information from students and then import into our new database. This part is still in development, but I am confident it will work. This will save us more data-entry time.
It is not only that we have saved 30K, plus some 5K annually, but we also were able to simplify and streamline the processes here in the office. One of our staffers jokingly asked, “if you automate everything, will we lose our jobs?” That’s just not the case. Our staff has a lot of expertise and experience, so they will be more closely involved with advising and guiding students through their programs. There is always more work to do; the trick is to replace boring, tedious work with more challenging, more interesting, and more useful work. Of course, in the meanwhile, I am sitting here designing forms, and it is not fun. But I am not complaining, and it is gratifying to see things that you first only imagined actually work.
The “shift left” move is very useful, and not only in software design. Nowadays, we tend to make things more complex, just because we can. We all need to learn to simplify, and to keep it simple.
The Helix is something faculty do not deal with at all: tracking student admissions to PTEP’s, their compliance with fingerprinting, TB testing, and other State requirements, keeping track of their admission to PTEP status, making sure they took all prerequisite courses before proceeding to the next level, etc. Then we place students for their multiple field experiences, and finally, recommend them for licensure. I have to admit, it took me a couple of months just to figure out what is it we do here in STE office. Again, this is something faculty do not necessarily know or involved in, and in a university, what does not concern faculty directly tends to move to background. Our dedicated staff plugs away at all this ever-more complicated processes, without much attention or appreciation of their work. However, all these tasks are essential for the School’s operations, and I had no choice but to try to figure out a solution.
Eugene, the Dean was actually supportive, although he blinked when he heard the amount we are likely to need to buy the new commercial product. “If you really needed, we can find the money,” he said. Then we explored several competing providers, and met with one of them to discuss a possible deal. That is when I got anxious. First, we would be buying a product that does not exactly fit our needs. Then, in order to modify it, we would have to have endless meetings with providers. You see, with the techies, you have to explain in detail what you want, and they will do it for you. But figuring out what you want is well over half of the solution, so you pay them for the work you did. And finally, the new system they were selling to us would be difficult to modify: we’d have to pay them every time we want some change. And who knows if the company will go bust, just like Helix did?
I called for help on our wonderful staff: Karon, Vicky, Marita and Layne, and we brainstormed a solution. What we came up with is an example of the “shift left” strategy, or improvement through radical simplification. Read about the meaning of the expression here. What many people do not realize is that software industry has a vested interest in selling people ever-more complicated, bloated products. I am playing with Office 2007 right now, and boy it is bloated. It has some nice features, and some features no one will ever use, but it is a memory hog I would never pay my own personal money for. Why do they sell us all those monsters? Part of it is explainable with simple lack of imagination, and lack of attention to consumers. However, a bigger part of it has to do with money – in the absence of cheaper alternatives, consumers have to shell out cash for complicated, cumbersome systems. Anyway, I am proud to say, we resisted, and here is our solution:
1. We invented the checkpoint courses – the fake courses in which students will bring whole packages of paperwork, and we will give them credit. Thus, we will be using the existing Registrar’s data base to perform a function that was not initially intended for it. Thanks to the Registrar people for being such good sports and supporting us on this. This took care of more than half of the old Helix functionality, and is going to save us a lot of time on data entry.
2. We bit the bullet, learned Access and developed our own smaller, simpler database. Of course, as the geekiest person in the office, I had to do most of the developing, but the idea is that at least two or three people here would know enough to tweak it when needed: to add or remove a field, to put together another form, etc. Access is unlikely to discontinue; it is a part of the standard Office, and we can always find an expert if we run into difficulties.
3. And finally, we are going to integrate Blackboard’s test feature to collect information from students and then import into our new database. This part is still in development, but I am confident it will work. This will save us more data-entry time.
It is not only that we have saved 30K, plus some 5K annually, but we also were able to simplify and streamline the processes here in the office. One of our staffers jokingly asked, “if you automate everything, will we lose our jobs?” That’s just not the case. Our staff has a lot of expertise and experience, so they will be more closely involved with advising and guiding students through their programs. There is always more work to do; the trick is to replace boring, tedious work with more challenging, more interesting, and more useful work. Of course, in the meanwhile, I am sitting here designing forms, and it is not fun. But I am not complaining, and it is gratifying to see things that you first only imagined actually work.
The “shift left” move is very useful, and not only in software design. Nowadays, we tend to make things more complex, just because we can. We all need to learn to simplify, and to keep it simple.
Jun 1, 2007
The 90/10 rule
The data technology revolution is here. Because we live in the midst of it, it is hard to see the magnitude of the changes we experience. Many industries have produced tremendous gains in productivity just because they found better ways of shuffling data. For example, truckers used to spend hours on the phone trying to get through to a dispatcher to assign them a load. Now they use their laptops and find a better load in no time. The global trade is impossible without informational flows on goods and capitals movement. Even fishermen and peasants benefit from improved information about market prices. People started to ask questions they have never asked before, and collecting information that was prohibitively expensive to collect before.
What about us? Has higher education, and teacher education in particular, caught up with the revolution? Has our work productivity increased significantly? The answer is no: the cost of K-12 and higher education continues to rise faster than inflation, while no one was able to demonstrate significant gains in academic achievement. So next time when you are buying a $30 DVD player in Wal-Mart, ask yourself why can’t you pay lower tuition for your kids’ colleges. The technological revolution somehow did not affect the core of educational business.
And it is not because universities do not invest in technology; they buy new gadgets all the time, dig tranches for fiber optics, purchase expensive database management systems, and hire consultants to figure out IT problems. In fact, increases in tuition are often justified with the need to purchase new technology. Yet something is wrong: phone companies do not increase their rates because they switch to computers; in fact, they will cut rates because of technology, and that is the whole idea. How come we buy computers and become less efficient? In education, many things are a lot more efficient now: it is easier to register for classes, to pay tuition, and to contact students. Why does it not translate into higher overall productivity? It is clear that the periphery of educational industry has become more efficient, but the core – the business of teaching – has not. K-12 education is a state monopoly, and thus inefficiency can be explained by lack of competition. However, higher education below Ivy League has a robust competition in many areas, and still tuition keeps climbing up. Why is that?
Part of the answer is in the very nature of teaching: it is individualized, laborious process. For example, for my students to make adequate progress in writing skills requires at least three papers a semester. When I grade them I use a number of technology tricks, like Auto Text entries, automatic calculation of total points, and e-mail notification. All these innovations I worked to implement save, oh, maybe 10 percent on my grading time. I still have to read all the papers, make sense of them, and give specific suggestions to each student on how to improve. No machine can do this yet (although ETS is experimenting with computer grading; it is not that good so far, I checked). While there are many ways to improve teaching, and make it more effective, there does not seem to be a way of increase student/teacher ratio without damaging the result; not without some horrendous Lancaster system or similar monstrosity.
Is there a way to do it though? Much of teaching is about information flows, although it is not always the type of information that is easily transferrable through computers. Teaching also involves highly individualized, and reciprocal information exchanges. In class, I can gauge how well students have learned whatever I want to teach, and quickly adjust to meet them half-way. They can ask questions, and engage in multiple participant discussions. So, the problem boils down to the kinds of informational exchanged, not the amount of information.
I believe there could be significant gains in productivity without the loss of quality, if only we overcome cultural and economic barriers. For example, there are no good training videos for teachers. I can find plenty of sugary videos with teacher stories, opinions, inspirational crap, etc. But can anyone point at a set of practical videos, where you would see, say, a first day of classes in elementary school? How about effective ways of dealing with disruptive students in middle school? How to engage a high school class in deep questioning? Given that we train a very large profession, with very high turnover rates, it is amazing to me no one has done it. We send all these hundreds of teacher candidates to more or less random classrooms, instead of carefully selecting truly best practices and showing them to all.
How about a simulation game? Pilots and military officers spend hundreds of hours in virtual reality simulation environments. Not all teaching can be simulated, but a lot of it can be.
And finally, let us take a hard and unsentimental look at what we, university professors, actually do in classrooms. Each one of us has a library of activities, phrases, little stories, and lecture bits we end up delivering again and again and again. Why not record all of this in either text or video format, so we can save energy on doing what we truly need to do differently for each individual student: answering unique questions, giving specific performance feedback, evaluating. Even then, most of the questions are the same, most of student errors and misunderstandings can be separated in a small number of specific groups, much of evaluation involves providing the same or similar feedback.
Of course, there are two barriers: first, the cultural one. We learned to value personal connection with students. They need personal connection when we intimidate them with unresponsive systems, and do not tell them what they need to learn and why. It is not like professors are so cool, students want to hang out with us. If we make our tremendous hoop-jumping machinery a bit more transparent and easy to go through, very few of us would be sought out for advising. Students need good, carefully selected information and specific feedback on how they are doing. Most professors believe (as in religious faith) that face-to-face interaction with a small class is the only form of effective teaching. Of course, none of them ever tried anything else, so their belief is not based on anything rather than blind faith with a pinch of general conservatism and unwillingness to change.
The second barrier is much more difficult to overcome: there is simply no time for me or any other professor to sit down and invest large amounts of time in designing a perfect course that can be delivered partially through video and other technologies. Such a course would require enormous resources. Of course, it could be then replicated hundreds of times, and can pay for itself over and over again. However, there is no mechanism for a university to make these sorts of investments and then benefit from its results.
In our office operations, I advocate a 90/10 rule: 90% of students should take 10% of time, while remaining 10% of students should take 90% of time. This is how things should work. We have to automate most of the processes so that those really needing individual help can have our time and energy. Well, the same rule should apply to teaching itself: 90% of it should only take 10% of professor’s time, so she or he is free to do what we can do best, and where individual attention is truly needed.
What about us? Has higher education, and teacher education in particular, caught up with the revolution? Has our work productivity increased significantly? The answer is no: the cost of K-12 and higher education continues to rise faster than inflation, while no one was able to demonstrate significant gains in academic achievement. So next time when you are buying a $30 DVD player in Wal-Mart, ask yourself why can’t you pay lower tuition for your kids’ colleges. The technological revolution somehow did not affect the core of educational business.
And it is not because universities do not invest in technology; they buy new gadgets all the time, dig tranches for fiber optics, purchase expensive database management systems, and hire consultants to figure out IT problems. In fact, increases in tuition are often justified with the need to purchase new technology. Yet something is wrong: phone companies do not increase their rates because they switch to computers; in fact, they will cut rates because of technology, and that is the whole idea. How come we buy computers and become less efficient? In education, many things are a lot more efficient now: it is easier to register for classes, to pay tuition, and to contact students. Why does it not translate into higher overall productivity? It is clear that the periphery of educational industry has become more efficient, but the core – the business of teaching – has not. K-12 education is a state monopoly, and thus inefficiency can be explained by lack of competition. However, higher education below Ivy League has a robust competition in many areas, and still tuition keeps climbing up. Why is that?
Part of the answer is in the very nature of teaching: it is individualized, laborious process. For example, for my students to make adequate progress in writing skills requires at least three papers a semester. When I grade them I use a number of technology tricks, like Auto Text entries, automatic calculation of total points, and e-mail notification. All these innovations I worked to implement save, oh, maybe 10 percent on my grading time. I still have to read all the papers, make sense of them, and give specific suggestions to each student on how to improve. No machine can do this yet (although ETS is experimenting with computer grading; it is not that good so far, I checked). While there are many ways to improve teaching, and make it more effective, there does not seem to be a way of increase student/teacher ratio without damaging the result; not without some horrendous Lancaster system or similar monstrosity.
Is there a way to do it though? Much of teaching is about information flows, although it is not always the type of information that is easily transferrable through computers. Teaching also involves highly individualized, and reciprocal information exchanges. In class, I can gauge how well students have learned whatever I want to teach, and quickly adjust to meet them half-way. They can ask questions, and engage in multiple participant discussions. So, the problem boils down to the kinds of informational exchanged, not the amount of information.
I believe there could be significant gains in productivity without the loss of quality, if only we overcome cultural and economic barriers. For example, there are no good training videos for teachers. I can find plenty of sugary videos with teacher stories, opinions, inspirational crap, etc. But can anyone point at a set of practical videos, where you would see, say, a first day of classes in elementary school? How about effective ways of dealing with disruptive students in middle school? How to engage a high school class in deep questioning? Given that we train a very large profession, with very high turnover rates, it is amazing to me no one has done it. We send all these hundreds of teacher candidates to more or less random classrooms, instead of carefully selecting truly best practices and showing them to all.
How about a simulation game? Pilots and military officers spend hundreds of hours in virtual reality simulation environments. Not all teaching can be simulated, but a lot of it can be.
And finally, let us take a hard and unsentimental look at what we, university professors, actually do in classrooms. Each one of us has a library of activities, phrases, little stories, and lecture bits we end up delivering again and again and again. Why not record all of this in either text or video format, so we can save energy on doing what we truly need to do differently for each individual student: answering unique questions, giving specific performance feedback, evaluating. Even then, most of the questions are the same, most of student errors and misunderstandings can be separated in a small number of specific groups, much of evaluation involves providing the same or similar feedback.
Of course, there are two barriers: first, the cultural one. We learned to value personal connection with students. They need personal connection when we intimidate them with unresponsive systems, and do not tell them what they need to learn and why. It is not like professors are so cool, students want to hang out with us. If we make our tremendous hoop-jumping machinery a bit more transparent and easy to go through, very few of us would be sought out for advising. Students need good, carefully selected information and specific feedback on how they are doing. Most professors believe (as in religious faith) that face-to-face interaction with a small class is the only form of effective teaching. Of course, none of them ever tried anything else, so their belief is not based on anything rather than blind faith with a pinch of general conservatism and unwillingness to change.
The second barrier is much more difficult to overcome: there is simply no time for me or any other professor to sit down and invest large amounts of time in designing a perfect course that can be delivered partially through video and other technologies. Such a course would require enormous resources. Of course, it could be then replicated hundreds of times, and can pay for itself over and over again. However, there is no mechanism for a university to make these sorts of investments and then benefit from its results.
In our office operations, I advocate a 90/10 rule: 90% of students should take 10% of time, while remaining 10% of students should take 90% of time. This is how things should work. We have to automate most of the processes so that those really needing individual help can have our time and energy. Well, the same rule should apply to teaching itself: 90% of it should only take 10% of professor’s time, so she or he is free to do what we can do best, and where individual attention is truly needed.
Subscribe to:
Posts (Atom)