In policy-making, the greatest temptation is clarity, and I am going to make the case for resisting it. The case in point is our College’s Retention, tenure and promotion policy that we just recently adopted, but the logic applies to the entire policy-making world.
What is a policy? - A tool that should help us make more consistent and thus fairer decisions. The biggest temptation is to write a policy can help us avoid making decisions. Wouldn’t it be great to simply look into the document, apply a policy, and make an automatic, and therefore fair and objective decision? No one wants to hurt other people’s feelings, or defend one’s own judgement. Well, it simply does not work that way. In the legal universe, this has been known for a long time, in both Roman and Common law systems. You do need a judge and/or jury to consider cases and apply law, no matter how detailed the law is. In fact, the attempt to pre-empt judgement mostly backfired, as it was the case with the three-strike laws and with the minimum sentencing laws. Laws and policies that are closed to interpretation always lead to absurd consequences. For example, a guy steals a pair of $2.50 socks gets a life sentence.
In our little case, thanks God, stakes are much lowers. However, they are also absurd. Most of our faculty members going through periodic reviews are outstanding. This brings the embarrassing question of what crowd exactly are they standing out of, and the whole Lake Woebegone effect. What happened? Well, people who were writing the policy tried to take all arbitrary judgment out of it. It basically, requires evaluation committees to see if there is evidence in various categories. If you do have evidence in so many categories, you’re good, if you have in so many more, you are outstanding. The policy does not ask question – what kind of evidence, because that would open the decision to subjectivism. The document removes the potential bias out of the decision, which guarantees to generate absurdity. The result is that if you have published a paper in AERJ, which is the world’s top education journal, it is the same if you paid $300 bucks to a predatory paper mill that publishes absolutely anything. Or, if you sat through a few meeting of an entirely decorative committee, it counts the same as if you were reviewing hundreds of student IRB applications. How is this fair?
We would not need judges or evaluation committees if it were possible to automate decision-making. We could have a student assistant to flip through evaluation portfolios, check a list, and be done with it. A computer algorithm would be even better. Alas, it does not work.
A good policy may set a few examples, but it has to use evaluative criteria, not finite lists, or numeric values. In some contracts and policies, there is a word “normally,” which irritates many people, but not me. What a great way it is to indicate that exceptions are still possible, if they are within the spirit of what is normally the rule. There is no way to get a holistic judgement call out of decision-making. Such judgments are inherently biased and prone to mistakes. However, they are much better than the bean-counting process that simply assure certain level of absurdity. Multiple layers of independent review, certain level of mutual trust, and professional ethics can significantly improve decision quality. Adding more precision to the document rarely can.
What is a policy? - A tool that should help us make more consistent and thus fairer decisions. The biggest temptation is to write a policy can help us avoid making decisions. Wouldn’t it be great to simply look into the document, apply a policy, and make an automatic, and therefore fair and objective decision? No one wants to hurt other people’s feelings, or defend one’s own judgement. Well, it simply does not work that way. In the legal universe, this has been known for a long time, in both Roman and Common law systems. You do need a judge and/or jury to consider cases and apply law, no matter how detailed the law is. In fact, the attempt to pre-empt judgement mostly backfired, as it was the case with the three-strike laws and with the minimum sentencing laws. Laws and policies that are closed to interpretation always lead to absurd consequences. For example, a guy steals a pair of $2.50 socks gets a life sentence.
In our little case, thanks God, stakes are much lowers. However, they are also absurd. Most of our faculty members going through periodic reviews are outstanding. This brings the embarrassing question of what crowd exactly are they standing out of, and the whole Lake Woebegone effect. What happened? Well, people who were writing the policy tried to take all arbitrary judgment out of it. It basically, requires evaluation committees to see if there is evidence in various categories. If you do have evidence in so many categories, you’re good, if you have in so many more, you are outstanding. The policy does not ask question – what kind of evidence, because that would open the decision to subjectivism. The document removes the potential bias out of the decision, which guarantees to generate absurdity. The result is that if you have published a paper in AERJ, which is the world’s top education journal, it is the same if you paid $300 bucks to a predatory paper mill that publishes absolutely anything. Or, if you sat through a few meeting of an entirely decorative committee, it counts the same as if you were reviewing hundreds of student IRB applications. How is this fair?
We would not need judges or evaluation committees if it were possible to automate decision-making. We could have a student assistant to flip through evaluation portfolios, check a list, and be done with it. A computer algorithm would be even better. Alas, it does not work.
A good policy may set a few examples, but it has to use evaluative criteria, not finite lists, or numeric values. In some contracts and policies, there is a word “normally,” which irritates many people, but not me. What a great way it is to indicate that exceptions are still possible, if they are within the spirit of what is normally the rule. There is no way to get a holistic judgement call out of decision-making. Such judgments are inherently biased and prone to mistakes. However, they are much better than the bean-counting process that simply assure certain level of absurdity. Multiple layers of independent review, certain level of mutual trust, and professional ethics can significantly improve decision quality. Adding more precision to the document rarely can.
No comments:
Post a Comment