Cataloging trouble

The Catalog working group, of which I was a member, just finished its tasks of redesigning the University catalogs. The group was great to work with; it was UNC at its best – willing to change, innovative, informal, and deeply caring about students. I think we made a lot of progress, and 2010/11 catalogs will be much more user-friendly, and easier to navigate. I am still wondering why a simple question – Which courses do I need to take? – requires such a complicated answer. We have the Catalogs, check-sheets in various offices, four-year plans, and a whole host of advisers who interpret the mysterious written word of the catalogs for students. The paradox is, by trying to make college easier, we make it a lot more complicated than it has to be. To off-set this complexity, we spend a whole lot of resources trying to explain the complexity away. Here are some examples:

• As one of our group members noted, the Catalog reflects the logic of program creators, not of program users. For example, faculty members divide courses in core and supporting, major and Liberal Arts Core, within the main discipline and in other disciplines. But very few of these categories matter to students; they simply need to know which courses they have to take for sure, and with which they have some choices. They also want to know which courses can be more useful for the future, which are easier to get into, which are fun, and which require too much work. It becomes a simple conflict of categorization. It's like going to a pharmacy and finding the medicine arranged by price rather than by the kind of trouble you might be experiencing.
  
• If we leave students to their own devices, their rate of error in planning the coursework would be very high. We then require them to see an advisor to get a PIN, so they can register. But getting to see an advisor is another obstacle, another hoop to jump through. Most of the advisors on campus are faculty, who are not here all the time. Every year, many of them are new, and have their own trouble reading and understanding of the catalog. As a result, in an effort to reduce errors, we introduce a whole new layer of complexity which generates new errors and more frustrations.
  
• The whole Liberal Arts Core idea is very old and venerable, and designed with the best intentions. Ideally, it should give every student a broad education, and allow easier transfer to another institution. However, people have been messing with the LAC for years. For example, when they simply wanted to expand the major, certain LAC courses became "Specified required LAC courses." This move, of course, defies the purpose of choice and transferability. Instead of calling it what it is – a larger major – these programs just add a level of confusion. Moreover, some of the major courses proper also satisfy the LAC requirements, and therefore can be double-counted. So the two categories overlap significantly, and make figuring them out very difficult. The LAC list itself is a maze of categories, subcategories, and rules.
  
• Then, of course, the State of Colorado requires the majors be different than teacher education programs (PTEPS). It comes from an obscure ideological stance requiring teachers to know enough content. Therefore, we list PTEP courses as a different set of courses, further confusing students. And of course, the difference between PTEP and major courses makes no sense, and the two categories overlap also. In some majors, some of methods PTEP courses are also counted as major courses, and in others, they are not.
  

It is not only that these things are difficult to explain to an 18-year old, who is fresh out of high school. What strikes me as absurd that we need to explain these things at all. They truly don't care, and should not care about these categories. What makes it complicated for students is that they also have no idea how often courses are offered, how hard is it to get in them, and which courses are "stacked," which means some are pre-requisites for others, and cannot be taken just at any time. Here are my conclusions/recommendations:

1. Advising is supposed to be important, and it connects students to faculty on a more personal level. But if much of it s spent on explaining the same catalog mysteries over and over again, perhaps it is not what we hope it is. We need to make another, more radical step in making the catalog easy to understand by purging the categories relevant to program creation. Maybe we should ask students for help.
   
2. Students need to have easy access to the same data faculty have access to: history of course offerings, how full do they tend to be in the past, as well as the schedule as it is being developed. Last year, I copied advanced schedules from the report portal and e-mailed it to two thousand teacher education students. Many were very grateful, because it allowed them to plan better. But why is this information hidden from them in the first place?
   
3. We need to learn to build logic models of student scheduling. Following the many complicated rules associated with course choices is better done by a computer than by a human mind. One simple step toward this: we already have a degree check feature in Ursa. However, there is no way for students to run their mock 4-year pan through the degree check, and see if they would graduate with a given set of courses. So if we allowed them to build a 4-year plan, and run through a pretend graduation, it would eliminate a lot of errors, and reduce the burden of advising.
   
4. In all out programs, we need to keep only those choices that make sense, and eliminate all of those choices that are only there to support the idea of choice. Many of the choices we announce are unobtainable because courses listed there are never offered, or limited to majors. In other cases, choices serve no discernable reason to exist, or are results of turf wars and turf peace-making. If we eliminate those, it will probably reduce the choices by 2/3. And when we keep the choice, we need to be able clearly explain what are advantages of each option, and what are implications of choosing one option over another.
   

 

The Spreadsheet fallacy

Perhaps people don't remember it, but VisiCalc, which was Excel's forefather, was one of the crucial forces to unleash the personal computer revolution. Excel is great; it allows running infinite number of scenarios quickly, make patterns and ratios visible, and generally, it is a good way of making a solid argument.

However, the whole idea of numbers came about when people started to deal with large quantities of essentially the same or very similar things, like bushels of grain, heads of sheep, or buckets of beer. Almost none of the hunting-gathering societies generally count beyond 3 or 4, because if you had 10 arrows, they were all different, and you remembered them all individually. If you saw a herd of animals, that's what you called it, and there was no need to count them. If you don't deal with identical things, you cannot count them. Or rather, they have to be identical in at least one practically important aspect. For example, you cannot count loafs of bread, if they are very different in size. I mean you can try, but won't be happy with outcomes, because you may end up with more loafs but less bread. So, the sameness of things you're counting is the most fundamental assumption of mathematics.

Here is where the Spreadsheet fallacy comes in. Excel spreadsheets are so compelling that people are tempted to count apples and oranges as pieces of fruit, but then make conclusions about their average skin-thickness. For example, we have some classes that are real classes, and other classes that are independent studies, and still others that are checkpoint courses (which are not classes at all, but tricks to make our registrar database keep the information we need). We have classes for each people are getting paid, and classes that are done as service. Some classes are methods, while others are theory; still others are tutoring classes. Some classes are co-taught by different content specialists, while others are co-taught because it is easier to have one large class than 4 smaller ones. These all require different enrollment caps, different forms of compensation, different rooms, etc. When you put all of these things into one spreadsheet, you must assume that 1 credit=1 credit, and 1 instructor=1instructor. But the basic assumption of sameness, of the consistent unit of measurement just does not work. If the only source of your information about reality is the numbers in the spreadsheet, you may see phantoms rather than the reality. By trying to be objective, you may actually become less objectives. Numbers are only good when they measure something real.

We receive a lot of spreadsheet reports from the University. I am always impressed with their authors' Excel skills, but can rarely see the raw data that goes into the calculations. In more than one occasion, I discovered that the input included numbers that cannot even be added together, because they compare incomparable things. Thank god, most of these reports have been inconsequential so far. However, when some actual decisions and policies can be made based on the Spreadsheet fallacy, remember the rule:

SHOW ME THE RAW DATA!

The limitations of grapevine

We all rely on informal information exchanges; I have written about it before. Here is another story that shows what are the strengths and the limitations of the grapevine as an information channel. In Spring, I have received several informal reports about one of our adjunct faculty having problems in interaction with students, with work ethic, and perhaps with competence. We are not obligated to provide work to any part-timers, and actually have a large reserve pool. My plan was to check on the rumors, and if they prove to be correct, look for someone else to hire.
We have collected student feedback from a series of surveys, so there was actually some data to verify the story. What I discovered is that just one unhappy student was the source of all the reports I received. Imagine how it works: this one student has several classes, and voices the same concern with several instructors. She may pose it in a way that implies other students were also treated badly. The instructors all tell me that there is something I should know about. Of course, they don't mention the student's name, because of confidentiality of the initial conversation, and because the source may have strengthen her case by implying it is a larger problem. Because the story changes with each transmission, soon it sounds like three different stories. Moreover, faculty talk among each other, and a person who have heard it from another faculty, tells me about the problem in yet another form. Considering that I actually receive an astonishingly small number of complaints, three or four comments about the same person sound like a lot.
From the students' comments, it transpires that we perhaps did not provide clear enough instructions and expectations to this new part time faculty. Not only am I no longer sure if there is a personnel problem in the first place; it may have been our program's problem. I am now thinking it may have been only a small personnel problem, which is easier and fairer to solve by providing a little more training and support to the instructor, rather than rushing to replace her or him with someone else. The new person may have other problems, after all. And who does not?
The grapevine is a great way to alert about a possible problem. It is not really good at determining cause or the extent of the problem, nor is it a good helper in making any decision. This blog is more of a note to self, because I don't want to discourage people from sharing what they know with me or with each other. This is how we improve, and develop our professional community.

Pushing back

Here is a story from last week that really made me feel good. I noticed that when we get graduates licensure applications, we make a hard copy, send the original to the State, then scan the copy into our digital archive, and then shred the hard copy. This seemed to me like a redundant process, because our copier that makes hard copies can also be used as a scanner. So, I figured, we can save a step by scanning the originals, and then simply uploading these copies to the digital archive. Three people are involved in the process: Vicky, Marissa and Lynette. Vicky did not like my idea right away, Lynette had a lot of doubts, and Marissa did not say anything, but I could tell she did not like it either. They thought it would actually be longer to scan everything right away. But I pushed hard, because I like new solutions, and because it just made sense to me to skip a step and save a little paper. After some discussion, we agreed that they will try the new process I developed (and I had to work out a few technical kinks; probably two hours worth of work).

They did try it, timed themselves, and have proven that the new process takes twice as much time as the old one. Lynette had the killer argument: the new process takes a lot of concentration, and at her busy front desk station she is likely to make more errors. Vicky and Lynette broke the news to me at the end of the day, so I was forced to retreat, and acknowledge that I was wrong. Licensure is very time-sensitive, because graduates need to get jobs, and every day of delay may affect someone's job prospects. The paper, however, is cheap. The illogical process actually works better, and reduces the time in limbo, when a particular record is inaccessible (there is a time gap between scanning and indexing). The possibility of an error is a big thing: we learned the hard way how costly such errors can be.

But what made me really happy and proud was the fact that they did not give up, and kept pushing back, because they could prove the point. A school director has a lot of administrative power over staff, and it is not easy to have a culture where people feel comfortable defending their ideas, and telling the boss he is wrong. In part, we have it just because who these people are: Vicky, Karon, Marita and Lynette all have many years of experience, and a good common and professional sense. They know what they are talking about. But I also felt like I was doing something right, because our little debate did happen, and because I was wrong this time. The last thing I want to do is to make staff's work more difficult.

We did improve a little part of it though, reducing one step where Marissa has to look up every applicant in Ursa. We tricked our SIMS database into generating ready cover sheets for majority of applicants.