User:KurtWatts/OKAIRP

What is the Office of Institutional Research?
An organization of people who do research for large and small colleges and universities to provide school administrators with data to help them make informed management decisions and pass effective policy.

The best way to describe the Office of Institutional Research
I have to wash the dishes before I load them into our automatic dishwasher at home. Automatic dishwashers only work when you put clean dishes in them. People buy automatic dishwashers because the very idea of a machine that automatically washes and dries your dishes is so practical and so desirable that the fact the machines don't actually work is not important. Automatic dishwashers are a myth created by science and technology, and perpetuated by human denial and wishful thinking.

The Office of Institutional Research is a lot like an automatic dishwasher. The very idea that big computers and complex scientific methods can automatically find workable solutions to the problems facing school administrators is so practical and so desirable that every university has an office devoted to research and assessment, even though most of the data collected by these offices have never actually been used to solve problems facing school administrators.

Why the rift between institutional research and school policy?
It's been like this for many years:

"Although assessment activity was observed, the university does not appear to integrate assessment outcomes information in decision-making purposes." "(Regarding) Assessment: Collected data is not effectively used in making decisions."

"It is unclear how the university uses collected data and results for organizational decision making and innovation. Information is gathered in great quantity, but it is unclear whether there is a systematic process to use data to guide strategic decision making and to enable innovation."

"...it's clear that the university has some ground to make up in the collection, analysis, and interpretation of data..."

Much of the research conducted by The Office of Institutional Research is unreadable, vague, and irrelevant
Most of the people doing institutional research for universities are not real researchers. They are 'data grabbers.' They have no formal training or experience in technical writing, research, or analysis. And it shows in this report written by the current director of Institutional Research at a large midwestern university:

"The trends that are evident in this document are indicative of many institutional, state, and national factors. These factors have contributed to both gained and declines in enrollment during this five year period. An analysis of these factors and appropriate planning and collaboration may contribute significantly to better management of enrollment at our university. This document was developed to promote such an enrollment management conversation." This is the summary from a ten-page report to the president of the university. No trends are specifically identified, no real analysis of the data is made, and no clear recommendations are offered. There are grammar and spelling errors throughout the document, and key graphs referred to in the text are missing from the report. The writing is wordy and pretentious, but the report doesn't really say anything:

"...It is imperative that a recruitment plan be developed to sustain and increase the number of students who are enrolled at the graduate level. It is important also that this plan incorporate the academic standards of quality deemed necessary for ultimate student retention and graduation."

The author does not sign her work. The pages of her report are not numbered. She is careful, however, to include numerous disclaimers so as not to offend anyone who might actually read her report:

"Even though these numbers may demonstrate potential trends in enrollment as it relates to headcount and credit hour production at our university, it is only a small piece of the institution’s enrollment landscape."

Those responsible for the low retention and graduation rates at our colleges and universities
Marketing schemes, motivational rhetoric, and leadership programs will not improve retention and graduation rates at our colleges. Only good teachers will improve retention and graduation rates at our colleges. But teaching is a subjective thing and cannot be measured scientifically. The art of teaching does not easily lend itself to scientific analysis.

Those truly responsible for our low retention and graduation rates belong to the Office of Institutional Research. They simply are not doing their job. As cited at the beginning of this section, several independent studies have reported complaints against the Office of Institutional Research time and time again. But the Office of Institutional Research continues to do the same old things that clearly don't work. Their methods are standardized, accepted, politically safe, and totally impotent in solving our current problems. No new information is allowed in because, "That's not the way we do things in the Office of Institutional Research."

If we are going to see any real improvement in retention and graduation rates at our colleges and universities, we must introduce new ideas into the system; new ways of doing and seeing things. Politics and personalities must be set aside, and our teachers must be given first priority.

Research is far more complicated than it needs to be
Complexity makes people feel important. When I'm called upon to perform some complex form of statistical measure like predictive modeling, the people asking for my services often don't know what they want to prove or disprove. They simply want something impressively complex and technical that can be incorporated into their recruiting or marketing plan to give it an air of sophistication and authority.

Predictive modeling does look impressive, but it can't prove anything. Like most computerized statistical measures, predictive modeling is based on probability theory. Tarot cards and astrology have a better track record than probability theory in predicting events. Wall Street analysts use probability theory extensively, and look where our economy has landed.

A brief and direct report with simple graphs showing clear patterns of data and how the data moves can often be of more value to policy makers than the verbose and often pompous literature mass-produced by the Office of Institutional Research.

People accept research as valid only if it proves what they already believe to be true
On those rare occasions when school administrators accept my research as valid, it's only because my research has proved something they already believe to be true. For example, if I were to build a complex predictive model that proved physically attractive recruiters are more successful than unattractive recruiters in luring prospective students to our university, I know exactly how recruiters would respond to my report. The homely recruiters would fervently negate my findings and label my research "unscientific and worthless." The good-looking recruiters, on the other hand, would enthusiastically support and defend my report as "a work of genius."

Simple rules for good research reporting
1. Reports should be as short and concise as possible.

2. Reports should be written in everyday language.

3. Reports should present a concise summary of findings on the front page.

4. Every page should be numbered and display the name of the author and report title.

5. Reports should deal with relevant issues of genuine value and interest to policy makers.