« Monday Morning Crabbiness | Main | Addendum to my last entry »

If it's broken, how can we make it worse?

I'm not really a happy camper in the world of higher education these days. The U of M is currently undergoing a Strategic Positioning process in order to become one of the top 3 research institutions in the world within the next 10 years (8 now). This is not new news, nor is the U alone in its endeavors. Many institutions are doing similar things, trying to boost their rankings in whichever ranking system they have targeted. Some may be trying to rate higher in the U.S. News and Bullsh*t..ahem, World...Report, while others have found other measurement scales upon which to rise in numbers. I can't even find the scale the U is using, but I've heard "Florida measurement system" thrown around a few times.

All of this is well and good, maybe. Student selectivity seems to be one of the criteria for many ranking scales: the more students who want to come get turned away, the higher in status the school rises, leading to more students wanting to attend. Lovely little circle, that, but it fundamentally seems to ignore some of the access and outreach missions of a land grant university.

The other large concern I have is on a national level: accountability for education. In other words, how can "we" [taxpayers, legislatures, the country] guarantee that our students are learning anything at "your" institution. Recent articles in Inside Higher Ed and The Chronicle of Higher Education (see the extended entry for examples) indicate that this is not a momentary trend, but is something here to stay.

The problem is that education is not a tangible thing that can be measured in charts and graphs, yet Education Secretary Margaret Spellings is looking for ways for "accreditors and colleges to provide tangible proof that they are educating their students." Many institutions look at graduation rates as an indicator of student education. Given some of the examples of reading and writing skills of students who have graduated from high school, I can pretty much guarantee that a diploma does not necessarily equal education. On the flip side, lack of a diploma does not mean a person is uneducated.

"Measuring student outcomes" appears frequently in the first article below from Inside Higher Ed. To what outcomes are we refering? Pascarella and Terenzini have spent 3 decades measuring student outcomes such as student change; verbal, quantitative, and subject matter competence; cognitive skills and intellectual growth; psychosocial change; attitudes and values; moral development; educational attainment and persistence; career and economic impacts of college; and quality of life after college. But it's crucial to note that these outcomes have only been measured for students who have attended college. There are no comparable studies for those who have not attended. While we can probably reasonably assume that the levels differ between students and non-students, we don't know for sure.

And how can they really be measured? Terenzini and Pascarella's work is widely accepted as one of the best, if not the best, examination of all of the research that has been done on student outcomes. But the time and money that would be required for each institution in the U.S. to do all of those measurements would end up breaking the bank.

I'm not going to go any further, though I could probably write a dissertation of just plain ranting on this topic. An alternate title to this entry could be "How my love affair with higher education ended." It's me, not you [higher education]. Forces beyond your and my control are pushing us in a direction I fear.

Dec. 5

Can You Say NACIQI?
In the alphabet soup of acronyms of Washington higher education, most people could probably go a long time before running across — or caring about — the federal panel known as NACIQI. But while the National Advisory Committee on Institutional Quality and Integrity is probably not poised to become a household name in the faculty lounge or the campus dining hall, it is clear that the profile of the panel that advises the U.S. education secretary on accreditation is about to enjoy one of its periodic moments of greater visibility.

Judging from Monday’s meeting of NACIQI (nuh-SEE-kee), the first since Education Secretary Margaret Spellings and the report of her Commission on the Future of Higher Education put major changes in accreditation near the top of their reform agenda, that visibility is likely to turn up the pressure on accreditors and colleges to provide tangible proof that they are educating their students.

Also likely to increase, if Monday’s meeting was any indication, are questions about just how hard the advisory committee can push in that direction without running afoul of federal laws and Congressional prerogative.

The advisory panel is charged, among other things, with recognizing the authority of individual accrediting agencies to operate, and therein lies its power: Without the approval of NACIQI, an accreditor’s stamp of approval of a college does not carry with it the all-important right for the institution’s students to receive federal financial aid. Although the work of the panel and a now-defunct predecessor have occasionally flared into controversy — such as when it sought to limit one regional accreditor’s imposition of a diversity standard in the early 1990s — the advisory committee has largely operated out of the limelight.

That may be about to change, because Spellings has made clear — most recently at last week’s forum she called on accreditation — that she sees the panel as one way the Education Department might be able to carry out its effort to compel colleges and universities to collect and report better data without the need for new laws or federal rules.

This view, which several participants at last week’s accreditation forum urged, holds that NACIQI can, using its existing standards for judging whether individual accrediting agencies deserve recognition, begin to force accreditors (and, by extension, colleges) to produce, collect and publish more and better information about student outcomes. Legal experts are divided on just how far the department might be able to go along those lines, but most agree that the NACIQI standards are broad and unspecific enough that there is “a lot of running room.”

At the start of Monday’s three-day meeting, Carol D’Amico, executive vice president of Ivy Tech Community College and chair of NACIQI, told her colleagues, “I think we’re now out of the closet in terms of our role in the whole issue of accreditation and quality. I think the secretary has high expectations for this group going forward.”

The advisory committee had already been pushing in the direction of more accountability for learning outcomes over the last year or two, but Monday’s meeting of the panel offered several signs that that trend may be accelerating. The panel’s new vice chair, Geri H. Malandra, interim executive vice chancellor for academic affairs at the University of Texas System, is closely tied to Charles Miller, who as chairman of the Spellings commission had tough words for accreditors and colleges on the learning outcomes issue.

Perhaps more importantly, some of the reports the panel’s staff prepared for this week’s meeting were perceived as pushing accreditors harder and further on measuring learning outcomes than they have been pushed before. And the one accreditor that had a chance to respond Monday — the Western Association of Schools and Colleges’ Accrediting Commission for Senior Colleges and Universities — was firm in pushing back.

The staff report for the Western accreditor found four areas in which the agency needed to improve, including a need to “clarify how it will evaluate the quality of an institution’s effectiveness based on the student outcomes data it collects and to outline in its procedures its expectations for institutional improvement (student learning) throughout the accreditation cycle.” Although the staff recommended that the Western association be re-recognized for the standard five years, it urged that the accrediting agency be required to report back in a year on its progress in fixing the perceived deficiencies.

When it was his turn to speak, Ralph A. Wolff, president and executive director of the Western association’s senior college commission, conceded three of the department’s four points but challenged the finding on student outcomes data, which he and other officials from the accrediting group said essentially would require it to tell colleges what performance measures they should meet. He defended the agency’s “exceptional record” in holding the colleges it accredits accountable for their performance in educating students, and accused the committee’s staff of changing the standards midstream.

“I want to say that you, as an advisory committee and decision making committee, when new rules are applied, I would raise the issue of consistency and fairness, that they be applied equally and consistently to all accrediting bodies,” Wolff said. “To have it applied singly to our agency, we would submit, unfairly burdens our institutions beyond what we are are already doing.”

Members of the committee, its staff and the Western association proceeded to spend nearly an hour trying to reach agreement on exactly what the commission was asking for, and how big a chance it represented from what the committee has asked previously. John Barth, the Education Department’s director of accreditation and state liaison, said that Western officials themselves had identified “triggers,” such as graduation rates, that they would use to gauge colleges’ performance at various points in the accreditation process. “What we are requesting of the agency” is that it identifies “a somewhat brighter line about how they’d let us know how they’re going to make a decision about [how a college has performed on] those triggers.”

Richard Winn, associate director of the Western association’s college commission, said it would be a “major new regulation” for NACIQI to ask accreditors to set what he called “bright line indicators” for what is acceptable performance for an institution. Requiring accreditors to set benchmarks for performance by the colleges they oversee, Wolff said, would represent a new and unacceptable level of federal intrusion. Referring to Spellings’s statement at last week’s accreditation forum that the department planned to “do this with you, not to you,” he added: “To impose that on a single agency at this point, with no further discussion, would be to us.”

A recent addition to the accreditation panel, Arthur Keiser, president of the for-profit Keiser Collegiate System, said he thought it was legitimate for accreditors to set minimum levels of performance. Citing a hypothetical institution that graduates as few as 3 percent of its students, he said, “I don’t think institutions can abrogate their responsibility.... It is not acceptable for us to say that that institution is demonstrating success. At some point we have to be able to say that that [level of performance] is just not acceptable. That’s our role. We can’t just ignore this thing.”

Lawrence J. DeNardis, president emeritus of the University of New Haven and one of the longest-serving members of the advisory committee, suggested a compromise in which the Western association, given its “pioneering work” in measuring student outcomes, would prepare a report about how its member institutions most effectively use data on student success to gauge their performance, in exchange for dropping the finding that it had fallen short of the NACIQI standard. “I would ask that they press ahead and provide some thoughts to us that might be useful systematically,” DeNardis said, encouraging the Western association to “be at the cutting edge.”

After a bit more discussion — including a warning from Keiser that “the [regional accrediting groups] cannot hide behind the notion that we cannot collect data” — the advisory panel unanimously approved DeNardis’s compromise.

Having helped to defuse that conflict, DeNardis had some more general words of caution for his fellow members of the advisory committee — and, implicitly, it seemed, for Spellings and her colleagues at the Education Department. “Clearly the secretary wants to move in certain directions, and there is strong support in this body for many of those recommendations, as they will be clarified over time,” he said. “But perhaps there are some things that we cannot do by administrative fiat, and that are better done by engaging the impressing accreditation system that we have.”

And while he acknowledged that the accreditation system is at a point where “major changes are indeed in order,” DeNardis, who once represented Connecticut in Congress, suggested that there may be real limits on how much change NACIQI, and the Education Department itself, can accomplish on its own. “Excellent concepts, even if they pass muster eventually, need to be institutionalized before they are operationalized,” he said. “And they need to be institutionalized first with members of Congress, who will want to play a role” in defining and approving them.

Day 2 of the advisory committee’s meeting today could bring more discussion and debate about how far the Education Department should go in requiring accreditors to measure student outcomes. Among the many issues raised in a highly critical audit of the American Academy of Liberal Education is this one: “The agency needs to more clearly define what AALE itself considers acceptable levels of institutional success with respect to student achievementr based on an evaluation that includes external outcomes, both quantitative and qualitative.”

— Doug Lederman

The original story and user comments can be viewed online at http://insidehighered.com/news/2006/12/05/naciqi.


Wednesday, December 6, 2006

Fears of Possible Federal Learning Standards Grow as Liberal-Arts Accreditor Is Penalized
By BURTON BOLLAG

Washington

An Education Department committee that oversees accreditation organizations has barred the American Academy for Liberal Education from accrediting new institutions or programs for at least a half year. The move, which must be approved by the department's assistant secretary for postsecondary education before it goes into effect, followed a determination that the small accreditor had been lax in not setting minimum standards for what students must learn at the colleges it accredits.

The action, taken by the National Advisory Committee on Institutional Quality and Integrity during a meeting here on Tuesday, was the strongest of several recent moves the panel has made against other accreditors for the same reason. Accreditors say they have growing concerns, as Education Secretary Margaret Spellings begins doing what she said she would: using the advisory committee to require them to make greater use of measurements of what students learn.

The problem with that, say accreditors and some college groups, is it could signal the start of a shift toward federal standards for what students must learn.

As an additional punishment against the American Academy for Liberal Education, the committee decided to postpone for six months consideration of the academy's re-recognition.ÊThe academy accredits eight institutions in the United States -- many of them colleges with a "great books" approach -- as well as five programs at American institutions and several institutions overseas. Those institutions' accreditation is unaffected by the punishment.

The committee made the decision on the basis of a review by the Education Department. The department official who presented that review at Tuesday's hearing suggested that the agency was strengthening its requirements for all accreditors -- including the six regional associations that accredit most higher-education institutions in the United States -- to give more weight to measures of what students learn.

"What is good enough?" the department official, Steve Porcelli, asked, referring to various indicators of student achievement. "This is a problem we've had" with the regional accreditors, he said. "They pretty much leave it up to the institutions. Well, this is no longer considered good enough." He added that there had to be "external expectations" of minimum standards that colleges must meet to keep their accreditation.

In comments to The Chronicle, leaders of several higher-education organizations expressed concerns about the committee's actions. "This notion of bright lines, yardsticks, or external validation -- I haven't heard that before," said Judith S. Eaton, president of the Council for Higher Education Accreditation, an umbrella group representing accreditors. "This is pretty far-reaching," she said, adding that she was anxious the government might eventually seek to set standards on what students should learn.

Susan K. Hattan, a senior official with the National Association of Independent Colleges and Universities, said she feared the committee's recent decisions might signal government moves "that could lead to a level of standardization that could harm the great diversity of American higher education."

On Monday, the first day of a three-day meeting here, the committee reviewed the Western Association of Schools and Colleges' Accrediting Commission for Senior Colleges and Universities, one of the six big regional accreditors. Although the accreditor was re-recognized for five years, it was told to report back before the committee's next meeting, in June, on how it would pay more attention to what students learn.

Later on Monday the accrediting arm of the American Bar Association came up for review. It was also told to pay more attention to student achievement -- in that case, the proportion of students from any law school who passed state bar examinations (The Chronicle, December 5).

At a meeting with accreditors and others last Wednesday, Education Secretary Margaret Spellings said she wanted accreditors to pay more attention to various measures of student achievement, such as graduation and employment rates, and standardized-test scores. Her intention is for the accreditors to use those indicators to push institutions with deficiencies to do better (The Chronicle, November 30).

Although the secretary has spoken only in general terms about the changes she would like to bring to higher education, some of the first concrete signs of what those changes might entail have been coming out of the committee that oversees the accreditors. It is crucial for accreditors to gain the committee's approval of recognition by the Education Department, since only then are the colleges they accredit eligible for various forms of federal funds, including student aid.

Accreditors also say the education secretary may be stretching the limits of her authority under current legislation. Moreover, they say, she appears to be seeking to make changes without consulting with the accrediting agencies.

During Monday's hearing, Ralph A. Wolff, executive director of the Western association's college-accrediting section, referred to a statement Ms. Spellings made to accreditors last week: "We're going to do it with you, not to you." Any moves to impose on accreditors requirements to set "bright lines," or minimum standards for student achievement, would be unacceptable, he said.

Jeffrey D. Wallin, president of the American Academy for Liberal Education, spoke bitterly after Tuesday's meeting at which his group was penalized. He said the committee was trying, at the Education Department's behest, to make "an end run" around Congress in an attempt to federalize higher education.

Earlier, however, there was palpable frustration in the meeting, as several committee members stressed that the academy had not resolved the panel's concerns despite repeated requests over the past five years.

"I'm troubled that your agency can't calibrate how to meet Education Department expectations," said George A. Pruitt, a committee member and president of Thomas Edison State College, in New Jersey. "You guys just don't seem to get it."

Echoing comments he had made at previous meetings of the committee, Mr. Wallin told the panel that his group had a deep interest in assessing what students learn, though he added that the task was particularly difficult in the liberal arts. After its founding in 1992, said Mr. Wallin, his group tried to get institutions to accept evaluations of what their students learned by external experts, but there was great resistance to the idea. "Schools simply wouldn't do it," he said, "so we backed off."

Mr. Wallin added that his group had tried repeatedly to satisfy the Education Department's requirements in student achievement, but the requirements were unclear and changed over time.

The Education Department's review of the accreditor listed 13 mostly minor problems. Mr. Porcelli, the official who presented the report, said most had recently been resolved. In an unusual move, however, the committee did not accept the report's recommendation to re-recognize the academy for five years but instead chose unanimously to show its disapproval by putting off a decision until the committee's next meeting, in June.

The academy must present a report to that gathering demonstrating that it has moved substantially to comply with department standards on the evaluation of student learning, as well as on two other minor issues involving the training of members of its board and evidence of compliance with its standards that the accreditor requires from colleges it oversees.

The committee had recommended last December, for the same reason, that the academy be barred from accrediting new institutions or programs (The Chronicle, December 9, 2005). But the assistant secretary who must approve such recommendations decided the move was unnecessary, since the accreditor was coming up for a full review at Tuesday's meeting.

Comments

This sounds all too familiar: No Child Left Behind, now for higher ed! Also very troubling is the rhetoric being thrown about (not new perhaps) about "training" college students for in-demand (read: science/engineering) jobs and having the industries influence what is being taught in the curriculum. Excuse me, but higher ed is not a four year training session!

Post a comment

The views and opinions expressed in this page are strictly those of the page author. The contents of this page have not been reviewed or approved by the University of Minnesota.