All About Institutional Effectiveness
All About Institutional Effectiveness (return)
Developing an Institutional Effectiveness (IE) or Outcomes Assessment Plan for Administrative and Educational Support Services Units
The primary purpose of developing and implementing an institutional effectiveness or outcomes assessment plan for an administrative or educational support services planning unit is to document how well the unit is fulfilling its purpose. This is accomplished by developing an annual IE plan to document the achievement of expected outcomes and continuous improvement of the colleges administrative and educational support services units. All administrative and educational support services units should assess the effectiveness of their operations, processes and programs on a continuous basis.
Cape Fear Community College adopted an online planning system, Strategic Planning Online (SPOL), beginning with the 1999-2000 planning year. The primary purpose of this system was to eliminate paper and pencil and provide unit managers with an electronic tool for documenting continuous improvement and archiving plans from year to year.
The steps in developing an outcomes assessment (institutional effectiveness) plan in the SPOL system for your unit are as follows:
Step 1 – State the mission or purpose of your unit
Identify the portions of the CFCC Mission Statement and College Goals that your unit supports. Your purpose statement serves as a direct link between the college mission and goals and your more specific unit objectives/outcomes. In writing your purpose statement, consider the following questions:
- Who do you serve?
- What are the primary functions of your unit?
- What are your core activities?
The purpose statement can be as brief as a sentence or as long as one or more paragraphs. Here are examples of two administrative unit purpose statements (not to be considered models):
Registrar’s Office
The purpose of the Registrar’s Office is to serve the students and employees of the college by receiving and processing applications for admission, processing and reporting student enrollment, maintaining registered student records, receiving and reporting grades, implementing successful graduations, and processing student transcript information.
Accounting Office
The Accounting Office seeks (1) to provide administrators with accurate and timely financial data to assist them in the management of the college’s resources, and (2) to ensure that financial records are maintained in accordance with generally accepted accounting principles and guidelines as established by State and Federal agencies
Step 2 – Write your unit’s Objectives / Intended Outcomes
Administrative objectives may be concerned with
- Level or volume of activity. Administrative units often justify their existence based on the amount of service provided or work accomplished.
- Efficiency with which your unit produces or works. This might include cost savings measures, turnaround time, improving a process, etc.
- Compliance with external standards or regulations, established Professional Standards, or “Standards or Good Practice’. There are many standards such as OSHA or fire marshal’s standards, health department requirements, American Library Association, Council for the Advancement of Standards (CAS), federal and state laws, regulations, audits, accounting practices, and peer reviews to name a few.
- Benchmarking performance by using comparative data collected from other colleges.
- Client/student outcomes are the gains you want those you serve to make. For example, what can a client do or know after interacting with your unit? What is the outcome of a student’s involvement with your unit?
- Client satisfaction - how do those you serve rate their satisfaction with your unit’s services?
In developing your objectives/intended outcomes, consider these key questions:
- What results should you expect after providing services?
- How do you know when your unit is both efficient and effective?
- What does the end user experience through interaction with your unit?
A few examples of administrative objectives/intended outcomes:
- The library will meet or exceed the standards established by the American Library Association.
- The college bookstore will provide textbooks to students and faculty in a timely manner.
- Food Services will receive superior ratings from monthly Health Department inspections.
- The Financial Aid Office will receive clean audits with no exceptions.
- Students will be able to successfully access library resources using a variety of formats upon completion of a library orientation workshop.
- Faculty will successfully implement video on demand following a training session provided by the instructional department.
- Graduates receiving help from the Career Center will be able to produce an acceptable resume.
- Employees and students will be satisfied with the cleanliness of the college facilities.
- Patrons will be satisfied with the LRC hours of operation.
- Students will be satisfied with the advising process.
- Purchasing and Receiving will reduce lost or stolen inventory by 2%.
A good method for developing specific administrative unit objectives/intended outcomes is for the unit manager to prepare a ‘long list’ of performance indicators that the unit uses to measure performance or how well the unit is functioning and achieving its mission or purpose. This list can then be shared and discussed with others in the unit for the purpose of selecting from 3 to 5 objectives/intended outcomes to be measured for the year. This list can be updated and reviewed annually.
Step 3 – Develop ‘Procedures/Assessment Methods’ for each objective/intended outcome
Once you have established your unit’s objectives/intended outcomes, decide how you will collect evidence and the source of the evidence to prove you are achieving your objectives/intended outcomes and fulfilling your unit purpose.
You may already have an inventory of information and data sources you could use such as audit reports or counts of unit services. You could start by making a list of those readily available to you. You could also ask your peers at other colleges how they assess their efficiency and effectiveness to help you find proven methods to use. Evaluations by an outside, third party such as auditors, professional standards, or experts in the field are excellent procedures/assessment methods.
Remember these common types of assessment:
- Attitudinal – measures of satisfaction from those you serve
- Direct – counts of unit services
- External - validation by a neutral party, auditor, professional standards, etc.
Step 4 – Establish the ‘Criteria for Success’ or targets to measure success
Always aim for a success level that stretches the unit’s performance. Your criteria for success should answer this question: “If our unit is functioning the way we think it ‘ought’ to function, what will be our target or score?” You should have more than one target or criteria for measuring success. Sample targets:
- At least 80% of eligible employees will participate in training.
- 90% of transcripts will be processed and mailed to students within three days.
- 90% of the forms processed will be free of errors.
- 99% of clients will be ‘satisfied’ or ‘very satisfied’ with the time it takes to receive print jobs.
Step 5 – Identify Other “Units Impacted”
Very often you will need the assistance of another department to fully carry out your objective/outcome. It might be that several different departments will be called upon to help you with different tasks. In these instances, your objective/outcome is going to have an impact on the resources of another planning unit. You are asked to indicate which planning unit(s) your objective/outcome will impact so that the unit manager(s) can plan accordingly.
Step 6 – Implement Your Plan/ Conduct Assessment Activities
Put your plan into action and begin collecting data. Determine which activities are to be conducted at the college level (such as surveys) and those to be carried out by your or others in your program. If you are using college surveys, check with the Research Office to make sure the questions relate directly to your objectives/intended outcomes so you will get back useful information. The Graduating Student Opinion Survey and the Faculty and Staff Survey are two surveys that might help you.
Step 7 – Report and Analyze your “Results”
Give a brief summary of your results, your findings and ‘what happened’. Your summary should be complete enough to convince the reader the plan took place. Make sure your ‘results’ relate back to the objective/intended outcome.
Analyze your results. What did the results tell you? Did you meet your targets for success? Is further planning and assessment needed?
Step 8 – “Close the Loop” by documenting “Use of Results for Improvement” of your planning unit
Describe how your unit used the results to improve the unit’s services or programs. For example, what changes, improvements or modifications will you make or have you already made that will help improve your unit? This improvement should relate directly back to your objective/intended outcome stated in Step 2. If the unit failed to meet its objective/intended outcome, then describe what future action the unit plans to take or has taken to insure the outcome is met.
This step is called “closing the loop” because it is the final step in the annual planning ‘cycle’ or ‘loop’. In the step you are documenting that continuous improvement is taking place in your unit. This step is accomplished by reporting any action, change, or improvement made as a result of how well your unit performed compared to what was expected. Here are a few examples of some typical changes or improvements you might see in an administrative assessment plan:
- Organizational changes
- Increasing time devoted to a service
- Changes in hours of operation
- Modifications to procedures
- Creation of new policies or procedures
- Clean audits or error free reports
- Increased client satisfaction
- Eliminating services or adding new services
- Increase in clients served
- Implementing suggestions from surveys
- Raising the ‘criteria for success’ the next planning cycle
Sometimes no action or changes are needed because the objective/intended outcome was met. In that case you would report no changes or improvements were needed.
Step 9 – Begin the Planning Cycle Again
At the close of the annual planning cycle, review your objectives/intended outcomes and select those to be used during the next cycle. Depending on what happened in the previous cycle, there might be objectives/intended outcomes that need to be repeated. Many administrative objectives/intended outcomes are on-going and can be measured each year or every other year to demonstrate continuous improvement in your unit.
Please see examples of annual administrative institutional effectiveness (IE) plans in Appendix B for illustration.
