AtD Main > Teams > Data Team > Proposal Narrative
Planning and Results. Describe data collection and student outcome data examined.
Quantitative data: North Central State College (NCSC) compiled the results of COMPASS assessments for math, reading and writing for the 2002-2005 cohorts. Success rates were analyzed for all developmental reading, writing and math courses, as well as high-enrollment, high-failure gatekeeper courses taken by students in the 2002, 2003 and 2004 cohorts. "Failure" was defined as withdrawal past the census/D/F. Stop-out rates were analyzed for students failing these courses. A progression analysis was performed which compared success rates in freshman composition for students who successfully completed developmental writing as opposed to students testing directly into freshman composition. Term-to-term and fall-to-fall persistence rates were analyzed for the 2002 cohort. Course success rates and persistence data were then disaggregated by race, age group, gender, and Pell status. A comprehensive assessment was performed on Integrated Postsecondary Educational Data System (IPEDS) financial aid indicators.
Qualitative data: NCSC collected qualitative data in three sweeps, each time narrowing the focus of priorities. First, engagement teams were formed to collect general input on perceived barriers to student success and persistence. The student engagement team met with five stakeholder groups and conducted a survey and received feedback from 264 students. The community engagement team met with six different groups of community representatives (N=40), including minority organizations, local ministers, community agencies, employers, and local educators. Summary reports were generated for each stakeholder group and common themes or issues were identified. Second, NCSC held a strategy retreat for 50 key faculty/staff in January 2006 in order to discuss data analysis and performance gap findings. Data was shared about performance gaps for specific student subgroups. Retreat activities challenged faculty and staff to ask "why"? and to hold candid conversations about the underlying reasons for student performance gaps. Finally, a consultant, Dr. Kenneth Gonzalez, conducted seven focus groups with 30 successful developmental students representing student subpopulations, and with eight developmental faculty.
In what ways did you engage faculty, staff, students and the community?
From the beginning, NCSC intentionally integrated its Achieving the Dream (ATD) work with the Board of Trustees' planning goals and with its accreditation work through the AQIP (Academic Quality Improvement Program) of the Higher Learning Commission. The ATD work has become an AQIP action project and will be evaluated as part of the reaccredidation process.
Approximately 50 faculty and staff (25% of full-time count) serve on committees related to ATD. Since February, a cross-functional Developmental Education team has been actively constructing strategies designed to improve student success rates in developmental education. Faculty and staff have participated in activities related to ATD at in-service trainings in September, January, and April. A recent informational ATD open forum was voluntarily attended by 127 employees who wanted to learn more or become actively engaged in ATD. Every employee attended at least one ATD meeting during the 05-06 academic year. As stated above, approximately 300 students have been engaged through meetings, surveys and focus groups. Moreover, student input was sought through the Community College Survey of Student Engagement (CCSSE), and advising questions were included from a consortium of ATD colleges. Though the latter results were not returned by the time this proposal was submitted, NCSC intends to study these responses once we receive them and use future CCSSE surveys as a benchmark to measure progress. As noted, candid discussions have been held with community members representing a variety of constituencies. In addition, NCSC has begun work with Mansfield's Community Development Department on strategies to expose underserved secondary students to college and career opportunities. The ATD Communications Team launched a website and started a monthly newsletter to keep stakeholders informed and engaged in the process. Information about ATD is regularly reported to the Board of Trustees. NCSC newsletters mailed to the area households, as well as a weekly student newsletter, report on ATD.
What priority issues arose from data analyses and discussions? Why did you choose these priorities? What were key contributing factors? What evidence led to this understanding?
Analysis and discussions revealed three significant issues that NCSC identified as priorities for ATD. They are: 1) to improve student success in developmental education, 2) to improve student success in gatekeeper courses and 3) to improve student persistence rates.
Priority 1: Improve student success in developmental education.
Why Selected: The analysis reveals that 66% cohort students were assessed at one or more developmental levels, and that students attempting these courses achieved a 68% success rate. 92% of cohort students who successfully complete developmental work enroll in additional courses. Of those students who are unsuccessful in developmental work, 47% immediately leave the institution and do not re-enroll. This has a significant impact on student persistence and completion rates at NCSC.
Key Contributing Factors: Disaggregating the data revealed three student subpopulations with significant performance gaps in the area of developmental education.
Quantitative Evidence: Pass rates for males, African Americans and students age 23-29 were 60%, 57% and 62%, respectively, compared to an overall 68% cohort average. Male students and students age 23-29 comprise 38% and 39.4%, respectively, of the multi-year cohort. While the African American composition is small at 5%, it is rapidly increasing and achieved 8% in fall 2005.
NCSC also analyzed the potential impact of not maintaining a mandatory placement policy for math. Of cohort students assessed into developmental math (three levels) and attempting a math course, 33% self-placed into the college level and 23% into a higher than assessed developmental level. The success rates of these students in the first math attempt was 69% and 68%, respectively. This compared to a success rate of 80% for students who attempted their first course at the assessed developmental level. Of the 160 students testing into the lowest developmental math level, yet attempting the highest developmental or college level, the pass rate was only 61%.
NCSC does have mandatory placement for students assessed into developmental reading and writing. Consequently, it analyzed students progressing into college-level gatekeeper English (ENG 101) against students directly placed into the college level. Students starting in lower-level writing (WRT 115) had a 67% success rate in ENG 101; students in upper-level (WRT 116) had a 73% success rate; students in lower-level reading (RDG 115) had a 41% success rate; and students in high-level reading (RDG 116) had a 68% success rate. Native ENG 101 students had an 81% success rate, confirming the need to improve developmental preparation.
Qualitative Evidence: Focus group work was done with NCSC students who have successfully completed developmental education. Barriers common for all three student subpopulations included: (a) balancing school, work, and family, (b) lack of available courses/scheduling, (c) teachers covering a lot of material too quickly, and (d) inadequate preparation for college-level work. Other barriers that were prevalent in the data included:" (a) instructors not getting down to students' level," (b) some instructors being "non-supportive/non-effective," (c) loss of knowledge for older students, and (d) adjusting to the process of returning to school.
There were a number of barriers that were unique to specific student subpopulations. Unique barriers for the male student subpopulation included: (a) coping with an ineffective advising system, (b) pride, and (c) lack of desire for learning among younger male students. Unique barriers for students 23 to 29 years of age included the wait list system for many of the health science programs. Finally, unique barriers for African American students included: (a) lack of access to computers, (b) lack of money/financial aid, and (c) instructors lacking cultural awareness. An aversion to taking developmental classes seems to be especially true for NCSC male students, whose focus groups estimated that 50-75% of men have too much pride to think that they need a developmental course.
Priority 2: Improve student success in gatekeeper courses.
Why Selected: A total of five high-enrollment, high failure courses were defined as "gatekeeper" courses at NCSC. Cohort enrollment for these five courses was 1.7 times the actual cohort size, and pass rates were 69%. The gatekeeper business class (BUS 121) posted a success rate of only 56%. Even a one-credit college success course (FYE 101) had a pass rate of only 77%. FYE 101 is required for all students with the exception of those who place into both developmental reading and writing, who must take a three-credit success course that utilizes the College Student Inventory (CSI) from Noel-Levitz. The analysis reveals that cohort students who successfully complete gatekeeper courses persist at a rate of 94%. Of those who fail one or more gatekeeper courses, 42% leave the college, including 47% who fail the success course. This has a significant impact on student persistence and completion rates.
Key Contributing Factors: Disaggregating data revealed two student subpopulations with significant performance gaps.
Quantitative Evidence: Male students and African American students posted success rates of 62% and 42%, respectively. Only 49% of males and 22% of African Americans passed BUS 121, and 69% of males and 56% of African Americans passed FYE 101. Further, students that had taken developmental reading and developmental writing failed BUS 121 at a rate of 60% and 68%, respectively.
Qualitative Evidence: Data was collected from different stakeholder groups with regard to barriers to success in gatekeeper courses. Issues commonly cited include (a) balancing responsibilities of (and at times conflicting priorities related to) school, work, and family, (b) a sometimes weak or ineffective course placement and advising system, (c) gatekeeper course content not seen as relevant to student long-term goals, (d) lack of flexibility in scheduling gatekeeper courses resulting in limited access, and (e) gatekeeper course environment not as nurturing as developmental course environment.
Faculty in BUS 121, which has no prerequisites, specifically noted that reading and writing skills are definite indicators of success. Also, African American students included (a) possible feeling of alienation with few minority peers and no African American faculty on campus, and (b) a feeling that some faculty teach only to "majority" and do not relate well to African American students. Finally, male students argued that FYE 101 is too demanding for the credit hours earned, and that it is inconsistently taught among sections.
Priority 3: Improve student persistence rates.
Why Selected: The Fall-to-winter persistence rate for 2002 cohort students is 70%. Fall-to-fall persistence rate for 2002 cohort students is 44.0%. Using data from the Ohio Board of Regents, the fall-to-fall persistence rate of freshman degree-seekers (full-time) averaged 51% for 2002 and 2003, compared to 55% at all Ohio technical colleges. Persistence has a direct impact on completion, transfer, and graduation rates. The campus interactions believed to be the most critical in determining student persistence are those that occur within the first six months or first year of college (Tinto, 1994, 1996). Moreover, Noel-Levitz reports that financial barriers are the primary reason students do not persist at college.
Key Contributing Factors: Quantitative Evidence: Disaggregating data revealed males, African Americans and students age 23-29 had term-to-term persistence rates of 64%, 60% and 63%, respectively. Likewise, these groups posted fall-to-fall persistence rates of 38%, 31%, and 37%, respectively. Financial aid statistics reviewed in IPEDS revealed the percentage of NCSC students receiving any aid consistently lagged peer technical colleges, and has fallen from 79% in 2001 to 63% in 2004. Likewise, cohort students receiving Pell grants fell from 47% in 2002 to 41% in 2004.
Qualitative Data: Qualitative evidence was collected from different stakeholder groups with regard to barriers to persistence. Issues commonly cited include: (a) peer pressure or a family culture that teaches (especially for first-generation students) that education is not the first priority, (b) numerous financial barriers and a financial aid application process that is confusing to student/family, (c) a sometimes weak or ineffective advising system, (d) class schedules that collide with work and family life, and (e) lack of flexibility in scheduling needed courses that results in limited access. Unique issues for African American students were access to technology and knowledge of
Implementation Plan: What measurable changes do you intend to achieve over the four-year period? How will you bring about these changes? What evidence or rationale suggests that your strategies will be effective in increasing student success?
Priority 1: Improve student success in developmental education.
Measurable changes: Success rates in developmental math courses will increase by 4% each year for FY07, FY08, FY09. Success rates in RDG and WRT courses will increase by 3% each year for FY07, FY08, FY09. Developmental students reporting positive advising experiences on CCSSE surveys and final focus groups will increase by 10%, including a 15% subgroup increase. Early alert referrals for students in developmental courses will increase by at least 8%. Students receiving grades of D/F, or "no-pass" will decrease by 10% as compared to the 02-04 cohort. Developmental students attending Noel Levitz College Student Inventory advising appointments will persist fall to winter at a rate of at least 10% greater than developmental students who do not follow up on their CSI appointments. Success rates for developmental students progressing into college-level gatekeeper English or math will increase by 4%, including 8% for the subgroups.
Strategies for change:
A. Strengthen the college advising system in relation to developmental courses Planned action steps related to advising include (1) implement mandatory placement for developmental math courses, (2) develop and distribute math assessment preparation materials, (3) utilize a math diagnostics tool to help students identify areas for skill improvement, (4) administer a pre-test the first day of developmental math classes to help the instructor determine if a student can still change levels, (5) require an intensive three-unit college success course utilizing the CSI from Noel-Levitz for every student taking developmental math, reading, and/or writing, (6) expand the DIRECTIONS program (intrusive advising for at-risk students) to any student taking developmental reading, writing or mathematics, (7) share information with students about the commonality of starting in a developmental course in an effort to reduce the stigma and increase acceptance by students, (8) strengthen the Academic Early Alert warning system for any student taking one or more developmental courses and (9) provide appropriate professional development opportunities to academic advisors.
B. Improve curriculum and instruction for developmental courses Planned action steps related to curriculum and instruction include (1) convert one lecture hour to lab hours in every developmental math course, (2) incorporate supplemental instruction and tutoring into developmental mathematics courses, (3) integrate manipulatives into math learning environment, (4) include computer assisted instruction in all developmental courses where appropriate, (5) add a mandatory tutoring component to the developmental writing courses, (6) eliminate the "flex-time" format of the developmental writing final exam to gain a week of instructional time, (7) develop individualized reading course textbooks that utilize material from college level social science courses, (8) make the lowest-level (and sparsely offered) developmental reading course a lab component of upper-level reading, and (9) provide appropriate professional development opportunities for faculty and tutors.
Evidence/Rationale: Bettinger and Long (2005) report that Ohio community college students placed in math remediation were 15% more likely to transfer to a four-year college than similar students not required to take developmental courses, and that these students also took 10 more credit hours. The National Association of Developmental Education (NADE) Best Practices in Developmental Mathematics (2003) recommends that colleges help prepare students for placement tests and provide first-day "readiness tests". Research by NCSC shows that students who complete the College Student Inventory and subsequent advising appointment are more likely to persist and maintain higher GPAs than students who do not attend the appointment. Boylan and Saxon (2001) recommend a counseling component integrated into course structures, ongoing student success courses, integration of classroom and lab, use of supplemental instruction and tutoring by well-trained tutors, use of sound cognitive theory and a variety of learning systems, consistency between developmental exit and credit course entrance, use of a variety of delivery systems as indicators of success, and ongoing professional development as contributors to developmental success. The Florida Department of Education is reporting dramatic outcome improvements in areas such as persistence for developmental students taking a 3-unit success course (Krazis, 2006).
Priority 2: Improve student success in gatekeeper courses.
Measurable changes: Success rates in gatekeeper courses will increase by 4%, including 8% for the student subgroups, each year for FY08 and FY09.
Strategies for change:
A. Strengthen the college advising system in relation to gatekeeper courses Planned action steps include (1) further analysis and correlation of COMPASS scores and success in gatekeeper courses, (2) review of existing prerequisites for gatekeeper courses, (3) additional analysis of the success rates of students progressing through developmental to college level writing and math courses to be certain the curricula are appropriately aligned, and (4) advisor training and professional development .
B. Improve curriculum and instruction for gatekeeper courses Planned action steps include (1) complete an instructional review of the FYE101 Orientation to College course, (2) maximize gatekeeper offerings and availability to students, (3) recruit more diverse faculty and (4) provide professional development for faculty who teach gatekeeper courses.
Evidence/Rationale: Leeward Community College (Hawaii) maintains a detailed prerequisite policy in the basic skills of reading, writing and math for courses other than reading, writing and math. It states "basics skills prerequisites shall be established for all content area courses, including the entry-level course in a series or sequence of courses within a discipline." It further created an "Academic Review Board", to facilitate interpretation of the policy and conduct a periodic evaluation on the impact of prerequisites. Stovall (2000) recommends that colleges carefully craft the delivery (class size, student interaction techniques, faculty contact outside of class) and core content areas within success courses. Barefoot & Fidler (1996) report that 70% of community colleges that offer student success courses offer some type of instructor training. Hollins (1996) recommends that when instructors and students do not share the same culture, the instructor "must be knowledgeable about the student's cultural background."
Priority 3: Improve student persistence rates.
Measurable changes: Fall-to-winter persistence will increase by 2% each year with an increase of at least 5% in each subgroup. Fall-to-fall persistence rates will increase by 2% each year. The percentage of students receiving any type of financial aid will increase 1.5% each year.
Strategies for change:
A. Increase access to and use of financial aid
Planned action steps include (1) perform process mapping of the student financial aid experience, (2) conduct a business process review of the financial aid area, (3) provide appropriate professional development and training to financial aid staff, (4) enlist assistance of community partners to help communicate and simplify the financial aid process for students, (5) partner financial aid staff with students to research funding for computers and access to technology.
B. Strengthen the college advising system
Planned action steps include (1) analyze enrollment patterns of students historically underserved by advising (e.g. pre-health majors and undeclared majors), (2) recommend new advising strategies for underserved students, (3) increase effective utilization of the Early Alert system and (4) implement a mandatory orientation for FY08.
C. Improve curriculum and instruction
Planned action steps include (1) continue to collect and evaluate data on course scheduling and availability, (2) where possible, implement alternatives such as weekend course offerings, "late start" courses, satellite campus offerings, and additional web- based courses, (3) require in-service training for all faculty and staff on topics related to increasing student persistence.
Evidence/Rationale: The CCSSE 2005 survey report recommends that since academic advising can have a significant effect on student retention and success, instructors who build these activities into their course requirements have the potential to reach students who otherwise would not receive this counsel. Kramer (2003) adds that faculty should be supported so that they know how to connect students with relevant campus services and resources; and that advising should involve a mix of technologically supported opportunities and personalized links between employees and students. Linsey (1997) recommends that institutional researchers study the viability of financial aid programs in view of institutional goals, as well as the efficiency and effectiveness of delivery of funds. Perin (2005) suggests a variety of new course formats designed to increase student achievement, including self-paced, tutor-based, online, accelerated, offsite, etc. Institutions such as Merritt Community College (CA) operate computer loan programs for students in need, leveraging partnerships with corporations.
How will you assess progress?
(1) Quantitative outcome changes will be tracked annually by comparing placement, developmental course success (including Nelson-Denny standardized reading test), progression through gatekeeper, persistence rates, financial aid usage, early intervention usage/effectiveness, etc. against baseline 2002-2004 data. (2) Surveys (including CCSSE) and focus groups of internal/external stakeholders will be employed on a formative basis, including a repeat of the subpopulation focus groups.
Who will be responsible for collecting and analyzing evaluative data?
The Office of Institutional Research/data team will supply progress reports on performance indicators to the core team periodically throughout the year to determine if strategy adjustments are warranted..
Institutionalizing the Plan: How will your college use the initiative to drive lasting change in core policies and practices at the college?
NCSC has linked the basic tenets of the ATD initiative to categories of the Academic Quality Improvement Program (AQIP) for institutional accreditation and transformation to "systems thinking". This also underscores the need to develop a culture of evidence and inquiry, and to focus on successful outcomes. This is being echoed by our state and federal funding sources. Our ATD work will become an integral part of our institutional strategic plan. Institutional priorities will continue to reflect our commitment to this work.
Who will lead this work and how will they engage others inside and outside the institution?
The core, data, communications and developmental committees will continue to implement strategy, review progress, and make necessary adjustments. In addition, an ATD advisory committee with strong representation from community and student stakeholders will be actively engaged in providing feedback, assessing progress and planning future strategies.
What will be the role of the president and the governing board?
President Ronald Abrams will ensure this initiative remains a top institutional priority, especially in terms of securing supplemental and continuation funding. For example, he is collaborating with the county human services department to obtain public assistance funding for services to students near the poverty level. He is also member of a legislatively appointed advisory council studying higher education reform issues, and on the Ohio ATD State Policy Team. From these posts, he can advocate initiative strategies on a statewide basis. Likewise, the Board of Trustees will monitor annual initiative progress reports and support institutional efforts to scale up strategies for transformational change - especially in continuing to communicate the benefits of this initiative to other north central Ohio stakeholders.
How will your plans influence the allocation/reallocation of college resources in the future?
NCSC demonstrated its commitment by self-funding the planning year of this initiative from its unrestricted funds ($50,000 line item), and intends to renew at least half this line item in FY 2006-07. Most course adjustments will be funded through tuition/financial aid. Strategies such as advising/tutoring will involve some reallocation of current resources. Activities related to tutoring and advising will receive partial support from Carl D. Perkins Two Year College funds. NCSC will keep the college and community foundations informed for continuation funding for grants or vouchers to pay for support services. NCSC believes that it can reallocate current computer equipment to create a dedicated developmental learning lab. Finally, activities such as data collection and evaluation (excepting final focus groups) will be funded through the Institutional Research Office's existing budget. This will be especially important in helping to offset reduction of coaching and data facilitation assistance.