California State University Bakersfield
Simplifying Signature Assignments
& Teacher Performance Assessments
California State University, Bakersfield (CA)
CSUB opened in September 1970 as the 19th member of the 23-campus California State University system. Some 7,800 undergraduate and graduate students attend CSUB at either the main campus in Bakersfield or the off-campus center in Antelope Valley. More than 90% of CSUB's tenured/tenure-track faculty members have doctoral degrees. The University is a comprehensive regional university and is committed to excellence in its four schools: Arts & Humanities, Business and Public Administration, Natural Sciences, Mathematics and Engineering, and Social Sciences & Education. The University is fully accredited by the Western Association of Schools and Colleges (WASC), with six programs also accredited by national accrediting organizations.
CSUB is composed of two campuses, geographically separated by over 90 miles, and because of this, the University encountered challenges regarding collecting and measuring student learning assessment data for accreditation as well as the submission, scoring, and feedback processes in place for the Teacher Performance Assessment (TPA).
Students were required to print out copies of their TPA's which were at least 25-35 pages each and deliver them in person to the campus of which they were part. The TPA hard copies then were collected and distributed to the assessors — these assessors located in a variety of geographic locations, many of whom are retired educators. Once the hard copies were scored, it was the assessors' responsibility to return the TPA hard copies to CSUB administrators who then had to distribute the scores to students. It seemed that administrators were spending large amounts of time and effort managing the logistics of distributing the students' work, handling obscene amounts of paper, overspending on the postage required to transfer the TPA's to and from assessors, and managing the hundreds of emails to students with scoring results. With limited resources and a reliance on manually managing this assessment process, the process was clearly vulnerable to human error.
California State University, Bakersfield chose to implement LiveText technology to ease the burdens of their former process of assessment and to support accreditation efforts.
CSUB has its NCATE visit in the fall of 2014. Each of the departments in the education program has developed an 8-point assessment plan. LiveText is most crucial for conducting signature assignments and collecting assessment data on specific assignments. For example, the department of curriculum and instructions is using signature assignments as part of their assessment piece for NCATE. They are also using it for their culminating portfolio which includes the capstone activity, which could be a thesis or a project. In teacher education at CSUB, 100% of the courses with signature assignments use LiveText.
In using LiveText's completely web-based learning assessment system, CSUB students prepare all of their CAL TPA work in LiveText, keeping all work centrally located and electronic in nature. Assessors no longer have to be on one of the campuses in order to receive the TPA's to score. Once submitted by the student in LiveText, the assessor can access the student's TPA submission, score it accordingly, and provide immediate electronic feedback to students.
Scores are published securely within LiveText so that students can see all of the comments, feedback, and evaluations made, including a Teacher Performance Expectation (TPE) rubric that shows students the specific areas of strength and weakness for which to improve upon before their next TPA submissions.
From a data collection perspective, the program is able also collect this TPA data and aggregate it over academic terms to see where curricular weaknesses are occurring. The LiveText system provides very specific data about the signature assignments and ties them directly to the teaching performance expectations. Administrators and faculty can look at the TPA data and cross check it with the signature assignments data to see where the gaps in instruction are occurring. Each summer the program has a half-day retreat during which they analyze this LiveText data, which has allowed the program not only to identify areas of curricular improvement but also to increase faculty engagement and dialogue.
LiveText has been used for 8 years at CSUB by approximately 40 faculty and 1,000 students at any one time. About 350 students register for LiveText each year.
- Dr. Randy Schultz
- Grant Administrator
- Edvention Partners TQP Grant
- School of Social Sciences and Education
Point Loma Nazarene University
Assessment Wheel: Transparency, Sustainability and Faculty Engagement
Point Loma Nazarene University (CA)
Point Loma Nazarene University offers more than 60 undergraduate areas of study and graduate program regional centers in Bakersfield, Mission Valley (San Diego), and the Inland Empire. Founded in 1902, PLNU serves more than 3,500 students. PLNU is accredited by the Western Association of Schools and Colleges (WASC).
Model used with permission from the National Institute for Learning Outcomes Assessment (2011).
Some of the challenges PLNU faculty were determined to address included the following: faculty ownership and control of the assessment process; moving from compliance to a continuous improvement culture; creating an environment of assessment transparency; developing a learning community around assessment; and engaging discussions around best practices. PLNU began implementing an assessment management system and needed to move assessment plans and activities to the next level of development by creating an open online forum to engage faculty, co-curricular staff, and others in campus-wide discussions around student learning.
As an institution, PLNU also needed a more efficient, effective way to aggregate assessment data over time and across multiple programs and campuses for program review, curricular changes, historical records, trends, and evidence of student learning. PLNU faculty leadership wanted to move away from storing key assessment data in a variety of locations, including hardcopy and digital files, to a more transparent, consistent, and easily accessible location to be used by multiple governance bodies such as curricular and program review committees.
In efforts to address these challenges, PLNU implemented the Assessment Wheel across academic and co-curricular units. The Assessment Wheel was developed by the National Institute for Learning Outcomes Assessment (NILOA) and adapted by PLNU faculty leadership to meet the institution’s unique needs. In addition, PLNU created an individual assessment wheel for each academic and co-curricular unit. The wheel has provided PLNU with a sustainable and intuitive framework that has encouraged faculty to have conversations among themselves regarding the different components of student learning assessment and improved transparency.
LiveText, a comprehensive assessment management system, was adopted in late 2008 by Point Loma Nazarene University. Point Loma uses LiveText for its myriad functions to aggregate assessment data across multiple programs and campuses, particularly for programs such as General Education, nursing, and business which have professional and regional accreditation requirements.
LiveText is seen as a critical companion piece to the assessment wheel, fostering collaboration among departments and streamlining the process of collecting institute-wide assessment data.
LiveText began as a grassroots movement at PLNU, with a small group of faculty converts that grew as word spread regarding the tool’s ease of use. More and more faculty began using the tool to assess course and program outcomes. Once the different program and department-specific rubrics were inserted into LiveText, it was very easy for faculty to assess students and, with just a couple clicks, generate instant reports on how their students were performing against these rubrics. This process also allowed faculty and administrators across different programs to finally have conversations about the kinds of rubrics they were using – now ensuring consistency in evaluations and increased transparency to students on how and what they would be evaluated.
Additionally, students in different programs, such as nursing, business, and music have started using LiveText to create e-Portfolios, eliminating any need for putting together cumbersome paper binders in order to meet portfolio requirements. The e-Portfolios allow students to exhibit the competencies and demonstrate program learning outcomes.
The use of LiveText continues to evolve and grow at PLNU. Faculty members are using it for their promotional and tenure packages. In addition, PLNU plans to take institutional transparency to next level by creating a stronger linkage between LiveText and the PLNU assessment wheel, allowing faculty to seamlessly upload end-of-semester assessment results into their assessment wheel from LiveText.
- Dr. Maggie Bailey
- Vice Provost for Program Development and Accreditation
- Stephanie Lehman
- LiveText Coordinator
- Office of Institutional Effectiveness
Direct Assessment to Demonstrate Student Performance
Brandman University, part of the Chapman University System, is a private, WASC-accredited, nonprofit institution serving the needs of working adults since 1958. Brandman offers over 50 bachelor's, master's and certificate programs in a blended format, combining online and blended in-class instruction at 26 campuses in California and Washington. Brandman boasts the second largest private university teacher credentialing program in California.
With multiple campuses and a variety of brick-and-mortar and online programs, Brandman University's School of Education required a sustainable strategy to collect and centralize student assessment data more efficiently in order to provide evidence of learning for multiple national, regional, and specialized accreditation bodies.
Before LiveText, the School of Education merely gathered samples of student learning data but administrators realized that 100% of student learning assessment data was needed to provide a more accurate picture for opportunities for improvement. The University needed both formative and summative assessment data that could be disaggregated by program, campus, delivery mode (hybrid vs. online), and point in time.
This data also needed to be captured and housed securely and easily so that the institution could generate multiple customized reports for various accrediting bodies.
Brandman chose LiveText for its comprehensive assessment solution and its ease of use. Hundreds of faculty and thousands of students in different geographical areas and online were able to purchase, register, log in and learn how to use LiveText with minimal effort.
Faculty and the administrators were also able to generate wide variety of course and program level data reports that could be used for continued course, curriculum, and program improvement. Such reports included data on Signature Assignments for Program Learning Outcomes (PLOs), Teacher Performance Expectations, California Teacher Performance Assessments, Fieldwork, Student Dispositions, and Demonstration of Mastery.
LiveText's integrated Field Experience Management module has also allowed Brandman to centralize, administer, and track its student teacher placements within its network of more than 500 schools, ensuring that student's backgrounds are always matched with an appropriate school. Moreover, it provides a single share workspace for the students, mentors and university supervisors for sharing work samples, recording hours and completing assessments from one page.
Finally, Brandman administration values that LiveText incurs all data housing and server management costs which in and of itself can be a massive ongoing expenditure for any university.
- Dr. Christine Zeppos
- Dean of the School of Education
- Brandman University
California Baptist University
Direct Assessment to Demonstrate Student Performance
Founded in 1950, California Baptist University is affiliated with the California Southern Baptist Convention and is fully accredited. CBU is the only Southern Baptist college or university on the West Coast. CBU is accredited by the Accrediting Commission for Senior Colleges and Universities of the Western Association of Schools and Colleges (WASC). Currently, LiveText is used by the institution to meet WASC accreditation guidelines, as well as by the School of Education and School of Christian Ministries.
California Baptist University sought to collect institutional assessment data and program-level accreditation data more effectively. Before LiveText, the process of collecting assessment data was much harder and mainly relied on survey and indirect evidence. The institution also faced other challenges in implementing student portfolios within the School of Education and School of Christian Ministries
LiveText is used on this campus to make the assessment process easier. LiveText allows program coordinators to demonstrate measurable evidence of how students are improving over time, and as an institution, CBU uses LiveText to collect data for WASC accreditation.
The School of Education uses LiveText to collect assessment data for program outcomes, which contributes to the institutional assessment data used to demonstrate meeting WASC accreditation guidelines, as well as the Education program’s CTC accreditation.
In the School of Christian Ministries Philosophy program, LiveText is used extensively in the text-heavy courses where students write a lot of essays. Students do all of their course submissions within LiveText, and instructors are able to assess these students using course-embedded rubrics. The course-embedded rubrics in LiveText make it clear to students what is expected in these courses.
The School of Christian Ministries requires each student to create and submit an e-Portfolio that is used to collect assessment data based on the program’s Student Outcomes. Required e-Portfolios contain key artifacts that are used to collect assessment data in specific areas, and faculty members are able to communicate feedback to students about the quality of their work. Students can self-assess their myriad assignments and have a permanent digital copy of their portfolios, which they then can share with graduate schools or potential employers using the LiveText Visitor Pass feature.
The data collected by programs is used as a foundation for the University’s Annual Program report.
- Dr. DawnEllen Jacobs
- Vice Provost
- California Baptist University
- Dr. Scott B. Key
- Professor of Philosophy
- School of Christian Ministries
- California Baptist University
Robust Assessment Management says Northwestern College, IA
About Northwestern College, Department of Nursing (IA)
Founded in 1882, Northwestern College is a Christian liberal arts college affiliated with the Reformed Church in America and located in Orange City, Iowa. Northwestern is fully accredited by the Higher Learning Commission of the North Central Association of Colleges and Schools, and its nursing program is accredited by the Commission on Collegiate Nursing Education (CCNE).
An Interview with Pam Reints
How do you use LiveText?
We adopted LiveText in 2006 and started with 2 faculty and 8 students in the department of nursing. The use of LiveText has significantly grown and is currently being used by 9 faculty and 120 students. Besides the nursing program, LiveText is also currently used in the education and athletics training departments.
The main reason for choosing and continuing to use LiveText is that it has allowed us to streamline our assessment process and to collect information that we utilize for improving our curriculum and providing concrete evidence to our accrediting agencies. As part of their graduate requirement, our nursing students develop an e-Portfolio in LiveText. The e-Portfolios are assessed by faculty and then later used by students to share their work with prospective employers.
With an upcoming Commission on Collegiate Nursing Education (CCNE) accreditation visit next year, our faculty members and department administrators have been able to save a lot of time collecting and reporting course and program level assessment data. Being able to tag outcome and accreditation standards to rubrics and to monitor and report on that assessment data over time has been the best feature for us in LiveText.
Faculty members and students also feel very comfortable and empowered using LiveText. Students like using portfolios to capture their ongoing reflections, and they love the intuitive design of the new student submission process. Additionally, LiveText’s inter-rater reliability feature has helped our faculty to be consistent when assessing and has allowed us to communicate to our faculty, especially adjuncts, what the expectations are in terms of assessment and how students meet our learning outcomes.
- Pamela Reints, B.A., M.Ed.
- Administrative Services Manager/Instructional Technology Specialist for the Department of Nursing
- Northwestern College
Framingham State University
Intuitive, Easy, Engaging e-Portfolios
About Framingham State University (MA) Department of Nursing
Accredited by the New England Association of Schools and Colleges, Framingham State University, founded in 1839, is a public, four-year, co-educational institution located 20 miles from Boston and offers 26 undergraduate and 24 graduate programs. Framingham's department of nursing is accredited by the Commission on Collegiate Nursing Education (CCNE).
An interview with Dr. Cynthia Francis Bechtel
How do you use LiveText?
We initially adopted LiveText in May 2012 as a pilot program for the Master of Science in Nursing program and are in the process of expanding this to the entire class this fall semester. We needed our master's students to develop professional portfolios as one of the main assignments in our Advanced Technology and Nursing Informatics course. The objective was to engage students directly in their learning. Using LiveText allowed students to centralize their work, making it easier for them to reflect on their learning process and to develop a career portfolio for employment or career advancement upon graduation. We will soon use LiveText to assess program outcomes and prepare for our CCNE accreditation in 2014.
Previous to LiveText, we used a portfolio tool that came as an extension to our existing learning management system. However, the solution was very cumbersome to use, and students were spending all their time on the technology part rather than using it as an effective learning tool. When LiveText was piloted as an alternative solution, students had no problems using its e-Portfolio capabilities. Both faculty and students felt the difference was "like night and day." With custom templates designed by LiveText for their use, students found it much easier to develop and make changes to their portfolios. Faculty also liked the fact that they could provide formative feedback to the students using the inline comment features.
- Dr. Cynthia Francis Bechtel
- Associate Professor, Master of Science in Nursing
- Framingham State University
Upper Iowa University
Upper Iowa University: General Education Outcomes Assessment
Upper Iowa University is a recognized innovator in higher education, offering accredited quality programs, through flexible learning experiences, including online and independent study. UIU has partnered with LiveText as the university moves to the next level of global academic assessment; the partnership is also a catalyst for the university's AQIP culture. Beginning with General Education Assessment in fall 2011, LiveText made it possible to efficiently streamline the collection of thousands of artifacts covering nine General Education outcomes across 18 centers in the U.S., and six international centers. In addition to individual faculty assessment and reflection, each term a general education outcome is chosen for evaluation by a task-specific team of four faculty members. With the "Blind Assessor(s)" feature in LiveText, this process has been revolutionized so that each faculty member of the task team can electronically evaluate random student artifacts on their own time, in their own location. Through the aggregate assessment reports, comparative analysis of faculty of records' evaluations and task team members' evaluations can be made, prompting feedback and possible course enhancements to student learning.
Looking ahead to 2012, UIU plans to utilize LiveText to facilitate the following assessment activities:
- Major/Program Assessment
- Senior Projects Portfolios/Assessment
- Wellness Strategies (FYE)
- Nursing Program
- Education Program
- Dr. William R. Duffy II
- Upper Iowa University
Athens State University
LiveText Assessment Data Provides Specific Analysis for Athens State University College of Business
LiveText provides assessment data that provides faculty and administrators with information to continually improve the curriculum.
About Athens State University
Athens State University is Alabama’s oldest institution of higher learning established in 1822. Athens State University, offering coursework at the junior and senior level, is the baccalaureate degree granting institution of the Alabama Community College System. The University, through quality teaching, individual attention, and a varied course delivery system, assists students in the timely achievement of their professional and career goals. In addition, Athens State University offers programs of continuing education and community services that provide a variety of cultural and professional opportunities.
Assessment Data Provides Specific Analysis
The faculty in the College of Business had a realistic view of the strengths and weaknesses of students in the areas of the core competencies, but they did not have specific data to substantiate their theories. LiveText bridged that gap with a consistent method to analyze writing, presentations, critical thinking, decision-making, understanding diversity, group interaction techniques and understanding ethical standards to name a few of the areas of assessment. Once the faculty determined specific areas that defined the elements of each competency to measure, developed the rubric, and determined what type of assignment to use in the process, they put procedures in place to both assess the students’ performance and collect samples in the portfolio.
They determined a common goal, across the university, to have 80% of our students perform at average or above average ranges in each competency measured. They are in their third year and findings are that in each of the competencies students have performed at levels of 89%-98%. They can identify specific deficiencies and regularly use the information to improve the curriculum. The data collected has also provided key information used to improve the presentation of material in distance learning formats.
Using the LiveText e-Portfolio and Reports in Longitudinal Studies
For the faculty and administrators who have previously been involved with providing longitudinal data for specific students, the LiveText portfolio is a quality of life enhancement. The previous methods normally involved randomly collecting samples of the work completed by approximately 10% of the students from the time they entered the university until they graduated. The work samples would then be assessed to determine improvement in different skill areas. More than one administrator could be quoted bemoaning the fact that it appeared that randomly selecting a group of students would most assuredly put them in a group less likely to complete their studies. At the end of the process, you could be left with far less than a sufficient representative sample for analysis. LiveText provides administrators with access to samples of work for all of the students by either reviewing it in the portfolio or by running specific reports on individual students.
Using LiveText as Part of the Quality Enhancement Plan
Athens State University has identified Building Success Through Writing as the Quality Enhancement Plan. LiveText has been instrumental in providing the data needed to assess the students’ improvement in their ability to write and also provided a method to collect reliable data to analyze improvement. Students prepare a research paper in a class required in their first and last semester. There is a common rubric used in both assignments and in most of the instances the same faculties are teaching both courses. The students have some type of writing assignment in most of the courses at Athens State University. They are in the second year of the Quality Enhancement Plan and have found that students improve their writing skills 98% of the time.
- Dr. Teresa Wanbaugh
- Assistant Professor of Management & Marketing
- Live Text Coordinator for the College of Business
University of Louisville
University of Louisville, College of Education and Human Development
Program Snapshot: University of Louisville, College of Education and Human Development; The CEHD and its teacher education programs are accredited by the National Council for Accreditation of Teacher Education (NCATE). This accreditation includes the initial teacher preparation and the advanced levels of professional education programs.
Challenges and Solutions
|Major Challenges||LiveText Solutions|
|Finding an integrated assessment and accountability solutions that worked for diverse programs in the college||The College of Education and Human Development uses Hallmark Assessment Tasks (HATs) across all of the departments within the college to ensure that there are baseline data being captured for the annual student learning outcomes (SLO) reports provided to the Office of the Provost and Institutional Effectiveness. The SLO process and the use of Hallmarks in LiveText ensure that every program is examining student performance in reference to specific program standards/outcomes with a goal toward program improvement.|
|Ensuring that the education preparation programs have common assessments to capture unit-wide data||With the ability to apply a rubric across multiple courses through LiveText, the college can now ensure that programs have common assessments in order to capture data on educator preparation programs. As the advanced programs in the educator preparation unit are so diverse, it has been critical to be able to apply common advanced program rubrics to courses from multiple disciplines in order to capture program and unit level data on student performance.|
|Continuous tracking of candidate performance throughout a program||LiveText's student progress reports will now allow for program faculty to see a snapshot of how well a student is performing on key unit assessments based on outcomes/standards. The program's faculty and P-12 partners have strategically designed rubrics to ensure that a student progress report clearly provides data on unit key assessments and also baseline course assessments with the ability to differentiate between the two.|
To learn more, click here.
- Katie Hunt
- LiveText Coordinator
- University of Louisville
South Suburban College
LiveText provides South Suburban College with assessment process that saves 67%* of faculty time when compared to old process.
SSC's Former Assessment Process
To assess student learning across the entire college, the SSC assessment committee developed a series of rubrics aligned with its institutional learning outcomes. Since fall of 2009, each faculty member received an emailed Excel Assessment Scoring Template at the beginning of the semester. Faculty members typed in the general course information, each student's name and ID number, and the results of the rubric.
Upon completing the scoring guide, faculty emailed a copy of the scoring guide to the department chair. The department consolidated scores into a master department form. The department chair then emailed the consolidated master form to the Institutional Assessment Coordinator. The Institutional Assessment Coordinator reviewed the data and emailed it to the Director of Institutional Research who further consolidated the department forms to create institutional assessment reports. In May 2010, SSC implemented LiveText as their new assessment system.
*The LiveText Solution for SSC
To demonstrate time efficiencies to faculty, a comparison time test between the current solution and new assessment processes was administered.
The results of the study do not include the time eliminated by department chairs and administrators who no longer needed to consolidate spreadsheets, clean up data, and merge demographic data. The data is entered directly into LiveText by the faculty, so reports are readily available. The results below show a significant savings in the time it took faculty to assess.
Faculty time savings and accuracy
|Faculty||Excel Test||Data Errors||LiveText Test||Data Errors|
- * Faculty handwrote results into the excel spreadsheet
- ** Faculty missed typing in and scoring a student.
- *** Faculty did not indicate the course number for each student
To learn more, click here.
- Diane Ostojic, D. A.
- Vice President, Academic Services
- South Suburban College
"LiveText is the data management system used by the whole Perkins College of Education for program improvement and to assess and monitor compliance to national accreditation standards. LiveText benefits students by collecting the data surrounding our program learning outcomes so we can evaluate how well or how poorly we are serving our students and make required adjustments to insure they are prepared for their future careers."
- Lynda Martin
- Director and Professor in the School of Human Sciences
- Stephen F. Austin State University
"I find LiveText to be invaluable in tracking student success, program learner outcomes, and continuous program improvement. Direct evaluation of student outcomes and indirect assessment of program standards have now become second nature to our faculty."
- Dr. Willis Walter, Jr.
- Vice President, Institutional Research, Planning and Accreditation
- Bethune-Cookman University
"With the recent adoption of both institutional and general education learning outcomes, LiveText allows us to easily gather and report on data telling us what students understand about multiple outcomes. Results are useful for institutional improvement and regional accreditation."
- Dr. Ruth Heinrichs
- Director of Institutional Effectiveness
- Point Loma Nazarene University
"Purchasing LiveText has been the best investment for our unit in the struggle to insure and be able to demonstrate that our programs are aligned with our conceptual framework, state, and national program standards. With continuously shrinking budgets and reductions in personnel, it is good to know that we have the infrastructure to maintain high quality assessment through LiveText. The best part of all this is the technical support. All our requests were answered within 24 hours and easy to implement"
- Dr. Beverly L. Downing
- Associate Provost
- Kentucky State University
"LiveText is a wonderful communication tool. It gives students and professors opportunities to share information with each other... LiveText empowered me to share my knowledge with others."
- Rosemary D. Scanlan
- Governors State University
- Master of Counseling program
"LiveText helps me organize my class work and prioritize my assignments. It's easy to use and fun."
- Andrew Gartside
- Social Studies Student
- The Citadel