The CSE Assessment Plan

The CSE department is committed to excellence in education in Computer Science and Engineering, and its fundamental applications. For that, our programs should always be in alignment to the modern trends and standards of the discipline and with the best recognized practices to produce successful professionals in the discipline. To achieve that goal to our best, we have established a continuous assessment and improvement process that periodically measures the level of achievement attained by individual courses as well as by the curricula as a whole along a set of Student Learning Outcomes.

Here we present the assessment plan for the two undergraduate programs that are hosted by the CSE department. These are the BS in Computer Science and Engineering, and the BS in Software Engineering. This continuous assessment and improvement process is fundamental for the following reasons:

  1. The Computer Science discipline is a rapidly evolving discipline, not only in deriving new knowledge but also in the impact that the discipline has in modern society.
  2. We must always guarantee our constituents that our alumni are capable of fulfilling their professional duties in a satisfactory and reliable manner. And in addition, that they will be able to evolve in knowledge as the discipline evolves to be able to continue their contribution to the growth of our society through the practice of the profession.

Student Learning Outcomes

Our continuous assessment and improvement process is designed to guarantee that our programs enable our students to acquire a set of fundamental skills that every professional in the discipline must possess.  Those skills are captured in the student learning outcomes (SLO) shared by the two academic programs. For both programs, the CSE department has decided those SLO to be the following:

(a) An ability to apply knowledge of computing and mathematics (in the solutions of problems…) appropriate to the program’s student outcomes and to the discipline

(b) An ability to analyze a problem, and identify and define the computing requirements appropriate to its solution, including designing and conducting experiments to this end

(c) An ability to design, implement, and evaluate a computer-based system, process, component, or program to meet desired needs, including:  economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability

(d) An ability to function effectively on multidisciplinary teams to accomplish a common goal

(e) An understanding of professional, ethical, legal, security and social issues and responsibilities

(f)   An ability to communicate effectively with a range of audiences

(g) An ability to analyze the local and global impact of computing on individuals, organizations, and society

(h) Recognition of the need for and an ability to engage in continuing professional development

(i) An ability to use current techniques, skills, and tools necessary for computing practice.

(j) An ability to apply mathematical foundations, algorithmic principles, and computer science theory in the modeling and design of computer-based systems in a way that demonstrates comprehension of the tradeoffs involved in design choices.

(k) An ability to apply design and development principles in the construction of software systems of varying complexity.

(l) A knowledge of contemporary issues.

Course Learning Outcomes

Since the curricula in our programs is the component that is responsible the most in fulfilling the above student learning outcomes, our plan concentrates on the continuous improvement of all our courses. That improvement will be based on results of the periodic assessment of courses as established by the assessment plan. Each course includes specific student learning outcomes referred to as Course Learning Outcomes (CLO), which are often specific to the topics that the course covers. All CLOs for our courses have been incorporated into the corresponding course syllabi which are publicly available on our website to all our students and constituents.  To guarantee that all student learning outcomes are addressed by at least one course, each CLO is mapped to the list of SLOs that it impacts. The following table displays part of that mapping; the full version is found here.

The table maps each course to the program’s SLOs. In the second column, we have included the number of CLOs that the course in the particular row has. Also, for each programs SLOs, the table includes one column. The number on each cell corresponds to the number of CLOs in the particular course that impact the corresponding program’s SLO. White cells imply that the particular course does not impact the particular program’s SLO.

Finally, we expect that our students are able to show attainment of the program’s SLOs at least when they work in their corresponding capstone course. Therefore, the capstone course will play a crucial role in the assessment of the whole program. From the results obtained on that course, we will be able to decide further improvement actions that might be required in the corresponding curricula.  

Plan for the Cyclical Assessment and Improvement Process of Courses

In order to guarantee timely and frequent assessment of all courses, while minimizing the additional workload imposed on our faculty, we have divided our courses into six groups according to Computing subdisciplines. Every year  we focus our improvement efforts on only one of this groups, thus the faculty involved in the assessment and improvement efforts will mostly consists of faculty within one disciplinary area. Course groups are evaluated in round-robin fashion. Thus all courses must have been assessed and improved every five years. We have identified the following five course groups that partition our courses in subdiciplinary areas as follows:

Course Group 1: Basic Courses

  • CIIC 3011: Introduction to Computer Programming
  • CIIC 4010: Advanced Programming
  • CIIC 4020: Data Structures

 

 

 

Course Group 4: Software Engineering

  • INSO 4101: Introduction to Software Engineering
  • INSO 4112: Software Reliability Testing
  • INSO 4115: Software Requirements
  • INSO 4116: Software Design
  • INSO 4118: Software Project Management (E)
  • INSO 4111: Human Computer Interface (E)

Course Group 2: Computer Architecture and Systems

  • CIIC 3081: Computer Architecture I
  • CIIC 4050: Operating Systems
  • CIIC 4082: Computer Architecture II
  • CIIC 5017: Operating Systems and Network Administration and Security (E)
  • CIIC 4019: High-Performance Computing (E)
  • CIIC 5029: Compilers (E)

Course Group 5: Advanced Requirements

  • CIIC 4030: Principles of Programming Languages
  • CIIC 4060: Databases
  • CIIC 4070: Computer Networks
  • CIIC 5018: Cryptography and Network Security (E)
  • CIIC 5015 Artificial Intelligence (E)

 

Course Group 3: Theory and Foundations

  • CIIC 3075: Foundations of Computing
  • CIIC 4025: Design and Analysis of Algorithms
  • CIIC 5045: Formal Language and Automata Theory

 

 

 

Course Group 0: Major Design Experience

  • Capstone Course

 

 

 

 

The direct assessment of the courses was initiated on the academic year 2017-2018. From that year on, we will concentrate in doing direct assessment on those courses belonging to Group 1. And the process will be repeated in a Round-Robin fashion every five years. The capstone course will be under assessment on every occasion that it is offered. The next figure summarizes the cyclical process that we have implemented to assess the courses and the whole programs. After those first five years, the process is repeated starting again with the set of courses in Group1, each five years after 2017.

On a particular academic year, our effort will concentrate in the group of courses that corresponds to that academic year. Assessment data and results will be collected by professors offering the particular courses. That data will be evaluated by the CAI Committee, in conjunction with the professors teaching the course. From that analysis, decisions are made for necessary improvement actions in each particular course. Those actions will be implemented on the next semester that the course is offered. The whole faculty will be informed of results, and consulted on those actions that may require departmental approval.

The following figure summarizes the general assessment and improvement cycle per academic year.

The following boxes describe the one year long course assessment and improvement process in detail, including the time frame for each activity:

Beginning of First Semester:

     1. CAIC meets with ALL professors teaching the course to:

          a. Review course learning outcomes.

          b. Select the student activities  and rubrics to be used to assess each CLO

          c. Develop a data collection plan using standardized tools such as forms and spreadsheets whenever possible.

Throughout First Semester:

     2. Professors carry-out assessment activities and collect data.

     3. The CIAC will also provide assistance to professors to tabulate data and store documents electronically if requested.

End of First Semester:

     4. Each professor submits a Faculty Course Assessment Report (FCAR) form capturing their overall assessment on course learning outcome performance, their impressions about the course based on their experience, and suggestions for improving the course.

     5. Professors submit tabulated CLO assessment data and student work samples that support those results.

     6. CAIC archives all submitted data electronically to facilitate future access by accreditation agencies.

     7. CAIC analyzes all the information submitted by professors.

Beginning of Second Semester:

     8. CAIC meets with professors to device specific CLO improvement actions to be implemented during the new semester.

     9. The CSE department will make sure that those improvement actions are carried out, even if new professors are assigned to the course.

Throughout Second Semester:

     10. Professors incorporate the improvement actions to their courses

     11. Professors will collect data once more as in Step 2 above, in order to assess the impact of the improvement action on students’ CLO performance.

End of Second Semester:

     12. At the end of the semester, professors will again submit results and supporting data and work samples.

     13. CAIC meets with professors to decide one of two outcomes depending on the results obtained on the second semester.

          a. If significant improvement on a CLO is observed, the improvement action will be adopted as part of the standard course practices

          b. If no significant improvement is observed, alternative improvement actions will be proposed and tested in future assessment cycles.

 

 

Departmental Retreat for Assessment and Continuous Improvement:

     14. Inform results to all faculty and discuss general issues regarding the assessment and continuous improvement process. All faculty members will have the opportunity to propose and approve new general improvement actions to both courses as well as the improvement process itself.

 

From the above process, the part that is specific to the assessment and improvement activities during the two semesters is summarized in the figure that follows.

In essence, individual course improvement is achieved fundamentally by measuring the performance in all of the course learning outcomes by those students approving the course. A subset of course learning outcomes achieving lowest performance levels are selected as candidates for improvement.  Then, an improvement plan, which may include modifying existing or incorporating new course activities, is developed to improve student performance on these candidate outcomes.

The Faculty Assessment Report (FCAR)

The data results that professors collect for his/her courses is finally presented through the specific FCAR for the course. The CSE department has established the policy that all sections of a given course must be coordinated among all professors offering the course simultaneously. Therefore, it is expected that only one FCAR is submitted summarizing the results in all sections of the course; either if it is a single professor or multiple professors offering the course during the semester. Also, discussion and decisions made for improvement in conjunction with the CSE CAI, will involve all the faculty teaching the course, and, whenever pertinent, also the faculty that will teach the course in the near future.

Although at this moment the FCAR is a working document, and therefore we should expect modifications in the near future, the form will include three essential parts. These are:

1. Coverage of Topics in the Course – This is a measure of the coverage level of the particular topic in the semester being reported, based on what is expected at the particular course. This is based on the idea that specific topics usually have certain number of subtopics that are usually covered in the course. If only 50% of those subtopics are covered, then the report should reflect that. We are actually using the range 0..5, where 0 is nothing and 5 is full coverage of the particular topic. The purpose of this data is to determine what topics are constantly being left out and make decisions based on that knowledge. This information is presented in the FCAR as a table, like the one shown here.

2. Assessment Course Learning Outcomes using Graded Work – This is a measure of how the students performed on each CLO based on graded work. For each CLO, the report for the course will include how it was measured and what was the level reached on each. The “how it is measured” part is fundamentally decided by the professors teaching the course. It may include homeworks, quizzes, exam parts, lab activities, course projects, oral presentations, etc. The “maximum score” is the sum of the maximum scores on each particular activity used to measure the particular CLO. Finally, the most important part is the “level reached.” This corresponds to the percentage of passing students who scored at least 70% of the maximum score possible. This percentage of students will be the criteria used to determine how the particular CLO was met in the particular course.  This information is captured in form of a table, such as the following.

3. Proposed Actions for Improvement – This are the actions that are preliminarily proposed for improvement of the performance of the course on each particular actions. It is initially proposed by the professor(s) themselves, but it is expected to be analyzed in conjunction the the CAI committee to eventually determine the final actions. Those final actions will be the same proposed actions proposed in this table, perhaps with some additional actions which might be decided in conjunction with CAI.

We expect that there will be proposed actions for at least those CLOs that exhibit low performance, even if that is above the established threshold.

How Do We Determine a Failing CLO

We measure the performance achieved on a particular CLO using the results of the graded activities that were selected by professors teaching the course as the activities to target that CLO. Those results are tabulated at the end of the semester using only the results of students passing the course.

We have established the threshold value of 70% as the minimum score that the student has obtained after counting his/her performance in all those graded activities. Therefore, any score representing less than 70% of the total possible score in the selected activities implies that the CLO was not reached by the particular student.

Using the data generated for each student on each CLO, we then determine what has been the performance of the particular course in regards to its CLOs. For this, we have also established the threshold of 70%. That is, a CLO is considered to be in good standing if at least 70% of those passing students have achieved a score of 70% or more in the graded activities targeting that CLO.

The following table summarizes what is described above.

Those CLOs that do not reach the good standing status will be considered as failing.

Assessment of Capstone Courses

Courses in our programs are under constant assessment in order to measure how each course needs to be improved for successfully fulfilling the role that particular courses have in the whole academic program. However, the final determination as to whether our graduating students are reaching the needed skills that the PLOs imply, is based on how they perform in their capstone courses. Hence the capstone courses in both programs will be under assessment on each occasion that they are offered. In them, we measure how students perform in every program student outcome.