In this document I will discuss three different types of assessment to gauge my teaching and student learning: 1) student evaluation of courses, 2) assessment of course outcomes based on exams, and 3) student surveys about the courses.

Student evaluations are now done every semester for all the courses. Sometimes in the past they were once a year after an instructor was tenured. I will look at my evaluations for all the courses I have taught at the University of Akron since I joined the University.

Using the other two forms of assessment mentioned above (course outcomes and student course surveys) I am trying to create a system where I can identify and isolate areas which students find difficult (or, may be, I make them so), I can then make modifications to just these areas in teaching, and then assess after making adjustments. In this process I will continuously improve my teaching and learning. I am still in the process of validating the process and ensuring its reliability. Once validated, I will be able to use it effectively.

Assessment of course outcomes based on student performance in exams can be influenced by many factors including, difficulty of the exam, student understanding of the material, and my own delivery of the material. These factors have not been partialed out in the analyses presented here. For this reason and because sometimes there was not enough data to draw any conclusions (too few students in a course or too few data points) all the results here are preliminary. The outcomes on which students have generally scored low are those which the students have found difficult or for which my delivery needs improvement.

In the student surveys I try to assess student satisfaction with the course and their perception of learning in the course. Here I can see that although students might feel that they find a topic to be difficult, they still feel that they have gained something out of the course in that particular topic or outcome. Their achievement may not be what I expected, yet the students may feel that they have learned.

In the next step, which has not yet been implemented I will try to correlate the above three types of measurements.

Student Evaluation of Teaching

In this section the student evaluations which are done every semester are discussed. These will be presented individually for the courses I have taught most often, for semesters, and averages for academic years.

These measurements are from the official instrument(s) that are being (or have been) used by the College. Although there has been a debate about their validity yet they do estimate some relevant aspects of the courses. While one or a few of these evaluations may not provide accurate results, their consistency over time does point to their temporal reliability.

Evaluations for individual courses:

The following table summarizes the values for all the courses I have taught at the University of Akron since I joined the University.

Table 1 Mean Student Ratings for the Courses I Have Taught

	Department and Course Number	Course Name	Number of Students[1]	Overall Mean Rating for the Course
1.	2440 (CIS)		1086	4.275
1.1.	101	Fundamental Computer Concepts	20	4.487
1.2.	102	Introduction to Windows	13	4.524
1.3.	103	Software Fundamentals	10	4.3
1.4.	105	Introduction to Computers & Application Software	82	4.462
1.5.	121	Introduction to Logic/Programming	65	4.179
1.6.	140	Internet Tools	115	4.316
1.7.	145	Operating Systems	112	4.140
1.8.	170	Visual Basic	215	4.334
1.9.	240	CIS Internship	6	3.779
1.10.	241	Systems Analysis & Design	77	4.221
1.11.	247	Hardware Support	24	3.998
1.12.	251	CIS Projects	132	4.428
1.13.	256	C++ Programming	139	4.177
1.14.	257	Microcomputer Projects	67	4.092
1.15.	268	Network Concepts	9	4.673
2.	2860 (EET)		52	4.263
2.1.	237	Digital Fundamentals	20	4.169
2.2.	238	Microprocessor Applications	8	4.208
2.3.	406	Communication Systems	10	4.388
2.4.	420	Biomedical Electronic Instrumentation	14	4.383
	Overall Mean	(for both Departments)	1138	4.274

Variations of student evaluations over the semesters for the courses I have taught most often are given below.

They are as follows:

1. Figure 1 “Introduction to Computers & Application Software (2440: 105)”,

2. Figure 2 “Programming Logic (2440: 120)”,

3. Figure 3 “Internet Tools (2440: 140)”,

4. Figure 4 “Operating Systems (2440: 145)”,

5. Figure 5 “Visual Basic (2440: 170)”,

6. Figure 6 “Systems Analysis & Design (2440: 241)”,

7. Figure 7 “CIS Projects (2440: 251)”,

8. Figure 8 “C++ Programming (2440: 256)”,

9. Figure 9 “Microcomputer Projects (2440: 257)”,

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Evaluations over the Semesters

Mean values over all courses for each semester from Fall 1999, when I joined the University, right up to Spring 2009 are shown below. Also superimposed on the graph are the trendline and the 95% confidence interval around the trendline. The trendline shows a gradual but very clear increase in overall mean student evaluation for semesters.

Figure 10

Mean annual evaluations

Mean student evaluations for calendar years are summarized here. Again a trendline and 95% confidence intervals are superimposed on the graph. As stated for the previous Figure the trendline is just like the one there and has the same the same characteristics. The trendline in Figure 10 is more reliable because it has more data points.

Figure 11

Measurement of Course Outcomes

I have measured achievement of students on course outcomes in some of my courses. The results presented here are direct measures of course outcomes. The different elements that are used to assess students in all my courses include exams, laboratory assignments, and attendance. All these elements of the courses were not analyzed. Only the multiple choice exams were analyzed to see how well students achieve the different course outcomes. Specifically the courses and the semesters when the outcomes were measured together with the number of students who took all the exams were:

Table 2 Courses for Which Exams Have Been Analyzed

Semester	Course	Number of Students in the Course
Fall 2006	Logic (Section 001)	18
Fall 2006	Logic (Section 801)	14
Fall 2006	Internet Tools	24
Fall 2006	Systems Analysis & Design	14
Fall 2006	C++ (online)	15
Spring 2007	Logic	22
Spring 2007	Operating Systems	24
Spring 2007	Visual Basic	10
Fall 2007	Internet Tools	27
Fall 2007	Operating Systems	26
Fall 2007	Digital Circuits	12
Fall 2007	Systems Analysis & Design	9
Spring 2008	Visual Basic	7
Fall 2008	Digital Circuits	11
Fall 2008	Operating Systems	27
Fall 2008	Systems Analysis & Design	7
Fall 2008	C++	14
Spring 2009	Visual Basic	12

Mean student scores on an outcome are being reported here for the courses together with the percentage of students who scored at least 70% on the relevant outcome.

Results for some of the individual courses that have been analyzed are presented now. One thing that is evident is that as the courses become move general to specialized, the student demographics changes and the level of understanding increases. I am arbitrarily using a 70% achievement level in the following. Probably it should be different in different courses.

Course outcomes for individual courses

Digital Circuits

The course outcomes for “Digital Circuits” (2860: 237) are:

Number	Course Outcome
1	I understand and can explain the basics of digital circuits (analog/digital representation, number systems, Boolean algebra, Karnaugh maps, basic gates)
2	I can differentiate the different digital ICs and their characteristics
3	I can analyze, design, build, and troubleshoot a broad range of combinational circuits using digital ICs
4	I understand and can explain flip-flops, one-shots, and timers
5	I can analyze, design, build, and troubleshoot a broad range of counters
6	I understand and can explain shift register basics, the various kinds, their operating characteristics, and applications
7	I can do computer modeling of digital circuits

The outcomes were analyzed for the percentage of students who scored 70% or above on each outcome and the mean student scores on each outcome. The results follow:

	Mean Student Score		Number of Students Achieving 70% or more on the Outcome
Outcome #	Fall 2007	Fall 2008	Fall 2007	Fall 2008
1	75%	80%	67%	82%
2	55%	60%	33%	18%
3	60%	69%	33%	64%
4	56%	71%	33%	73%
5	67%	76%	67%	82%
6	47%	64%	25%	27%
7	58%	73%	33%	46%

Introduction to Computers & Application Software

The course outcomes for “Introduction to Computers & Application Software” (2440: 105) are:

Number	Course Outcome
1	Basic Concepts of Windows
2	Concepts of E-mail programs
3	Concepts of the Internet and the Internet Explorer
4	Basics of computer hardware
5	Concepts of MS Word
6	Concepts of MS Excel
7	Concepts of MS Access
8	Concepts of MS PowerPoint

The outcomes were analyzed for the percentage of students who scored 70% or above on each outcome and the mean student scores on each outcome. The results follow:

For the semester from Spring 2008 through Summer III 2008 I used to deliver content via lectures and then have the students do exercises, in Spring 2009 I started using myITLab together with the book which changed the method of content delivery. The exercises were still assigned from the book and I was still not using skills testing in the myITLab. I started using skills testing in myITLab in Summer 2009 and the results for these have not been analyzed yet.

	Mean Student Score					Number of Students Achieving 70% or more on the Outcome
Outcome #	Spr 2008	Sum I 2008	Sum III 2008	Spr 2009 (001)	Spr 2009 (002)	Spr 2008	Sum I 2008	Sum III 2008	Spr 2009 (001)	Spr 2009 (002)
1	71%	63%	93%	81%	85%	65%	67%	100%	100%	88%
2	71%	49%	71%	69%	73%	65%	44%	56%	63%	54%
3	65%	53%	77%	79%	87%	62%	56%	78%	100%	96%
4	59%	48%	72%	62%	68%	39%	22%	33%	31%	33%
5	47%	29%	58%	50%	54%	23%	22%	33%	0%	4%
6	39%	34%	44%	57%	60%	12%	33%	33%	42%	42%
7	40%	33%	29%	41%	43%	15%	22%	11%	10%	4%
8	43%	30%	48%	45%	45%	19%	22%	33%	5%	17%

Programming Logic

The course outcomes for “Programming Logic” (2440: 121) are:

Number	Course Outcome
1	I can explain and use the basics of programming (data hierarchy, variables, data types, flowcharts, pseudocode, etc.)
2	I can explain and use program structure in terms of the 3 basic programming constructs
3	I can explain and use planning and can document programs
4	I can explain and use the Decision making structure
5	I can explain and use the Repetition structure
6	I can explain and use Arrays
7	I can explain and use modules in programs
8	I can explain and use Control Breaks in programs

The outcomes were analyzed for the percentage of students who scored 70% or above on each outcome and the mean student scores on each outcome. The results follow:

	Mean Student Score			Number of Students Achieving 70% or more on the Outcome
Outcome #	Fall 2006 (Sec 001)	Fall 2006 (Sec 801)	Spring 2007	Fall 2006 (Sec 001)	Fall 2006 (Sec 801)	Spring 2007
1	68%	81%	79%	39%	57%	46%
2	59%	85%	77%	50%	57%	41%
3	58%	81%	81%	28%	57%	36%
4	63%	81%	66%	28%	57%	23%
5	50%	69%	59%	22%	36%	36%
6	41%	59%	56%	6%	36%	14%
7	70%	87%	81%	33%	64%	55%
8	50%	64%	59%	33%	36%	32%

Internet Tools

The course outcomes for “Internet Tools” (2440: 140) are:

Number	Course Outcome
1	I can explain and use the basics of HTML (The elements in HTML, creation of Web pages and Web sites, links, etc.)
2	I can explain and use cascaded style sheets (CSS)
3	I can explain and use Web Tables
4	I can explain and use Web Forms
5	I can use frames in Web pages
6	I can explain and use multimedia in Web pages
7	I can explain and use XHTML
8	I know the basics of Javascript (before the 2 constructs)

The outcomes were analyzed for the percentage of students who scored 70% or above on each outcome and the mean student scores on each outcome. The results follow:

	Mean Student Score		Number of Students Achieving 70% or more on the Outcome
Outcome #	Fall 2006	Fall 2007	Fall 2006	Fall 2007
1	72%	72%	50%	59%
2	69%	65%	42%	41%
3	65%	56%	38%	19%
4	63%	63%	33%	22%
5	73%	71%	42%	37%
6	75%	72%	58%	44%
7	61%	77%	29%	56%
8	65%	78%	29%	59%

Operating Systems

The course outcomes for “Operating Systems” (2440: 145) are:

Number	Course Outcome
1	I can explain the basics of the Operating Systems Theory (structure of an OS, the 3 layers, their functions, etc.)
2	I can describe the history of UNIX, its various systems and its features
3	I can explain and use basic UNIX commands(login, logout, passed, etc)
4	I can explain and use vi editor
5	I can explain and use the UNIX file system
6	I can explain and use UNIX shells
7	I can explain and use UNIX communication (mesg, wall, mailx, news, etc.)
8	I know the basics of program development and can write simple UNIX scripts
9	I know and can use some common UNIX utilities (cal, date, du, etc)

The outcomes were analyzed for the percentage of students who scored 70% or above on each outcome and the mean student scores on each outcome. The results follow:

	Mean Student Score			Number of Students Achieving 70% or more on the Outcome
Outcome #	Spring 2007	Fall 2007	Fall 2008	Spring 2007	Fall 2007	Fall 2008
1	75%	49%	47%	63%	8%	11%
2	68%	78%	78%	33%	62%	59%
3	75%	66%	74%	50%	46%	52%
4	49%	58%	56%	4%	35%	19%
5	57%	62%	68%	21%	42%	44%
6		48%	47%		12%	4%
7		70%	80%		62%	56%
8			63%			30%
9			57%			26%

Visual Basic

The course outcomes for “Visual Basic” (2440: 170) are:

Number	Course Outcome
1	I can explain and use the elements of Visual Basic (The GUI, simple controls, the basic form, the integrated development environment, etc.)
2	I can explain and use variables and the 3 basic programming construct in relation to VB
3	I can explain and use procedures in VB
4	I can add forms to a VB project and can use lists, etc with programming in VB
5	I can use the printing controls and code them in VB
6	I can explain and use arrays in VB
7	I can use Web Forms in VB
8	I can connect to a database using VB and can code to navigate through and access the database
9	I can use file management methods in VB
10	I understand classes in VB

The outcomes were analyzed for the percentage of students who scored 70% or above on each outcome and the mean student scores on each outcome. The results follow:

	Mean Student Score			Number of Students Achieving 70% or more on the Outcome
Outcome #	Spring 2007	Spring 2008	Spring 2009	Spring 2007	Spring 2008	Spring 2009
1	88%	88%	81%	100%	100%	92%
2	71%	66%	68%	60%	43%	50%
3	71%	75%	67%	80%	71%	50%
4	71%	77%	69%	40%	86%	58%
5	80%	43%	67%	60%	14%	67%
6	53%	70%	67%	20%	72%	25%
7
8	75%	56%	71%	50%	43%	50%
9	76%	65%	56%	70%	86%	25%
10		68%			57%

Systems Analysis & Design

The course outcomes for “Systems Analysis & Design” (2440: 241) are:

Number	Course Outcome
1	I understand and can explain the context of systems development projects
2	I understand and can use the important Systems Analysis methods using the traditional approach
3	I understand and can use the important Systems Design methods using the traditional approach
4	I understand the concepts in construction and implementation of complete systems

The outcomes were analyzed for the percentage of students who scored 70% or above on each outcome and the mean student scores on each outcome. The results follow.

For Fall 2006 the text was by Satzinger, et. al. and for Fall 2007 and Fall 2008 the text was by Whitten & Bentley.

	Mean Student Score			Number of Students Achieving 70% or more on the Outcome
Outcome #	Fall 2006	Fall 2007	Fall 2008	Fall 2006	Fall 2007	Fall 2008
1	84%	71%	79%	93%	56%	71%
2	75%	65%	72%	79%	33%	43%
3	77%	64%	73%	79%	33%	57%
4	67%	58%	67%	57%	33%	57%

C++ Programming

The course outcomes for “C++ Programming” (2440: 256) are:

Number	Course Outcome
1	I understand and can explain the basics of C++; C++ program structure; Integrated Development Environment
2	I understand and can explain the 3 basic programming constructs in relation to C++
3	I understand and can use functions
4	I understand and can use classes (excluding multiple inheritance, etc.)
5	I understand and can use arrays in C++
6	I understand and can use basic pointers
7	I understand and can use basic file management concepts in C++

The outcomes were analyzed for the percentage of students who scored 70% or above on each outcome and the mean student scores on each outcome. The results follow.

The Fall 2006 course was an online offering using the text by Deitel & Deitel, while the Fall 2008 course was face-to-face delivery using the text by Gaddis.

	Mean Student Score		Number of Students Achieving 70% or more on the Outcome
Outcome #	Fall 2006 (online course)	Fall 2008 (face-to-face course)	Fall 2006 (online course)	Fall 2008 (face-to-face course)
1	82%	80%	80%	59%
2	61%	69%	33%	41%
3	57%	81%	20%	47%
4	54%	65%	13%	29%
5	53%	81%	33%	64%
6	40%	71%	0%	41%
7	61%	76%	20%	53%

Estimation of Student Perceptions of the Courses

I have measured student perceptions of course outcomes in some of my courses. This was done to see if the students themselves feel that they have learned something in the relevant courses. The perceptions (or opinions) were measured in the beginning of the courses and then again at the end of the courses and then compared to see if there was a significant change in the students’ rating of their knowledge. Some of the results presented here are indirect measures of course outcomes. Specifically, the courses and the semesters when the outcomes were measured together with the number of students completing the surveys were:

Table 3 Courses in Which Students Were Surveyed

Semester	Course	Number of Students in the Course[2]
Fall 2007	Digital Circuits	10
Fall 2007	Systems Analysis & Design	8
Spring 2008	Microprocessor Applications	10
Fall 2008	Digital Circuits	11
Fall 2008	Systems Analysis & Design	7
Fall 2008	C++	14
Spring 2009	Visual Basic	13
Spring 2009	Communication Systems	13
Spring 2009	Biomedical Electronic Instrumentation	16

The students rated the course outcomes for all these courses using a 4-point Likert scale.

Description of Rating	Rating
Never heard of it	1
Had heard of it, but didn’t really know what it meant	2
Had some idea of what this meant, but not too clear	3
Had a clear idea of what this meant and could explain it	4

Results for individual courses are presented now. For my purposes I arbitrarily use 70% level as a good bar, that is, if 70% or more students feel that they have gained on a particular outcome I think that I am doing well on that outcome. If less than 70% students feel that they did not learn on an outcome then I think I need to work on that outcome.

Student perceptions of individual courses

Digital Circuits

The course outcomes for “Digital Circuits” (2860: 237) are presented here again:

Number	Course Outcome
1	I understand and can explain the basics of digital circuits (analog/digital representation, number systems, Boolean algebra, Karnaugh maps, basic gates)
2	I can differentiate the different digital ICs and their characteristics
3	I can analyze, design, build, and troubleshoot a broad range of combinational circuits using digital ICs
4	I understand and can explain flip-flops, one-shots, and timers
5	I can analyze, design, build, and troubleshoot a broad range of counters
6	I understand and can explain shift register basics, the various kinds, their operating characteristics, and applications
7	I can do computer modeling of digital circuits

The course outcomes surveys were analyzed for the means ratings on each outcome pre-course and post-course. At the end of the course the two were compared for significance. The results follow:

Outcome #	Mean Ratings
	Fall 2007		Fall 2008
	Pre-Course	Post-Course	Pre-Course	Post-Course
1	2.92	3.48	3.02	3.61
2	1.49	2.27	1.57	2.85
3	1.71	3.38	2.11	3.65
4	1.39	3.40	2.19	3.57
5	1.46	3.42	2.13	3.52
6	1.27	3.22	1.71	3.51
7	1.65	2.15	1.66	2.93
p-Value	0.002242		8.12E-05
Confidence Level	99.0%		99.0%
Significance @ a = 0.01	Significantly different		Significantly different
Power of the test @ a = 0.01	0.937		1.0

The following table summarizes the percent of students who felt that they gained (significantly) for an outcome (rated at least a 3 for the outcome) in the course.

Outcome number	Percent of Students Rating 3 or above in Post-Course Survey
	Fall 2007	Fall 2008
1	80%	91%
2	10%	36%
3	90%	91%
4	90%	91%
5	80%	82%
6	60%	82%
7	0%	64%

Microprocessor Applications

The course outcomes for “Microprocessor Applications” (2860: 238) are presented here:

Number	Course Outcome
1	I understand and can explain the various types of memory systems
2	I can analyze and design memory and memory decoding systems
3	I understand the fundamentals of microcomputing environment (e.g., hardware functions, processor architecture, etc.)
4	I can analyze, design, build and test hardware and software applications
5	I can use assembly language

The course outcomes surveys were analyzed for the means ratings on each outcome pre-course and post-course. At the end of the course the two were compared for significance. The results follow:

Outcome number	Mean Ratings
	Spring 2008
	Pre-Course	Post-Course
1	2.67	3.60
2	1.73	2.56
3	1.71	2.80
4	1.62	2.33
5	1.83	2.79
p-value	0.000152724
Confidence Level	99.00%
Significance @ a = 0.01	Significantly different
Power of the test @a = 0.01	1.0

The following table summarizes the percent of students who felt that they gained (significantly) for an outcome (rated at least a 3 for the outcome) in the course.

Outcome number	Percent of Students Rating 3 or above in Post-Course Survey
Outcome number	Spring 2008
1	90%
2	30%
3	30%
4	10%
5	20%

Communication Systems

The course outcomes for “Communication Systems” (2860: 406) are presented here:

Number	Course Outcome
1	I understand and can explain different types of traditional modulation
2	I can explain systems to digitize and transform analog data to digital format
3	I can explain the basics of Information Theory
4	I understand and can explain different types of data encoding
5	I can explain the functioning of transmission lines wave propagation
6	I can explain the functioning and components of digital television
7	I can explain the concepts of satellite systems, spread spectrum techniques, etc.
8	I can explain the concepts and functioning of optical fibers

The course outcomes surveys were analyzed for the means ratings on each outcome pre-course and post-course. At the end of the course the two were compared for significance. The results follow:

Outcome number	Mean Rating
	Spring 2009
	Pre-Course	Post-Course
1	2.32	3.41
2	2.24	3.58
3	2.16	3.37
4	1.72	3.18
5	1.76	3.41
6	1.76	3.36
7	1.28	3.59
8	1.67	3.15
p-value	8.11764E-06
Confidence Level	99.00%
Significance @ a = 0.01	Significantly different
Power of the test @a = 0.01	1.0

The following table summarizes the percent of students who felt that they gained (significantly) for an outcome (rated at least a 3 for the outcome) in the course.

Outcome number	Percent of Students Rating 3 or above in Post-Course Survey
Outcome number	Spring 2009
1	69%
2	85%
3	77%
4	62%
5	77%
6	69%
7	85%
8	77%

Biomedical Electronic Instrumentation

The course outcomes for “Biomedical Electronic Instrumentation” (2860: 420) are presented here:

Number	Course Outcome
1	I can define a medical device
2	I understand and can explain the functioning of transducers, sensors, electrodes, etc.
3	I understand and can explain biopotentials
4	I understand and can explain the functioning of bioamplifiers
5	I understand and can explain the functioning of shocks (macro and micro); also concerns of safety
6	I understand and can explain the functioning of ECG,VCG
7	I understand and can explain the functioning of EEG
8	I can explain other cardiac parameters and how to measure them (BP, etc)
9	I understand different types of imaging and can explain their functioning

The course outcomes surveys were analyzed for the means ratings on each outcome pre-course and post-course. At the end of the course the two were compared for significance. The results follow:

For Spring 2009:

Outcome number	Mean Rating
	Spring 2009
	Pre-Course	Post-Course
1	2.29	3.74
2	2.29	3.46
3	1.66	3.58
4	2.10	3.87
5	2.20	3.59
6	1.40	3.58
7	1.20	3.60
8	1.26	3.17
9	2.19	4.00
p-value	7.91735E-07
Confidence Level	99.00%
Significance @ a = 0.01	Significantly different
Power of the test @a = 0.01	1.0

The following table summarizes the percent of students who felt that they gained (significantly) for an outcome (rated at least a 3 for the outcome) in the course.

Outcome number	Percent of Students Rating 3 or above in Post-Course Survey
Outcome number	Spring 2009
1	94%
2	88%
3	81%
4	94%
5	88%
6	88%
7	94%
8	53%
9	100%

Visual Basic

The course outcomes for “Visual Basic” (2440: 170) are presented here again:

Number	Course Outcome
1	I can explain and use the elements of Visual Basic (The GUI, simple controls, the basic form, the integrated development environment, etc.)
2	I can explain and use variables and the 3 basic programming construct in relation to VB
3	I can explain and use procedures in VB
4	I can add forms to a VB project and can use lists, etc with programming in VB
5	I can use the printing controls and code them in VB
6	I can explain and use arrays in VB
7	I can use Web Forms in VB
8	I can connect to a database using VB and can code to navigate through and access the database
9	I can use file management methods in VB
10	I understand classes in VB

The course outcomes surveys were analyzed for the means ratings on each outcome pre-course and post-course. At the end of the course the two were compared for significance. The results follow:

Outcome number	Mean Rating
	Spring 2009
	Pre-Course	Post-Course
1	2.50	3.68
2	2.03	3.61
3	2.16	3.43
4	1.85	3.60
5	2.00	3.77
6	1.68	3.27
7	1.64	3.22
8	1.83	3.33
9	1.69	3.35
10
p-value	1.29228E-08
Confidence Level	99.00%
Significance @ a = 0.01	Significantly different
Power of the test @a = 0.01	1.0

The following table summarizes the percent of students who felt that they gained (significantly) for an outcome (rated at least a 3 for the outcome) in the course.

Outcome number	Percent of Students Rating 3 or above in Post-Course Survey
Outcome number	Spring 2009
1	92%
2	92%
3	92%
4	92%
5	100%
6	62%
7	77%
8	85%
9	85%
10

Systems Analysis & Design

The course outcomes for “Systems Analysis & Design” (2440: 241) are presented here again:

Number	Course Outcome
1	I understand and can explain the context of systems development projects
2	I understand and can use the important Systems Analysis methods using the traditional approach
3	I understand and can use the important Systems Design methods using the traditional approach
4	I understand the concepts in construction and implementation of complete systems

The course outcomes surveys were analyzed for the means ratings on each outcome pre-course and post-course. At the end of the course the two were compared for significance. The results follow:

	Mean Ratings
	Fall 2007		Fall 2008
Outcome #	Pre-Course	Post-Course	Pre-Course	Post-Course
1	1.56	3.09	1.71	3.58
2	1.79	3.05	1.92	3.62
3	2.47	3.20	2.31	3.60
4	2.40	3.20	1.86	3.64
p-Value	0.00249		0.000971658
Confidence Level	99.0%		99.0%
Significance @ a = 0.01	Significantly different		Significantly different
Power of the test @a = 0.01	0.968		1.0

The following table summarizes the percent of students who felt that they gained (significantly) for an outcome (rated at least a 3 for the outcome) in the course.

Outcome number	Percent of Students Rating 3 or above in Post-Course Survey
	Fall 2007	Fall 2008
1	75%	100%
2	75%	86%
3	75%	86%
4	5%	100%

C++ Programming

The course outcomes for “C++” (2440: 256) are presented here again:

Number	Course Outcome
1	I understand and can explain the basics of C++; C++ program structure; Integrated Development Environment
2	I understand and can explain the 3 basic programming constructs in relation to C++
3	I understand and can use functions
4	I understand and can use classes (excluding multiple inheritance, etc.)
5	I understand and can use arrays in C++
6	I understand and can use basic pointers
7	I understand and can use basic file management concepts in C++

The course outcomes surveys were analyzed for the means ratings on each outcome pre-course and post-course. At the end of the course the two were compared for significance. The results follow:

Outcome number	Mean Rating
	Fall 2008
	Pre-Course	Post-Course
1	2.59	3.92
2	2.71	3.86
3	2.58	3.71
4	2.02	3.66
5	2.65	3.87
6	2.21	3.60
7	2.07	3.64
p-value	1.59549E-06
Confidence Level	99.00%
Significance @ a = 0.01	Significantly different
Power of the test @a = 0.01	1.0

The following table summarizes the percent of students who felt that they gained (significantly) for an outcome (rated at least a 3 for the outcome) in the course.

Outcome number	Percent of Students Rating 3 or above in Post-Course Survey
Outcome number	Fall 2008
1	100%
2	100%
3	93%
4	93%
5	100%
6	79%
7	93%

Conclusions

The data presented above has sometimes not had enough students in a course and sometimes there are not enough data points for analysis. I have still done some analyses. In some cases the results may not be statistically reliable. Yet from the overall data some conclusions can be drawn.

From student evaluations there is a very clear trend going slightly upward over the years which can be taken as an improvement over the years.

I am still working on direct and indirect measures of outcomes. Using the objective measure I have tried to assess how well the student actually achieve an outcome, while using surveys I try to measure how well the student are satisfied and if they feel they have gained anything out of the course. The results from objective measures (exams) depend on several factors, including student’s class level (freshman, sophomore, etc), difficulty of the exams, the language capabilities of students, among others. I am using these measurements to approximately trace areas which students find “difficult” for some reason and the areas in which students feel they are learning the least.

In the case of the indirect measures I have can assess the students’ feeling of learning. I have assumed for the purposes of statistical tests that the student responses are samples from a normal distribution. Since usually the number of outcomes is less than 10, I have estimated the power of the t-tests besides the significance at a 99% level. Besides these the consistency of results together with their correspondence with the results from the objective measures is important.

I have just presented some results above without going into details of which areas the student find “more difficult”, since that is not the purpose of this document. This information is easily obtained from combining the results from objective and indirect measures of outcomes and is more relevant to the areas I have taught, namely, Computer Information Systems (CIS) and Electronic Engineering Technology (EET). They would also be of more interest to only the faculty in these areas.

Future direction

To present a brief taste of the future of these studies I am including integration of the results for the first outcome (Basic knowledge of Windows) in the “Introduction to Computers and Application Software” course over the offerings in 2008 and 2009. Rest of the study is not finished yet. In the graph below 0 (zero) represents results without taking the course and 1 (one) is the maximum a student can achieve after taking the course in this outcome.

In this diagram the “Std. Mean Difference” and the “SE” (Standard error) columns are results obtained from students’ responses to exam questions. The rest of the three columns on the right of the “SE” column are the results of analyses. The results of the analyses indicate that the five groups of students were very likely samples from the similar student populations and that there was significant learning. On the average students scored 87% on this outcome with a confidence interval from 73% to 100% at the 95% confidence level. Integrating results from different semesters I can in this way increase the reliability and power of the statistical tests and hence my conclusions that I may draw from the data.

[1] This column enlists the number of students filling out the evaluations. It is not the same as the students enrolled in the class or who took all the exams.

[2] This refers to the number of students who completed the surveys. It is not necessarily the same as the number of students enrolled in the course or the number of students who took the exams.