REPORT ON QUANTITATIVE SECTION OF THE SURVEY QUESTIONS
(Morgan Jennings and Debra Dirksen, Metropolitan State College, Denver, CO)
Overall there seemed to be correlation between responses from the entire sample and the responses of just the professors who reviewed the program. In the following sections, the findings are presented and some similarities and differences are presented.
The respondents felt that the goals and objectives were clear, well organized (96% strongly agree/agree) and innovative and significant (94% strongly agree/agree). However, the respondents indicated that the textual data-mining component does not integrate with the overall objectives of the CIMEL project. Only 69% strongly agree/agree with the statement and 31% are neutral.
The professor respondents were in strong agreement over the clarity of the goals and their significance (100 % strongly agree/agree). They favored the textual data-mining component more than the total-group of respondents, with 83% (strongly agree/agree) indicating that the component integrated will with the overall objectives.
CIMEL Acronym
Total-group responses for the use of the CIMEL acronym was somewhat well received. Sixty-three percent responded strongly agree/agree, 33% were neutral and 6% responded strongly disagree/agree. This split was similar to the professor responses (67% strongly agree/agree, 33% neutral).
Depth and Breadth
The breadth and depth of the content covered was well accepted for an introductory course in computer science (89% strongly agree/agree) and the respondents indicated that the core material was sufficient for this course (81% strongly agree/agree). Using the program to span freshman to graduate level was not as well received, with only 64% strongly agree/agree, 21% neutral, and 15% strongly disagree/disagree. There does not seem to be overwhelming support for this idea, especially as only 71% strongly agree/agree that the content is at the right level for upper division students. The respondents were split in their support of a multimedia program for website development and JavaScript, with only 45% strongly agree/agree and 55% either strongly disagree, disagree, or neutral on the question.
According to the professor's, the breadth of chapters (100% strongly agree/agree) as well as the first nine topics of content listed (80% strongly agree/agree) are sufficient for this course. There was a discrepancy in the views of the professors and those of all responses regarding the appropriateness of the content for use with upper level students. One hundred percent (strongly agree/agree) of the professors supported the view that the content was at the right level, while only 71% of all respondents indicated that the content was at the right level. The professors were also split in their support of a multimedia program for website development and JavaScript, with only 60% strongly agree/agree and 40% neutral on the question. Using the program to span freshman to graduate level was also split, with only 50% strongly agree/agree, 25% neutral, and 25% strongly disagree/disagree.
Design
The design aspect of the program was positive. Ninety-four percent (strongly agree/agree) positively indicated that the material was presented in a logical order with new concepts building on previous concepts. The respondents also believed that distinguishing between introductory and advanced topics is a good idea (92% strongly agree/agree). However, the respondents were not as supportive of the design of the introductory page. While 79% of the respondents strongly agree/agree that the summary of the prerequisites, topics, and exercises were effective as a lead-in for the introductory page, 22% strongly disagree, disagree, or were neutral about the statement.
There was unanimous support by the professors for the design aspects of the program (100% strongly agree/agree). They indicated that the introductory page was a great lead-in, the instruction was presented in a logical order, and the distinguishing between introductory and advanced topics was a good idea.
Interface
The interface seemed to be well designed overall, with respondents finding the narration to be easy to read (86% strongly agree/agree), text fonts to be clear and readable (78% strongly agree/agree), interface to be visually appealing (73% strongly agree/agree), and functional (71% strongly agree/agree). The use of a content menu, showing lessons and screen topics was also well supported (82% strongly agree/agree). There was little support for the idea of allowing narration text to overlay the content menu with only 44% strongly agree/agree and 35% strongly disagree/disagree.
Review of the interface was somewhat mixed by the professors responding, with 100% (strongly agree/agree) indicating that the text is easy to read and the content menu should always be present on the screen. A majority of the professorial respondents, 80% (strongly agree/agree) indicated the user-interface included all the needed functionality. However, only 67% (strongly agree/agree) found the interface visually appealing and only 50% (strongly agree/agree) thought that the text fonts were clear and readable. On the other hand, the entire group of respondents rated the text fonts much higher. Narration was not received favorably, with only 50% (strongly agree/agree) supporting the view that narration text should overlay content menu. Sixty-seven percent (strongly disagree/disagree) of the professors disagreed with the statement, "voice-over narration should be different from the text on the screen." This aligns with cognitive research. Interestingly, the total-group respondents supported the use of voice-over narration that was different than the text.
Interaction
There was much support for the use of interaction in the program, with 71% strongly agree/agree that interactive exercises help students learn content and 86% (strongly agree/agree) indicating that more interactive exercises would be beneficial. Not many of the respondents' felt that the program as it stands provide opportunities for learning by doing (55% strongly agree/agree). There was a negative response to the statement, "The exercise and demonstration of the hot topics system shows an effective way to simulate inquiry-based learning," with 49% of the respondents strongly disagreeing or disagreeing with this statement. The respondents indicated that the multimedia did provide guidance for further investigation (80% strongly agree/agree) and were supportive a quiz at the end of each lesson (83% strongly agree/agree).
The use of interactivity was supported by the professor's responses, indicating that interactive exercises help students learn the content (80% strongly agree/agree), quizzes should end the lesson (80%), the program contained interactive exercises (83% strongly agree/agree), and more interactive exercises would be beneficial (83% strongly agree/agree). The total-group of respondents was much less supportive of the view that the program contained interactive exercises. In regarding the ability of the multimedia to provide guidance for further investigation, the professors were somewhat split, with only 67% (strongly agree/agree) supporting this view and 34% disagreeing or neutral to this view. The professors agreed with the total group of respondents being split over the view that the exercise and demonstration of the hot topics system shows an effective way to stimulate inquiry-based learning (33% strongly disagree/disagree, 33% neutral, 33% strongly agree/agree).
Self-paced
The respondents were very supportive of keeping the program self-paced, with 59% strongly disagreeing or disagreeing with the idea of making the program advance automatically. Self-paced components were well received, with 73% (strongly agree/agree) indicating that the program did allow them to work at their own pace, and 88% (strongly agree/agree) supportive of letting instructors or learners decide what topics should be covered. To clarify this question, however, it should be split into two questions so that we will know which is preferred, instructor control or student control.
There was strong agreement between the professor's responses and the total responses on the self-paced nature of the program. While 80% (strongly agree/agree) of the professors did find the program allowed self-pacing and 83% (strongly agree/agree) felt that the interface should allow for the student or instructor to decide topic coverage, 67% (strongly disagree/disagree) did not think the screens should advance automatically.
Collaboration
The show me demo does not seem to support interactivity and collaboration to the degree that would be desirable. Forty-nine percent disagreed with the statement that the demo stimulated inquiry-based learning and only 67% indicated that it stimulated collaboration. The interface of the program as a whole seemed to support collaboration, with 90% strongly agreeing and agreeing that the outline for the interface was effective for supporting collaboration.
In the area of collaboration, the professors again generally aligned with the total group responses with the professors not showing support of the hot topics system (50% strongly agree/agree and 50% neutral). They strongly aligned on the effectiveness of an outline for a collaborative interface (86% strongly agree/agree).
Multimedia
The use of multimedia to support the learning of course content was perceived as being valuable (90% strongly agree/agree), however the program was not perceived as being particularly strong in this area. Only 78% (strongly agree/agree) indicated that the multimedia made good use of graphics to explain the content, and only 73% (strongly agree/agree) indicated that the use of sound supports the learning of the content. The persona were not well accepted with the respondents being split in their perception that the persona were effective, 27% strongly disagree/disagree, 40% neutral, and 33% strongly agree/agree. Only 55% (strongly agree/agree) found it to be helpful to have a persona read the text on the screen.
There was total agreement by the professors in the view that multimedia is valuable in learning course content, use of sound to support learning, and having a persona read screen text (100% strongly agree/agree). Also, there was strong support for the view that the multimedia made good use of graphics to explain the content (83% strongly agree/agree). The professors (67%) found the effectiveness of the persona to be twice as high as did the total group response. Additionally, 67% of the professors indicated that a wide variety of media were effectively used to teach the course content.
Difficulty
The total-group respondents were well split over the difficulty of the content, indicating that a "happy" medium has been achieved. 35% (strongly agree/agree) that the material was too introductory, and 6% (strongly agree/agree) found the material to be too advanced. While in the converse, 27% (strongly disagree/disagree) disagreed with the view that the material was too introductory, and 55% (strongly disagree/disagree) disagreed with the view that the material was too advanced.
As with the total group responses, the professors were ambiguous over the difficulty of the content. Thirty-tree percent marked strongly agree/agree that much of the content of the multimedia was too introductory and 17% indicated that this same content was too advanced. Sixty-seven percent were neutral in their response to this question.