CIMEL Review Panel
March 15, 2001
Goals of today’s meeting:
n Review the web-based survey
of
students, faculty and others about our goals, analysis and prototype
n A starting point for brainstorming
n A basis for planning our alpha
version
Why an audio tape?
n For our friendly evaluators:
n Morgan Jennings and Debra Dirksen
n Education professors at the
Metropolitan College of Denver, CO
n They helped prepare for this
meeting by analyzing the survey data
n They will also help us assess
what we accomplish today
CIMEL project goals (from web site):
n To design a multimedia framework
for constructive, collaborative, inquiry-based learning, with multiple tracks.
Definitions:
n Constructive learning goes beyond learning
by receiving knowledge, to learning by building systems, with immediate, visual
feedback.
n Collaborative learning encourages students
to interact with instructors and librarians, via both live links and remote-controlled
"show me" sessions or by reviewing a multimedia FAQ of recorded
"show me" sessions .
n Inquiry-based learning guides the student
into pursuing exploratory research in a community of students and scholars.
n Multiple tracks means that we want deliver
content for students at different levels, for introductory and upper level
computer science courses, and with different learning styles.
Break-out groups:
n Each will discuss a set of
questions.
n Appoint someone to take notes.
n Each group gets a laptop.
n You’ll have about half an hour.
n Get as far as you can.
Break out group 1: User
interface issues
(Glenn
Blank, Paul Krasicky, Sasidhar Mukkamalla)
Should we try to improve the appearance of the user interface?
Any suggestions?
- The
user interface is visually appealing. (73% strongly agree/agree, mean 3.80)
- “The
user interface was very clear and visually appealing; it looked very neat
and orderly”
- “Your
bottom nav bar should look more like a nav bar, and not like a timeline.”
- Is
the bottom navigation bar effective? Are
the colored boxes on the button bar sufficient are should different icons
be used for the different functions?
- “I
don't like the help feature. I think the buttons should still function when
you are in help mode. They can still show the "Tool tip" as well.”
- Should
the help button be replaced by tool-tips that appear on mouse-over?
Does the user interface include all the needed functionality?
·
The user interface includes all the needed functionality.
(71%
strongly agree/agree)
· “Have the forward button become brighter or change
color when the prof is finished speaking and its time to be pushed.”
· Should the forward button be an arrow or should
it flash to signal ready to go to next page?
· “Also include a pause button, so the lesson can
be paused and student can do other things.”
· Are other functions needed, such as pause or minimize
window?
How can the content menu be more effective, e.g., giving
them a better awareness of context and progress?
- A
content menu, showing lessons and screen topics, should always be present
on the screen. (82% strongly agree/agree, mean 4.06)
- “I
think that too much of the screen is being used up by the menus.”
- Is
this a good use of screen real estate?
- “Divide
the sections into screens for navigation purposes and add screens to menu.”
- Would
more detail make the content menu more effective?
- For
technical reasons, this menu may need to move from the left-hand side to
the right—is this OK?
Notes from Sasidhar Mukkamalla and Glenn Blank:
- Pause
(stopping action) and hide (minimizing
screen) buttons would be desirable.
- The
interface bar (at the bottom) should be reorganized, so that all the navigation
functions (Menu, Pause, Hide, Exit, Back, Forward) should be above the bar
while all the other buttons (Find, Glossary, Explore, Preferences, Experts)
should be beneath the bar.
- The
back and forward buttons should be closer together and could be arrows;
highlighting the forward button (signaling ready to advance page) should
be more conspicuous.
- The
interface bar should be moved further down the screen.
- Help
can be replaced by tool tips which should not prevent the buttons from functioning.
The tool tip explanations should be closer to the respective buttons.
- The
navigation menu should remain, providing links to screens in a lesson, highlighting
the current screen (for example, see Authorware help pages). It would be better to keep the menu on
the left. Try to reduce the space
allocated for this menu and possibly make it possible to take it off the
screen.
Break out group 2: Multi-tracking, constructive exercises and “hot topics”
system
Brian
Courtney, David Gevry, Bill Pottenger
and Soma Roy
How can we design a user interface that lets instructors
and students choose their own tracks through the content?
- A
framework with multiple tracks, suitable for an audience ranging from freshmen
to graduate students, is desirable. (mean 3.81)
- “One
piece of software that has a very wide range of audience (as long as it
is applicable to all audiences) is appealing. If it is an interesting piece
of software a freshman who is accustomed to using it will continue to do
so.”
- “The
challenging part is cohesively organizing all the data into one product,
yet have it remain relevant to all audiences. It seems like the content
is there.”
- Distinguishing
between introductory and advanced topics in the menu is a good idea. (mean 4.62)
- Should
this distinction be made in a top level menu or should it be made by different
tracks through the content, either pre-made and user-selectable or creatable
by instructors and students?
- How could we better support learning
by construction?
- There
were interactive exercises that supported learning by doing or constructing.(Mean
3.64)
- Interactive
exercises help students learn content and 86% (strongly agree/agree) indicating
that more interactive exercises would be beneficial.
- Not
many of the respondents' felt that the program as it stands provide opportunities
for learning by doing (55% strongly agree/agree).
- See
Soma Roy’s constructive exercise, not available in the original prototype.
Is this a good example of a constructive exercise?
- Are
there other approaches or examples of constructive exercises that we should
consider? (Think about the subject matter, object-oriented software engineering.
- How
can the purpose of the hot topics system (stimulating inquiry-based learning
by doing research) be made clearer?
- The
exercise and demonstration of the hot topics system shows an effective way
to stimulate inquiry-based learning. (mean 2.80, with 49% of the respondents
strongly disagreeing or disagreeing with this statement.)
- “Overall,
the data mining component is a good idea, but the instruction given in the
prototype was too long and too detailed to be digested all at once.”
- “I
think that trend detection will integrate well with constructive learning
exercises.”
- Are there other ways to stimulate
inquiry-based learning that are not hinted at in the prototype?
Notes from
David Gevry:
In our group we looked at the constructive exercises, Hot Topic System, and the multi-tracking
aspects of the CIMEL project.
Constructive Exercises:
After looking at Soma’s demo, the main improvements our
group suggested would be more user cues as to whether the user had made a
correct assumption in the exercise or not.
Additionally these cues would need to be sound independent incase the
user did not have sound enabled, and provide a longer window for the user to
see that they were wrong in making their assumption. The message stating that the user was wrong flashed by a little
too quickly. Also, there should be a
method of pausing or slowing the exercise down so that the user can examine
things that are moving too quickly for them.
Hot Topic System:
The major issue discussed on this point due to time
restraints was the fact that the name of the Hot Topic System was confusing in
it's actual purpose. The Hot Topic
System is similar to technology forecasting in the fact that it identifies
topics in the literature source it is linked too that are emerging over time
and are potentially new technologies or concepts that have a high likelihood of
importance. However the Hot Topic system
does not show important topics in the course material nor is it a search engine
for document retrieval. A better name
would be along the lines of Current Research Hot Topics or Trends. During the panel meeting we agreed that the
Hot Topics engine should be accessible, along with the Web browser, via an
Explore button on the Interface bar.
Multi-tracking:
From the prototype it was hard to see whether the
information could be scaled appropriately to a range of introductory to
graduate level material. Potentially if
the individual lessons were short enough one could assemble to programs from a
common bin of lessons however the pace of the lessons would not really
change. For a graduate level student
the information might been already know too some extent and lessons moving at a
slightly faster pace would be more useful.
Additionally the need for a modular system from which an instructor
could assemble a course in the order they desire that also yields the student
some control in the exploration of the topic were examined. The need for the lesson modules to have prerequisite
knowledge also was brought up. If a
user/student has control over what they
look at then the lessons need to be able to tell the user what they might need
to understand the material inside. Different
ways of accomplishing this involved potentially having a instructor mapping and
a student schedule planning program. In
this the student would be given a schedule suggested by the instructor but be
able to drag and drop modules together to plan a schedule tailored to their
needs. This could also involve the instructor marking required and optional sections
to the student could then have a pool of extra material to work with if they
desired more from the class.
Break
out group 3: Personae and collaborative interface
Drew Kessler, Qiang Wang and
Tianhao Wu
What is effective and ineffective in regard to the persona?
- The
presence of personae (such as a professor or librarian) with human faces
on the screen is effective. The personae were not well accepted with the
respondents being split in their perception that the persona were effective,
27% strongly disagree/disagree, 40% neutral, and 33% strongly agree/agree.
(mean 3.04) The professors (67%) found the effectiveness of the persona
to be twice as high as did the total group response.
- “Professor
Blank's face sees really weird with Pottenger’s voice.”
- “Maybe
a selection of different voices.”
- “I
would try to use 1 uniform voice throughout the whole piece.”
- We
didn’t use Blank’s voice because
it probably isn’t clear enough. Should we use other people as models for
our professor to avoid this effect?
- Should
we let students or instructors choose the faces they want for the personae?
- How
can personae convey the idea that there is someone who cares about the student,
available to help them learn?
- How
should we combine voice and corresponding text?
- Voice-over
narration should be different from the text on the screen. (mean 3.00) 67%
of the professors disagreed. Jennings notes that this aligns with cognitive
research, because it’s hard to process two things at one time.
- There
was little support for the idea of allowing narration text to overlay the
content menu with only 44% strongly agree/agree and 35% strongly disagree/disagree.
- Should
text replicating voice-over narration be in printable PDF files?
- Should
user have the option of having text replicating voice-over narration in
a movable window?
- How
can we make the “show me” collaborative learning environment effective?
- The
show me demo, suggested by the reference librarian's explanation of hot
topics system, appears to be an effective way stimulate collaborative learning.
(mean 3.80).
- “I
know there were times throughout college when I wish I could have interacted
with a prof. or expert in real time, but had to suffer and wait for an email
reply.”
- “It
is not clear that a protocol for tracking low-level IO events (e.g., mouse
movements or button clicks) is the best approach to collaboration. These
are low-level IO events. Perhaps more appropriate would be a publish &
subscribe user interface where, if two user interfaces subscribe to the
same events, their two users get identical views of the session.”
- Is
it helpful to have the show me interface circle the part of the screen that
the expert is pointing at (in addition to the cursor)? Should the show me system run in a web
browser or at the operating system level?
Notes by Drew Kessler:
Effectiveness of persona:
• Voice not matching known persona
not a big problem
– Let user choose voice? Maybe not, if professional voice, or familiar
professor used
– Option to say “I don’t understand
this voice, give me another.”
– Maybe too expensive.
• Appearance of persona
– Not important to have image
of specific instructor
– User select from pictures at
a “configuration” step
– Maybe not important to see
the persona at all
– If many classes, may be more
important to have separate persona images, and more important to match with
real person
– Students could be identified
by persona images
More on persona
•
Voice and text
– Content should be the same
– Text is important, esp. for
international students
– Having different presentations
of content, occasionally will be valuable
•
Try and find out!
Show-me Interface:
•
Should not “grab” user’s cursor
•
Circles should be more regular,
and in brighter color
Interface comments:
•
Provide separate program for
asking configuration questions, or “Options” button
– Sound/no sound
– Read full text or not
•
For long animations/slides
provide
– Progress indicator (at bottom?)
– Thumbnails (in progress indicator)
to jump to points in animation
•
Show-me demonstration
needs fast-forward button?
Expert button:
– Need listing of instructors,
TA’s, other experts
– Need one category for connection
(includes conference (bi-directional audio? default?), email, IM, demonstration,
voice-mail, etc.)
– Allow specification of one-to-one,
allow listeners, can join existing conference, etc.
– May need a more complex interface