[379] in Athena User Interface
Fwd: Re: Beta, system, and usability testing, p.s.
daemon@ATHENA.MIT.EDU (Janet Littell)
Wed Aug 23 11:02:41 2000
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Message-Id: <v04020a0bb5c73acbed57@[18.152.0.213]>
Date: Wed, 23 Aug 2000 11:02:38 -0400
To: beland@mit.edu, aui@mit.edu
From: Janet Littell <jlittell@MIT.EDU>
Cc: kcahill@mit.edu, sbjones@mit.edu (Susan Jones)
B,
Sorry that this response to your message (of 8/14, 6:22PM) was delayed
until I returned from the Usability Professionals' Association UPA 2000
conference.
>>With regard to further testing, I'm not sure I understand exactly what
>>you mean, Janet.
I only mentioned usability testing during the non-public and public beta
phases because they seem to be the stages when e-mail and file management
are more likely to be included. Susan, Kathy, and I feel that it would
really be nice to have those two pieces before we test AUI widely.
Are the Nautilus file manager and Evolution e-mail expected in Nov. or at
the end of IAP, or are these dates not clear yet? Who are the third-party
developers?
If you want us to test in September, perhaps the tasks could be limited to
navigating the AUI menus to find, open, and close various applications,
using only those parts of an application that are related to gnome.
If you include usability-oriented questions for system testers, the
responses should be taken with a grain of salt. How people use a piece of
software to do work and how they _say_ they use it are often very
different, so it might be unwise to base any design modifications on these
responses. Usability testing really needs to have someone observe while a
tester performs assigned tasks, either on a real system, or on paper screen
prototypes.
>>So it seems logical to come back and do more formal testing during
>>IAP, no?
Susan, Kathy, and I -- backed up by the usability group -- can try to set
up tests for at least 1-2 users at any major milestone, provided that the
AUI team is willing and able to consider incorporating the test results, as
appropriate, into the next build/release/version.
As Susan suggested, cardsorting -- putting all the menu options on cards
and have a few users group and label the card groups -- could be useful at
the current prototype stage, in the fine tuning of menus.
Susan and I tried gnome again yesterday and had good experiences, and Kathy
is going to try today. Because of the rollover instructions, I did not get
stuck even once! I like the fact that the submenus repeat the title of the
parent menu, Utilities, Applications, etc. Susan used gmc successfully to
clean up her directories.
I notice that the Applications menu currently includes several editors. It
might be nice to have a submenu called "Editors" or "Word Processing".
Susan, Kathy, and I will stand by. Let us know how you'd like to proceed.
Janet
Beland's original message:
>>To: aui@MIT.EDU, jlittell@MIT.EDU, kcahill@MIT.EDU, sbjones@MIT.EDU
>>Subject: Re: Beta, system, and usability testing, p.s.
>>Date: Tue, 15 Aug 2000 06:22:20 -0400
>>From: "Christopher D. Beland" <beland@MIT.EDU>
>>
>>Sorry, yes, WebMoira is not a mail reader, it's an interface to
>>mailing lists. Other such interfaces include the listmaint,
>>mailmaint, moira, and blanche commands. 8)
>>
>>With regard to further testing, I'm not sure I understand exactly what
>>you mean, Janet.
>>I interpret the usability testing schedule to
>>consist of the following phases:
>>
>>1. Formal usability testing in August or September. This involves
>>running through a fixed version of the procedures we discussed at the
>>meeting with Susan, and perhaps a few other "insiders", and then
>>running the tests on a number of students and perhaps other
>>volunteers.
>>
>>2. Non-public beta testing. This is where we tell a select group of
>>mostly experienced Unix users (say, maybe SIPB, the OLC consultants,
>>and the rest of I/S) to use the system for daily work. We warn them
>>they may run into glitches, and that we may add (hopefully not break)
>>functionality as time goes on. We collect feedback about both system
>>and usability problems (basically, whatever they want to tell us).
>>
>>3. Public beta testing. Let any Athena user who wants to participate
>>use the system, and collect whatever feedback they will give us.
>>
>>Following this plan, there's no real way to separate the informal
>>parts into system vs. usability testing, since we're just giving it to
>>people to use and give feedback. We could give them directed
>>questions to answer, but I think we'd be interested in open-ended
>>feedback about both aspects.
>>
>>You're right; doing *formal* usability testing on a stable system is
>>most useful. Right now, our system is stable as long as subjects
>>don't stray too far from the list of tasks we give them. Having
>>people use the system on a regular basis (and giving developers time
>>to work) during the term should iron out the most frequently
>>encountered bugs.
>>
>>So it seems logical to come back and do more formal testing during
>>IAP, no? I'd certainly like to be able to go back and test some of
>>the things that won't be ready until then. And we might be able to
>>incorporate improvements into the Spring term opt-in public beta, if
>>we did testing at the beginning. (On the other hand, if we did it
>>near the end, we might be able to squeeze in Naulilus, the graphical
>>file manager, and Evolution, the mail reader, assuming these are
>>delivered early enough by the third-party developers.)
>>
>>I think there will be plenty of time, and plenty of beta-testing to
>>stabilize changes betwixt IAP and summer. (Hmm, famous last words.)
>>
>>-B.
