|
|
StarOffice 5.1a Usability StudyIssue 1.1, March 24, 2000
|
|
Scenario 4a (enter and format text, put 'Invitation to Party' in a box) |
Scenario 4b (enter additional text, insert a computer image, create a header) |
Scenario 5 (create a table, insert an image and add a border, create a footer ) |
|
|
Totals |
# who took 16 minutes or more: 0/11 # who took 15 minutes or less:11/11 # who took 10 minutes or less:11/11 |
# who took 16 minutes or more: 1/11 # who took 15 minutes or less:10/ 11 # who took 10 minutes or less:10/ 11 |
# who took 16 minutes or more: 3/10 # who took 15 minutes or less: 7/10 # who took 10 minutes or less: 2/10 # who did not complete: 1 |
Completion Table for Word Processing Tasks
|
Participant |
Scenario 4a (enter and format text, put 'Invitation to Party' in a box) |
Scenario 4b (enter additional text, insert a computer image, create a header) |
Scenario 5 (create a table, insert an image and add a border, create a footer ) |
|
pilot |
4 minutes |
9 minutes |
18 minutes table not finished (not sized) |
|
1
|
14 minutes |
16 minutes HINT: told there is a way to apply the date for header, HINT: told to look in Insert to apply header |
18 minutes HINT: told to look in Insert to apply footer |
|
2
|
7 minutes |
7 minutes |
19 minutes |
|
3 |
6 minutes |
8 minutes |
8 minutes |
|
4
|
5 minutes |
5 minutes |
15 minutes HINT: told she should be able to see the border on the picture |
|
5
|
4 minutes |
9. 5 minutes HINT: told to deselect her picture (made other menu items unavailable) |
10.5 minutes
|
|
6
|
5 minutes |
5 minutes |
15 minutes HINT: told footer field already in document (she did not realize she had already put it in) |
|
7
|
3 minutes |
6 minutes |
9 minutes |
|
8
|
5.5 minutes |
6 minutes |
12 minutes HINT: asked to check what footer was on the doc; she used Page count without realizing it |
|
9
|
7 minutes |
9 minutes |
Did not complete 16 minutes spent on header with 2 hints, footer completed, picture skipped HINT: see it there's something about a table HINT: told how to insert a table cell |
|
10
|
4 minutes |
9 minutes |
13.5 minutes |
|
Participant |
Scenario 4a (enter and format text, put 'Invitation to Party' in a box) |
Scenario 4b (enter additional text, insert a computer image, create a header) |
Scenario 5 (create a table, insert an image and add a border, create a footer ) |
Issues for Word Processing Scenarios
Issues related to the word processing scenarios are described below.
Tables
Participants had some problems resizing table columns to make the table proportional. "Using arrows was not easy, I should be getting grab boxes, only one side of the table changes" (P1, P2, P3, P4, P5, P6, P8, P10).
When trying to delete table cells, some participants selected the cells and then used the Delete key/menu. However, only text in the selected cells (if there was any) was deleted (P1, P6, P7, P9).
When autoformatting was on, participants had some problems knowing how to change it, and had problems overriding the autoformatting for the italicized header, the date format, and the autocaps (Pilot, P2, P1, P5, P7, P8).
Design Recommendations for Tables:
Increase the "hot spot" area (it is currently 1 pixel) where the arrows appear for table resizing. It was often hard to find and hard to select because of its small size.
When the contents of a table cell or cells is selected (a column, multiple rows, etc.) and the user performs a delete action (using a keyboard key or menu item etc.), perhaps the action should delete the cells contents AND the table cells.
For the italicized header for the first row of the table, remove 'Header on' as the default. Most participants did not notice this was set to 'on' and did not use it to change the italics. P1 stated "I want to control it" (she felt that the application should not control it).
Explore options for autoformatting defaults; some people like it on and others like it off. When it is on, user actions should always override it; for example if the user selects the text and types in a specific date and format, the program should not change it.
Using Borders
Participants sometimes chose the color pull down menu for 'shadow style' (in the right area of the dialog box) instead of using the 'color' pull down for 'Line' in the center of the dialog (P1, P4, P10).
Participants did not select the Preset box (left area of dialog) when applying their border (P1, P4). Participant 4 could not figure it out and after a while resorted to drawing a box on top of the picture, and then used an option to make the picture show through.
Participants did not seem to notice the Preview area (left area of the dialog box) when they were creating the border.
Design Recommendation for Borders:
If the user chooses a border element (line, shadow) and makes a selection, the border in the Preset box (second box from the left showing a complete square) should be turned on. The Preview area should be updated to show the 'complete square' selection.
Increase the visibility of the color pull down menu for Lines. Some possibilities are: add some white space, or remove some items from the Style pull down menu above it, and add some items to the color menu.
Inserting Clip Art Images
The inserted picture is "big" (P1, P2, P8).
A more direct method than Insert->Picture-> from File would be better, "since I often use clip art to insert images" (P9, P10).
Design Recommendations for Pictures:
Reduce the size of an inserted clip art image to approximately 2 x 2 inches, or at least something proportional to the current size of the opened document. The image should appear where the cursor is located.
When multiple pictures are inserted, do not completely overlap them; they should all be visible (or at least a portion of each one should be visible so the user can better select any particular one).
Explore the idea of having a menu item for inserting clip art from Gallery more directly (ex. Insert-> Clip Art).
Footers and Headers
Of the six hints that were provided for the word processing scenarios, four dealt with headers and footers (see Completion Table for details).
When inserting a footer (page number), some participants chose Page Count by mistake, likely because the two menu items are close together (P6, P8).
Some participants chose Insert-> Page Number, but had not yet created a footer field (P6, P8).
P9, P6 thought there should be a more direct method for applying the header, and P10 thought the field and header functionality should be combined more, using the More button in the header dialog.
Design Recommendations for Footers and Headers:
Explore some design options to combine headers/footers in one area, so two steps in two separate areas are not needed. (Currently, the method is to insert a field using one menu item/action, then insert a footer using another menu item/action.)
Explore some design options for the Insert-> Field menu to possibly restructure the menu items, such as separating the menu items for page count and page number. Investigate how other applications have structured this.
Using the URL Locator (Load URL field)
Some participants used this area on their own (P2, P5, P7, P8, P9), although P2 used it incorrectly by typing a word into it that she wanted help on.
Two participants (P2, P10) commented that this is a "busy area". P5 (an external, non-StarOffice user), who used the URL Locator quite often, was asked what it was and she stated, "files that I've used today".
In order to collect some preliminary data on this functionality, P8, P9, P10 were asked to complete a simple scenario where they:
opened a new document using any method they wanted
described the URL Locator area
used the URL Locator field to return to the document they had been working on
These participants described the area as:
the path where you find a file
different applications and directories
a history of recent files and path names
For the most part, the concept of the URL Locator was understood and well received as evidenced by the results of this simple scenario.
However, there are some issues with it. When P10 tried to use it, it did not show her files. And in extended use of it amongst members of the HCI group, various issues do occur, such as duplicate file names, or nonexistent file names for recently used files, and different file names appearing in it and in the user's work area for the same file. Such 'extended use' issues warrant a more in-depth review of this functionality.
Presentation scenarios
Scenarios 4-6 dealt with:
creating 2 colored boxes with text in them
creating a footer for all the slides
change an existing slide
create a bar graph by (including entering cell data)
Completion times for Presentation scenarios
For scenarios 4, 5, and 6, the table below describes the completion times and whether the scenario was not completed. Completion times are approximate.
If a hint was given, a brief description of the hint is also provided in the table. Hints were only provided after the participant had worked for at least 3 minutes on the scenario, or sub-task of the scenario.
Completion Table for Presentation Scenarios: Overview
There were a greater number of incompleted scenarios for the presentation portion than the word processing portion of this study. In addition, the number of hints provided was considerably higher.
It should be noted that even among participants who had StarOffice experience, had done a similar task before, and in some cases had taken a StarOffice course, difficulties were still experienced here (especially with the footer scenario).
|
Scenario 4 (create cover slide with 2 boxes, text) |
Scenario 5 (modify an existing slide, add footer) |
Scenario 6 (create bar graph) |
|
|
Totals |
# who took 16 minutes or more: 2/9 # who took 15 minutes or less: 7/9 # who took 10 minutes or less: 6/9 # who did not complete: 2/11 |
# who took 16 minutes or more: 0/6 # who took 15 minutes or less: 6/6 # who took 10 minutes or less: 4/6 # who did not complete: 5/11 |
# who took 16 minutes or more: 1/8 # who took 15 minutes or less: 5/8 # who took 10 minutes or less: 3/8 # who did not complete: 2/8 # who skipped scenario: 3/11 |
Completion Table for Presentation Scenarios
|
Participant |
Scenario 4 (create cover slide with 2 boxes, text) |
Scenario 5 (modify an existing slide, add footer) |
Scenario 6 (create bar graph) |
|
pilot |
8 minutes |
did not complete in 10 minutes: footer was applied to only one page HINT: told to use background mode, HINT: told to use Insert-> Field |
8.5 minutes HINT: told to use chart data, HINT: told to select chart and use chart data |
|
1
|
17 minutes HINT: told twice to create new page (not presentation), HINT: to use Insert->Slide from menu, HINT: to try text icon on left to create boxes |
Did not complete; 13 minutes HINT: told to use background mode, HINT: told to go to a single slide, and not use Background-> Notes, HINT: told to use Background-> Drawing, HINT: told she's in the background mode where she wants to be |
Skipped |
|
2
|
10 minutes |
14 minutes HINT: told to use background mode, |
Did not complete; 14 minutes (title and Y axis not done) * close to completion, but she had to leave |
|
3
|
8 minutes |
5 minutes |
Did not complete; 15 minutes . HINT: told about Chart Data, HINT: told to use View-> Chart Data in menu |
|
4
|
12 minutes |
15 minutes HINT: asked if footer on all slides, HINT: told to use background mode, HINT: told to use Background-> Drawing in menu |
10 minutes (Y axis not completed)
|
|
5
|
Did not complete HINT: told to create a new slide, HINT for creating boxes |
Did not complete skipped footer |
Skipped |
|
6
|
7 minutes |
6 minutes |
8 minutes HINT: told how to access Y axis/Scale area |
|
7
|
7.5 minutes |
7.5 minutes HINT: told to put page number on all slides (was on one slide only) |
11 minutes |
|
8
|
7.5 minutes |
7 minutes |
16 minutes |
|
9
|
17 minutes HINT: provided on how to curve edges for boxes |
was asked to do footer only; Did not complete, 11 minutes HINT: told to use background mode, HINT: told section of manual being read applied to word processing, and not presentation |
15 minutes *completed except for Y axis |
|
10
|
Did not complete HINT: about how to make/change boxes |
Did not complete (did footer only, due to time. She did not remember how to complete) |
Skipped |
|
Participant |
Scenario 4 (create cover slide with 2 boxes, text) |
Scenario 5 (modify an existing slide, add footer) |
Scenario 6 (create bar graph) |
Issues for Presentation Scenarios
Creating a chart
Of the five hints that were provided for the scenario about creating a bar chart, four dealt with telling participants to "use Chart Data" (See Completion Table for details).
Some participants had difficulty finding the Chart Data dialog box, or knowing that 'chart data' was an area they needed to use. P3 tried a right mouse click, but did not find Chart Data there. The Pilot and P3 tried Chart Type and AutoFormat. The Pilot also tried Format.
When Chart Data was found, participants were surprised it was in the View Menu (P3, Pilot).
When participants selected the 'chart' slide to create a chart, double clicking on the area (as described in the text to 'double click to create chart' did not always work (Pilot, P2, P9). P9 stated "It took 10 double clicks to open the chart."
Some participants (P2, P8) were surprised about the default chart that automatically appeared. "I didn't ask for this or want this chart, Where did it come from?"
Design Recommendations for creating and changing charts:
Automatically open the 'Chart Data' box when the user double clicks on the chart slide to create a chart, or when a chart is imported into a slide. Whenever the user opens any chart (or does some action to indicate they are creating a new chart or working on an existing one) always provide the Chart Data box.
Put Chart Data in the Edit menu, instead of the View menu. Some participants noted that 'View' did not imply that data could be changed. Consider using a different name than Chart Data, since some participants did not think the name was appropriate. See what other spreadsheet applications use for this term.
Investigate why double clicking to add a chart when inserting a new slide (when the 'Double click to add a chart' message appears) does not always work. It may be possible that the "hot spot" for the clicking area is too small.
Explore some design options about additional aid that could be provided when the user is creating a new chart. People did not always understand where the default chart came from since it was not data they had entered. Some additional text could be added stating that it is an example of a chart that can be changed. Or a brief explanation that Chart Data can be used for making changes could also be helpful. Other applications should be reviewed in more detail to see how they deal with this functionality.
Entering cell data and applying cell data to charts
Entering cell data and applying cell data to a chart was somewhat problematic.
When entering chart data, some participants stated they wished there was an easier or more standard way to enter the info (P6, P9) than having to type the data into the input field at the top of the chart data area.
The functionality and terminology of the 'Transfer Data' icon was not recognized (Pilot, P1, P4, P8, P9). Some participants (P1, P9) tried the Apply Data icon (even when it was grayed out) instead of Transfer Data.
Two participants closed the chart data dialog box (using the small Close box in the upper right corner of the box) instead of using the Transfer Data icon. Their changes were lost (P4, P9).
Design Recommendations for applying cell data:
For the default chart, consider using a chart with fewer data points, so the user has fewer required changes to make. Since the data is not their own, a simpler default chart would be perceived as easier to manage and change.
Look at other spreadsheet applications to see what they call the 'Transfer Data' function. One participant suggested using a more detailed mouse over description.
Look at other spreadsheet applications to see if standard methods for entering cell data can be incorporated.
If the user tries to close the Chart Data box before transferring their entered data, provide a confirmation dialog telling the user they have unsaved work, and asking the user if they want to save their changes.
Whenever the user changes their input focus in the chart data dialog (for example from one cell to another), update the chart to show the changes. The user should not have to continually 'apply' their changes with the Transfer Data icon.
Y Axis changes
For the three participants who added a Y axis, there were some issues in the 'Scale' Dialog box, where the 'Automatic' checkbox was turned on (the default) for each item (P6, P9).
The relationship between the 'automatic' checkbox and entering data in its corresponding field was not obvious.
The layout of the fields in the dialog box (intervals for axis) and how they related to the chart took some time to understand.
P9 got an error message that described 'origin', but she did not see the term origin anywhere in the scale dialog where she had been working; "Where is origin"?
Design Recommendations for Y Axis:
For the 'Scale' dialog, consider having only one 'automatic' checkbox instead of one for each interval item. This would reduce the choices and number of key strokes required.
When the user enters a value into an interval field, automatically deselect the 'automatic' checkbox.
Order the interval fields in the dialog box so they match the order of the Y axis on the chart that is being shown, such that minimum and maximum are the first and last fields, respectively (for example).
Ensure terminology used in error and other messages is consistent with the dialog/work area that the user is working in.
Apply any design changes (as appropriate) to both the X and Y axes, to maintain consistency.
Footer and background mode
There were numerous issues concerning footer creation. More than any other scenario, participants tried online Help and the book that was supplied. Even participants who had performed this functionality before had problems.
ALL of the 13 hints that were provided for this scenario dealt with how to create a footer.
Participants thought a footer item would appear in the Insert menu (P1, 4, 8, 9).
When Footer was not found in Insert, various other menu selections were tried: Insert Page Setup (P8), Insert Field (P9), Format ->Page-> Background (P4).
Some consistent mistakes were made: the footer was applied to only one page (P2, P4, P9, Pilot), Page 2 was placed on each slide (P6, P7).
Participants commented that it was "different than using Writer and Calc" (Pilot, P4, P8) "It would have been nice if header (from word processor) had been in the presentation program "(P1).
Participants tried View-> Maste View, or looked for some variation, such as Master sheet or Master Slide or Master Page (Pilot, P4, P9, P10).
Participants chose Background ->Notes (P2, P9) before they tried 'Drawing'. "Drawing refers to an image, Drawing doesn't make sense."
The 'background' concept was not obvious to most users: "Background isn't obvious" (P2), "Why don't they call it footer?" (Pilot) "They don't make it easy do they? "(P1). "I've exhausted my options, I'd have to ask someone" (P4).
Participant 4 commented that when she was using the background mode, the Preview box (View menu->Preview can also show this box) did not match what her slide was showing. She noted that this was inconsistent with what the preview box usually showed (when she was not in the background mode).
Design Recommendation for Footer and Background Mode:
Rethink the entire background mode operation. Explore a "master slide" function that participants suggested, as well as how other presentation applications support this functionality. Make sure the similar terminology of 'Master View' (used currently in StarOffice) and 'Master Slide' are understood conceptually by users. One or both areas may need additional thought.
Apply the model of 'Insert Footer' that StarWriter uses to StarImpress. Make header and footer functionality consistent across all StarOffice applications.
Ensure the same or similar terminology is not used across menus to describe different things; for example, Format->Page->Background, and Background mode (footer usage) refer to different things.
Core StarOffice 5.1a Issues
Some of the functionality used by the parcitipants in the study is common across StarOffice. Since this study focused on StarWriter and StarImpress, and did not address other areas StarOffice 5.1a functionality, it was recognized that not all issues would be studied. Therefore, only some preliminary design recommendations are made for these core areas, with the intent that additional discussions can occur in the future.
Read only mode
Two participants (P5, P9) got into viewing 'read only' documents. They did not know how they had gotten there, nor did they recognize that they were in a read only state (and why they could not therefore make changes).Footer/background mode
As discussed earlier in this report, there were numerous issues around creating a footer in StarImpress. Additional design work needs to be done in this area.URL Locator (Load URL field)
As discussed earlier in this report, extended use of this functionality has shown some issues. Additional design work needs to be done in this area.Layout/appearance
There were some comments that the fonts on the screen appeared bold (Pilot, P2, P4, P6, P8, 9), and that fonts and pictures looked different when they were printed (P6).Two participants noted that a footer takes up too much space on the page (P6, P8), and one described how they moved things around during their normal use of StarOffice so it would print out properly (although the online document then looked funny).
It was noted that non-U.S. standard margin defaults were used (Pilot, P2, P4, P9, P10), and that paper sizes have to be changed when using StarOffice 5.1a.
Online help
Most participants used and liked the mouse over tool tips.Most participants did not like the pop-up help that automatically appeared when they were working (Pilot, P1, 2, 4, 8). P7 noticed a pop-up and then commented "I don't know what why that appeared." Participants usually closed the pop-up information, sometimes with a corresponding negative comment.
Most of the time, participants commented that the online help was not helpful or "user friendly" (Pilot, P1, P2, P3, P5, 9, 10), or that it was misleading, "I can't find the icon it mentions" (Pilot, P3, P5). Some mentioned that they have learned not to use it in their previous work (P6, P10).
Two participants commented that there was not a search area at the top level of help (P5, P9). The pilot typed 'footer' in the URL locator when looking for help.
Selection areas
In addition to the table sizing comments, there were some instances where selecting items was tricky. It was sometimes hard for participants to perform the desired action; for example, selecting the text box to move or stretch it, or selecting text within the text box to edit (Pilot, P1, P2, P3, P5, P9).Sometimes, participants seemed to have trouble noticing the small right arrow glyph beside the box icon (to select different box shapes).
Anchors
It was unclear whether participants understood the model that was used when pictures were inserted and anchored (P1, P6, P9, P10). Participant 10, an experienced user, right-clicked on an anchor hoping it would provide some help in completing the scenario she was working on.
Design Recommendation for Core Issues:
When a read-only document is opened, present a confirmation dialog to the user explaining what is happening.
Consider placing Online help pop-ups in the lower right area of the screen. At a minimum, they should not appear in the area the user is working in. In the study, a pop-up often obscured the ongoing work.
Reassess the entire online help strategy, with additional feedback from users and other designers.
Investigate the pull-right arrow glyph used beside some icons to see why some people had trouble noticing it.
Further investigate the functionality for some 'customized' features. When changing the autoformatting, autocaps etc. feature, we did not know the difference between the [M] and [T] controls in the Tools->AutoCorrect/Autoformat menu dialog. When we asked P10 what these were for (an engineering manager), she replied, "They do the same thing".
Comparison of StarOffice 5.1a to applications participants usually use
At the end of all the presentation scenarios, as well as at the end of the word processing scenarios, participants were asked to compare StarOffice 5.1a to the applications they normally use.
The question was: "Compared to what I normally use, StarOffice was easier to use, the same, or harder to use".
For the word processing scenarios, of the 7 participants who responded,
2 said: it was easier to use
3 said: it was harder to use
2
said: it was about the same
For the presentation scenarios, of the 5 participants who responded,
1 said: it was easier (since it did not crash during the session)
4 said: it was harder
Note: Some participants did not provide a rating. For example, if the study participant had not completed enough of the tasks in the scenarios to answer the question, or the question was not applicable to them (if what they normally used was StarOffice).
Positive characteristics of StarOffice 5.1a
Some specific changes, as noted in the report, should result in improved usability. Participants were generally able to complete some or all aspects of the scenarios with which they were tasked. Particularly, changing sizes on boxes proved simple for the participants. Adding and deleting chart and table rows were also generally accomplished without problems.
We also observed participants using multiple methods to complete their actions. For example, participants reordered slides using both the Tabs on the bottom as well as the slide sort method. We also saw the majority of the participants use the Open file method when adding a picture, while some participants used the Gallery (P4, 6, 7). The multiple methods available to complete a task did not appear to pose a problem to the study participants. In fact, it is likely that this aided a number of users.
Related to the use of multiple methods, experienced users made use of keyboard shortcuts, and expected the same shortcuts to be functional across applications as well as across platforms. For example, Control + End and Control + Return were used a number of times, to create a new page or go to the end of a page. Users are accustomed to these methods from other applications and enjoyed applying these skills to their present situation.
