CSE 450 The Review of ¡°Bringing Order to the Web: Automatically Categorizing Search Results¡± by Liangjie Hong This paper represents a novel user interface that organizes web search results into hierarchical categories. The authors use the Support Vector Machine algorithm, a fast and effective algorithm for text classification problems, to automatically classify arbitrary search results into existing categories. The strength of the work is that it shows that the classification of search results and the categorized user interface can have significant impact on the search time of users. Differing from previous works, the authors mainly emphasize the user interface aspect not the classification techniques. Their final result of the user study also shows that the categorized user interface, which uses hover-text and overlay techniques, assists users to quickly locate and focus on task-relevant information rather than list user interface. The authors do a good job to give the comparison between their designs of interface and previous works especially Custom Folders by Northern Light and tree structures by Johnson. Besides, in order to contain more information on the first screen, they also provide a distilled information display, a carefully designed interface, to guide users to effectively locate their path or further exploration. However, this paper suffers several drawbacks, which would lessen the result of their work. Firstly, for their underlying classifier, they only used 13,352 web pages to train the model, which is too small for training a real-world web classifier, compared to the huge web content. Additionally, 70% accuracy achieved by their classifier is relatively low in my opinion while they also didn¡¯t provide the comparison between the SVM algorithm and other classification algorithms on the same training data, which would make their underlying classifier unreliable. Secondly, in their user study, they only chose eighteen subjects with intermediate web ability, which cannot represent all types of users. The number of subjects is too small and so-called ¡®intermediate ability¡¯ is difficult to determine. In fact, from their user study, some people can finish all search tasks in a median of 37 seconds while some of them can complete all tasks in 142 seconds, showing that not all users are in the same technology skill level. Furthermore, those subjects are all adult. How about children and senior people? The authors cannot ignore these types of users. In addition, subjects are all residents in Seattle area, which is a high-tech area. How about people in mid-west states or in other countries? Thirdly, their experiment only consists of 30 search tasks. The number of tasks is too small to make a conclusion for a new user interface. Besides, these tasks are carefully designed with reasonably unambiguous answers. How about queries with ambiguous answers? If the classifier cannot categorize the result correctly, the total search time of the new interface could become longer. And at this point, the user interface would no longer be the key element to influence the search time but the correctness of the classifier would be the key actor. However, authors didn¡¯t discuss this situation. Additionally, in the questionnaire section, the authors mentioned that the most popular web search service among their subjects was Yahoo. We know that Yahoo is a categorized search engine. If subjects showed bias on categorized user interface before the experiment, it is more likely for them to prefer categorized user interface in the experiment, which makes the experiment useless. At last but not the least, they showed that their new interface shorten the search time of users. However, they didn¡¯t provide the experiment to show that whether this improvement is mostly achieved by the techniques of classification or the new interaction-style interface like hovering, page views and ¡®ShowMore¡¯ button. Since the new interface is the key point of their work, they didn¡¯t convince us that their new design can really boost the search time without comparing to other search services which also provide some sorts of category interface like Yahoo. In conclusion, this paper gives an interesting new user interface using classification techniques as the foundation. To make the paper more convincible, they should provide a more careful comparison between the classifier they used and other classification algorithms. In addition, their user study needs to be revised to include much more subjects ranging from children to senior people with different background in various geographical areas. Besides, they should provide more search tasks with unambiguous answers and also ambiguous answers. And they also should compare the search time by using their interface and by using other categorized search engines.