Scott Weber
CSE 498

Review of "Manageability, Availability and Performance in Porcupine: A Highly Scalable, Cluster-based Mail Service"

This paper describes the design and development of the Porcupine mail system.  It specifically targets developing a mail service that scales well across a group of commodity PCs and is easier to manage and outperforms existing solutions to handle very large email systems.  While there already exist many systems that help Internet servers scale over a group of computers, most of these handle read-centric servers that can be significantly improved through creative caching of data.  An email system is much more write-intensive than these other types of servers and so requires a different set of optimizations to allow it to scale effectively.

Porcupine provides both mail delivery (SMTP) and mail retrieval (POP/IMAP) functionalities.  The services provided by each node in the system are identical, but the data stored there may not be the same.  A dynamic load balancing system forwards requests to the least busy node with the requested data.  There is much discussion about the load balancing once a request gets to the system, but no mention of how to balance the incoming connections.  In a system that expects most likely tens of thousands and perhaps as much as millions of simultaneous connections, this issue is very important as a single node simply cannot handle that many at once.  Perhaps the authors meant for this type of balancing to be done by some "out of band" means, but it seems too key a part of system scalability to leave it out.

A user's mailbox in Porcupine is divided and replicated across several servers to allow for reliability in the face of failure and performance in the face of node overload.  Thus, when a user accesses his mail, it will cause a flurry of communication between nodes as the headers for all the messages are located and sent back to the single point of external communication.  Depending on how the user has organized his email, it seems that this inter-node communication overhead could become a significant factor in the overall performance of the system.  Perhaps by storing the headers for all messages in a given mailbox in every location for that box, the overhead could be reduced and the overall response time improved.

Automatic detection of and recovery from node failure and node insertion helps the system maintain high performance and reliability.  At the same time, loose requirements on consistency among the nodes helps keeps performance high, but also causes a few problems.  For example, a user could delete a message from his mailbox and see it reappear later if the node it was deleted from crashes after sending the deletion confirmation, but before it has a chance to notify the replicating nodes of the delete.  Under a similar circumstance, multiple deliveries of the same message may occur.  The authors dismiss these problems as inconsequential by claiming they already occur in standard mail systems much more often than they would in Porcupine.  No data or reference is given to back up these statements which I can only assume are the opinion of the authors, misrepresented as fact.  I do not believe that these inconsistency issues can be disregarded withou! t further investigation.

The load balancing algorithm implemented uses statistics about disk activity to determine which node is least busy at any given moment.  The argument given for using only disk activity is that, in the testing done, the disks were always the bottleneck and, as the slowest part of the system, caused the most overhead.  This is another place where the lack of consideration for thousands of simultaneous connections causes problems.  If a host has the lowest disk activity, but is servicing several thousand SMTP and IMAP connections, it may not be appropriate to choose that host as the least busy.  Especially as the system scales and the number of connections grows, this will probably become a bigger issue.

In testing, several simplifying assumptions were made, including the disabling of user authentication, client domain lookup and system log auditing and the reallocation of emptied mailboxes.  These assumptions are not related to a realistic situation to analyze how the modifications might change the validity of the results obtained.  For example, if in reality only five in 100 users empties their entire mailbox on a regular basis, how does this affect the performance gain obtained through reallocation of mailboxes upon empty?  Even if the assumptions do not change the validity of the results, this fact should be stated and backed up with reasons and evidence.

This paper provides a good first look at a system to optimize the scalability of a write-intensive system such as email.  I would suggest that others read this paper as a good place to start, but note that some issues such as consistency requirements and proper load balancing still need to be investigated further.

REFERENCE
Yasushi Saito, Brian Bershad, and Henry Levy. "Manageability, Availability and Performance in Porcupine: A Highly Scalable Internet Mail Service."  ACM Transactions on Computer Systems, August 2000.