Scale and Performance in a Distributed File System

Scale and Performance in a Distributed File System

1987 | John H. Howard, Michael L. Kazar, Sherri G. Menees, David A. Nichols, M. Satyanarayanan, Robert N. Sidebotham, Michael J. West
The Andrew file system is a distributed file system developed by Carnegie Mellon University and IBM. Its primary goal is to support up to 7000 workstations while providing users with the amenities of a shared file system. A key design decision is to transfer whole files between servers and workstations rather than smaller units like records or blocks, which is different from most other distributed file systems. This paper examines the consequences of this and other design decisions that affect the scalability of Andrew. The prototype of the Andrew file system was intended to validate the basic architecture and obtain design feedback. It had about 400 users sharing 100 workstations and was implemented with six servers. The system allows most applications to use files in Vice without being recompiled or relinked, demonstrating that it is possible to emulate 4.2BSD file system semantics using caching and whole file transfer. However, performance was usually acceptable up to about 20 active users per server. Performance improvements were made, including reducing the frequency of "stat" operations and introducing a callback mechanism to notify Venus when file status information changes. These changes significantly reduced Venus-Vice traffic and the resultant network and CPU loads. Additionally, symbolic pathname resolution was moved to workstations, and a Fid (File Identifier) was introduced to facilitate communication between Venus and Vice. The Andrew file system was compared with NFS, a block-oriented file system. The results showed that Andrew is far less load sensitive than NFS. The Andrew system handles 50 users with capacity to spare, while NFS is more heavily loaded. The volume mechanism allows for file migration, space quotas, replication of seldom-changing system files, and on-the-fly backup. Volumes are glued together at Mount Points to form the complete name space. The Andrew file system is designed to support a large number of workstations and users, with the current design allowing nearly doubling the number of workstations. Future improvements include moving Venus and server code into the kernel, converting the kernel intercept mechanism to an industry standard, and implementing replication of writable files. Monitoring, fault isolation, and diagnostic tools will also become increasingly important. Decentralized administration and physical dispersal of servers will be necessary in a very large system.The Andrew file system is a distributed file system developed by Carnegie Mellon University and IBM. Its primary goal is to support up to 7000 workstations while providing users with the amenities of a shared file system. A key design decision is to transfer whole files between servers and workstations rather than smaller units like records or blocks, which is different from most other distributed file systems. This paper examines the consequences of this and other design decisions that affect the scalability of Andrew. The prototype of the Andrew file system was intended to validate the basic architecture and obtain design feedback. It had about 400 users sharing 100 workstations and was implemented with six servers. The system allows most applications to use files in Vice without being recompiled or relinked, demonstrating that it is possible to emulate 4.2BSD file system semantics using caching and whole file transfer. However, performance was usually acceptable up to about 20 active users per server. Performance improvements were made, including reducing the frequency of "stat" operations and introducing a callback mechanism to notify Venus when file status information changes. These changes significantly reduced Venus-Vice traffic and the resultant network and CPU loads. Additionally, symbolic pathname resolution was moved to workstations, and a Fid (File Identifier) was introduced to facilitate communication between Venus and Vice. The Andrew file system was compared with NFS, a block-oriented file system. The results showed that Andrew is far less load sensitive than NFS. The Andrew system handles 50 users with capacity to spare, while NFS is more heavily loaded. The volume mechanism allows for file migration, space quotas, replication of seldom-changing system files, and on-the-fly backup. Volumes are glued together at Mount Points to form the complete name space. The Andrew file system is designed to support a large number of workstations and users, with the current design allowing nearly doubling the number of workstations. Future improvements include moving Venus and server code into the kernel, converting the kernel intercept mechanism to an industry standard, and implementing replication of writable files. Monitoring, fault isolation, and diagnostic tools will also become increasingly important. Decentralized administration and physical dispersal of servers will be necessary in a very large system.
Reach us at info@study.space