SystemImager and BitTorrent: a peer-to-peer approach to large scale OS deployment
von Erich Focht (NEC HPC Europe), Andrea Righi (CINECA), Brian Finley (Argonne National Laboratory), Bernard Li (Pharma startup in Bay Area)
Donnerstag, 31.05.2007, Saal 4: Kaiserslautern , 11:00-12:00 Uhr
This paper describes the integration of BitTorrent as a transport method to deploy OS images to a large number of clients using SystemImager (www.systemimager.org). Traditionally, the classical client-server model does not scale well, thus large deployments, such as big clusters or complex grid-computing environments, need to be staged. An alternative scalable and reliable method for image distribution needs to be implemented and BitTorrent seems to be a natural fit. Measurements with big clusters of hundreds of nodes show that the SystemImager BitTorrent deployment scales as expected and dramatically exceeds the performance of the existing multicast and rsync deployment methods. Experimental results of deployment time and bandwidth analysis on a large installation are documented in the paper to confirm the advantages of the peer-to-peer approach with respect to the other transports. The speed of the deployment and the support for heterogenous harware provides the means to deploy virtual clusters which are built on the same resource pool, where it is possible to dynamically re-initialize workspaces on user demand.
Über den Autor Erich Focht:
– studied physics at the RWTH Aachen
– PhD work in theoretical Physics (computational methods) at RWTH Aachen and von Neumann Institute for Computing (NIC) Juelich
– started to work for NEC as Application Analyst in 1996, tuning of numerical algorithms and CAE simulation codes for parallel vector supercomputers
– worked as Linux kernel developer for IA64 and NUMA systems, involved in ATLAS64 and LSE projects aiming at scalability of Linux on large systems
– currently working as solution architect, doing research and development for cluster and grid software
– open source projects involvement: OSCAR (currently chair), SystemImager, SystemInstaller, SystemConfigurator, Ganglia, Kerrighed SSI
Über den Autor Andrea Righi:
Andrea Righi was born in Colle di Val d’Elsa a wonderful town in the hearth of Tuscany (Italy) on 19th March 1978. He received the Laurea degree in Computer Engineering at University of Siena on 2004. Currently he is a system administrator in CINECA (www.cineca.it/en/index.htm), the most important supercomputing centre in Italy. He is also involved in the deployment and maintenance of the DEISA Supercomputing Grid Infrastructure (www.deisa.org), the European grid. Andrea is one of the active core developers of SystemImager (wiki.systemimager.org), a popular tool for installing and updating Linux clusters. He is an activist of OSS and a Linux user since 1999.
Über den Autor Brian Finley:
Brian Elliott Finley has specialized in operating systems, system architecture, and system integration since 1990. He holds a number of technical certifications, writes articles for industry publications, and is the creator of SystemImager, the popular Linux installation and update software. In 1993 he fell in love with Linux and has been active in the open source community ever since. Mr. Finley is the Linux Strategist for Argonne National Laboratory, where he is focused on effective use of open source technologies for general IT infrastructure, as well as specific scientific applications. Mr. Finley lives in Naperville, IL, US with his wife, four children, and two large dogs.
Über den Autor Bernard Li:
Bernard has been actively involved with HPC open source software since 2003. He was the chair of the OSCAR project from 2005-2006 and is still an active developer. He is also involved with the SystemImager and Ganglia projects.
He has authored various papers related to OSCAR and HPC in general. His article titled "OSCAR Cluster and Bioinformatics" was published in the November 2004 issue of Linux Journal.
He is interested in all things HPC, from cluster deployment, to cluster file distribution, scheduling systems and ad-hoc grid applications such as Apple’s Xgrid.