One of the problems with the wide adoption of clusters for mainstream high performance computing is the difficulty in building and managing the system. There are many efforts in solving this problem by building fully automated, integrated software distribution from several open source software. However, these sets of software come from many sources and never been designed to work together as a truly integrated system. So, some problem is still remaining unsolved.
With the experiences and tools developed to build many clusters on our site, we decided to build an integrate software tool that is easy to use for cluster user community. This software tool, called SCE (Scalable Computing Environment), consists of a cluster builder tool, complex system management tool (SCMS), scalable real-time monitoring, web base monitoring software (KCAP), parallel Unix command, and batch scheduler (SQMS). This software run on top of our cluster middleware that provides cluster wide process control and many services. MPICH are also included. SCE are truly integrated since our group builds all tool but MPICH. SCE also provides more than 30 APIs to access system resources information, control remote process execution, ensemble management and more. These APIs and the interaction among software components allow user to extends and enhance SCE in many ways. To make things easy, the installation and configuration in SCE are fully automated completely by GUI
As shown in Figure 1.1, SCE consists of 4 main components. First, Beowulf builder is a software tools that create cluster and maintain cluster configuration. User use Beowulf Builder to automatically create all necessary configurations that allows a set of diskless nodes to remotely boot from master node. Once user finishes the installation, a middleware layer called KSIX controls normal operations of a cluster. KSIX always run in background and provides many services to upper layer software tools. There are 2 main software systems running on top of KSIX, that is SCMS cluster management system and SQMS batch scheduling system. SCE also include MPICH, one of the most widely use MPI implementation so user can start programming in parallel under SCE immediately after the installation finish. In the following chapter, each part of the system will be explained in more detail.
The clear advantage of SCE approach in integrating all cluster software tools together are as follows.
