The structure of bwDataArchive
Tape technology as mass storage
The technical realization of archive long-term storage takes place via mass storage, which consists largely of magnetic tapes - a proven technology with a much lower error rate compared to hard disks. Large data centers rely heavily on this reliable and durable storage technology, which is also undergoing continuous development. In addition, the acquisition and operating costs of tape technologies in the petabyte range are up to ten times less expensive than rotating disks. The disadvantage of longer access times (up to two minutes) with linear media is offset by advantages of longevity and low costs for power and cooling.
Data are securely stored on magnetic tape. We use the following tape drives and technologies: STK 10kC (max. 4 TB per tape) and STK 10kD (max. 8 TB per tape), TS1160 (~20 TB per tape).
HPSS as an archiving solution
An all-encompassing system must be deployed to manage archival storage in combination with appropriate metadata management and provision of functions, for example, to ensure integrity. The SCC chose to install and operate IBM's High Performance Storage System(HPSS). The HPSS allows storage of millions of files into the exabyte range. Disks and tape storage are combined in a virtual file system to create a high-performance storage management system that automatically migrates data from disk and tape.
In Germany, HPSS users include the German Climate Computing Center (DKRZ), the Max Planck Society (MPG), and the German Weather Service (DWD), in addition to HLRS. HPSS is also in use at several large research and HPC facilities in the United States.
Application areas in the SCC
The bwDataArchive project will provide interfaces and archive storage from HPSS for the developments of the RADAR project and for other applications that require a platform for long-term storage. The SCC will continue to establish and expand the tape system to store simulation results from HLRS through LHC measurement data from CERN to Lustre file systems at the SCC.
Archived Data