bwDataArchive - long-term archival storage of research data

bwDataArchive is a service providing long-term data archiving of research data for members of academia and public scientific institutions in Baden-Württemberg, the Helmholtz association and European data infrastructures.
Interior of StorageTek tape library at NERSC (2),  Foto: Wikimedia Commons, Derrick Coetzee

 

The Service bwDataArchive

The service bwDataArchive provides archival storage for research and other public institutions in the state of Baden-Württemberg, the Helmholtz association and European data infrastructures. The service is not an online storage system rather an archive! Data is stored in technical infrastructure of the Karlsruhe Institute of Technology (KIT) and comprises trustworthy large data storage for a period of ten years and more. The service enables a qualified implementation of the recommendation of the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) regarding the safeguarding and storing of research data.

User registration: https://www.rda.kit.edu/bwDA

 

Motivation

Data from scientific experiments, from measurements, analysis and simulations, have to be stored long-term to guarantee accessibility after scientific projects have ended. The storage of this data isn't only important because of legal reasons, it also commits because of its historical value and its possible scientific value in the future. At KIT's Steinbuch Centre for Computing (SCC) the quantity of these data will increase as KIT will become the central archive location of the state of Baden-Württemberg. Within the states project 'bwDataArchiv' (as project name without 'e'), supported by the Baden-Württemberg Ministry of Science, Research, and the Arts (MWK) the SCC expands its technical infrastructure for the long-time scientific data storage of data coming from research institutions, libraries, and also from the high performance computing center of the University of Stuttgart (HLRS).
Within the 'bwDataArchiv' project, technologies for a structured, reliable and secure long-term storage of more than an Exabyte have been developed and deployed. The project has been promoted till 2020 and will be further extended.

The service bwDataArchive addresses among others, these questions:

  • How to simplify the data storage and archiving process for a non IT confident scientific community?
  • Which safety-related aspects are important for the long-term storage?
  • How to efficiently ensure the data integrity of hundreds of Petabytes of data?

Promotion and Cooperation

The 'bwDataArchiv' project was supported by the Ministry of Science, Research and the Arts Baden-Württemberg and it cooperates with DFG project RADAR, the state projects bwDataInMotion (bwDIM) and bwDataDiss, with many different scientific communities, and also with the international projects EUDAT, the Human Brain Project (HBP) and the LHC Computing Grid (WLGC).