CSC's heavy computing environment expands with a massive data management system

he CSC Kajaani ecosystem will significantly expand when the European Meteorological Satellite Agency EUMETSAT acquires a massive Data Lake system for the Destination Earth initiative.

The Data Lake is particularly suitable for distributed big data workflows and storage of big data. It will provide researchers with seamless access to data from different sources, such as satellites and instruments, and the data produced by the Digital Twins of the Destination Earth initiative. The concepts applied in the Data Lake service will provide harmonization of data access federation beyond anything that exists today.

The distributed Destination Earth data lake has an edge-computing footprint (data bridge) at CSC Kajaani to allow near data processing of digital twin data generated at the European LUMI supercomputer with a storage capacity of 30 petabytes and a physical size of about 30 rack cabinets, one rack being the size of a tall refrigerator. The data bridge will be installed in Kajaani in the data center of the joint European LUMI supercomputer hosted by CSC.

Destination Earth is an ambitious initiative of the European Commission within the Digital Europe Programme to develop high-precision digital models of the Earth in a broad collaboration. The Digital Twins will allow modelling of climate change, local impacts of climate change, and extreme weather events with unprecedented accuracy and provide data to support decision-making.

– This acquisition is a significant addition to our Kajaani ecosystem and a key part of Destination Earth. The DestinE Data Lake will enable researchers using Digital Twins to access the large amounts of data they need and run smooth computations on the LUMI supercomputer. We want to increase the number of  European research projects and international collaboration in our ecosystem, expand our ecosystem and develop our competence. This project offers excellent opportunities for that, says Jukka-Pekka Partanen, Director of the Kajaani Datacenter Program at CSC.

– Within the overall context of Destination Earth and also with regards to the DestinE Data Lake the EuroHPC JU sites like Lumi play a fundamental role. We consider their expertise and infrastructure contribution as essential and are looking forward to a lasting relationship with us and also vis a vis our industrial prime CloudFerro, says Lothar Wolf, Destination Earth Programme Manager at EUMETSAT.

Three entrusted entities will jointly implement the Destination Earth initiative: the European Centre for Medium-Range Weather Forecasts (ECMWF), the European Space Agency (ESA), and the European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT). 

• ESA has the responsibility for the Core Service Platform that is interfacing with the Destination Earth users
• ECMWF is responsible for two Digital Twins: Extreme Weather Digital Twin and Climate Adaption Digital Twin. CSC leads the international partnership that delivers the Climate Adaption Digital Twin
• EUMETSAT  has end-to-end responsibility for the Destination Earth Data Lake

The DestinE Data Lake service development, implementation and operations will be supplied by CloudFerro, a provider of innovative cloud solutions. The installation at Kajaani centre will start in spring and will start early operations from June 2023.

– The data center sector is one of Kajaani's top industries, and we have worked closely with CSC to develop it. This procurement of EUMETSAT is an impressive international result and expands Kajaani's data center ecosystem in a significant way. The City of Kajaani has invested a lot in collaboration with the CSC and is ready to continue to do so, both with the CSC and its partners, says Teuvo Hatva, Chairman of the Kajaani City Council.

This article was first published on 13 March by CSC.