Square Kilometre Array prepares for the ultimate big data challenge

22 Sep 2016 | News
The world’s most powerful radio telescope will collect more information each day than the entire internet. Major advances in computing are required to handle this data, but it can be done, says Bernie Fanaroff, strategic advisor for the SKA

The Square Kilometre Array (SKA), the world’s most powerful telescope, will be ready from day one to gather an unprecedented volume of data from the sky, even if the supporting technical infrastructure is yet to be built.

“We’ll be ready – the technology is getting there,” Bernie Fanaroff, strategic advisor for the most expensive and sensitive radio astronomy project in the world, told Science|Business.

Construction of the SKA is due to begin in 2018 and finish sometime in the middle of the next decade. Data acquisition will begin in 2020, requiring a level of processing power and data management know-how that outstretches current capabilities.

Astronomers estimate that the project will generate 35,000-DVDs-worth of data every second. This is equivalent to “the whole world wide web every day,” said Fanaroff.

The project is investing in machine learning and artificial intelligence software tools to enable the data analysis. In advance of construction of the vast telescope - which will consist of some 250,000 radio antennas split between sites in Australia and South Africa - SKA already employs more than 400 engineers and technicians in infrastructure, fibre optics and data collection.

The project is also working with IBM, which recently opened a new R&D centre in Johannesburg, on a new supercomputer. SKA will have two supercomputers to process its data, one based in Cape Town and one in Perth, Australia.

Recently, elements of the software under development were tested on the world’s second fastest supercomputer, the Tianhe-2, located in the National Supercomputer Centre in Guangzhou, China. It is estimated a supercomputer with three times the power of Tianhe-2 will need to be built in the next decade to cope with all the SKA data.

In addition to the analysis, the project requires large off-site data warehouses. These will house storage devices custom-built in South Africa. “There were too many bells and whistles with the stuff commercial providers were offering us. It was far too expensive, so we’ve designed our own servers which are cheaper,” said Fanaroff.  

Fanaroff was formerly director of SKA, retiring at the end of 2015, but remaining as a strategic advisor to the project. He was in Brussels this week to explore how African institutions could gain access to the European Commission’s new Europe-wide science cloud, tentatively scheduled to go live in 2020.

Ten countries are members of the SKA, which has its headquarters at Manchester University’s Jodrell Bank Observatory, home of the world’s third largest fully-steerable radio telescope. The bulk of SKA’s funding has come from South Africa, Australia and the UK.

Currently its legal status is as a British registered company, but Fanaroff says the plan is to create an intergovernmental arrangement similar to CERN. “The project needs a treaty to lock in funding,” he said.

Early success

On SKA’s website is a list of five untold secrets of the cosmos, which the telescope will explore. These include how the very first stars and galaxies formed just after the Big Bang.

However, Fanaroff, believes the Eureka moment will be something nobody could have imagined. “It’ll make its name, like every telescope does, by discovering an unknown, unknown,” he said.

A first taste of the SKA’s potential arrived in July through the MeerKAT telescope, which will form part of the SKA. MeerKAT will eventually consist of 64 dishes, but the power of the 16 already installed has surpassed Fanaroff’s expectations.

The telescope revealed over a thousand previously unknown galaxies. “Two things were remarkable: when we switched it on, people told us it was going to take a long time to work. But it collected very good images from day one. Also, our radio receivers worked four times better than specified,” he said. Some 500 scientists have already booked time on the array.

Researchers with the Breakthrough Listen project, a search for intelligent life funded by Russian billionaire Yuri Milner, would also like a slot, Fanaroff said. Their hunt is exciting and a good example of the sort of bold mission for which SKA will be built. “It’s high-risk, high-reward territory. If you search for aliens and you find nothing, you end your career with no publications. But on the other hand you could be involved in one of the biggest discoveries ever,” said Fanaroff.

Golden age

SKA has helped put South Africa's scientific establishment in the shop window says Fanaroff, referring to the recent Nature Index, which indicates the country’s scientists are publishing record levels of high-quality research, mostly in astronomy. “It’s the start of a golden age,” Fanaroff predicted.

Not that the SKA does not have its critics. With so much public funding going to the telescope, “Some scientists were a little bit bitter at the beginning,” Fanaroff said.  “But that has faded with the global interest from science and industry we’re attracting. The SKA can go on to be a platform for all science in Africa, not just astronomy.”

Never miss an update from Science|Business:   Newsletter sign-up