TU Berlin: Revolutionising data centres

The incessant upswing in the use of artificial intelligence and data-heavy applications in our day-to-day lives is increasing the demand on modern data centers with the volume of communications they process increasing dramatically. In order to address these challenges, a team of researchers at Technische Universität Berlin is developing a new type of network technology that can adapt dynamically to real-time data traffic. The work is part of the "Self-Adjusting Networks" project funded by the European Research Council (ERC) and will significantly increase the computing capacity of the systems and save on resources. Initial findings and an overview of the research field were recently published in the Communications of the ACM journal.

Communication networks that adapt to data streams

To date, traditional networks in data centers have been static. Server racks – groups of computers with processors such as graphics processors that are particularly well suited for intensive computing – are connected to each other via hard-wired paths, regardless of how much data is being exchanged in that moment. Professor Dr. Stefan Schmid, head of the Chair of Internet Architecture and Management and his team, including Professor Chen Avin of the Ben-Gurion University in Israel, are taking a more dynamic approach in which network connections can be flexibly reconfigured and adapted to the data traffic structure in real time. The concept is similar to the Golden Gate zipper, a special machine that moves the median barrier on the Golden Gate Bridge in San Francisco to create more lanes in the direction more traffic is heading to. In data centers, these new networks shorten connection paths, for example by creating direct connections between racks with high levels of communication. This reduces the amount of wasted bandwidth, allows more data to be transferred, and increases the overall performance of data center networks. At the same time, the networks also have potential when it comes to energy efficiency and robustness.

Relevance through AI applications and new technologies

The new approach is highly relevant given that machine learning and artificial intelligence are generating enormous volumes of data in data centers. These data flows also follow certain patterns – structures that can be technically leveraged.

The new, dynamic network connections use a modern technology called optical switches. These switches can change connections in the network extremely quickly – in just a few millionths of a second. Unlike traditional devices that rely on electricity, optical switches use light transmitted through fiber-optic cables. This can then be controlled, for example, using different light colors or programmable digital mirrors that direct the signals precisely in the network. This allows the network to react flexibly and quickly to new demands. This technology also saves energy since the light does not need to be converted into electrical signals first.

Basic research with a practical application

Several years ago, the researchers at TU Berlin began their basic research in the Self-Adjusting Networks project, where they developed mathematical models and methods to better understand and increase network capacities. A key focus is on an information-theoretical approach. The researchers are investigating how patterns in data traffic can be identified and leveraged. As with language, certain communication pairs occur more frequently than others, just as the letter E appears much more often in texts than a Q does. Such recurring patterns can be compressed or bundled particularly well. And that is exactly what the new networks do: They identify recurring or easily predictable data streams and optimize them, making the network faster, more efficient, and saving on resources. "Our research shows which architectures and control mechanisms are needed to adapt data centers of the next generation of Google and other tech giants to the flow of data traffic," says Stefan Schmid.

This article was first published on 12 June by TU Berlin.