Company

Danish Center for Scientific Computing Analyzes CERN Data with Force10 Networks

As CERN turns its Large Hadron Collider (LHC) on in October to study the results of colliding particles, the Danish Center for Scientific Computing at the University of Copenhagen (DCSC/KU) will be in the middle of the action, analyzing the resulting data to further the study of the universe and the particles that make it up.  To make sure they don’t miss a thing, the DCSC deployed Force10 Networks®.

The DCSC at the University of Copenhagen deployed the Force10 TeraScale E-Series® family of switch/routers and the S-Series family of access switches to support its newest and most powerful supercomputer that processes and analyzes scientific data resulting from the collisions of millions of particles at CERN’s LHC.

Selected for its reliability and performance, the Force10 equipment supports more than 400 compute nodes and nearly 400 Terabytes of raw storage. The new supercomputer is part of the Nordic DataGrid Facility (NDGF) Tier 1 termination point for the LHC at CERN, one of only 10 such points worldwide. As part of the NDGF, the University of Copenhagen’s computing cluster collects, stores and processes massive amounts of data produced by the LHC and shares that data with Tier 2 points around the world. 

While the high energy physics research group analyzes data from CERN, the university’s chemistry department, which houses the supercomputer, will also utilize the computing capacity to advance state-of-the-art research. Additionally, researchers at the Niels Bohr Institute at the University of Copenhagen will study astrophysics using the power of the newest supercomputer. 

Proven Track Record

Over three years, the growth of the Force10 deployment mirrored the expansion of DCSC/KU’s computing clusters. Initially, the center deployed several S50s. During that time, they experienced no hardware or software issues with the S50s, resulting in 100 percent network uptime. Nearly three years later, as they were building the newest supercomputer, which would become part of the NDGF, network administrators knew they needed a robust core to support the large amounts of data expected from the LHC as well as the research from the chemistry and astrophysics departments. 

“Based on our history with Force10, we knew that their switches don’t fail in high pressure environments, so when we began planning for the new supercomputer, we requested that all bids include Force10 at the core,” said Daniel Kalici, system manager at the DCSC/KU. “Force10 continues to prove that we made the correct choice.”
 

Reliability = More Research

As network availability relates to studying the collisions of microscopic particles, the formula is simple: Less network downtime = More research.  For network administrators, more research started when the Force10 TeraScale E300 was put into production within two hours of receiving it, and it has been running ever since.

The Force10 TeraScale E300 – with its 288 Gigabit Ethernet ports – is interconnected to seven S50 switches to form the foundation of the university’s newest computing cluster. Leveraging Force10’s VirtualScale technology, the DCSC/KU can manage multiple S50s as a single switch and extend reliability from the computing edge to the core, reducing the costs of managing the infrastructure and further ensuring network uptime. 

“The VirtualScale technology really allowed us to simplify how we build out our infrastructure,” said Kalici. “It gives us both the scalability to expand as we need to and provides another layer of reliability that extends throughout the infrastructure.”

In the core, the support for up to 48 Ten Gigabit Ethernet ports on the TeraScale E300 enables the center to implement a high performance core that can scale to accommodate an increase in data from the LHC or demand from the astrophysics and chemistry departments. Moreover, the low latency, non-blocking core created by the Force10 solution enables the astrophysics department to more efficiently leverage the computing power to conduct advanced research with complex codes that previously required Infiniband.

“The low-latency and non-blocking bandwidth enables some of our astrophysics and chemistry researchers to conduct more experiments than we could previously while leveraging our existing investment,” says Professor John Renner Hansen.

Scaling for Growth

This application readiness and scalability gave DCSC/KU administrators considerable confidence in their ability to increase computing capacity as needed. 

“With the E300, we are able to scale our cluster to a much larger capacity, thereby providing the researchers with the possibility to expand the scope of their research accordingly,” says Hansen.

Conducting this research are astrophysicists at the Niels Bohr Institute at the University of Copenhagen who will utilize the supercomputer to advance research into the visible universe, such as planets, stars and galaxies, as well as the invisible universe and the presence of dark energy and matter. Within the chemistry department, researchers are investigating and understanding complex chemical systems and molecular dynamics.

The high energy physics research group is analyzing data collected by two CERN experiments on the LHC, A Large Ion Collider Experiment (ALICE) and A Toroidal LHC Apparatus (ATLAS). The ALICE experiment will collide lead ions to recreate conditions just after the Big Bang and will enable physicists to study a state of matter known as quark. ATLAS is a particle physics experiment that will search for new discoveries in the head-on collisions of protons of high energy. 

Robust Network Infrastructure Going Forward

The DCSC/KU’s supercomputer provides scientists with the tools to study some of the most pressing and complex issues of our world. Force10 provides the infrastructure the center needs to ensure its researchers continue to advance state-of-the-art science.