Christofari

Christofari — are Christofari (2019), Christofari Neo (2021) supercomputers of Sberbank based on Nvidia corporation hardware Sberbank of Russia and Nvidia. Their main purpose is neural network learning. They are also used for scientific research and commercial calculations.

The supercomputers are named after, the first customer of Sberbank, holder of the Bank's first savings account passbook. The supercomputers are listed in the Top 500 ranking of most powerful commercially available computer systems.

Development
Sberbank presented the supercomputers together with its subsidiary SberCloud. In December 2019, Sberbank and SberCloud commercially launched the Christofari supercomputer. Within a year, the power of Christofari became the foundation of a cloud based ML Space platform. It was configured to work with machine learning models. Sberbank and SberCloud announced this platform in December 2020.

The more powerful Christofari Neo supercomputer was presented at the AI Journey international conference in November 2021 by David Rafalovsky, the CTO of Sberbank Group. Currently David Rafalovsky is not a member of Sberbank Group.

Usage
The supercomputers can be used by scientific, commercial and government organizations working in the various sectors of the economy. The machines were developed to work with artificial intelligence algorithms, neural network learning, and inference of various models.

Sber uses Christofari for internal tasks e.g. speech recognition and autoresponder voice generation in a call center (40% of customer inquiries are already answered automatically by bots ). Also they use it for analysis of CT scan images of the lungs. The SberDevices and Sber AI teams were the first who received access to Christofari Neo. They developed the first service based on the DALL-E neural network that generates images from queries in Russian.

The power of supercomputers is also provided to other organizations when connecting the services of the cloud platform SberCloud ML Space.

Christofari
The first supercomputer was presented by Herman Gref, CEO of Sberbank, and David Rafalovsky, CTO of Sberbank Group, on 8 November 2019, at the AI Journey conference in Moscow.

As of March 2020, this is the only supercomputer in Russia designed specifically for working with artificial intelligence algorithms. It is capable of training software models based on complex neural networks in unprecedentedly short time, it's Russia's fastest supercomputer.

The Christofari machines are based on Nvidia DGX-2 nodes equipped with Tesla V100 graphics accelerators. The InfiniBand network based on Mellanox hardware is used for an interconnection. Effective performance is about 6.7 petaflops, which makes it the 40th most powerful system in the world (at the time of launch on 8 November 2019), the 7th in Europe, and the 1st in Russia (the results of the previous Russian leader — the Lomonosov-2 system — were exceeded by more than two times). This is the first supercomputer of a financial organization in the world available for third-party users. It is also the first supercomputer certified in Russia to work with personal data.

DGX-2 single node specifications
The DGX servers are connected via Mellanox switches with 36-ports, supporting up to four InfiniBand EDR connections at 100 Gbit/s.
 * Maximum Power Usage — 10 kW
 * CPU — Dual Intel Xeon Platinum 8168, 2.7 GHz, 24-cores
 * GPUs — 16X NVIDIA Tesla V100
 * GPU Memory — 512 GB total
 * NVIDIA CUDA Cores — 81920
 * NVIDIA Tensor cores — 10240
 * System Memory — 1.5 TB

Almost the entire machine learning stack in use is oriented on the Ubuntu operating system as the base platform. The machines utilize a modified server version of an Ubuntu 18.04 LTS operating system. It is supported by Nvidia including graphics accelerators drivers, network software stack, and the necessary tools for maintenance and diagnostics. The package also includes proprietary software from Nvidia CUDA Toolkit, cuDNN, NCCL, and Docker Engine Utility for GPU Nvidia (the entire main machine learning stack runs in containers).

The supercomputer is located in the Sberbank data center (DC) in the Skolkovo Innovation Center in Moscow, Russia. It occupies one machine room and was built in less than a year.

Positions in rankings

 * September 2021 — 1st of the 50th most powerful computers in CIS;
 * June 2022 — 80th among the 500th most powerful computers in the world (November 2021 — 72nd).

Christofari Neo
The second supercomputer is also built on the basis of Nvidia technologies and is equipped with Nvidia A100 GPUs with 80 GB of memory. For an interconnection is used the high-speed switching InfiniBand network, which provides the speed of data exchange up to 1600 GB/s per each compute node and minimal latency. The actual performance is 11,95 petaflops.

DGX A100 single processor specifications

 * Maximum Power Consumption — 6,5 kW
 * CPU — Dual AMD Rome 7742, 128-cores, 2.25 GHz (base), 3.4 GHz (maximum)
 * GPUs — 8X Nvidia A100, 80 GB
 * GPU memory — 640 GB
 * System memory — 2 TB

Positions in rankings

 * November 2021 — 7th place in the HPL-AI rating of supercomputers and artificial intelligence;
 * June 2022 — 46th place among the 500 most powerful computers in the world (November 2021 — 43rd).