Database system

Replace Linux with a Database System

When it comes to building large-scale computing platforms, there are many options for system design, but in supercomputing the only practical choice for an operating system is Linux.

A team from MIT and Sandia took note of this imbalance, noting that while traditionally it was an operating system’s job to manage hardware, now the control processor and compute engines are much more separate. In other words, the operating system is more like a resource tracker that manages the usage of hardware resources. They draw parallels between this and a database management system and thus turned a database into a kind of operating system.

On the surface, it may not seem logical, or more precisely, as if it would work. But simulations of the database system (called TabulaROSA) on a 32k mainframe supercomputer give impressive first results with a 20x measured performance increase over Linux while handling 2000x more processes in fully searchable tables.

“Today’s mainstream operating systems date back 50 years to computers designed for basic office functions running on serial, local, homogeneous and deterministic hardware operating in benign environments. But increasingly, these traditional operating system platforms are at best witnesses and at worst obstacles to the use of specially designed processors, ”the team explains, pointing to the GPU and others. accelerators in which the user must interact with the operating system to use specialized processors.

There are many overlaps between database systems and functions required for an operating system on a cluster, in the case of TabulaROSA, however, the functions are handled in database tables instead.

While there are Linux variants that support users with heterogeneous systems, the team identified areas where other operating system redundancies could be removed and replaced by their database approach.

TabulaROSA explores the “potential benefits of implementing operating system functions in a way that takes advantage of the power and mathematical properties of database systems”, with the idea that core systems data already does much of what an operating system manages, including ingesting and cleaning up data, transforming, analyzing, and moving unified data to other systems.

The team defines the key functions of the operating system in terms of “rigorous mathematical semantics (algebra of associative arrays) that are directly translatable into database operations.” Since the mathematical operations of database tables are based on a linear system on the union and intersection semi-ring, these operations have a number of mathematical properties that are ideal for parallel operating systems. ensuring accuracy over a wide range of parallel operations.

Number of processes managed out of total number of forkers for TabulaROSA D4M simulation and Linux operating system running on over 32,000 system cores.
Number of processes managed out of total number of forkers for TabulaROSA D4M simulation and Linux operating system running on over 32,000 system cores.

“The fork simulations in TabularROSA are performed using an associative array implementation and are compared to Linux on a supercomputer with over 32,000 cores. Using over 262,000 forkers handling over 68,000,000,000 processes, simulations show that TabulaROSA has the potential to perform operating system functions on a massively parallel scale. TabulaROSA simulations deliver 20 times better performance than Linux, while handling 2,000 times more processes in fully searchable tables ”

Efficiencies can be easily measured in simulations and theory can be better than practice. Running Linux on large supercomputers isn’t just about tracking and allocating resources. Nonetheless, it shows that there are other ways to continue shrinking the operating system and let resource management offload onto software running in the background. Much more depth in the full article.

Subscribe to our newsletter

Featuring the week’s highlights, analysis, and stories straight from us to your inbox with nothing in between.
Subscribe now

Source link