GPTune

GPTune is an autotuning framework that relies on multitask and transfer learning to help solve the underlying black-box optimization problem using Bayesian optimization methodologies. In particular, GPTune is designed to tune high-performance computing (HPC) application codes as "black-boxes", running them for carefully chosen tuning parameter values and building a performance model based on the measured performance. GPTune provides a number of advanced autotuning features outlined as follows.

• High-performance and parallel tuning: GPTune is designed to tune applications running on large-scale cluster systems and can exploit distributed memory parallelism for accelerating surrogate modeling.
  • Example: Autotuning ScaLAPACK's PDGEQRF using distributed parallel surrogate modeling
• Multitask learning-based autotuning: GPTune supports multitask learning that allows us to tune multiple tuning problems simultaneously. Multitask learning would predict an optimal tuning parameter configuration using a fewer number of evaluations compared to single task autotuning by modeling the linear dependence of the multiple tasks.
  • Example: Multitask learning-based autotuning of ScaLAPACK's PDGEQRF
• Transfer learning-based autotuning: GPTune supports transfer learning to tune the given tuning task by leveraging already available performance data collected for different tasks. Different tasks can mean different input problem sizes or the same input problem on different machine and/or software settings.
  • Example: Transfer learning from a different input problem for autotuning ScaLAPACK's PDGEQRF
  • Example: Transfer learning from a different machine setting for autotuning ScaLAPACK's PDGEQRF
• GPTuneBand (multi-fidelity autotuning): Multi-fidelity tuning uses multiple fidelity levels to guide sampling (generating many cheap samples from lower-fidelity levels). GPTuneBand combines multitask learning with a multi-armed bandit strategy to guide sampling of the given tuning problem.
  • Example: Multi-fidelity tuning of STRUMPACK's Kernel Ridge Regression (need to set the tuner option to "GPTuneBand")
• Multi-objective tuning: GPTune supports multi-objective tuning that uses NSGA2 algorithm to maximize multiple EI functions for multiple objectives. For an objective, users can also specify whether they want to optimize (minimize) the objective within the given range, or they just want the objective is within the given range.
  • Example: Multi-objective tuning of SuperLU_DIST
• Sensitivity analysis: Using obtained function evaluation results, users can run a surrogate model-based sensitivity analysis that estimates importance of each tuning parameter.
  • Example: ScaLAPACK
• Unified interface for different autotuners: GPTune uses a unified Python interface and supports using several different autotuners.
  • Example: Comparing GPTune, HpBandSter, and OpenTuner for a synthetic function

History Database

The success of (Bayesian optimization-based) autotuning depends on collecting sufficient performance data samples to build an accurate surrogate performance model or to explore the search space. To enhance reusabiltiy of performance data for autotuning, we provide a history database which is a shared repository and application programming interface (API) to share autotuning performance data between multiple users at different sites. Our shared repository in this website provides two useful interfaces (1) an interactive web dashboard that allows users to navigate the database from a web browser (2) a programmable API (called crowd-tuning API) which allows users to write a program query to download or upload performance data. The history database is a part of the GPTune project. Performance data in the history database is compatible with GPTune, and the database interfaces are incorporated into GPTune. The database feature can also be used by other autotuners (this is a future work).

Our history database enables following use cases.

• Checkpointing and restarting: Users can perform checkpointing and restarting of (long-running) autotuning.
• Query tuning results: Users can query the best tuning parameter configuration to run their applications.
  • Example (web-interface): To browse performance data of ScaLAPACK's PDGEQRF in our history database, 1. Access https://gptune.lbl.gov/repo/dashboard, 2. Choose "PDGEQRF" and click on "Search", 3. You should be able to see the data table that contains the historical data of ScaLAPACK's PDGEQRF.
• Sensitivity analysis: Using historical data, users can run a surrogate model-based sensitivity analysis that estimates importance of each tuning parameter.
  • Example (web-interface): To run a sensitivity analysis of SuperLU_DIST, 1. Login at http://gptune.lbl.gov, 2. Access https://gptune.lbl.gov/repo/dashboard, 3. Choose "SuperLU_DIST-pddrive_spawn" and click on "Search", 4. You should be able to see the table, and there should be a button "Sensitivity analysis", you can click on this and follow the instructions there.
  • Example (crowd-tuning API): Run a sensitivity analysis of SuperLU using crowd-tuning API
• Performance prediction: Using historical data, users can run a surrogate model-based performance prediction for a given tuning parameter configuration.
  • Example (web-interface): To run a sensitivity analysis of SuperLU_DIST, 1. Login at http://gptune.lbl.gov, 2. access https://gptune.lbl.gov/repo/dashboard, 3. Choose "SuperLU_DIST-pddrive_spawn" and click on "Search", 4. You should be able to see the table, and there should be a button "Make prediction", you can click on this and follow the instructions there.
• Transfer learning: Users can download historical performance data and use the knowledge to tune a similar but different tuning problem using transfer learning-based autotuning.
  • Example: GPTune's transfer learning-based autotuning for a synthetic function using the history database. It uses the crowd-tuning API to access and download historical data from the history database.

Unregistered users can only view publicly available performance data. This means that they cannot view data disclosed to authorized users or upload any kind of data into out database. Please Sign Up to access more data in our history database! Registered users can download data disclosed to registered users. There is another privilege level called "certified" for registered users. The GPTune team will update the user status as "certified" if the user has entered user profile information correctly. Certified users can access data for certified users and create collaboration groups to share data within your group members. For more information about our membership, click here.

About Us

GPTune is a joint research project between Lawrence Berkeley National Laboratory and University of California at Berkeley, and is part of the xSDK4ECP effort supported by the Exascale Computing Project (ECP).

         

Acknowledgements

This research was supported by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of the U.S. Department of Energy Office of Science and the National Nuclear Security Administration. We used resources of the National Energy Research Scientific Computing Center (NERSC), a U.S. Department of Energy Office of Science User Facility operated under Contract No. DE-AC02-05CH11231.

Copyrights

GPTune Copyright (c) 2019, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy) and the University of California, Berkeley. All rights reserved.
If you have questions about your rights to use or distribute this software, please contact Berkeley Lab's Intellectual Property Office at IPO@lbl.gov.
NOTICE. This Software was developed under funding from the U.S. Department of Energy and the U.S. Government consequently retains certain rights. As such, the U.S. Government has been granted for itself and others acting on its behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software to reproduce, distribute copies to the public, prepare derivative works, and perform publicly and display publicly, and to permit other to do so.