Skip to content

Add benchmark scripts #173

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Add benchmark scripts #173

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e8fe852
Add benchmark scripts
sirmarcel Feb 17, 2025
6a5c392
Remove benchmark from manifest
sirmarcel Feb 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions MANIFEST.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ prune examples
prune tests
prune .github
prune .tox
prune benchmark

exclude CODE_OF_CONDUCT.md
exclude CONTRIBUTING.rst
Expand Down
23 changes: 23 additions & 0 deletions benchmark/README.rst
View file
Open in desktop
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add a comment that we run only single precision (because we showed in the paper that it makes no difference!)

Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
``benchmark``: performance sanity checking
==========================================

This provides a basic benchmarking script intended to catch major performance regressions (or improvements!). It emulates a "training"-style workload of computing energy and forces for a number of similar systems with pre-computed neighborlists and settings. In particular, we run PME forward and backward calculations for supercells of cubic CsCl crystals, in different sizes. Results are stored as a time-stamped ``.yaml`` file, together with some basic system and version information (which is just the output of ``git`` for now).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We run PME or P3M? I think the latter is preferred.

You can take the version of torch-pme. Thanks to setuptools-scm the version will contain the version and the commit!



Usage
-----

First, make sure to install ``torch-pme`` with the right dependencies in a fresh enviroment:

.. code-block:: bash

pip install .[metatensor,examples]

Then, run

.. code-block:: bash

python run.py


Please include the output ``.yaml`` file in any major pull request. We recommend using a H100 as benchmark GPU (internally, we use the ``kuma`` cluster).
11 changes: 11 additions & 0 deletions benchmark/geometry.in
View file
Open in desktop
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could just for consistency in the project use a .xyz. But up to you.

Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#===============================================================================
# Created using the Atomic Simulation Environment (ASE)

# Fri Sep 20 14:44:08 2024

#=======================================================
lattice_vector 1.0000000000000000 0.0000000000000000 0.0000000000000000
lattice_vector 0.0000000000000000 1.0000000000000000 0.0000000000000000
lattice_vector 0.0000000000000000 0.0000000000000000 1.0000000000000000
atom 0.0000000000000000 0.0000000000000000 0.0000000000000000 Cs
atom 0.5000000000000000 0.5000000000000000 0.5000000000000000 Cl
Loading