Exploration with MACE¶

Use MACE for forces and local-environment descriptors in APE. This walkthrough uses examples/ape/pd_surface_mace/ and builds on the preparation and monitoring steps in First exploration with EAM.

The repository example supplies the structure and configuration. You must provide the MACE model files and an APE build with native MACE support for your target GPU or APU. In the installation selector, choose APE, MACE, and your hardware. A Python-only MACE installation does not supply this native force interface.

1. Prepare a separate run¶

Select the matching build and copy the example inputs:

export SODA_ROOT="$HOME/soda"
export SODA_BIN="$SODA_ROOT/build/your-mace-prefix/soda"
source "$SODA_ROOT/build/your-mace-prefix/modules.rc"

export SODA_EXAMPLE="$SODA_ROOT/examples/ape/pd_surface_mace"
export SODA_RUN="$HOME/soda-runs/ape-pd-mace"
mkdir -p "$HOME/soda-runs"
mkdir "$SODA_RUN"
cp "$SODA_EXAMPLE/config.ini" "$SODA_EXAMPLE/pos.con" \
  "$SODA_EXAMPLE/SoftFixAtomIDs.dat" "$SODA_RUN/"
cp -R "$SODA_EXAMPLE/potfiles" "$SODA_RUN/"
cd "$SODA_RUN"

Use a new run directory when changing potential. Optimize the starting structure with the selected MACE model, preserving the intended fixed atoms and cell conditions, and use the optimized structure as pos.con or pos.xyz. A structure relaxed with EAM may not be a minimum for MACE. Follow the input-format conventions and keep only the chosen structure format in the run directory.

2. Set the two model paths¶

The example uses two representations of a MACE model:

Input	Used for
`[Descriptors] mace_path` in `config.ini`	Python MACE model that provides local-environment descriptors for the DECAF classifier in APE.
`pair_coeff` in `potfiles/in.lammps`	Exported LibTorch model for the native LAMMPS `pair_style mace` force calculation.

For this walkthrough, use a compatible model for Pd and its corresponding LAMMPS export. Keep both files on storage accessible to every compute node; absolute paths avoid ambiguity after worker files are staged.

The MACE package installed by the recipe provides an export command for this interface. If you do not already have the export, run it in that environment:

python -m mace.cli.create_lammps_model /absolute/path/to/your.model \
  --format libtorch

It writes /absolute/path/to/your.model-lammps.pt. For a model with multiple heads, select the head appropriate to your calculation during export. The example uses pair_style mace, so use the LibTorch export for that interface.

Edit the existing descriptor entries in config.ini:

[Descriptors]
descriptor = mace
mace_path = /absolute/path/to/your.model
mace_device = gpu

gpu is also used with the ROCm build on an AMD APU; apu is an accepted alias. Edit the existing pair_coeff line in potfiles/in.lammps:

pair_style mace no_domain_decomposition
pair_coeff * * /absolute/path/to/your.model-lammps.pt Pd

Keep the other LAMMPS settings from the example. The Python descriptor path and native force path are separate settings; changing one does not update the other. Installing MACE also does not fix the potential choice for every future run: each run directory selects its own potential.

3. Set a short exploration limit¶

The supplied MACE config.ini has no [APE] section. Add:

[APE]
max_cycle = 5
sampling_strategy = energy
soft_fix = true

Retain the other example settings. The default cycle limit is otherwise zero, which leaves exploration uncapped by max_cycle. This five-cycle limit is for checking the complete setup, not assessing convergence.

Run the input check after replacing the model placeholders:

python "$SODA_ROOT/src/python/soda_analyzer.py" check-run-folder . \
  --soda-exe "$SODA_BIN" --check

Resolve blocking findings, then check the model with a small calculation on the intended hardware. File checks cannot establish model suitability or runtime compatibility.

4. Launch on a GPU allocation¶

For a workstation or an existing GPU allocation using Open MPI, a minimal layout has one head/master and one GPU worker. Allocate two physical CPU cores and at least one compatible GPU:

export OMP_NUM_THREADS=1
export OMP_PLACES=cores
export OMP_PROC_BIND=close
export MKL_NUM_THREADS=1
export OPENBLAS_NUM_THREADS=1
mpirun -n 2 --map-by slot:PE=1 --bind-to core --report-bindings \
  "$SODA_BIN" \
  --num-gpu-workers-per-node=1 \
  --numjobsperwgroup=2 \
  --numjobsdonetotriggerupdate=2 \
  --storesummariesinfile=1 \
  --flushafterjobsdone=1 \
  --flushafterminutes=0

The time-based flush condition is disabled, so regular writes occur after more than one collected job summary. Reaching max_cycle also forces a final write. See APE output settings for alternatives that combine job-count and elapsed-time conditions.

For APE with this GPU force/descriptor setup, every process-search worker must be GPU-capable. Requesting extra CPU-only workers does not provide an alternative place to execute these searches. Keep lammps_threads at its default for the native MACE GPU recipe, which does not enable LAMMPS’s OPENMP package.

The example also provides submitjob.raven and submitjob.viper. Copy the appropriate script from $SODA_EXAMPLE into your run directory. Edit SODA_ROOT and BUILD_NAME near its top, review the allocation, and submit it with sbatch. Both source build/$BUILD_NAME/modules.rc and set SODA_EXEC from that build.

Template	Supplied layout
`submitjob.raven`	Three nodes, 13 ranks per node: one head and 12 GPU workers sharing four GPUs on each node.
`submitjob.viper`	Two nodes, six ranks per node: one head and five GPU workers sharing two MI300A APUs on each node.

These larger layouts enable GPU sharing. Check model memory use with a small run before increasing the worker count. Change both the Slurm allocation and srun arguments when adapting a template. Review fresh-run output cleanup in the selected script before submitting.

5. Check and analyze the result¶

Follow APE progress. In addition to ordinary workflow progress, check GPU detection and assignment messages and the [LAMMPS runtime] entries in worker logs. Confirm that both model paths loaded successfully; increasing the cycle limit will not fix a model or device mismatch.

Use Analyzing APE output to read states, processes, barriers, and environments from this run with SodaLibrary(..., workflow="ape"). Descriptor clusters depend on the chosen representation, so cluster numbers need not correspond to those from an EAM/default-descriptor run. Differences in discovered processes can also reflect the changed force model and sampling.

See the APE references and Citing SODA for method citations. Also cite MACE and the particular trained model used for your calculation. Further tutorials on model selection and descriptor settings are coming soon; APE settings provides a short reference for the current options.