# Exploration with MACE

Use MACE for forces and local-environment descriptors in APE. This walkthrough
uses `examples/ape/pd_surface_mace/` and builds on the preparation and monitoring
steps in [First exploration with EAM](ape-eam.md).

The {repo}`repository example <examples/ape/pd_surface_mace/README.md>` supplies
the structure and configuration. You must provide the MACE model files and an
APE build with native MACE support for your target GPU or APU. In the
[installation selector](../install.md), choose **APE**, **MACE**, and your
hardware. A Python-only MACE installation does not supply this native force
interface.

## 1. Prepare a separate run

Select the matching build and copy the example inputs:

```bash
export SODA_ROOT="$HOME/soda"
export SODA_BIN="$SODA_ROOT/build/your-mace-prefix/soda"
source "$SODA_ROOT/build/your-mace-prefix/modules.rc"

export SODA_EXAMPLE="$SODA_ROOT/examples/ape/pd_surface_mace"
export SODA_RUN="$HOME/soda-runs/ape-pd-mace"
mkdir -p "$HOME/soda-runs"
mkdir "$SODA_RUN"
cp "$SODA_EXAMPLE/config.ini" "$SODA_EXAMPLE/pos.con" \
  "$SODA_EXAMPLE/SoftFixAtomIDs.dat" "$SODA_RUN/"
cp -R "$SODA_EXAMPLE/potfiles" "$SODA_RUN/"
cd "$SODA_RUN"
```

Use a new run directory when changing potential. **Optimize the starting
structure with the selected MACE model**, preserving the intended fixed atoms
and cell conditions, and use the optimized structure as `pos.con` or
`pos.xyz`. A structure relaxed with EAM may not be a minimum for MACE. Follow
the [input-format conventions](preparing.md#check-structures-and-conventions)
and keep only the chosen structure format in the run directory.

## 2. Set the two model paths

The example uses two representations of a MACE model:

| Input | Used for |
| --- | --- |
| `[Descriptors] mace_path` in `config.ini` | Python MACE model that provides local-environment descriptors for the DECAF classifier in APE. |
| `pair_coeff` in `potfiles/in.lammps` | Exported LibTorch model for the native LAMMPS `pair_style mace` force calculation. |

For this walkthrough, use a compatible model for Pd and its corresponding
LAMMPS export. Keep both files on storage accessible to every compute node;
absolute paths avoid ambiguity after worker files are staged.

The MACE package installed by the recipe provides an export command for this
interface. If you do not already have the export, run it in that environment:

```bash
python -m mace.cli.create_lammps_model /absolute/path/to/your.model \
  --format libtorch
```

It writes `/absolute/path/to/your.model-lammps.pt`. For a model with multiple
heads, select the head appropriate to your calculation during export. The
example uses `pair_style mace`, so use the LibTorch export for that interface.

Edit the existing descriptor entries in `config.ini`:

```ini
[Descriptors]
descriptor = mace
mace_path = /absolute/path/to/your.model
mace_device = gpu
```

`gpu` is also used with the ROCm build on an AMD APU; `apu` is an accepted
alias. Edit the existing `pair_coeff` line in `potfiles/in.lammps`:

```text
pair_style mace no_domain_decomposition
pair_coeff * * /absolute/path/to/your.model-lammps.pt Pd
```

Keep the other LAMMPS settings from the example. The Python descriptor path
and native force path are separate settings; changing one does not update
the other. Installing MACE also does not fix the potential choice for every
future run: each run directory selects its own potential.

## 3. Set a short exploration limit

The supplied MACE `config.ini` has no `[APE]` section. Add:

```ini
[APE]
max_cycle = 5
sampling_strategy = energy
soft_fix = true
```

Retain the other example settings. The default cycle limit is otherwise zero,
which leaves exploration uncapped by `max_cycle`. This five-cycle limit is for
checking the complete setup, not assessing convergence.

Run the input check after replacing the model placeholders:

```bash
python "$SODA_ROOT/src/python/soda_analyzer.py" check-run-folder . \
  --soda-exe "$SODA_BIN" --check
```

Resolve blocking findings, then check the model with a small calculation on
the intended hardware. File checks cannot establish model suitability or
runtime compatibility.

## 4. Launch on a GPU allocation

For a workstation or an existing GPU allocation using Open MPI, a minimal
layout has one head/master and one GPU worker. Allocate two physical CPU
cores and at least one compatible GPU:

```bash
export OMP_NUM_THREADS=1
export OMP_PLACES=cores
export OMP_PROC_BIND=close
export MKL_NUM_THREADS=1
export OPENBLAS_NUM_THREADS=1
mpirun -n 2 --map-by slot:PE=1 --bind-to core --report-bindings \
  "$SODA_BIN" \
  --num-gpu-workers-per-node=1 \
  --numjobsperwgroup=2 \
  --numjobsdonetotriggerupdate=2 \
  --storesummariesinfile=1 \
  --flushafterjobsdone=1 \
  --flushafterminutes=0
```

The time-based flush condition is disabled, so regular writes occur after
more than one collected job summary. Reaching `max_cycle` also forces a final
write. See [APE output settings](ape-settings.md#queue-and-output-settings)
for alternatives that combine job-count and elapsed-time conditions.

For APE with this GPU force/descriptor setup, every process-search worker must
be GPU-capable. Requesting extra CPU-only workers does not provide an
alternative place to execute these searches. Keep `lammps_threads` at its
default for the native MACE GPU recipe, which does not enable LAMMPS's
OPENMP package.

The example also provides `submitjob.raven` and `submitjob.viper`. Copy the
appropriate script from `$SODA_EXAMPLE` into your run directory. Edit
`SODA_ROOT` and `BUILD_NAME` near its top, review the allocation, and submit
it with `sbatch`. Both source `build/$BUILD_NAME/modules.rc` and set
`SODA_EXEC` from that build.

| Template | Supplied layout |
| --- | --- |
| `submitjob.raven` | Three nodes, 13 ranks per node: one head and 12 GPU workers sharing four GPUs on each node. |
| `submitjob.viper` | Two nodes, six ranks per node: one head and five GPU workers sharing two MI300A APUs on each node. |

These larger layouts enable GPU sharing. Check model memory use with a small
run before increasing the worker count. Change both the Slurm allocation and
`srun` arguments when adapting a template. Review fresh-run output cleanup
in the selected script before submitting.

## 5. Check and analyze the result

Follow [APE progress](ape-eam.md#3-follow-progress). In addition
to ordinary workflow progress, check GPU detection and assignment messages
and the `[LAMMPS runtime]` entries in worker logs. Confirm that both model
paths loaded successfully; increasing the cycle limit will not fix a model
or device mismatch.

Use [Analyzing APE output](ape-analysis.md) to read states, processes, barriers,
and environments from this run with `SodaLibrary(..., workflow="ape")`.
Descriptor clusters depend on the
chosen representation, so cluster numbers need not correspond to those from
an EAM/default-descriptor run. Differences in discovered processes can also
reflect the changed force model and sampling.

See the [APE references](ape.md#references) and [Citing SODA](../citations.md)
for method citations. Also cite MACE and the particular trained model used
for your calculation. Further tutorials on model selection and descriptor
settings are coming soon; [APE settings](ape-settings.md) provides a short
reference for the current options.