# Preparing custom factories for SODA–EZGA
Supply mutation, crossover, calculation, and scoring functions through the
EZGA YAML. SODA distributes calculation tasks and collects their results.
Start with the [FCC example](ezga-fcc.md) and
[EZGA settings](ezga-settings.md) for a working run configuration.
## Where your functions run
An agent maintains the population and generates candidates; workers run the
calculations. The linked YAML fields below select your functions.
The six hooks use **native EZGA callable interfaces**. Follow the
linked wiki for examples and the contracts here for SODA's pinned version
and integration limits.
```{raw} html
One generation after initial-population
evaluation. Geometry and duplicate filters are omitted. Population constraints
apply before dispatch and again to returned structures.
```
Results admitted at step 6 are scored at step 1 of the next generation.
Parent selection and convergence use supported settings rather than general
factory hooks.
## Connect a factory to the run
Connect three files: `config.ini` permits imports and selects the YAML;
the YAML selects factories; Python defines them. A simple **run directory** is:
```text
my-run/
├── config.ini
├── native_ezga.yaml
├── my_ezga.py
└── seed_structures.json
```
First, point SODA at the YAML and allow your module in `config.ini`:
```ini
[Main]
job = ezga
[EZGA]
config_yaml = native_ezga.yaml
seed_structures_json = seed_structures.json
trusted_import_prefixes = my_ezga
```
`trusted_import_prefixes = my_ezga` allows imports from that module or package.
It does not install the code or select a function.
**The environment running SODA must be able to execute `import my_ezga`.**
Activate the environment created for your SODA build, normally
`$SODA_ROOT/venv/$BUILD_NAME`, through its activation script
([Preparing a run](preparing.md#select-the-installation)). Add the absolute
run-directory path to `PYTHONPATH` so workers can import `my_ezga.py`:
```bash
source "$SODA_ROOT/build/$BUILD_NAME/modules.rc"
export SODA_RUN="/absolute/path/to/my-run"
export PYTHONPATH="$SODA_RUN${PYTHONPATH:+:$PYTHONPATH}"
```
Use this setup in your batch script for every rank. The absolute path keeps
imports working when workers change directory. An installable package can
instead be installed into this environment with `python -m pip`.
Next, select the factory and its configuration arguments in
`native_ezga.yaml`. For the single file `my_ezga.py`, use:
```yaml
mutation_funcs:
- factory: my_ezga:make_mutation
kwargs:
displacement: 0.05
```
`module:factory` identifies the import. Define `make_mutation` in
`my_ezga.py`; it receives the YAML settings and returns a **runtime callable**:
```python
# my_ezga.py — signature outline; implement your mutation in the inner function.
def make_mutation(displacement):
def mutate(structure, rng=None):
# Use displacement from the enclosing factory to modify structure.
...
return structure
return mutate
```
Check the interpreter and import in the activated environment:
```bash
python -c "import sys, my_ezga; print(sys.executable); print(my_ezga.__file__)"
```
SODA–EZGA then makes these two calls conceptually:
```python
# During configuration loading:
from my_ezga import make_mutation
mutation = make_mutation(displacement=0.05)
# Later, when EZGA selects this operator:
child = mutation(structure, rng=ezga_rng)
```
The returned function captures `displacement`; EZGA supplies `structure`
and `rng` at runtime. The tables below describe that **runtime callable**.
YAML `args` and `kwargs` configure the factory, whose argument names you define.
For more functions, you can organize `my_ezga` as a package directory instead
of a single file:
```text
my-run/my_ezga/
├── __init__.py
└── operators.py # defines make_mutation
```
`__init__.py` can be empty. Now `my_ezga.operators:make_mutation` means
`from my_ezga.operators import make_mutation`. The same import permission applies.
Most YAML fragments below use this **package layout**, with functions in
`operators.py`, `worker.py`, and `scoring.py`. The population-constraints
example uses the single-file layout. Adjust paths for your layout and merge
repeated sections into your run configuration.
## Shared data conventions
Here, **A** is the number of atoms in one structure, **B** the number of
structures in a dataset, and **F** and **O** the numbers of feature and objective
columns. Numeric arrays should contain finite values with consistent units;
the supplied atomistic calculators normally use Å and eV.
(structure-object)=
### Understand a single structure
`structure` is an argument holding a **SAGE (`sage_lib`) `SingleRun`**: one
atomic configuration. Access its atomic data through
`structure.AtomPositionManager`. See the native
[structure interface](https://gitlab.mpcdf.mpg.de/fhi-theory/EZGA/-/wikis/custom_modules/Custom_Mutation_Crossovers),
section 1, for positions, symbols, and cell vectors.
In SODA, `atomicConstraints` normally holds boolean flags `(A, 1)`, with `True` meaning fixed;
`pbc` holds three boundary flags; `metadata` is a dictionary of properties
and bookkeeping. Use SAGE setters such as `set_atomPositions(positions)`
to keep geometry caches consistent. Preserve metadata and keep per-atom arrays aligned
when changing atom counts or ordering. Mutation and crossover must return
SAGE structures, even if they use ASE internally.
(dataset-object)=
### Understand the dataset object
`dataset` normally holds a **SAGE `Partition`**, the same collection type
used by native EZGA:
```text
dataset SAGE Partition containing B structures
└── containers
├── [0] SAGE SingleRun
│ └── AtomPositionManager positions, symbols, cell, energy, metadata
├── [1] SAGE SingleRun
└── ...
```
See [Database Handling](https://gitlab.mpcdf.mpg.de/fhi-theory/EZGA/-/wikis/EZGAResults/Database-Handling),
section 8, for iteration and atomic fields. SODA supplies this object to
`compute(dataset)`; you do not open a database inside the callback.
Caching can supply a subset, sometimes as a sequence of structures. Use
`getattr(dataset, "containers", dataset)` to iterate either form. Treat the
input as read-only and return one row per structure in input order, with a
fixed column count. EZGA combines outputs from multiple feature or objective
functions by column.
(mutation)=
## Mutation
**`mutation_funcs` changes one candidate structure at step 2.**
See native [Mutation & Crossover Modules](https://gitlab.mpcdf.mpg.de/fhi-theory/EZGA/-/wikis/custom_modules/Custom_Mutation_Crossovers),
sections 2–5 and 8, for mutation examples and configuration.
| Contract | Requirement |
| --- | --- |
| Call | `mutate(structure, rng=None)` |
| Input | One [SAGE `SingleRun`](#structure-object); optional NumPy `Generator` named `rng`. |
| Output | One SAGE `SingleRun`, or `None` if the attempt produces no candidate. |
```yaml
mutation_funcs:
- factory: my_ezga.operators:make_mutation
kwargs:
displacement: 0.05
```
The normal loop supplies a copy of the parent, which you may modify.
Returning `None` skips that attempt and retains the current structure;
avoid modifying the input before returning `None`.
For proposal probabilities, the native loop also accepts
`(structure, log_q_fwd, trace, log_q_rev_func)`. The last three entries are a
numeric log probability, JSON-compatible trace dictionary, and optional reverse-proposal callable
`log_q_rev_func(child, trace) -> float`.
Declare `rng` to receive EZGA's NumPy generator, or `seed` for an integer
seed; `**kwargs` alone requests neither. Some guided-variation calls pass
only the structure, so keep these arguments optional and define fallback
randomness deliberately.
(crossover)=
## Crossover
**`crossover_funcs` combines two parents into two children at step 2.**
See native [Mutation & Crossover Modules](https://gitlab.mpcdf.mpg.de/fhi-theory/EZGA/-/wikis/custom_modules/Custom_Mutation_Crossovers),
sections 6–8. **Adapt its single-child examples:** the pinned native loop
expects two children.
| Contract | Requirement |
| --- | --- |
| Call | `cross(parent_a, parent_b, rng=None)` |
| Input | Two [SAGE `SingleRun` objects](#structure-object); optional `rng` or `seed` as for mutation. |
| Output | A pair `(child_a, child_b)` of SAGE `SingleRun` objects. |
```yaml
crossover_funcs:
- factory: my_ezga.operators:make_crossover
kwargs: {}
```
Copy parents with `copy.deepcopy` before editing; the caller passes them
directly. If either returned child is `None`, EZGA falls back to copies of
both parents. Use `crossover_funcs: []` to omit crossover.
(calculator)=
## Worker calculation
**`simulator.calculator` performs the calculation on one candidate at step 4.**
Return a native EZGA calculator callable; no base-class inheritance is
required. It can perform relaxation, MD, or an external calculation.
The native [Custom Calculators guide](https://gitlab.mpcdf.mpg.de/fhi-theory/EZGA/-/wikis/custom_modules/Custom_Calculators)
uses `register_calculator` to adapt ASE functions to the array interface
below. Import that optional adapter from
`ezga.simulator.calculators.calculator_interface`. Returning an ASE `Atoms`
object directly requires this adapter.
For each call, the worker supplies one candidate's `symbols`, `positions`,
`cell`, and `fixed` flags, plus the context keywords below. You receive
arrays, not a population or a parent structure object. Use them to run your
backend and return its results; apply the fixed-atom flags when configuring
your calculation. Model paths and other factory settings remain available
through the enclosing factory.
```python
def calculate(
*, symbols, positions, cell,
fixed=None, constraints=None, sampling_temperature=0.0,
output_filename=None, seed=None, parent_hash=None, shared_state=None,
):
...
return new_positions, new_symbols, new_cell, energy, metadata
```
| Input | Type and meaning |
| --- | --- |
| `symbols` | Chemical-symbol array or sequence, `(A,)`. |
| `positions`, `cell` | Floating-point arrays, `(A, 3)` and `(3, 3)`. |
| `fixed` | Boolean array `(A, 1)` from SODA; `True` means fixed. Accept `None` for use elsewhere. |
| `constraints` | `None` in the current SODA worker call. |
| `sampling_temperature` | Numeric sampling temperature. |
| `output_filename` | String path for optional scratch output; SODA does not read this file. |
| `seed` | Integer from YAML `simulator.seed`, or `None`; not SODA's per-task worker seed. |
| `parent_hash` | Optional reference ID from the candidate's metadata, normally a string; see below. |
| `shared_state` | `None` in the current SODA worker call. |
`*` makes the arguments keyword-only. Accept the listed names, or collect
unused runtime keywords in `**kwargs`. YAML factory settings are separate.
Capture atom-selection predicates and boundary conventions in the factory;
the runtime supplies neither predicates nor a `pbc` argument.
`parent_hash` lets you label the origin of a calculation's results. EZGA
uses the input candidate's stored `metadata["hash"]`; when missing, SODA
tries a recorded parent ID, then `parent_hash`, `structure_hash`, or `id`
from its metadata. It is `None` if no reference is available. Treat it as a
label: it neither contains parent coordinates nor lists all crossover
parents, and it is not a unique job ID. You may ignore it for an energy or
relaxation calculation, or log it and return it under your own metadata key,
such as `"source_reference"`. SODA supplies it automatically; no YAML entry
is needed. With `register_calculator`, access it through
`atoms.info.get("parent_hash")` instead.
SODA seeds Python's `random` and NumPy's legacy global RNG per task. This
does not seed a new `np.random.default_rng()` or change the `seed` keyword.
Return a **five-element tuple** in this order, keeping **A equal to the
input atom count**:
| Output | Type and meaning |
| --- | --- |
| `new_positions` | Finite floating-point Cartesian array `(A, 3)`; preserve atom indices to keep inherited per-atom fields aligned. |
| `new_symbols` | Array or sequence of valid chemical symbols `(A,)`, one per position. Elements may change; see below. |
| `new_cell` | Finite floating-point cell matrix `(3, 3)`, or `None` to retain the cell and boundary flags. See the cell-update behavior below. |
| `energy` | Finite scalar float for the returned geometry and composition, in your objectives' units. |
| `metadata` | Dictionary of JSON-compatible results, merged into structure metadata; use `{}` if empty. It does not replace atomic arrays. |
```yaml
simulator:
calculator:
factory: my_ezga.worker:make_calculation
kwargs:
model_path: /absolute/path/to/model
fmax: 0.05
```
`model_path` and `fmax` configure `make_calculation`. Capture them in the
returned function as in the mutation example above; they do not become
runtime arguments to `calculate`.
The following limits apply to the current SODA worker interface:
| Change made by the calculator | Current behavior |
| --- | --- |
| Add or remove atoms | **Unsupported through this return.** The inherited fixed-atom array keeps its input length, causing a shape mismatch even if all atoms are mobile. |
| Introduce a new element | **Supported at unchanged atom count.** Your calculator, features, objectives, and element-dependent atomic fields must support it. |
| Change the cell | Each structure has its own cell; no list of recognized boxes applies. See the cell-update behavior below. |
| Return several structures | **Requires branching settings.** Return a list of five-element tuples within the configured limit; each result must satisfy the same contract. |
Candidates may have different atom counts. Make such changes in mutation
or crossover, updating all per-atom arrays together; worker `metadata`
cannot update them. Symbol changes also leave inherited masses unchanged.
For example, `[Au, Ag]` may become `[C, Ag]` even when seeds contain no carbon:
SODA recognizes H through Og independently of the seeds. Your potential and
scoring must support C. For native `feature_composition_vector`, include C
in `IDs: [Au, Ag, C]` from the start if you need its count; feature columns
do not expand automatically.
```{note}
**Cell updates: behavior to be aware of.** In the current native simulator,
applying a nonzero `new_cell` also rescales the returned positions and sets
`pbc` to `[True, True, True]`. For example, doubling the cell vectors also
doubles those positions, so the stored geometry may differ from the geometry
used to calculate the returned energy. The boundary flags are set even if
the cell is unchanged.
Return `new_cell=None` when the cell is unchanged to preserve positions and
boundary flags. An all-zero matrix is skipped and cannot clear a cell.
Preserving final Cartesian positions and boundary flags when changing the
cell requires additional handling in the simulator integration.
```
For a calculator returning two structures, add this to the existing
`simulator` section of your YAML, alongside `calculator`:
```yaml
simulator:
branching:
enabled: true
max_branches_per_work_item: 2
allowed_branch_kinds: [branch]
```
The corresponding return has this form:
```python
return [
(positions_a, symbols_a, cell_a, energy_a, {"branch_kind": "branch"}),
(positions_b, symbols_b, cell_b, energy_b, {"branch_kind": "branch"}),
]
```
SODA enforces the branch count and allowed kinds; the maximum configurable
count is 128. Returning `[]` produces no structures; `None` is invalid.
All results still undergo agent-side validation.
You may ignore `output_filename`. SODA collects the **returned tuple or
list**, and does not read or archive this file. The worker's scratch
directory is deleted after the simulation. If your backend writes results
to files, parse them inside `calculate` and return the needed values in the
tuple or `metadata`; returning a filename does not preserve its contents.
Create the parent directory of `output_filename` before writing to it.
(objectives)=
## Objectives
**`evaluator.objectives_funcs` computes the scores used for selection and
objective convergence at step 1.**
See native [Custom Objectives](https://gitlab.mpcdf.mpg.de/fhi-theory/EZGA/-/wikis/custom_modules/Custom_Objectives)
for scoring examples. Its single-score examples also extend to multiple
columns in the pinned native evaluator.
With objective caching disabled, SODA supplies the **retained population
for the current stage** at the start of each generation: initialized seeds
plus admitted results, minus pruned structures.
| Contract | Requirement |
| --- | --- |
| Call | `compute(dataset)` |
| Input | An EZGA-supplied [SAGE `Partition`](#dataset-object) containing B structures; `.containers` gives the structures in row order. Caching can supply a subset instead. |
| Output | Numeric NumPy array `(B, O)`, or `(B,)` for one objective. |
```yaml
evaluator:
objectives_funcs:
- factory: my_ezga.scoring:make_objective
kwargs:
scale: 1.0
```
Lower scores are preferred; negate a quantity to maximize it. Keep
`evaluator.cache_objectives: false` for scores that depend on the whole
dataset, such as a hull-relative score, unless you have verified the cache
behavior: cached evaluation can pass only a subset.
(features)=
## Features
**`evaluator.features_funcs` describes structures for scoring and validation
at steps 1, 3, and 5, and optional information convergence at step 6.**
See native [Custom Features](https://gitlab.mpcdf.mpg.de/fhi-theory/EZGA/-/wikis/custom_modules/Custom_Features)
for the batched composition-vector example and column names. Use the
dataset contract below; the guide's scalar/per-structure overview does not
match the pinned evaluator.
| Contract | Requirement |
| --- | --- |
| Call | `compute(dataset)` |
| Input | An EZGA-supplied [dataset](#dataset-object) containing B SAGE structures. |
| Output | Numeric NumPy array `(B, F)`, or `(B,)` for one feature. |
```yaml
evaluator:
features_funcs:
- factory: my_ezga.scoring:make_features
kwargs: {}
```
Features used before dispatch must be computable before the calculation.
Keep columns stable; for multiple columns, attach
`compute.get_feature_index_map()` returning `{name: column_index}`, as in
the native example. Return finite values; the evaluator can replace callback
exceptions with NaNs.
(population-constraints)=
## Population constraints
**`population.constraints` accepts or rejects candidates using their features
at steps 3 and 5.** It applies to generated candidates, supported injected/guided
proposals, and each returned calculator product.
See native [Population Constraints](https://gitlab.mpcdf.mpg.de/fhi-theory/EZGA/-/wikis/3.Configuration/1Population/Constraints)
for feature bounds. The guide uses legacy `bansga` imports; in the pinned
package, import `ConstraintGenerator` from `ezga.DoE.DoE`.
| Contract | Requirement |
| --- | --- |
| Call | `check(features)` |
| Input | One NumPy feature row `(F,)` with integer or real floating-point values, supplied by EZGA's evaluator. Treat it as read-only. |
| Output | One `bool` or `numpy.bool_`: `True` passes this predicate; `False` rejects. Arrays and other truthy values are errors. |
For the single-file layout above, add this factory to `my_ezga.py`:
```python
def make_bounds(feature_index, lower, upper):
def check(features):
return bool(lower <= features[feature_index] <= upper)
return check
```
Then select it in `native_ezga.yaml`. This example checks the atom count:
```yaml
evaluator:
features_funcs:
factory: ezga.evaluator.features:feature_total_atoms
population:
constraints:
- factory: my_ezga:make_bounds
kwargs:
feature_index: 0
lower: 1.0
upper: 100.0
```
The YAML arguments configure `make_bounds`; SODA later calls `check(features)`
with the candidate's current feature row. Indices refer to the combined
feature columns. Native `ConstraintFactory` predicates can also use names
from the feature functions' index maps; SODA resolves these against the owning
agent and stage. For bounds alone, you can use the
[built-in factory example](ezga-settings.md#optional-population-constraints)
without writing a custom module.
The predicate receives features, not a structure or dataset. Returning `True`
passes this predicate; the candidate must still pass the other applicable
validation checks before it is accepted.
All predicates must pass (`logic="all"`), with native short-circuit evaluation.
An empty list or `null` disables these additional checks. Explicit YAML and
HiSE stage settings replace predicates supplied through Python; an absent
setting preserves them. Invalid feature names, malformed feature rows,
predicate exceptions and non-Boolean returns fail the session with context.
Keep predicates deterministic functions of the features and factory settings:
SODA rebuilds them on resume, and does not checkpoint mutable closure state.
Keep your factory module available in the SODA environment when restarting.
Constraint rejects receive native `out_of_doe` operator penalties, so rejection
can change mutation/crossover weights as well as population membership.
Saved rejection records combine constraint and `ef_bounds` failures in the
`native_validation_failed` category; observable blacklist hits are identified
separately. See [Inspect rejected candidates](ezga-analysis.md#inspect-rejected-candidates)
for the record contents. Rejecting every proposal or result still allows
generation completion.
Initial seed admission is separate, and inbound migrants retain the native
hash-only admission policy. These checks therefore do not guarantee that every
retained seed or migrant satisfies the receiving agent's constraints.
Bayesian optimization (BO) target generation does not apply these predicates
directly; materialized candidates still pass through the supported validation
points.
Predicates run on the agent using existing feature rows; the extra computation
depends on your function. They add no worker arguments or matrix transfers,
and keep the existing output schema. Rejection metadata uses existing
generation records, and native rejection XYZ files remain suppressed.
Population constraints do not fix atoms during relaxation or populate the
worker calculator's `constraints` argument, which remains `None`.
Atom-selection predicates take
`predicate(atom_index, structure) -> bool`. Supply those through a receiving
factory's `kwargs.constraints` when it supports that argument.
## Integration boundaries
Enable optional fields such as tags or masses through
`[EZGA] sage_structure_fields` in the [settings guide](ezga-settings.md#soda-configuration-ezga).
Arbitrary attributes and private factory state are not automatically
transported or restored on restart. Extra structure fields do not add
calculator arguments. `generative.custom` is currently disabled.
The SODA service serializes agent-session calls within each process. Direct
Python integrations must likewise serialize engine construction and session
calls so feature-name mappings remain isolated.
These contracts target the pinned `ezga-lib==1.2.31` and
`sage_lib==0.1.9.29` versions in the
{repo}`native authority manifest `.
For integration details, see the
{repo}`configuration inventory ` and
{repo}`SAGE structure contract `.
Use [EZGA analysis](ezga-analysis.md) to inspect saved results.