# Merge

Merge indexed reflection data from one or more indexing runs into a single set of structure-factor amplitudes using CrystFEL's `partialator`. The output is a merged `crystfel.hkl` file ready for export to downstream crystallography software.

## Inputs

- One or more `.stream` files from indexing runs (`indexingintegration_*` or `ici_*` run folders)
- A unit cell (inferred automatically from the stream header or the matching indexing run folder)

## Using the GUI

Open the **Merge** window from the toolbar or menu. The window has three tabs:

| Tab | Contents |
|---|---|
| **Select Run** | Choose an indexing run, view its stream files, and combine them. |
| **Settings** | Configure Partialator parameters, start the merge, and inspect live and post-merge results. |
| **Compare** | Side-by-side comparison of all completed merge runs. |

### Select Run tab

The left panel shows a dropdown listing all discovered indexing runs (`indexingintegration_*` and `ici_*`). Select a run to see its stream files.

| Button | Effect |
|---|---|
| **Refresh** | Re-scan for indexing runs. |
| **Combine →** | Concatenate all streams from the selected run into a new `<stem>_merge_<timestamp>/` folder; the combined stream is added to the Settings tab. |
| **Latest All →** | Combine the most recent mergeable stream from each INI in the workspace into one combined stream. |
| **Use Selected** | Load a previously combined stream from the right panel and switch to the Settings tab. |

The right panel (**Combined Streams**) lists all existing `<stem>_merge_*` folders and their stream files.

### Settings tab

The top of the tab shows the active stream (dropdown), the **inferred cell parameters** parsed from the stream header or matched indexing run, and — when a workspace-refined unit cell is available — a **Cell for merge analysis** selector to choose between the indexing cell and the workspace refined cell.

Parameters are on the right panel:

| Parameter | Default | Description |
|---|---|---|
| Point Group | — | Symmetry point group for merging (`-y`). |
| Unique Axis | — | Unique axis suffix for point groups where CrystFEL accepts one. |
| Threads | cpu_count | Parallel threads (`-j`). |
| Min Resolution | 5.0 Å | Per-crystal resolution sanity filter; tick **∞** to include all crystals. |
| Iterations | 25 | Number of scaling/merging cycles. |
| Model | offset | Partiality model (`offset`, `xsphere`, `unity`, `ggpm`). |
| Min Measurements | 5 | Minimum observations per reflection. |
| Max ADU | ∞ | Saturation cutoff; untick **∞** to set a numeric limit. |
| Push Resolution | 0 | Extend merging beyond the observed resolution. |
| Use B-scaling | off | When unchecked, passes `--no-Bscale` to Partialator. |
| Save Partialator Diagnostic Logs | off | Write per-crystal diagnostic files to `pr-logs/`. Not needed for normal merging or export. |

> Polarisation is fixed to `none` for electron diffraction and is not user-configurable. COSEDA always asks Partialator to write intermediate HKL files after each cycle.

**Buttons:**
- **Save Settings** — persists settings to the `.ini` and a JSON file next to the combined stream.
- **Merge** — creates a `merge-run<N>/` subfolder in the combined stream's folder and starts Partialator. If the stream has been merged before, COSEDA clones it into a fresh timestamped folder first so previous results are preserved.
- **Abort** — terminates the running Partialator process.

The inner panel shows three sub-tabs that update live as the merge runs:

**Merging sub-tab** — scaling table updated after each Partialator cycle:

| Column | Description |
|---|---|
| Overall CChalf (%) | Per-cycle overall CC½. |
| Reflections | Number of reflections used. |
| deltaCChalf | Change in CC½ from the previous cycle. |
| deltaCChalf Error (%) | Uncertainty on ΔCC½. |
| Bad Crystals | Crystals rejected this cycle. |
| OK Crystals | Crystals accepted this cycle. |
| Negative delta CC½ | Count with negative ΔCC½. |

Click any column header to select it for plotting (up to two columns, dual y-axis).

**Statistics sub-tab** — resolution shell statistics from `check_hkl`, populated automatically after the merge finishes:

Center 1/nm · # refs · Possible · Compl (%) · Meas · Red · SNR · Mean I · d(Å) · Min 1/nm · Max 1/nm

Click selectable metric headers to plot vs. resolution.

**Split Half sub-tab** — split-half statistics from `compare_hkl` (requires `crystfel.hkl1`/`crystfel.hkl2` output):

d(Å) · CC1/2 · CC* · Rsplit (%) · # refs · Center 1/nm · Min 1/nm · Max 1/nm

A **Console Output** panel at the bottom streams live Partialator output. Use **Clear** to reset or **Save Log…** to write it to disk.

### Compare tab

Shows all completed merge runs in a summary table:

Run · Point Group · Input Crystals · Used Crystals · CC1/2 Final (%) · Compl ~2Å (%) · Red ~2Å · SNR ~2Å · CC1/2 Split ~2Å · CC* ~2Å · Rsplit ~2Å · Min Res · Push Res · Min Meas · Model · Iterations · Max ADU · B-scale · Partialator Logs · Output

Select one or more rows and choose a metric to compare their resolution-shell curves in the plot below. Use **Refresh** to re-scan for completed runs.

## Outputs

Each merge run creates a `merge-run<N>/` subfolder inside the combined stream's folder:

- **`crystfel.hkl`** — merged reflection intensities in CrystFEL format.
- **`crystfel.hkl1`**, **`crystfel.hkl2`** — final split-half HKL files written by Partialator.
- **`harvest.json`** — detailed per-cycle Partialator statistics.
- **`stdout.log`** — full Partialator console output.
- **`shells.dat`** — resolution shell statistics from `check_hkl`.
- **`compare_hkl_CC.dat`**, **`compare_hkl_CCstar.dat`**, **`compare_hkl_Rsplit.dat`** — split-half shell files from `compare_hkl`.
- **`merging_table.csv`**, **`merging_plot.png`** — exported Merging sub-tab data.
- **`statistics_table.csv`**, **`statistics_plot.png`** — exported Statistics sub-tab data.
- **`split_half_table.csv`**, **`split_half_plot.png`** — exported Split Half sub-tab data.
- **`pr-logs/`** — Partialator per-crystal diagnostic files (only when **Save Partialator Diagnostic Logs** is enabled).
- **Settings JSON** (`<stem>_merge_settings.json`) — full parameter snapshot next to the combined stream.
- **NXprocess metadata** — written to the source HDF5 files.

## Tips

- **Point group first.** Set the correct symmetry before merging. An incorrect point group will produce misleading statistics.
- **Trigonal notation.** CrystFEL does not accept bare trigonal symbols such as `32`; use explicit forms such as `321_H`, `312_H`, or `32_R`.
- **Convergence.** Watch Overall CChalf stabilize across cycles in the Merging sub-tab. If it keeps changing, increase the iteration count.
- **Re-running.** Clicking **Merge** on a stream that already has results clones it to a fresh timestamped folder, so all previous results are preserved.
- **Multiple datasets.** Use **Latest All →** to combine the best stream from each loaded dataset into a single merge for improved completeness.
- **Workspace refined cell.** If a unit cell fit has been run, the Cell source selector appears in the Settings tab, letting you use the refined cell for `check_hkl` and `compare_hkl` statistics without re-merging.