# Indexing & integration

Index diffraction patterns and integrate reflection intensities using CrystFEL's `indexamajig`. COSEDA wraps this into a GUI that manages geometry and cell definitions, run folders, and live metrics — so you can monitor hit rates and indexing rates as the job progresses.

> This page covers the standard **Simple Indexamajig** workflow. For the experimental iterative center-shift refinement loop, see [ICI Orchestrator](7b_ici_orchestrator.md).

> This stage requires center finding and refinement to have run first (`center_x`, `center_y`, `det_shift_x_mm`, `det_shift_y_mm` must exist in the HDF5).

## Inputs

- COSEDA working file (`<dataset>.h5`) with:
  - `entry/data/images`
  - `entry/data/center_x`, `center_y` (from center finding / refinement)
  - `entry/data/det_shift_x_mm`, `det_shift_y_mm` (used for geometry file generation)
  - optional `entry/data/index` (if frames were stripped)
- Dataset `.ini` with acquisition details and processing parameters
- A **cell file** (`.cell`) describing the expected unit cell. CrystFEL accepts `rhombohedral` or `hexagonal` lattice types for trigonal cells; do not use `trigonal` as the `lattice_type`.
- A **geometry file** (`.geom`) describing the detector layout

## Parameters

### Threading

| Parameter | Default | Description |
|---|---|---|
| Worker threads | 12 | Number of parallel indexamajig workers (`-j`). |
| Max indexer threads | 1 | Fixed internally for Simple Indexamajig. Use worker threads to control parallelism. |

### Resolution & integration

| Parameter | Default | Description |
|---|---|---|
| Push resolution | 0.5 | Integrate beyond the apparent resolution of each pattern (1/nm). |
| Integration radii | 4,5,8 | Radii for the three integration rings: inner, middle, outer (px). |

### Peak selection

| Parameter | Default | Description |
|---|---|---|
| Min peaks | 25 | Minimum peaks for a frame to be attempted. |

### Tolerances

| Parameter | Default | Description |
|---|---|---|
| Tolerance | 5,5,5,1.5 | Lattice parameter tolerances (a, b, c in %, angles in °). |
| XGandalf tolerance | 0.02 | Relative tolerance of lattice vectors (2%). |

### XGandalf

| Parameter | Default | Description |
|---|---|---|
| Sampling pitch | 5 | Sampling density of reciprocal space (0–7, higher = finer). |
| Gradient descent iterations | 2 | Refinement iterations (0–5). |
| Min lattice vector length | 3 Å | Shortest allowed lattice vector. |
| Max lattice vector length | 30 Å | Longest allowed lattice vector. |

### Optional flags

| Flag | Default | Effect |
|---|---|---|
| Use fixed profile radius | off | Lock integration to a fixed profile radius (requires a value). |
| No non-hits in stream | on | Exclude non-indexed patterns from the output stream. |
| No revalidate | on | Skip filtering peaks too close to detector edge, double peaks, or saturated pixels. |
| No check peaks | on | Disable peak sanity checks when reading peaks. |
| No half-pixel shift | on | Disable half-pixel offset when reading images. |
| No retry | on | Do not retry failed frames with weakest reflections removed. |
| No check cell | on | Skip unit cell comparison against reference cell. |
| No refine | on | Disable post-indexing unit cell refinement. |
| Overpredict | on | Predict reflections beyond the observed peaks. |

## Using the GUI

Open the **Indexing & Integration** window from the toolbar or menu.

The window has a **workspace tree** on the left (showing datasets and their run history) and a **method selector** with **tabbed panel** on the right. Select **Simple Indexamajig** from the method drop-down to use the standard workflow.

| Tab | Contents |
|---|---|
| **Settings** | Indexing parameters, method selector, flags |
| **Cell File** | Unit cell editor — load, edit, and save `.cell` files |
| **Geom File** | Geometry editor — load, edit, and save `.geom` files |
| **Run** | Start/stop controls, live metrics, progress bar |

1. **Select a dataset** from the workspace tree. The tabs populate with any previously saved settings.
2. **Prepare cell & geometry files.** Use the **Cell File** and **Geom File** tabs to load or edit existing files, or create new ones.
3. **Configure parameters** in the **Settings** tab. Click **Save Settings** to persist them to the `.ini` and make them available for batch runs.
4. **Start a run** from the **Run** tab:
   - **Start Current** — creates a timestamped run folder and starts indexamajig for the selected dataset.
   - **Start All in Batch** — runs all workspace datasets sequentially, each in its own timestamped folder.
5. **Monitor progress.** The Run tab shows live metrics:
   - Hit rate (%)
   - Indexing rate (%)
   - Throughput (frames/s)
   - Indexed frame count
   - Progress bar
6. **Stop** at any time. Partial results are preserved in the run folder.


## Outputs

- **Stream file:** `indexingintegration_<timestamp>/<run_name>.stream` — CrystFEL stream with indexed reflections.
- **Settings snapshot:** `index_settings.json` in the run folder — full parameter dump for reproducibility.
- **NXprocess metadata:** Written into the source HDF5 files documenting the indexing step.
- **Run history:** Each run appears in the workspace tree for quick re-inspection or restart.

## Tips

- **Cell file first.** The indexer needs a cell file to know what lattice to look for. If you do not have one, use the cell file editor to define the expected unit cell.
- **Start conservative.** Begin with a single dataset and review hit/indexing rates before running the full batch.
- **Sampling pitch.** Higher values (6–7) give finer sampling but are slower. Start at 5 and increase if indexing rates are low.
- **Tolerances.** If indexing rates are very low, try relaxing the general tolerance or increasing the XGandalf tolerance.
- **Run folders.** Each run is timestamped and self-contained. You can compare different parameter sets by inspecting their run folders side by side.