Remove empty frames

Purpose. Reduce dataset size by permanently removing frames with too few detected peaks. This frees disk space and speeds up all downstream processing (center finding, indexing, merging).

Always keep backups of your raw data! Stripping is irreversible — make sure your peak-finding settings gave sensible results before running this step.

How it works

  1. Classify frames. Each frame’s nPeaks value is compared against a threshold. Frames below the threshold are marked for removal.

  2. Build an index mapping. An index dataset (same length as the original frame count) is created, filled with -1 for stripped frames and the new compact index for kept frames. An image_key dataset maps compact indices back to their original frame positions.

  3. Reduce image-indexed datasets. All datasets tied to the compact image stack (images, nPeaks, peakTotalIntensity, peakXPosRaw, peakYPosRaw, center_x/y, etc.) are rewritten with only the kept frames.

  4. Preserve dense frame metadata. Full-length per-original-frame datasets are copied unchanged. This includes stage-position streams (stagepos_*, stage_position*), tilt angles, frame intensity datasets (frame_mean_intensities, frame_total_intensities, frame_radial_intensities, plus legacy intensity datasets), streak labels, and the detector mask.

  5. Rebuild NeXus views. NXdata wrapper groups (peaks, peak_counts, image_data) are regenerated with correct axis dimensions pointing to the reduced datasets.

  6. Write to a new file. The reduced data is written directly to a new HDF5 file (no copy-then-delete), so the output file size reflects only the kept data. The original file is kept as a .backup until the process completes successfully, then deleted.

Because stage positions, streak labels, frame intensities and radial intensity profiles full-length, the full atlas can still be regenerated after stripping. Existing saved atlases are copied through as well.

Safety checks

  • Disk space. The function estimates the output file size and refuses to proceed if insufficient space is available.

  • Force flag. If more than 90% of frames would be removed, COSEDA aborts and asks you to review your peak-finding results. Set the Force flag to override this check when you are confident the settings are correct.

Using the GUI

Open Remove Empty Frames from the toolbar or menu.

Controls

Control

Description

Mode

Current file only or All files in workspace.

Minimum Peaks per Frame

Frames with fewer peaks are removed.

Only remove frames with 0 peaks

When checked, the threshold is locked to 1 (removes truly empty frames only).

Force even if many frames are removed

Override the 90% safety check. A confirmation dialog appears when combined with batch mode.

Start Stripping

Writes strip settings into the .ini(s), then starts the stripping process.

Abort and Restore Backup

Stops the current operation and restores the .backup file for the last processed dataset.

Progress

  • Single file: A progress bar shows frame-by-frame progress.

  • Batch mode: Two progress bars appear — Overall (file X of Y) and Current file (frame progress within that file). The main window status bar shows “Stripping empty frames…” with an animated ellipsis.

  • On completion, a summary shows how much storage was freed.

Parameters (INI [Parameters])

  • strip_threshold — minimum nPeaks value to keep a frame

  • strip_force — True to override the 90% safety check

CLI

from coseda.initialize import write_stripsettings
from coseda.pipeline.strip_h5 import strip_h5_batch, strip_h5

# Batch: write settings, then strip
ini = "/path/to/dataset.ini"
write_stripsettings(ini, strip_threshold=5, strip_force=False)
strip_h5_batch(ini)

# Single file (standalone)
strip_h5("/path/to/dataset.h5", threshold=20, force=False)

Tips

  • Check peak finding first. If your threshold is set too high or peak finding was too aggressive, you may discard useful data.

  • Batch stripping. Use All files in workspace to strip every dataset in one go. Files can be in different directories.

  • Re-stripping. You can strip an already-stripped file again with a higher threshold. The index mapping and all image-indexed datasets are updated correctly.

  • Frame intensities. If intensity datasets are missing, COSEDA will offer to calculate them (for the current file or all workspace files) when you next open a dataset.

  • Atlas safety. Stripping removes image frames, not the dense stage/intensity streams used by the Atlas Viewer, so atlas reconstruction should give the same result before and after stripping.