Importing instrument data

Bringing instrument data into an experiment in Clarida means importing the Cq export your instrument software already produced. If that import is well annotated, that is all it takes to start downstream analyses. If the experiment was designed in Clarida, two extra steps link the Cq data to the pre-generated annotation and resolve any annotation conflicts. We describe what kind of data Clarida imports today and how the importer adapts to the workflow: a single step when going straight to analysis, two more when there is a design to reconcile against.

After a qPCR run finishes, the instrument software produces a table of quantification cycle (Cq) values, one per well or well-dye combination. The Import & annotate section brings that table into a Clarida experiment, where the data can be quality-controlled, normalized, and explored. The input is the export file the instrument software produces; the Cq values it contains are used as recorded.

This page describes what data the importer reads today, and why the import consists of one to three steps depending on the experiment. The terms used throughout — quantification cycle (Cq), dye, and the sample-type codes — follow the MIQE guidelines (Bustin et al., 2009) and the RDML data standard (Lefever et al., 2009), the reference vocabularies for reporting qPCR.

Clarida currently imports Cq export files: per-well tables of quantification cycle values, as produced by a qPCR instrument or its analysis software. The Cq values determined by that software are used directly; the raw amplification curves are not re-read.

This reflects the current scope rather than a final design. Reading instrument run files directly, and deriving Cq values from amplification curves within Clarida, is planned. Until then, importing the Cq export reuses the baseline, threshold, and amplification calls already made in the instrument software, and leaves those values unchanged.

The importer reads tabular exports — .xlsx, .csv, .txt, or .tsv — with one row per measured well, or per well-dye combination on a multiplex run. Two columns are required: the well position and the Cq value. The remaining columns are optional; when present, they are used to annotate the run or to match the import against an experiment's design.

★ Required. All other columns are optional.

Instrument vendors use no common export format, so header names are matched case-insensitively against a list of known aliases; a column labelled "Ct Value", for example, is recognised as Cq. When the headers in a file cannot be recognised automatically, the source is flagged and its columns are mapped by hand instead of the import failing. Most instrument exports are read without modification.

Two of the optional columns — Dye and Sample type — are matched against fixed sets of recognised values rather than taken verbatim. Both tolerate missing or unrecognised entries instead of failing the import.

The export reports a dye — the reporter on the probe or reaction, the term used by MIQE and RDML. Each dye corresponds to a detection channel on the instrument, and on a multiplex well that reports several dyes, each dye becomes its own measurement (its own row in the export).

The Dye value on each row is matched, case-insensitively, against Clarida's recognised channels: FAM, VIC, HEX, CY5, CY5.5, SYBR, ROX, TAMRA, and Texas Red.

A value that does not match one of these is not rejected: the measurement is still imported, but assigned to the default channel (SYBR) and flagged with a warning. When no Dye column is present at all, the run is treated as singleplex and every measurement is assigned to that same default channel.

The dye also resolves which assay part a measurement belongs to. For a singleplex assay there is a single part, so this is unambiguous; for a multiplex assay, each measurement is matched to the part that uses its dye.

The Sample type value classifies each well as a sample or a control. It is normalized case-insensitively to one of the eight RDML sample types, accepting common variant spellings:

These are the sample types defined by the RDML data standard, several of which (NTC, NAC, NRT) are reproducibility-relevant controls under MIQE. All eight are accepted on import and preserved. A value that matches none of them, or a row with no Sample type at all, is treated as Unknown, and an unmatched value is recorded as a warning.

The Clarida interface currently lets you view and set four of these directly — Unknown, Standard, NTC, and Positive. The other control types (NAC, NRT, NTP, OPT) are imported and retained, but for now display as Unknown in the plate view; surfacing them in the UI is planned. Sample type is not required for import, and wells left unclassified can be set later in the experiment.

The importer has three steps — Import, Link, and Reconcile — that address three separate questions:

• Import: what data is in the file? A file is not always a single dataset. An Excel workbook may contain one sheet per run, several unrelated sheets, or sheets that hold no Cq data. The import step reads each file, and each sheet, as a candidate source, and parses the ones that are selected.
• Link: which run does the data belong to? An imported source carries measurements but no record of which run produced them. Linking assigns each source to a run defined in the design, or promotes it to a new standalone run when no designed run applies.
• Reconcile: which annotation applies where they differ? When a source is linked to a designed run, two annotations of the same wells exist: the one defined in the design and the one carried in the imported file. Matching wells are resolved automatically; conflicting wells are resolved by an explicit choice between the imported and the designed annotation.

Linking and reconciliation only have something to act on when the experiment carries a design, so the number of steps that need attention depends on the experiment.

In an analysis-only experiment — created to process data, with no design — there is nothing to link or reconcile against. Sources are promoted to runs automatically and the file's annotation is used as recorded, so steps two and three complete without user action. The import is effectively a single step.

In an experiment designed in Clarida — plates laid out, samples and assays named — the imported data is checked against that design, and that is where linking and reconciliation earn their place. Because both are explicit, the imported file does not have to match the design exactly: sample names that differ, a workbook holding several runs, or a re-run that replaces earlier data are handled by assigning and reconciling sources rather than by demanding a fixed file format or naming scheme. Each action is reversible — a staged link can be cleared before it commits, and a merged run can be reset — so corrections do not require restarting the import.

After reconciliation is committed, the imported Cq values are attached to their runs, and the experiment proceeds to quality control, normalization, and exploration.