Working with data

A CORNETO analysis brings together a biological network and experimental observations. The network describes what can interact; the observations describe what was measured in one or more samples or experimental conditions.

CORNETO represents these observations with the Data class. A Data object keeps related measurements together, records which sample they belong to, and provides a common data model for CORNETO methods.

import corneto as cn

Samples and features

A Data object contains named samples. Each sample contains features, such as measured proteins, genes, metabolites, or phenotypes. Every feature has three common fields:

• id identifies the measured entity.

• value stores its observed value.

• mapping says whether the feature corresponds to a network vertex, a network edge, or is currently mapped to none.

Features can also carry information such as the assay, units, or a biological role. These additional fields are kept with the feature. Their meaning depends on the method that uses the data, so each method documents the fields it requires.

Creating a dataset

Here we record measurements from untreated cells and cells stimulated with EGF. Protein measurements map to vertices in a network. Cell viability is retained as part of the experiment but is not mapped to the network.

observations = {
    "untreated": {
        "EGFR": {"value": 0.18, "mapping": "vertex"},
        "MAPK1": {"value": 0.22, "mapping": "vertex"},
        "cell_viability": {
            "value": 0.96, "mapping": "none", "assay": "phenotype"
        },
    },
    "EGF": {
        "EGFR": {"value": 0.91, "mapping": "vertex"},
        "MAPK1": {"value": 0.84, "mapping": "vertex"},
        "cell_viability": {
            "value": 0.94, "mapping": "none", "assay": "phenotype"
        },
    },
}

data = cn.Data.from_cdict(observations)
data

Data(n_samples=2, n_feats=[3 3])

The sample names are preserved, and each sample contains its own feature values.

list(data.samples)

['untreated', 'EGF']

data.samples["EGF"].features

[Feature(id=EGFR, value=0.91, mapping=vertex),
 Feature(id=MAPK1, value=0.84, mapping=vertex),
 Feature(id=cell_viability, value=0.94, mapping=none, assay=phenotype)]

Querying measurements

Queries make it possible to select features without changing the original dataset. For example, we can retain only measurements that map to network vertices.

network_measurements = (
    data.query.filter_features(
        lambda feature: feature.mapping == "vertex"
    ).collect()
)
network_measurements

Data(n_samples=2, n_feats=[2 2])

Copying and extending data

Use copy() when you want to modify a dataset while keeping the original unchanged. Features can then be added to an individual sample.

extended_data = data.copy()
extended_data.samples["EGF"].add(
    cn.Feature(
        id="AKT1",
        value=0.73,
        mapping="vertex",
        assay="phosphoproteomics",
    )
)

len(data.samples["EGF"].features), len(extended_data.samples["EGF"].features)

(3, 4)

Saving and loading data

A Data object can be saved and loaded without losing sample names, feature values, mappings, or additional annotations.

from pathlib import Path
from tempfile import TemporaryDirectory

with TemporaryDirectory() as directory:
    data_path = Path(directory) / "experiment.json.xz"
    data.save(data_path)
    restored_data = cn.Data.load(data_path)

restored_data.to_dict() == data.to_dict()

True

Using data with methods

Data is the general representation used by CORNETO methods. If your observations are already stored as a Data object, pass them to the method through build_from_data:

problem = method.build_from_data(network, data)

For common analyses, methods also provide shorter interfaces. build accepts the scientific inputs for one condition, while build_many accepts inputs for several named conditions:

problem = method.build(network, ...)
problem = method.build_many(network, ...)

These convenience methods validate their inputs, create the corresponding Data object, and then build the optimization problem. The biological meaning and names of the inputs differ between methods, so they are explained in each method’s guide.