Managing datasets | BioLM Docs

Local inventory for protocol results, finetuning inputs, and related artifacts.

A dataset in the BioLM SDK is a self-describing directory: a dataset.yaml plus whatever files you put beside it (usually under data/). Datasets are discovered under ~/.biolm/datasets and ./.biolm/datasets (and optional extra roots). They are addressable by id so other workflows can refer to them later. Open sequence tables with Working with SeqFrame (Dataset.open_seqframe() / SeqFrame.from_dataset() when the dataset holds SeqFrame Parquet).

Optional backends such as MLflow only extend push / pull. Local create, list, show, and add work with no extras installed.

Layout and discovery

text

~/.biolm/datasets/finetuning-v1/
├── dataset.yaml
└── data/
    └── train.csv

biolm dataset create writes that layout for you. biolm dataset init PATH drops a dataset.yaml into an existing directory without moving files. Schema details live in Dataset Schema Reference.

Creating and adopting datasets

bash

biolm dataset create finetuning-v1 --tag finetune
biolm dataset init ./training-data --id finetuning-v1 --tag finetune

Add files:

bash

biolm dataset add finetuning-v1 train.csv
biolm dataset add finetuning-v1 ./more-data --recursive

Listing and inspecting

bash

biolm dataset list
biolm dataset list --type files --tag finetune
biolm dataset show finetuning-v1
biolm dataset show ./training-data

Python API

python

from biolm.datasets import DatasetClient

client = DatasetClient()
ds = client.create("finetuning-v1", tags=["finetune"])
ds.add("train.csv")
for item in client.list(tag="finetune"):
    print(item.id, item.path)

Push and pull (optional backends)

Sync with a remote backend when you need shared storage. MLflow requires the optional extra and platform login:

bash

pip install "biolm-sdk[mlflow]"
biolm account login

biolm dataset push finetuning-v1 --backend mlflow
biolm dataset pull finetuning-v1 --backend mlflow
biolm dataset pull finetuning-v1 --backend mlflow --path ./my-copy

pull defaults to ~/.biolm/datasets//.

Workspaces, datasets, and volumes

A workspace is account/environment context ({account}/{environment}).
A dataset is a local (or pushed) bag of files with dataset.yaml.
A runtime volume is server-side storage for Jupyter/protocol runs — not a local SDK storage API.

Use biolm workspace for platform context and biolm dataset for local artifact inventory (with optional remote sync).

Dataset next steps

Dataset Schema Reference — dataset.yaml field reference.
Working with biological data — reading sequences and structures from files.
Protocol Workflows — protocol inputs/outputs that pair with datasets.
biolm dataset — full biolm dataset command reference.