Configuring Semantic Segmentation Experiments in One YAML File with rplatypus and pyplatypus
The previous post took one trained model and one dataset and went looking for the places it fails. This one zooms out to the whole toolkit: rplatypus in R and pyplatypus in Python, both built for semantic segmentation, where every pixel gets a class and the score is overlap. The API isn't the hard part. Choosing among the options is.
Most of what follows is about training, even though the code here never trains anything. To run on any laptop, the examples only read configurations and build networks, using the four nuclei images from the first post. The scores come from full training runs of the same configurations.
One file holds the experiment
The cleanest way to see the toolkit is the file both packages read. One YAML file describes an entire training run:
nuclei.yaml
task: semantic_segmentation
data:
train_path: nuclei
validation_path: nuclei
mode: nested_dirs
colormap:
- [0, 0, 0] # background
- [255, 255, 255] # nucleus
models:
- name: unet
architecture: u_net
input_shape: [256, 256]
blocks: 4
filters: 16
loss:
name: cce_dice
metrics:
- name: dice
include_background: false
optimizer:
name: adam
learning_rate: 0.001
callbacks:
- name: early_stopping
monitor: val_dice
patience: 5
augmentation:
- name: HorizontalFlip
params: {p: 0.5}
- name: RandomRotate90
params: {p: 0.5}
epochs: 30
batch_size: 8Read it top to bottom. The task line decides what else the file is allowed to contain. Semantic segmentation is one of four tasks; classification, instance segmentation and object detection each get their own posts.
The data block says where the images live and how to read them:
train_pathandvalidation_pathpoint to the training and validation data.modedescribes the layout.nested_dirsexpects one directory per image, each withimages/andmasks/inside.config_fileexpects a CSV listing the files, which is whatsplit_dataset()writes.colormapmaps mask colours to classes, one entry per class, background first.
models is a list, so a single file can train several models on the same data and compare them. Each entry needs a name, an architecture and an input_shape. Two optional settings matter most: blocks sets the depth, and filters sets the width of the first block, which doubles at each level down.
weights and fit are absent here because this run starts from scratch. weights names a checkpoint, either a published name from available_weights(), a Hugging Face reference or a local file, and training starts from those values instead of random ones. fit: false skips training entirely. The weights load, and the model goes straight to predict() and evaluate(). That's how the last post scored the published nuclei model.
The remaining blocks (loss, metrics, optimizer, callbacks, augmentation) each take a name plus that component's own arguments. Every name comes from a catalogue the package can print, and that catalogue is most of what's left in this post.
Reading the file produces the same object as building it from arguments. Here it is in both languages:
# R
library(rplatypus)
spec <- platypus_spec("nuclei.yaml")
spec
#> platypus specification
#> from : nuclei.yaml
#> data : nuclei
#> classes : 2
#> models : 1
#> unet u_net 256x256, cce_dice, 30 epochs
# Python
import pyplatypus as pp
spec = pp.from_yaml("nuclei.yaml")
[(m.name, m.architecture.value, m.input_shape) for m in spec.models]
#> [('unet', 'u_net', (256, 256))]The file gets checked before a single image is read or a network is built. A typo costs you a second, not an epoch. Ask for something the architecture can't do and you get refused, with the offending field named:
# R
platypus_spec(
data = segmentation_data("nuclei", "nuclei", colormap = binary_colormap),
models = list(u_net("x", input_shape = c(256, 256),
deep_supervision = TRUE))
)
#> Error in `platypus_spec()`:
#> ! The specification has a problem:
#> * models[[1]]: deep_supervision requires architecture 'u_net_plus_plus'; 'u_net' has no intermediate outputs to supervise
# Python
pp.from_dict({
"task": "semantic_segmentation",
"data": {"train_path": "nuclei", "validation_path": "nuclei",
"colormap": [[0, 0, 0], [255, 255, 255]]},
"models": [{"name": "x", "architecture": "u_net",
"input_shape": [256, 256], "deep_supervision": True}],
})
#> ConfigError: The configuration has 1 problem:
#> - models[0]: deep_supervision requires architecture 'u_net_plus_plus';
#> 'u_net' has no intermediate outputs to superviseFour architectures
The package ships four segmentation architectures:
# R
available_models(task = "semantic_segmentation")$name
#> [1] "u_net" "u_net_plus_plus" "res_u_net" "linknet"
# Python
[m["name"] for m in pp.available_models("semantic_segmentation")]
#> ['u_net', 'u_net_plus_plus', 'res_u_net', 'linknet']Under the hood they share one design. Each is a U: an encoder that halves the image and doubles the feature count blocks times, then a decoder that climbs back up, with skip connections carrying each level's detail across. What separates them is what sits inside each block and how the skips reach the decoder.
Blue blocks are the encoder, green are the decoder, and each arrow is a connection. The input is a stained tissue image from the same Data Science Bowl collection, and the output is its ground-truth mask.
U-Net
This is the plain U. Each skip connection is concatenated onto the decoder's features, so a decoder block sees both what came up from below and what the encoder saw at its own scale. If you're new to this, start here.
U-Net++
The gap between encoder and decoder fills up with nested blocks. Each one takes every earlier block in its row plus the block diagonally below, so the top row, which is the decoder, connects to the encoder at every depth rather than just one. It's also the only one of the four that accepts deep_supervision, which adds a loss on each block in the top row. That's the setting the refusal above was about.
Res-U-Net
Same U as before, but with residual blocks. Each block's input skips over its two layers and gets added back at the end. The layers only have to learn a correction instead of a full new representation. The usual reason to try it is depth: when adding more blocks stops helping, this is the next thing to test.
LinkNet
Residual blocks in the encoder, and skip connections that get added to the decoder's features rather than concatenated onto them. Fewer channels reach each decoder block, which makes the decoder lighter. LinkNet is pitched on memory and speed, not accuracy.
Cost at the same depth and width is quick to measure. The cheapest way to build all four is one epoch on four small images:
# R
one_epoch <- function(architecture, name) {
architecture(name, input_shape = c(64, 64), blocks = 4, filters = 16,
epochs = 1, batch_size = 2)
}
four <- platypus_spec(
data = segmentation_data("nuclei", "nuclei", colormap = binary_colormap),
models = list(one_epoch(u_net, "u_net"),
one_epoch(u_net_plus_plus, "u_net_plus_plus"),
one_epoch(res_u_net, "res_u_net"),
one_epoch(linknet, "linknet"))
)
built <- platypus_fit(four, verbose = FALSE)
evaluate(built)[, c("model", "parameters")]
#> model parameters
#> 1 u_net 1942594
#> 2 u_net_plus_plus 2263730
#> 3 res_u_net 2029682
#> 4 linknet 1746754
# Python
def one_epoch(architecture):
return {"name": architecture, "architecture": architecture,
"input_shape": [64, 64], "blocks": 4, "filters": 16,
"epochs": 1, "batch_size": 2}
engine = pp.build_engine(pp.from_dict({
"task": "semantic_segmentation",
"data": {"train_path": "nuclei", "validation_path": "nuclei",
"colormap": [[0, 0, 0], [255, 255, 255]]},
"models": [one_epoch(a) for a in
["u_net", "u_net_plus_plus", "res_u_net", "linknet"]],
}))
engine.fit()
[(row["model"], row["parameters"]) for row in engine.evaluate()]
#> [('u_net', 1942594), ('u_net_plus_plus', 2263730),
#> ('res_u_net', 2029682), ('linknet', 1746754)]LinkNet comes out about 10% smaller than U-Net here, not half. At 16 filters and four blocks, the shared skeleton accounts for most of the parameters, so the lighter decoder saves only a sliver.
Which one wins
When the nuclei weights were published, all four architectures were trained on the same split of the 2018 Data Science Bowl images, with the same specification, for 60 epochs:
| architecture | parameters | mean Dice |
|---|---|---|
| U-Net | 1,942,594 | 0.9217 |
| U-Net++ | 2,263,730 | 0.9212 |
| LinkNet | 1,746,754 | 0.9203 |
| Res-U-Net | 2,029,682 | 0.9187 |
The four means sit just 0.0030 apart. Within any single architecture, the scores across individual images spread by 0.054. Retraining the same U-Net with a different random seed moved its mean by 0.0012, which is 40% of the entire gap between architectures.
So on this dataset, the architecture isn't where the score comes from. That's also why only one set of weights was published instead of four. Other datasets will behave differently, but the lesson transfers: start with a plain U-Net and look for gains in the data, the loss or the resolution before you swap the network.
Losses and metrics
Once the network is chosen, the decisions multiply. The loss function and the metrics are where most of the remaining options live. The catalogue for segmentation currently offers eleven losses and five metrics:
# R
available_losses(task = "semantic_segmentation")$name
#> [1] "boundary" "cce" "cce_dice"
#> [4] "combo" "dice" "focal"
#> [7] "focal_tversky" "generalized_dice" "iou"
#> [10] "lovasz" "tversky"
available_metrics(task = "semantic_segmentation")$name
#> [1] "cldice" "dice" "iou" "surface_dice"
#> [5] "tversky"
