Understanding Autonomous Driving Datasets by Describing Differences between Image Subsets in Natural Language

Understanding the composition of large-scale autonomous driving datasets is essential for safety, robustness, and reliable operation across domains. For example, domain shift between locations could lead to the operating environment being misaligned with the training data, resulting in potentially dangerous performance degradation. Yet, existing data analysis pipelines largely rely on metadata, predefined labels, or manual inspection, which provide limited semantic insight or do not scale. This paper studies set difference captioning: given two subsets of images, the goal is to produce a natural-language hypothesis describing differences between the target and reference set. Building on a two-stage formulation, we adapt the method to autonomous driving by focusing on object-centric patches derived from object detection, which simplifies aggregation and enables attribution of differences to specific object instances or categories. To evaluate this setting in-domain, we introduce a new benchmark, AD-Diff Bench. Low-concentration experiments assess the suitability of set-difference-captioning approaches to sparse, real-world differences. We restrict our experiments to open-weight models to support reproducibility and ease of deployment. The proposed benchmark and analysis provide a step towards practical, human-interpretable dataset introspection for autonomous driving datasets. Our implementation and benchmark dataset are available at https://github.com/KIT-MRT/AD-Diff

Shortcut Learning in a Public Grape Disease Dataset: Annotation Granularity as a Modulator, Not a Cause

Public datasets for agricultural disease detection are usually judged fit for use from reported metrics, which say nothing about whether the annotation scheme is internally consistent. On one public grape disease dataset (3288 images, 11995 boxes, 6 classes), varying model capacity, input resolution and detection paradigm yields a test-set mAP50 range comparable to seed-to-seed noise, with the bottleneck at small objects across all five architectures. The finding lies on the data side: one class is annotated at whole-leaf level (median box area 43.16% of the image) while the other five are annotated at lesion level. On 5156 cross-species images containing no grape, 65.7% of the false-positive boxes fall into that one class, an over-representation of 13.41x relative to its share of the training annotations. Counterfactual retraining establishes a causal effect of granularity on the magnitude of the shortcut: shrinking only that class's boxes cuts its cross-species false positives by 66%, and a placebo control confirms the effect is specific to the manipulated class. A manipulation in the opposite direction, with criteria registered in advance, returns a negative result: coarsening the finest class to whole-leaf level (0.57% to 40.37%), matched in box count and share of annotations and with higher in-distribution AP, still leaves its cross-species false positives at zero boxes, while the unmanipulated original class holds 50.0% of them. Annotation granularity is therefore a modulator of this shortcut, not its cause: it can amplify or attenuate a sink that already exists, but cannot create one, and what fixes the destination remains open. We also give a granularity screening statistic requiring neither images nor training, and show airborne lesion-level detection to be optically out of reach. The failure mode is invisible to in-distribution evaluation.

Semantic Context Matters: Analysis of Color Names Across Domains

Color naming is influenced not only by physical color values but also by the semantic context in which colors are used. This paper investigates context-dependent color naming by mapping color-name datasets from Cosmetics, Crayola, and Car-color vocabularies onto the 86 fuzzy color categories of the COLIBRI color model. Contextual variation is analyzed using category coverage, Shannon entropy, and maximum lift. The results show that the three contexts occupy the COLIBRI color space differently: Cosmetics covers 48 of 86 fuzzy categories, Crayola covers 50, and Car colors cover 40. The results demonstrated that Crayola provides the broadest and most balanced use of the fuzzy color space, Cosmetics is mainly concentrated around warm-tone regions, and Car colors are more specialized around blue and achromatic regions. These findings show that color naming cannot be fully explained by numerical color similarity alone and that semantic context plays an important role in human color interpretation. The proposed framework supports the development of context-aware color models for design analytics, product search, recommendation systems, and human-centered artificial intelligence.