Author

Amorocho, José ; Balsa, Ana ; Giraldo-Huertas, Juan José ; Bloomfield, Juanita ; Patrone, Paula ; Cid, Alejandro

Date

2026

Extensión

31 p.

Abstract

Observational coding of parent–child interactions is a gold standard in developmental science but remains unscalable. Multimodal generative AI could help, yet its reliability and failure modes are not well characterized. We benchmarked a multi-agent pipeline (GABRIEL) against a multi-rater expert consensus when scoring 22 PICCOLO items on 156 ten-minute free-play interactions from Uruguay. Agreement was summarized with Percent Agreement (PA) and Cohen’s κ, and disagreement with a unit-free normalized mean squared error (nMSE = MSE/Var(Yitem)). A priori item classes indexed cognitive inference (Low/Medium/High). Final calibration yielded modest agreement (PA = 50.7%, κ = .216). Disagreement was chiefly structured by inference (Kruskal–Wallis H = 308.70, p<.001), a pattern that persisted in late iterations. The model also overused the middle category (1) and underused “2.” No systematic differences in nMSE emerged by sex, age quartile, or maternal education. We conclude that generative AI is promising for scalable detection of concrete, low-inference behaviors, whereas high-inference judgments still require expert adjudication. A human-in-the-loop, co-intelligence workflow aligns current strengths with ethical oversight and supports equitable deployment at scale.

Keyword

Generative AI Parent-Child Interaction Observational Coding Inter-Rater Reliability PICCOLO Multimodal AI

Tipo

Artículo

Derechos

Abierto

Versión

Aceptada

Collections

• Artículos [21]