This paper explores the potential use of a social robot as tutor and playing partner in a primary school mathematics classroom. Our approach is using a math card game, based on the understanding of fractions, to study the pupil-robot interaction. We follow an iterative process consisting of three consecutive phases: (1) data collection involving a human playing partner, (2) a remotely controlled social robot as tutor and playing partner, and (3) a fully autonomous robotic tutor and playing partner. We present data collected from the phase 1 and 2 trials involving a total of 13 pupils (ages 10–12). The lessons learned from the first two phases are presented, as well as how these lessons can help inform future steps towards meeting the challenges presented by an autonomous robotic playing partner (phase 3).