Cognitive and Normative Features of a “Knowledge-Based Regulation Tool”: The Case of OECD’s PISA

Printable version of the article  
Send this article by mail  

OECD’s PISA is a knowledge-based and knowledge-generating device that combines technical and social components, takes part in establishing regulations for the coordination and control of public action in education, operates several core categories of schooling, and to a certain extent rewrites the educational model for contemporary societies.

PISA is more than a new type of international comparative assessment created in order to assist policy-makers with “robust” data and information, for us, it is a knowledge-policy instrument (Carvalho, 2009). The Programme serves as a site for the continued interpretations of ‘reality’ and ‘appropriate agency.’ The logic through which students and education systems are described, compared and assessed, displays cognitive and normative frames for acting in the educational sector. More specifically, the discourse emanating from PISA redefines students (as life-long learners), as well as redefines the relationship between teaching and learning, and school and knowledge. It also sustains particular cultural dicta about contemporary policy-makers and policy-making.

It is clear to us that the indicators do not result only from technical choice, but are also socially constructed by an agency of elite experts and policy-makers. Their success depends also on the ability to make compatible the specific demands of these universes (Morgan, 2007). And, it is equally clear to us that the purpose of these indicators is not merely functional, as the criteria and indicators through which national societies and education systems are described, compared and assessed operate on an ‘ontological’ level and generate epistemic communities (Alasuutari, 2005). According to Popkewitz (2000: viii) they can be understood as “discursive practices whose distinctions and categories order the world, define the places of individuals, and establish rules and standards through which expression and practices are made possible.”

Literacy and Competence: redefining categories of schooling

PISA is not only a device for comparative monitoring, it also establishes an outline for a different way of looking at education and schooling, and adapting to the needs of preparing the workforce for integration into a “knowledge-based economy.” Central items in this approach are the concepts of literacy and competency, or perhaps better said, the notion of competencies of literacy: “… the capacity of young adults to access, manage, integrate and evaluate information, to think imaginatively, to hypothesise and discover, and to communicate their thoughts and ideas effectively” (Schleicher (2007: 3) [1].

PISA undertakes, therefore, to assess how the pupils can use their reading skills to learn and interpret the different kinds of written materials they come across every day; the way they implement their mathematical knowledge and skills to solve the mathematical challenges and problems, and also the way they use their scientific knowledge and skills to understand, decipher and solve scientific situations and challenges (OECD, 1999, 2003, Schleicher, 2006: 35, see also OECD/PISA website).

Thus, the “competences of literacy” framework does not only generate scripts of knowledge but it determines the characteristics and (expected) modes of action, repertoires of behaviours; hence, it takes part in fabricating human beings for an imagined global society. The political stakeholders of PISA seem to share this hope about the diagnostic and prognostic power of the PISA tool, as they intend that it provides “proof” that it is a valid tool to measure the quality of preparation for adulthood [2].

The pupils are registered in a broader category that groups them and redefines them – lifelong learners and the same thing happens as regards school learning (this broadened in itself – including the so-called ‘transversal competencies’): “Students cannot learn everything they need to know in school. In order to be effective lifelong learners, young people need not only knowledge and skills, but also an awareness of why and how they learn. PISA measures both student performance in reading, mathematics and science literacy and also asks students about their motivations, beliefs about themselves and learning strategies” (OECD/PISA website, see also Schleicher, 2006: 31, OECD, s.d.: 7).

This redefinition of the school learners – and simultaneously what is important to know about them – goes hand in hand with their placement in a broader category – that of a life-long learner, which itself is subject to being re-updated. Nikolas Rose’s Powers of Freedom has been cited to clarify these discursive changes. Rizvi & Lingard (2006: 253), analysing the OECD’s action in the educational field noted that in the 1990s there was a confrontation between “the humanistic, social democratic opportunity construction of life-long learning and the individualistic, neo-liberal self-capitalizing individual construction”, and they went on to associate the preponderance of the latter in major OECD’s mid-nineties texts in which the aims of education emphasise “preparing people for the world of work and a life of self-capitalization” and view the citizens of the world as “flexible mobile lifelong learners who have cosmopolitan dispositions and are able to deal effectively with cultural diversity, endemic change, and innovation”.

The redefinitions mentioned above are based on or invoke a particular scientific perspective on the concept of learning, whose adoption is legitimate in the light of the changes in the economy – shifting from the industrial age to the post-industrial age. Indeed, the need to shift from a behaviourist theory of learning to a cognivitist theory of learning – briefly and crudely put, one of which is centred on considering the chains of stimulation-responses-reinforcement, and the other on the mental structures, schemes and operations – has been emphasised by those who call for the school system to be adapted to the mutations from the economy of industrialisation to post-industrialisation – e.g. contrasting the nature of the work and the typical roles: “physical activity” versus “mental activity”; “transformation of material objects” versus “information gathering and problem solving”; “prescribed tasks and routine activities” versus “non-routine tasks”; “humans as appendages of machines” versus “machines are tools”; “low interaction rates” versus “high interaction rates”; “roles resistant to change” or “roles frequently and substantially redefined” (examples from Weeres & Kerchner, 1996: 146) [3].

The consequences of adopting a cognitive perspective translates into descriptions and propositions about changes in the ways of organising the teaching-learning: from the “passive” learner who is “encouraged to be passive” to the “experimenting and engaged” learner, from the teacher who informs and corrects within a “hierarchical learning structure” to the teacher who uses organic models (“branching and networking”) and “develops authority and responsibility” in the pupil, from the exterior world of knowledge (bodies of knowledge sequentially transmitted) to the interior world of knowledge construction (the learning begins with the establishment by the pupil of a “connection with what he already knows” upon which he will study (in the words of Weeres and Kerchner, 1996: 147).

In the light of the various citations of texts under the authorship of PISA (or their representatives), which we have used herein as regards literacy and competence, we do not hesitate to say that the PISA discourse enacts these kinds of guidelines. In their background, the mutations in how to view learning suppose organizational mutations. As Mangez points out (2008: 103), these discursive changes involve other categories – such as knowledge and school knowledge and involve the reassessment of the utility of the knowledge and the school knowledge [4]. As such, the new focus that PISA brings to the international assessments – assessment of the degree of preparation of the pupils, at the end of their compulsory schooling, to face up to the challenges of the knowledge society (OECD, s.d.: 6) – the building blocks of literacy and competence, together, provoke a shift in the survey from the field of the school subjects and the knowledge of the school programs: “Rather than examine mastery of specific school curricula, PISA looks at students’ ability to apply knowledge and skills in key subject areas and to analyze, reason and communicate effectively as they examine, interpret and solve problems”.

This shift is, in itself, a double move away: (i) on the one hand, from the national choices about teaching materials and about learning through the teaching material; (ii) on the other hand, from the studies carried out previously by other agencies (e.g. the IEA studies). This movement endows the PISA project with a redoubled distinction: of the conventional reflection of the school system on itself and the conceptualization established of the compared scientific studies about the performances of the pupils which depend on the categories belonging to the educational systems. But the deviations do not affect only the particular categories of schooling. They imply the schooling itself. The redefinitions in the discursive categories imply, as written by Martin Lawn (2003: 333), in relation to lifelong learning, the rethinking of the “concerns, purposes and structures of education”.

We can, based on this point of view, talk about another and broader shift than PISA: the establishing, looking at and debating of the effects of the school systems as regards their ‘outcomes’, directly connecting the contexts and ways of teaching/learning with the “demands” of the school system environment to the learning. This enterprise includes the resumption of the so-called scientific movement/justification towards ‘opening black boxes’. But it is not a question of either the classroom or the school, but rather the school model itself. The black box is, here, the (national) school institution which remains black or closed in on itself because it describes and assesses only taking into consideration its inputs and/or its outputs (results of tests and exams based on the goals and content of the teaching materials). This last case introduces ways of internal retro-feeding, and as such does not allow the educational system to evaluate its actual impact on the social system. What it manages to do with the pupils in relation to what we establish as the learning aims is the question that is raised, when it should be what our pupils manage to do with what they acquire in their school years.

From this point of view the question of the school institution’s assessment practices and ceremonies is also raised, and in turn, the ways of organizing the learning in the school space. The PISA vision is that the schooling should not, therefore, have its product assessed by using its own references and conventional categories (aims of the teaching materials – examining the acquisition of knowledge), frequently decoupling what is being reiterated – the various and not necessarily concordant missions and objectives of schooling, such as democratic equality, social efficiency, social mobility (see Labaree, 1999) – what is being done (the reproduction of teaching-learning organizational models and practices that belong to the industrial model) and the consequences of what is done, not providing accountability regarding what it does to the other systems.

As a lever to open this black box – and its myths – PISA can be understood as the bearer of potential for change; it can operate as a critical and de-institutionalizing force, operating on the key categories of schooling. But it can constitute – simultaneously – a rewriting of the school model, a semantic geared towards its adaptation to the logic of the knowledge society and the economy (and responding to these), or be part of a broader rewriting of the educational model of contemporary societies. On this matter, and noting the similarity between the discourses of the European Commission and the discourses of the OECD and the World Bank concerning the problem of learning and its role in contemporary societies and economies, Lawn (2003: 331) reminds us of the vision implemented by the Study Group for Education and Training (group of specialists nominated in the mid-nineties by the then Commissioner for Education): “The future of Europe was to be constructed by several shifts: from objective to constructed knowledge; from an industrial to a learning society; from instruction to personal learning; from formal educational institutions towards new organizational structures for learning (yet to be determined)" (European Commission, 1997:7).

These discourses open and close pathways and establish possibilities. It is not necessarily a question of convergence, because social actors that use them operate choices regarding the guidelines - of control and/or emancipation - to pursue and follow on. Furthermore, the discursive categories that circulate on a worldwide scale and even carry the authority of being universal categories, when mobilized in other spaces, are reworked by and held up to rewrite systems of meaning and local relations of power, producing particular effects (see Popkewitz, 2000, 2003) [5].

PISA incorporates and creates spaces for the politics of cognition, for the agency about the visions of the world, the perceptions of the reality and the preferences. In any event, PISA’s intervention is not limited to framing the problem of the ‘educational reality’. The regulatory intervention also exercised in the field is analyzing and reordering the action of their potential users (and their cognitions and norms).

Comparison as a way of governing (and politicians as learners)

“…and remember: Without data, you are just another person with opinion” [6].

We believe we can extend Lindblad & Popkewitz’s arguments (2004: xx-xxi) about the kind of imagined/fabricated pupil that these studies put forward – the problem-solver, the lifelong learner – to the case of the political actor and state that devices such as PISA incorporate and put into practice modes of thinking that can be and are interiorized by the participating actors and which enable them to take part “naturally” in the exercises of monitoring and international comparison, as coherent practices that are appropriate for their identities and roles.

In this background PISA presupposes and also defines a certain kind of political decision maker: the problem solver, the learner, the reasonable policy maker who governs with/through (a specific, scientific) reason/reasoning. In texts that describe, assess and thus prescribe the relations of the politicians with PISA itself, we can find these rules that qualify-disqualify the political decision maker. We now transcribe an interview given by Andreas Schleicher to a Brazilian journal with a big circulation: “(…) Brazil began to have the chance to advance at the moment in which it began to map out the problems in an objective way – and not based on the intuition of some governors. This is basic. There is no way of improving something that has not even been measured - hence, the importance of international comparison. In looking at the rankings, parents, educators and authorities can start to make comparisons and see the obvious: their schools are lagging far behind the OECD countries”. (…) “Upon learning about the fiasco of the latest rankings, some politicians and specialists with a more backward mentality phoned me in a rage. They said: ‘You are demanding that the pupils speak about situations that are too far removed from their reality. It is unfair’. The myopia of these people, which prevents them from grasping the fact that Chinese or American students have the answer to such questions, not only reveals the lack of preparation of the Brazilians, but also shows how they are at a disadvantage when competing against others”. (…) “Only recently did the authorities begin to use this extremely effective measurer to ascertain faults and draw up solutions based on experiences that have worked in other countries. (…) The Chinese do not hold back in copying what works in other countries. Quite the opposite: they are stimulated by this (…) China, of course, still has a lot to improve in education – but it is moving forward fast”.

The good political decision maker is one who governs by diagnosing objectively his world or sector, and one who is guided by searching for competitive advantages by measuring the outcomes of the school system, who identifies weaknesses and adopts solutions based on ‘what works’ in other countries, who learns about and copies competitors so as to progress more quickly. But the good politician is also one who makes the education sector a political priority and who thinks in line with economic reasoning – he is governed by the principle of competitiveness (see above), putting forward effective measures, and efficiently generating the budget sums for the educational sector. But the good political decisions maker is not only one who manages the system; he is one who changes the system, who hauls the educational sector out of its “backwardness” – by comparing it with other sectors, which are considered equivalent – who makes his sector produce more and better, who moves from the industrial model to the post-industrial model. This is why it is necessary that one thinks the educational sector in another way, based on the objective and relevant scientific knowledge.

It is inside such web of ‘causes and effects’, of ‘conditions and consequences’ (present and future) that the action of the politicians is understood and the criteria for their assessment are fixed. The joint-Director of the Education Department of the OECD wrote (Hugonnier, 2008: 61): "Nous vivons à l’heure de l’économie de la connaissance et de la mondialisation. L’une et l’autre exigent une plus haute efficience des services publics, ce qui ne peut être atteint que si l’ont peu comparer les investissements réalisés avec leurs résultats, et si les décisions politiques sont en pleine connaissance de cause. PISA répond à ces deux exigences en fournissant des informations inédites sur les performances comparées de leur système éducatif. (…) Sans ces informations [Hugonnier refere-se ao PISA e a outras informações estatísticas da OCDE, criadas ou ainda por criar] (…) les décisions politiques fondées davantage sur des perceptions que sur les réalités présentent de forts risques d’échec, avec les coûts considérables en termes financiers mais aussi humains que l’ont peut imaginer". In the background of these arguments the ‘PISA results’ supply the political decision maker with a basis of secure comparison to learn about not only the place of ‘their’ educational system in relation to the worldwide competitive space, but also its place in a timeline that travels from the industrial society to the knowledge society. With the ‘PISA data’ and the PISA ways of thinking, the political decision makers can guide their own movements and can move their country to the ‘time’ (physical and symbolic) of “tomorrow’s world”, in line with a prospective future backed up by data.

With or through PISA, the reasonable policy maker is the policy-maker who governs with/through (a specific) comparative reason. Thus, tools such as PISA institute the philosophy of comparison as a mode of government: “(…) la construction d’échelles et d’indicateurs internationaux instaure des principes d’évaluation mutuelle, invitant chaque pays (et chaque citoyen) à se comparer perpétuellement a l’autre. Il est évident qu’on ne parle jamais d’”homogénéisation” ou d’"uniformisation ”, mots qui ne sont plus admis dans le jargon du discours expert. Les temps sont à la “concertation”, à la “lisibilité des systèmes”, au “transfert” ou à la “facilité de communication”. Et pourtant, on assiste à l’émergence d’une “pensée mondiale” qui gouverne la manière de poser les problèmes et de construire les politiques” (Nóvoa & Tariv-Mashal, 2003: 36).

[1] For example, the competencies of literacy in mathematics, the following skills are considered: (i) identifying, understanding and engaging in mathematics; (ii) make well-founded judgements about the role that mathematics plays in an individual’s current and future private life, occupational life, social life with peers and relatives, life as a constructive, concerned and reflective citizen. And these competencies are considered dependent on (i) mathematical knowledge and skills, (ii) ability to think and work mathematically, (iii) ability to apply the knowledge in a wide variety of contexts (see OCDE, 2003).

[2] “The BPC (…) noted the importance of validating, over the medium term, the effectiveness of PISA in measuring the preparedness of students for adult life from both national and international levels" (OECD/DEELSA, 1999: 2).

[3] The relevance of the knowledge of the neurosciences and the invocation of change from an industrial society to a post-industrial society is contained in the discourse of the PISA managers (see Weinberg, 2008). On the importance of the neuroscientists’ arguments in the formulation of the knowledge-worker notion (see Mangez, 2008: 103-104).

[4] On the curricular changes built around the notion of competence, see Mangez (2008: 102-103).

[5] This can be understood in Lawn’s critical analysis (2003: 330) – in the European Union the discourse about the ‘usefulness of knowledge’, although not specific, is used around the construction of Europe’s particular project, in which the policy of life-long learning is connected to the vision of the knowledge society, and both to the construction of a transnational governance.

[6] Source: Schleicher presentation: ‘Seeing School through the prism of PISA’ (10th OECD seminar, Tokyo, June 2005).

CARVALHO L.M. [with E. COSTA] (2009), Production of OECD’s- Procramme for International Students Assessment" (PISA), KNOWandPOL Report.

Read document

  • ALASUUTARI Pertti (2005), The governmentality of consultancy and cooperation: the influence of OECD, Paper presented at the 37th World Congress of the International Institute of Sociology, Stockholm, July 5-9.
  • CARVALHO Luis M. with E. Costa (2009), The production of OECD: Programme for International Student Assessment (PISA), KNOWandPOL Report.
  • HUGONNIER Bernard (2008), Les acquis des adolescents, Futuribles, 344, 47-61.
  • LABAREE David (1999), The chronic failure of curriculum reform, Education Week, 16:36 (May 19), 42-44.
  • LAWN Martin (2003) The ‘usefulness’ of learning: the stuggle over governance, meaning and the education space, Discourse, 24(3), 325-36.
  • LINDBLAD Sverker & POPKEWITZ Thomas S. (2004), Educational Restructuring: (re) thinking the problematic of reform, in S. Lindblad & T.S. Popkewitz (Eds), Educational Restructuring: perspectives on traveling policies, IAP.
  • MANGEZ Eric (2008), Knowledge economy, knowledge policy and knowledge regimes, in Delvaux Bernard & Mangez Eric, Towards a Sociology of the Knowledge-Policy Relation. KNOWandPOL Literature Review – Integrative Report, 98-118. MORGAN Clara (2007), The OECD Programme for International Student Assessment: Unraveling a knowledge network, Doctoral Thesis, Ottawa: School of Public Policy and Administration, Carleton University. NÓVOA António & YARIV-MASHAL Tali (2003), Vers un Comparatisme Critique, Lisboa : Educa. OECD (s.d.), PISA – The OECD Programme for International Student Assessment, PISA Brochure, OECD (1999), Measuring Student Knowledge and Skills – A New Framework for Assessment, Paris: OECD. OECD (2003), The PISA 2003 Assessment Framework: Mathematics, Reading, Science and Problem Solving Knowledge and Skills, Paris: OECD. OECD/DEELSA (1999), 6th Meeting of the Board of Participating Countries (BPC), Tokyo, 1-3 March, [DEELSA/PISA/BPC/M (99)1]. POPKEWITZ Thomas S (2000), Preface and Globalization/regionalization, knowledge, and the educational practices, T.S. Popkewitz (Ed), Educational Knowledge: Changing relationships between the state, civil society, and the educational community (vii-x, 3-27), Albany: State University of New York. POPKEWITZ Thomas S (2003), National imaginairies, the indigenous-foreigner, and power, Schriewer Jürgen (Ed), Discourse Formation in Comparative Education, 2nd revised edition, Frankfürt: Peter Lang, 261-264. RIZVI Fazal & LINGARD Bob (2006), Globalisation and the changing nature of the OECD‟s educational work, In Lauder Hugh " al. (Eds), Education, Globalisation and Social Change, Oxford: Oxford University Press. SCHLEIDER Andreas (2006), Fundamentos y cuestiones políticas subyacentes al desarollo de PISA, Revista de Éducación, extraordinário 2006: 21-43. SCHLEIDER Andreas (2007), Can competencies assessed by PISA be considered the fundamental school knowledge 15-year-olds should possess?, Journal of Educational Change, 8(4), 349-357. WEERES Joseph & KERCHNER Charles (1996), This time it’s serious: post-industrialism and the coming change in education, in Crowson Robert L., Boyd William L., & Mawhinney Hanne B. (Eds), The Politics of Education and the New Institutionalism, London: Falmer Press, 135-151. WEINBERG Monica (2008), Entrevista: Andreas Schleicher - Medir para avançar rápido, Veja, Agosto 17, 20-21,

© 2011 Knowandpol Designed and Powered by platanas