Programme for International Student Assessment
Encyclopedia
The Programme for International Student Assessment (PISA) is a worldwide evaluation
in OECD member countries (currently there are 65 member nations) of 15-year-old school pupils' scholastic performance, performed first in 2000 and repeated every three years. It is coordinated by the Organisation for Economic Co-operation and Development
(OECD), with a view to improving educational policies and outcomes. Another similar study is the Trends in International Mathematics and Science Study
, which focuses on mathematics and science but not reading.
(IEA). Much of PISA's methodology follows the example of the Trends in International Mathematics and Science Study
(TIMSS, started in 1995), which in turn was much influenced by the U.S. National Assessment of Educational Progress
(NAEP). The reading component of PISA is inspired by the IEA's Progress in International Reading Literacy Study (PIRLS).
PISA aims at testing literacy
in three competence fields: reading, mathematics, science.
The PISA mathematics literacy test asks students to apply their mathematical knowledge to solve problems set in various real-world contexts. To solve the problems students must activate a number of mathematical competencies as well as a broad range of mathematical content knowledge. TIMSS, on the other hand, measures more traditional classroom content such as an understanding of fractions and decimals and the relationship between them (curriculum attainment). PISA claims to measure education's application to real-life problems and life-long learning (workforce knowledge).
In the reading test, "OECD/PISA does not measure the extent to which 15-year-old students are fluent readers or how competent they are at word recognition tasks or spelling". Instead, they should be able to "construct, extend and reflect on the meaning of what they have read across a wide range of continuous and non-continuous texts"
Every period of assessment focusses on one of the three competence fields reading, math, science; but the two others are tested as well. After nine years, a full cycle is completed: after 2000, reading is again the main domain in 2009.
PISA is sponsored, governed, and coordinated by the OECD. The test design, implementation, and data analysis is delegated to an international consortium of research and educational institutions led by the Australian Council for Educational Research
(ACER). ACER leads in developing and implementing sampling procedures and assisting with monitoring sampling outcomes across these countries. The assessment instruments fundamental to PISA's Reading, Mathematics, Science, Problem-solving, Computer-based testing, background and contextual questionnaires are similarly constructed and refined by ACER. ACER also develops purpose-built software to assist in sampling and data capture, and analyses all data. The source code of the data analysis software is not made public.
To fulfill OECD requirements, each country must draw a sample of at least 5,000 students. In small countries like Iceland
and Luxembourg
, where there are less than 5,000 students per year, an entire age cohort is tested. Some countries used much larger samples than required to allow comparisons between regions.
In selected countries, PISA started also experimentation with computer adaptive testing.
Germany does this in a very extensive way: on the day following the international test, students take a national test called PISA-E (E=Ergänzung=complement). Test items of PISA-E are closer to TIMSS than to PISA. While only about 5,000 German students participate in both the international and the national test, another 45,000 take only the latter. This large sample is needed to allow an analysis by federal states. Following a clash about the interpretation of 2006 results, the OECD warned Germany that it might withdraw the right to use the "PISA" label for national tests.
of item response theory
(IRT). According to IRT, it is not possible to assess the competence of students who solved none or all of the test items. This problem is circumvented by imposing a Gaussian prior probability
distribution of competences.
One and the same scale is used to express item difficulties and student competences. The scaling procedure is tuned such that the a posteriori distribution of student competences, with equal weight given to all OECD countries, has mean 500 and standard deviation 100.
The following table gives the mean achievements of OECD member countries in the principal testing domain of each period:
In the official reports, country rankings are communicated in a more elaborate form: not as lists, but as cross tables, indicating for each pair of countries whether or not mean score differences are statistically significant
(unlikely to be due to random fluctuations in student sampling or in item functioning). In favorable cases, a difference of 9 points is sufficient to be considered significant.
In some popular media, test results from all three literacy domains have been consolidated in an overall country ranking. Such meta-analysis is not endorsed by the OECD. The official reports only contain domain-specific country scores. In part of the official reports, however, scores from a period's principal testing domain are used as proxy for overall student ability.
between PISA 2003 and TIMSS 2003 grade 8 country means is 0.84 in mathematics, 0.95 in science. The values go down to 0.66 and 0.79 if the two worst performing developing countries are excluded. Correlations between different scales and studies are around 0.80. The high correlations between different scales and studies indicate common causes of country differences (e.g. educational quality, culture, wealth or genes) or a homogenous underlying factor of cognitive competence. Western countries perform slightly better in PISA; Eastern European and Asian countries in TIMSS. Content balance and years of schooling explain most of the variation.
, Belgium
, Canada
, the Czech Republic
, Finland
, Japan
, South Korea
, New Zealand
and the Netherlands
spent less but did relatively well, whereas the United States spent much more but was below the OECD average. The Czech Republic, in the top ten, spent only one third as much per student as the United States did, for example, but the USA came 24th out of 29 countries compared.
Another point made in the evaluation was that students with higher-earning parents are better-educated and tend to achieve higher results. This was true in all the countries tested, although more obvious in certain countries, such as Germany.
It has been suggested that the Finnish language plays an important part in Finland's PISA success.
International testing, including both PISA and TIMSS, has been a central part of many recent analyses of how cognitive skills relate to economic outcomes. These studies consider both individual earnings and aggregate growth differences of nations.
In 2010, the 2009 Program for International Student Assessment (PISA) results revealed that Shanghai students scored the highest in the world in every category (Math, Reading and Science), stunned educators. The OECD described Shanghai as a pioneer of educational reform, noting that "there has been a sea change in pedagogy". OECD point out that they "abandoned their focus on educating a small elite, and instead worked to construct a more inclusive system. They also significantly increased teacher pay and training, reducing the emphasis on rote learning and focusing classroom activities on problem solving."
The results from PISA 2003 and PISA 2006 were featured in the 2010 documentary Waiting for "Superman".
Economic researchers studied single educational policy factors like central exams (John Bishop), private schools or streaming between schools at later age (Hanushek/Woessman). An extensive literature related to cross-countries difference in scores has developed since 2000.
The stable, good results of Finland have attracted a lot of attention. According to Hannu Simola the results are not due to attributes of the educational system, but are due to disciplined students, the respected status of teachers (attracting good students to the teaching profession), high quality of teachers due to professional teacher education, conservative direct instruction ("teaching ex cathedra", "pedagogical conservatism"), low rates of immigration, fast diagnosis of learning problems and treatment of them including special schools, and the culture of a small border country (as in Singapore and Taiwan) feeling that the people could survive only with effort. Others have suggested that Finland's low poverty rate is a reason for its success.
Systematic analyses across different paradigms (culture, genes, wealth, educational policies) for 78 countries were presented by Heiner Rindermann and Stephen Ceci: They report positive relationships between student ability and educational levels of adults, amount and rate of preschool education, discipline, quantity of institutionalized education, attendance at additional schools, early tracking and the use of central exams and tests. Rather negative relationships were found with high repetition rates, late school enrollment
and large class sizes.
In their opinion the results suggest that international differences in cognitive competence could be narrowed by reforms in educational policy.
is closely related to American poverty
, but the same reasoning applies to other countries. Riddile also shown that when adjusted for poverty, the richest areas in the US, especially areas with less than 10% poverty can perform an average PISA score of 551 (higher than any other country). In essence, the criticism isn't so much directly against the Programme for International Student Assessment itself, but against people who use PISA data uncritically to justify measures such as Charter school
s.
The table below summarizes the scores of American schools by their poverty rates and compares them to countries with similar poverty rates.
(NAEP). Two studies have linked the performance in individual states to national scores on PISA. In the first, comparisons were made between those scoring at the advanced level in mathematics and reading according to NAEP with the corresponding performance on PISA for the "High School Class of 2009." Overall, 30 nations did better than the U.S. in producing students at the advanced level of mathematics. Six percent of U.S. students were advanced in mathematics compared to 28 percent in Taiwan. Perhaps more significantly, the highest ranked state in the U.S. (Massachusetts) was just seventeenth in the world rankings. In the second, U.S. students in the "Class of 2011" who were proficient on the NAEP in mathematics (32 percent) ranked thirty-second among the nations participating in PISA. Massachusetts was again the best U.S. state, but it ranked just ninth in the world.
Comparisons with results for the Trends in International Mathematics and Science Study
(TIMSS) appear to give different results -- suggesting that the U.S. states actually do better in world rankings. The difference in apparent rankings is, however, almost entirely accounted for by the sampling of countries. PISA includes all of the OECD countries, while TIMSS is much more weighted in its sampling toward developing countries.
's report, there are 18.9 million students enrolled in four-year higher education institutions (non-vocational, etc.) in 2007, making the figure roughly 2% of the total Chinese population.
, which scored quite low, over the method used in its PISA test. Although being a trilingual country (Luxembourgish, French and German), in 2000 the test was not allowed to be done in Luxembourgish
, the mother tongue of the majority of students (77%).
15-years old student was for many years underrated and underachieving in terms of reading literacy, mathematics and science knowledge in the OECD, nearly tied with the Italian and just above those from countries like Greece, Turkey and Mexico. However, since 2010, PISA results for Portuguese students improved dramatically. The Portuguese Ministry of Education announced a 2010 report published by its office for educational evaluation GAVE (Gabinete de Avaliação do Ministério da Educação) which criticized the results of PISA 2009 report and claimed that the average Portuguese teenage student had profund handicaps in terms of expression, communication and logic, as well as a low performance when asked to solve problems. They also claimed that those fallacies are not exclusive of Portugal but indeed occur in other countries due to the way PISA was designed.
Evaluation
Evaluation is systematic determination of merit, worth, and significance of something or someone using criteria against a set of standards.Evaluation often is used to characterize and appraise subjects of interest in a wide range of human enterprises, including the arts, criminal justice,...
in OECD member countries (currently there are 65 member nations) of 15-year-old school pupils' scholastic performance, performed first in 2000 and repeated every three years. It is coordinated by the Organisation for Economic Co-operation and Development
Organisation for Economic Co-operation and Development
The Organisation for Economic Co-operation and Development is an international economic organisation of 34 countries founded in 1961 to stimulate economic progress and world trade...
(OECD), with a view to improving educational policies and outcomes. Another similar study is the Trends in International Mathematics and Science Study
Trends in international mathematics and science study
The Trends in International Mathematics and Science Study is an international assessment of the mathematics and science knowledge of fourth- and eighth-grade students around the world...
, which focuses on mathematics and science but not reading.
Framework
PISA stands in a tradition of international school studies, undertaken since the late 1950s by the International Association for the Evaluation of Educational AchievementInternational Association for the Evaluation of Educational Achievement
The International Association for the Evaluation of Educational Achievement is an association of national research institutions and government research agencies related to education. The IEA is an independent organization. It was founded in 1958 and is headquartered in Amsterdam...
(IEA). Much of PISA's methodology follows the example of the Trends in International Mathematics and Science Study
Trends in international mathematics and science study
The Trends in International Mathematics and Science Study is an international assessment of the mathematics and science knowledge of fourth- and eighth-grade students around the world...
(TIMSS, started in 1995), which in turn was much influenced by the U.S. National Assessment of Educational Progress
National Assessment of Educational Progress
The National Assessment of Educational Progress is the largest continuing and nationally representative assessment of what our nation’s students know and can do in core subjects. NAEP is a congressionally mandated project administered by the National Center for Education Statistics , within the ...
(NAEP). The reading component of PISA is inspired by the IEA's Progress in International Reading Literacy Study (PIRLS).
PISA aims at testing literacy
Literacy
Literacy has traditionally been described as the ability to read for knowledge, write coherently and think critically about printed material.Literacy represents the lifelong, intellectual process of gaining meaning from print...
in three competence fields: reading, mathematics, science.
The PISA mathematics literacy test asks students to apply their mathematical knowledge to solve problems set in various real-world contexts. To solve the problems students must activate a number of mathematical competencies as well as a broad range of mathematical content knowledge. TIMSS, on the other hand, measures more traditional classroom content such as an understanding of fractions and decimals and the relationship between them (curriculum attainment). PISA claims to measure education's application to real-life problems and life-long learning (workforce knowledge).
In the reading test, "OECD/PISA does not measure the extent to which 15-year-old students are fluent readers or how competent they are at word recognition tasks or spelling". Instead, they should be able to "construct, extend and reflect on the meaning of what they have read across a wide range of continuous and non-continuous texts"
Development and implementation
Developed from 1997, the first PISA assessment was carried out in 2000. The results of each period of assessment take about one year and half to be analysed. First results were published in November 2001. The release of raw data and the publication of technical report and data handbook took only place in spring 2002. The triennial repeats follow a similar schedule; the process of seeing through a single PISA cycle, start-to-finish, always takes over four years.Every period of assessment focusses on one of the three competence fields reading, math, science; but the two others are tested as well. After nine years, a full cycle is completed: after 2000, reading is again the main domain in 2009.
| Period | Main focus | # OECD countries | # other countries | # students | Notes |
|---|---|---|---|---|---|
| 2000 | Reading | 28 | 4 | 265,000 | The Netherlands disqualified from data analysis. 11 additional non-OECD countries took the test in 2002 |
| 2003 | Mathematics | 30 | 11 | 275,000 | UK disqualified from data analysis. Also included test in problem solving Problem solving Problem solving is a mental process and is part of the larger problem process that includes problem finding and problem shaping. Consideredthe most complex of all intellectual functions, problem solving has been defined as higher-order cognitive process that requires the modulation and control of... . |
| 2006 | Science | 30 | 27 | ||
| 2009 | Reading | 34 | 33? | Results made available on 7 December 2010 |
PISA is sponsored, governed, and coordinated by the OECD. The test design, implementation, and data analysis is delegated to an international consortium of research and educational institutions led by the Australian Council for Educational Research
Australian Council for Educational Research
The Australian Council for Educational Research , established in 1930, is an independent educational research organisation based in Camberwell, Victoria and with offices in Sydney, Brisbane, Perth, Adelaide, Dubai and India...
(ACER). ACER leads in developing and implementing sampling procedures and assisting with monitoring sampling outcomes across these countries. The assessment instruments fundamental to PISA's Reading, Mathematics, Science, Problem-solving, Computer-based testing, background and contextual questionnaires are similarly constructed and refined by ACER. ACER also develops purpose-built software to assist in sampling and data capture, and analyses all data. The source code of the data analysis software is not made public.
Sampling
The students tested by PISA are aged between 15 years and 3 months and 16 years and 2 months at the beginning of the assessment period. The school year pupils are in is not taken into consideration. Only students at school are tested, not home-schoolers. In PISA 2006 , however, several countries also used a grade-based sample of students. This made it possible also to study how age and school year interact.To fulfill OECD requirements, each country must draw a sample of at least 5,000 students. In small countries like Iceland
Iceland
Iceland , described as the Republic of Iceland, is a Nordic and European island country in the North Atlantic Ocean, on the Mid-Atlantic Ridge. Iceland also refers to the main island of the country, which contains almost all the population and almost all the land area. The country has a population...
and Luxembourg
Luxembourg
Luxembourg , officially the Grand Duchy of Luxembourg , is a landlocked country in western Europe, bordered by Belgium, France, and Germany. It has two principal regions: the Oesling in the North as part of the Ardennes massif, and the Gutland in the south...
, where there are less than 5,000 students per year, an entire age cohort is tested. Some countries used much larger samples than required to allow comparisons between regions.
The test
Each student takes a two-hour handwritten test. Part of the test is multiple-choice and part involves fuller answers. In total there are six and a half hours of assessment material, but each student is not tested on all the parts. Following the cognitive test, participating students spend nearly one more hour answering a questionnaire on their background including learning habits, motivation and family. School directors also fill in a questionnaire describing school demographics, funding etc.In selected countries, PISA started also experimentation with computer adaptive testing.
National add-ons
Countries are allowed to combine PISA with complementary national tests.Germany does this in a very extensive way: on the day following the international test, students take a national test called PISA-E (E=Ergänzung=complement). Test items of PISA-E are closer to TIMSS than to PISA. While only about 5,000 German students participate in both the international and the national test, another 45,000 take only the latter. This large sample is needed to allow an analysis by federal states. Following a clash about the interpretation of 2006 results, the OECD warned Germany that it might withdraw the right to use the "PISA" label for national tests.
Data Scaling
From the beginning, PISA has been designed with one particular method of data analysis in mind. Since students work on different test booklets, raw scores must be scaled to allow meaningful comparisons. This scaling is done using the Rasch modelRasch model
Rasch models are used for analysing data from assessments to measure variables such as abilities, attitudes, and personality traits. For example, they may be used to estimate a student's reading ability from answers to questions on a reading assessment, or the extremity of a person's attitude to...
of item response theory
Item response theory
In psychometrics, item response theory also known as latent trait theory, strong true score theory, or modern mental test theory, is a paradigm for the design, analysis, and scoring of tests, questionnaires, and similar instruments measuring abilities, attitudes, or other variables. It is based...
(IRT). According to IRT, it is not possible to assess the competence of students who solved none or all of the test items. This problem is circumvented by imposing a Gaussian prior probability
Prior probability
In Bayesian statistical inference, a prior probability distribution, often called simply the prior, of an uncertain quantity p is the probability distribution that would express one's uncertainty about p before the "data"...
distribution of competences.
One and the same scale is used to express item difficulties and student competences. The scaling procedure is tuned such that the a posteriori distribution of student competences, with equal weight given to all OECD countries, has mean 500 and standard deviation 100.
Historical league tables
All PISA results are broken down by countries. Public attention concentrates on just one outcome: achievement mean values by countries. These data are regularly published in form of "league tables".The following table gives the mean achievements of OECD member countries in the principal testing domain of each period:
In the official reports, country rankings are communicated in a more elaborate form: not as lists, but as cross tables, indicating for each pair of countries whether or not mean score differences are statistically significant
Statistical significance
In statistics, a result is called statistically significant if it is unlikely to have occurred by chance. The phrase test of significance was coined by Ronald Fisher....
(unlikely to be due to random fluctuations in student sampling or in item functioning). In favorable cases, a difference of 9 points is sufficient to be considered significant.
In some popular media, test results from all three literacy domains have been consolidated in an overall country ranking. Such meta-analysis is not endorsed by the OECD. The official reports only contain domain-specific country scores. In part of the official reports, however, scores from a period's principal testing domain are used as proxy for overall student ability.
2000–2006
Top results for the main areas of investigation of PISA, in 2000, 2003 and 2006.2006
Top 10 countries for Pisa 2006 results in Math, Sciences and Reading.2009
Top 30 countries for Pisa 2009 results in Math, Sciences and Reading. For a complete list, see reference.Comparison with other studies
The correlationPearson product-moment correlation coefficient
In statistics, the Pearson product-moment correlation coefficient is a measure of the correlation between two variables X and Y, giving a value between +1 and −1 inclusive...
between PISA 2003 and TIMSS 2003 grade 8 country means is 0.84 in mathematics, 0.95 in science. The values go down to 0.66 and 0.79 if the two worst performing developing countries are excluded. Correlations between different scales and studies are around 0.80. The high correlations between different scales and studies indicate common causes of country differences (e.g. educational quality, culture, wealth or genes) or a homogenous underlying factor of cognitive competence. Western countries perform slightly better in PISA; Eastern European and Asian countries in TIMSS. Content balance and years of schooling explain most of the variation.
Topical studies
An evaluation of the 2003 results showed that countries that spent more on education did not necessarily do better. AustraliaAustralia
Australia , officially the Commonwealth of Australia, is a country in the Southern Hemisphere comprising the mainland of the Australian continent, the island of Tasmania, and numerous smaller islands in the Indian and Pacific Oceans. It is the world's sixth-largest country by total area...
, Belgium
Belgium
Belgium , officially the Kingdom of Belgium, is a federal state in Western Europe. It is a founding member of the European Union and hosts the EU's headquarters, and those of several other major international organisations such as NATO.Belgium is also a member of, or affiliated to, many...
, Canada
Canada
Canada is a North American country consisting of ten provinces and three territories. Located in the northern part of the continent, it extends from the Atlantic Ocean in the east to the Pacific Ocean in the west, and northward into the Arctic Ocean...
, the Czech Republic
Czech Republic
The Czech Republic is a landlocked country in Central Europe. The country is bordered by Poland to the northeast, Slovakia to the east, Austria to the south, and Germany to the west and northwest....
, Finland
Finland
Finland , officially the Republic of Finland, is a Nordic country situated in the Fennoscandian region of Northern Europe. It is bordered by Sweden in the west, Norway in the north and Russia in the east, while Estonia lies to its south across the Gulf of Finland.Around 5.4 million people reside...
, Japan
Japan
Japan is an island nation in East Asia. Located in the Pacific Ocean, it lies to the east of the Sea of Japan, China, North Korea, South Korea and Russia, stretching from the Sea of Okhotsk in the north to the East China Sea and Taiwan in the south...
, South Korea
South Korea
The Republic of Korea , , is a sovereign state in East Asia, located on the southern portion of the Korean Peninsula. It is neighbored by the People's Republic of China to the west, Japan to the east, North Korea to the north, and the East China Sea and Republic of China to the south...
, New Zealand
New Zealand
New Zealand is an island country in the south-western Pacific Ocean comprising two main landmasses and numerous smaller islands. The country is situated some east of Australia across the Tasman Sea, and roughly south of the Pacific island nations of New Caledonia, Fiji, and Tonga...
and the Netherlands
Netherlands
The Netherlands is a constituent country of the Kingdom of the Netherlands, located mainly in North-West Europe and with several islands in the Caribbean. Mainland Netherlands borders the North Sea to the north and west, Belgium to the south, and Germany to the east, and shares maritime borders...
spent less but did relatively well, whereas the United States spent much more but was below the OECD average. The Czech Republic, in the top ten, spent only one third as much per student as the United States did, for example, but the USA came 24th out of 29 countries compared.
Another point made in the evaluation was that students with higher-earning parents are better-educated and tend to achieve higher results. This was true in all the countries tested, although more obvious in certain countries, such as Germany.
It has been suggested that the Finnish language plays an important part in Finland's PISA success.
International testing, including both PISA and TIMSS, has been a central part of many recent analyses of how cognitive skills relate to economic outcomes. These studies consider both individual earnings and aggregate growth differences of nations.
In 2010, the 2009 Program for International Student Assessment (PISA) results revealed that Shanghai students scored the highest in the world in every category (Math, Reading and Science), stunned educators. The OECD described Shanghai as a pioneer of educational reform, noting that "there has been a sea change in pedagogy". OECD point out that they "abandoned their focus on educating a small elite, and instead worked to construct a more inclusive system. They also significantly increased teacher pay and training, reducing the emphasis on rote learning and focusing classroom activities on problem solving."
Reception
For many countries, the first PISA results were surprising; in Germany and the United States, for example, the comparatively low scores brought on heated debate about how the school system should be changed. Some headlines in national newspapers, for example, were:- "La France, élève moyen de la classe OCDE" (France, average student of the OECD class) Le MondeLe MondeLe Monde is a French daily evening newspaper owned by La Vie-Le Monde Group and edited in Paris. It is one of two French newspapers of record, and has generally been well respected since its first edition under founder Hubert Beuve-Méry on 19 December 1944...
, December 5, 2001 - "Miserable Noten für deutsche Schüler" (Abysmal marks for German students) Frankfurter Allgemeine ZeitungFrankfurter Allgemeine ZeitungThe Frankfurter Allgemeine Zeitung , short F.A.Z., also known as the FAZ, is a national German newspaper, founded in 1949. It is published daily in Frankfurt am Main. The Sunday edition is the Frankfurter Allgemeine Sonntagszeitung .F.A.Z...
, December 4, 2001 - "Are we not such dunces after all?" The TimesThe TimesThe Times is a British daily national newspaper, first published in London in 1785 under the title The Daily Universal Register . The Times and its sister paper The Sunday Times are published by Times Newspapers Limited, a subsidiary since 1981 of News International...
, United Kingdom, December 6, 2001 - "Economic Time Bomb: U.S. Teens Are Among Worst at Math" Wall Street Journal December 7, 2004
- "Preocupe-se. Seu filho é mal educado." (Be worried. Your child is badly educated.) VejaVeja (magazine)Veja is a Brazilian weekly newsmagazine published in São Paulo and distributed throughout the country by the media conglomerate Grupo Abril. It is the leading weekly publication in the country, and one of the most influential outlets of the Brazilian press...
November 7, 2007 - "La educación española retrocede" (Spanish education moving backwards) El País December 5, 2007
- "Finnish teens score high marks in latest PISA study" Helsingin SanomatHelsingin SanomatHelsingin Sanomat is the largest subscription newspaper in Finland and the Nordic countries, owned by Sanoma. Except after certain holidays, it is published daily. In 2008, its daily circulation was 412,421 on weekdays and 468,505 on Sundays...
November 30, 2007
The results from PISA 2003 and PISA 2006 were featured in the 2010 documentary Waiting for "Superman".
Research on causes of country differences
PISA, TIMSS and PIRLS, their organizers and researchers, are restrained in giving reasons for the large and stable country differences. Cautiously they leave this task to other researchers, especially from the economic sciences and psychology.Economic researchers studied single educational policy factors like central exams (John Bishop), private schools or streaming between schools at later age (Hanushek/Woessman). An extensive literature related to cross-countries difference in scores has developed since 2000.
The stable, good results of Finland have attracted a lot of attention. According to Hannu Simola the results are not due to attributes of the educational system, but are due to disciplined students, the respected status of teachers (attracting good students to the teaching profession), high quality of teachers due to professional teacher education, conservative direct instruction ("teaching ex cathedra", "pedagogical conservatism"), low rates of immigration, fast diagnosis of learning problems and treatment of them including special schools, and the culture of a small border country (as in Singapore and Taiwan) feeling that the people could survive only with effort. Others have suggested that Finland's low poverty rate is a reason for its success.
Systematic analyses across different paradigms (culture, genes, wealth, educational policies) for 78 countries were presented by Heiner Rindermann and Stephen Ceci: They report positive relationships between student ability and educational levels of adults, amount and rate of preschool education, discipline, quantity of institutionalized education, attendance at additional schools, early tracking and the use of central exams and tests. Rather negative relationships were found with high repetition rates, late school enrollment
Enrollment
Enrollment or enrolment may refer to:* Matriculation, the process of initiating attendance to a school...
and large class sizes.
In their opinion the results suggest that international differences in cognitive competence could be narrowed by reforms in educational policy.
United States
Critics, such as Mel Riddile say that low performance in the United StatesUnited States
The United States of America is a federal constitutional republic comprising fifty states and a federal district...
is closely related to American poverty
Poverty in the United States
Poverty is defined as the state of one who lacks a usual or socially acceptable amount of money or material possessions. According to the U.S. Census Bureau data released Tuesday September 13th, 2011, the nation's poverty rate rose to 15.1% in 2010, up from 14.3% in 2009 and to its highest level...
, but the same reasoning applies to other countries. Riddile also shown that when adjusted for poverty, the richest areas in the US, especially areas with less than 10% poverty can perform an average PISA score of 551 (higher than any other country). In essence, the criticism isn't so much directly against the Programme for International Student Assessment itself, but against people who use PISA data uncritically to justify measures such as Charter school
Charter school
Charter schools are primary or secondary schools that receive public money but are not subject to some of the rules, regulations, and statutes that apply to other public schools in exchange for some type of accountability for producing certain results, which are set forth in each school's charter...
s.
The table below summarizes the scores of American schools by their poverty rates and compares them to countries with similar poverty rates.
| Country | Poverty Rate | PISA score |
|---|---|---|
| United States | < 10% | 551 |
| Finland | 3.4% | 536 |
| Netherlands | 9.0% | 508 |
| Belgium | 6.7% | 506 |
| Switzerland | 6.8% | 501 |
| United States | 10%–24.9% | 527 |
| Canada | 13.6% | 524 |
| New Zealand | 16.3% | 521 |
| Japan | 14.3% | 520 |
| Australia | 11.6% | 515 |
| United States | 25–49.9% | 502 |
| Estonia | 501 | |
| Poland | 14.5% | 500 |
| United States | 50–74.9% | 471 |
| Austria | 13.3% | 471 |
| Turkey | 464 | |
| Chile | 449 | |
| United States | > 75% | 446 |
| Mexico | 425 | |
| NASSP |
Performance of U.S. states in international comparisons
Given the wide variation in performance of students in different states within the United States, several comparisons have been made by calibrating international assessments to assessments in the United States. The U.S. has regularly tested students in mathematics and reading for individual states since the early 1990s in its National Assessment of Educational ProgressNational Assessment of Educational Progress
The National Assessment of Educational Progress is the largest continuing and nationally representative assessment of what our nation’s students know and can do in core subjects. NAEP is a congressionally mandated project administered by the National Center for Education Statistics , within the ...
(NAEP). Two studies have linked the performance in individual states to national scores on PISA. In the first, comparisons were made between those scoring at the advanced level in mathematics and reading according to NAEP with the corresponding performance on PISA for the "High School Class of 2009." Overall, 30 nations did better than the U.S. in producing students at the advanced level of mathematics. Six percent of U.S. students were advanced in mathematics compared to 28 percent in Taiwan. Perhaps more significantly, the highest ranked state in the U.S. (Massachusetts) was just seventeenth in the world rankings. In the second, U.S. students in the "Class of 2011" who were proficient on the NAEP in mathematics (32 percent) ranked thirty-second among the nations participating in PISA. Massachusetts was again the best U.S. state, but it ranked just ninth in the world.
Comparisons with results for the Trends in International Mathematics and Science Study
Trends in international mathematics and science study
The Trends in International Mathematics and Science Study is an international assessment of the mathematics and science knowledge of fourth- and eighth-grade students around the world...
(TIMSS) appear to give different results -- suggesting that the U.S. states actually do better in world rankings. The difference in apparent rankings is, however, almost entirely accounted for by the sampling of countries. PISA includes all of the OECD countries, while TIMSS is much more weighted in its sampling toward developing countries.
China
Education professor Yong Zhao has said the high scores in China are due to an excessive workload and testing, and added that it's "no news that the Chinese education system is excellent in preparing outstanding test takers, just like other education systems within the Confucian cultural circle—Singapore, Korea, Japan, and Hong Kong." Zhao also noted that most major Chinese media outlets did not pay much attention to this story. Others have criticized Shanghai as an outlier among China, while most of the country has a lower quality of education. According to Chinese state-media, Xinhua News AgencyXinhua News Agency
The Xinhua News Agency is the official press agency of the government of the People's Republic of China and the biggest center for collecting information and press conferences in the PRC. It is the largest news agency in the PRC, ahead of the China News Service...
's report, there are 18.9 million students enrolled in four-year higher education institutions (non-vocational, etc.) in 2007, making the figure roughly 2% of the total Chinese population.
Luxembourg
Criticism has ensued in LuxembourgLuxembourg
Luxembourg , officially the Grand Duchy of Luxembourg , is a landlocked country in western Europe, bordered by Belgium, France, and Germany. It has two principal regions: the Oesling in the North as part of the Ardennes massif, and the Gutland in the south...
, which scored quite low, over the method used in its PISA test. Although being a trilingual country (Luxembourgish, French and German), in 2000 the test was not allowed to be done in Luxembourgish
Luxembourgish language
Luxembourgish is a High German language spoken mainly in Luxembourg. About 320,000 people worldwide speak Luxembourgish.-Language family:...
, the mother tongue of the majority of students (77%).
Portugal
According to OECD's PISA, the average PortuguesePortugal
Portugal , officially the Portuguese Republic is a country situated in southwestern Europe on the Iberian Peninsula. Portugal is the westernmost country of Europe, and is bordered by the Atlantic Ocean to the West and South and by Spain to the North and East. The Atlantic archipelagos of the...
15-years old student was for many years underrated and underachieving in terms of reading literacy, mathematics and science knowledge in the OECD, nearly tied with the Italian and just above those from countries like Greece, Turkey and Mexico. However, since 2010, PISA results for Portuguese students improved dramatically. The Portuguese Ministry of Education announced a 2010 report published by its office for educational evaluation GAVE (Gabinete de Avaliação do Ministério da Educação) which criticized the results of PISA 2009 report and claimed that the average Portuguese teenage student had profund handicaps in terms of expression, communication and logic, as well as a low performance when asked to solve problems. They also claimed that those fallacies are not exclusive of Portugal but indeed occur in other countries due to the way PISA was designed.
See also
- Teaching And Learning International SurveyTeaching And Learning International SurveyThe Teaching And Learning International Survey is a worldwide evaluation on the conditions of teaching and learning, performed first in 2008. It is coordinated by the Organisation for Economic Co-operation and Development , with a view to improving educational policies and outcomes....
(TALIS) - Trends in International Mathematics and Science StudyTrends in international mathematics and science studyThe Trends in International Mathematics and Science Study is an international assessment of the mathematics and science knowledge of fourth- and eighth-grade students around the world...
(TIMSS)
Official websites and reports
- OECD/PISA website (Javascript required)
- OECD (1999): Measuring Student Knowledge and Skills. A New Framework for Assessment. Paris: OECD, ISBN 92-64-17053-7 http://www.oecd.org/dataoecd/45/32/33693997.pdf
- OECD (2001): Knowledge and Skills for Life. First Results from the OECD Programme for International Student Assessment (PISA) 2000.
- OECD (2003a): The PISA 2003 Assessment Framework. Mathematics, Reading, Science and Problem Solving Knowledge and Skills. Paris: OECD, ISBN 978-92-64-10172-2 http://www.oecd.org/dataoecd/46/14/33694881.pdf
- OECD (2004a): Learning for Tomorrow's World. First Results from PISA 2003. Paris: OECD, ISBN 978-92-64-00724-6 http://www.oecd.org/document56/0,2340,en_2649_201185_34016248_1_1_1_1,00.html
- OECD (2004b): Problem Solving for Tomorrow's World. First Measures of Cross-Curricular Competencies from PISA 2003. Paris: OECD, ISBN 978-92-64-00642-3
- OECD (2005): PISA 2003 Technical Report. Paris: OECD, ISBN 978-92-64-01053-6
- OECD (2007): Science Competencies for Tomorrow's World: Results from PISA 2006 http://www.pisa.oecd.org/document/2/0,3343,en_32252351_32236191_39718850_1_1_1_1,00.html
About reception and political consequences
- General:
- A. P. Jakobi, K. Martens: Diffusion durch internationale Organisationen: Die Bildungspolitik der OECD. In: K. Holzinger, H. Jörgens, C. Knill: Transfer, Diffusion und Konvergenz von Politiken. VS Verlag für Sozialwissenschaften, 2007.
- France:
- N. Mons, X. Pons: The reception and use of Pisa in France.
- Germany:
- E. BulmahnEdelgard BulmahnEdelgard Bulmahn is a German politician from the Social Democratic Party of Germany . Bulmahn entered the German Bundestag after the 1987 elections. She was Federal Minister of Education and Research from 1998 to 2005...
[then federal secretary of education]: PISA: the consequences for Germany. OECD observer, no. 231/232, May 2002. pp. 33–34. - H. Ertl: Educational Standards and the Changing Discourse on Education: The Reception and Consequences of the PISA Study in Germany. Oxford Review of Education, v 32 n 5 pp 619–634 Nov 2006.
- E. Bulmahn
- United Kingdom:
- S. Grek, M. Lawn, J. Ozga: Study on the Use and Circulation of PISA in Scotland. http://www.ces.ed.ac.uk/PDF%20Files/K%2BPWP12.pdf
Criticism
- Books:
- S. Hopmann, G. Brinek, M. Retzl (eds.): PISA zufolge PISA. PISA According to PISA. LIT-Verlag, Wien 2007, ISBN 3-8258-0946-3 (partly in German, partly in English)
- T. Jahnke, W. Meyerhöfer (eds.): PISA & Co – Kritik eines Programms. Franzbecker, Hildesheim 2007 (2nd edn.), ISBN 978-3-88120-464-4 (in German)
- R. Münch: Globale Eliten, lokale Autoritäten: Bildung und Wissenschaft unter dem Regime von PISA, McKinsey & Co.McKinsey & CompanyMcKinsey & Company, Inc. is a global management consulting firm that focuses on solving issues of concern to senior management. McKinsey serves as an adviser to many businesses, governments, and institutions...
Frankfurt am Main : Suhrkamp, 2009. ISBN 9783518125601 (in German)
- Websites:
- J . Wuttke: Critical online bibliography