Q: As a Senior Vice President of ETS’s Policy Evaluation & Research Center as well as the former Vice President for Assessment of the University of Tennessee Systems, what do you think is the role of education policy? In fact, what is education policy?
Dr. Nettles: Education public policy represents the actions taken by government to affect on education institutions and the people who participate in them. It ranges from the funding provided through annual appropriations to decisions made to regulate and control behavior. It encompasses laws passed by elected officials and established and enforced by bureaucrats to implement the laws that are enacted by the three branches of government.
We think about education public policy in the context of the public policy cycle which has three major components: First there is the actions which are taken to establish laws or policies – I distinguish laws from policies, because very often the requirements of a government are not just the laws, but also the rules, regulations and requirements established by bureaucratic entities. The second element of the policy cycle is implementation. Once we have laws or decisions made by government entities, the process of implementation has to be undertaken by either government or external entities or both. Third important element is evaluation. That’s why we have evaluation in the title of the policy evaluation and research center at ETS, because what happens periodically after laws and policies are enacted and implemented, there’s a period of asking whether they actually worked and how well they worked, and the answers to these questions feeds back into the process to inform of re-enactment and modification. Monitoring and evaluation are critical activities at ETS, from a research perspective, because we want to be able to apply our research and the results of our research to inform decision makers and policy makers about the actions that should be taken in the future.
Q: How do you evaluate a good policy? What do you consider a good policy for the education system?
Dr. Nettles: We actually try to conduct evaluations that are scientific, so that the evidence we produce is defensible and credible.
Lately, we’ve been conducting evaluations of programs and investments by private foundations. I’ll give you two examples. One evaluation that we recently conducted was related to investments made by the Goldman Sachs Foundation to increase the rate at which disadvantaged populations applied and were admitted to selective colleges and universities in the U.S., the U.K., Ireland and South Africa. Goldman Sachs is one of the largest investment banks in the world, and it invested 50 million dollars into the program that were designed to preparing children outside of conventional schools and school days, to enter and succeed in selective colleges and universities. Goldman Sachs was equally interested in knowing the return on its investment. They asked us to evaluate those programs and to give them a report on the return for their investment. They were primarily interested to know the economic benefit from the investment of 50 million dollars. So that was one of our projects and our report presented answers to that question.
Presently, we are conducting an evaluation of an investment made by the Robert Wood Johnson Foundation to examine the return for its investment into addressing the nation’s nursing shortage. In the US, we have a short supply of nurses, and one idea in which the Robert Wood Johnson Foundation considers valuable in addressing the problem is to recruit people who already have bachelor’s degrees in fields other than nursing and to recruit them to return to a participating university to pursue a bachelor’s or master’s degree in nursing. The Foundation asked us to generate the evidence required to inform them about the merits of that idea. So we’re in the process of conducting that evaluation now. Both evaluations have public policy implications, because they represent issues that are important to government at the national, state, and local levels. So our findings in these evaluations can be useful to public policy makers in making decisions about the types of policies they might pursue.
Q: I know that you are concerned about the growing education achievement gap too, is that correct?
Dr. Nettles: That is correct, I am indeed concerned about the growing education achievement gap, and I know that Lily Hsieh, the director of the Taitung project of improving English learning achievements in elementary schools, has the same concern. We invited her to come to the US to make a speech in San Antonio, Texas, last February, about a project that is being conducted in Eastern Taiwan. As a matter of fact, I plan to visit Taitung when I come to Taiwan next.
Twice each year since 2003, ETS has been convening symposiums on the topic of education achievement gaps. At this point, we have convened 16 symposiums since 2003. But we have had a long history at ETS of looking of measuring achievement gaps so that we can quantify them for the public. So one of the things that we take seriously at ETS is how well we measure the differences in achievement by various population groups.
Let me give you an example. As early as the 1980s, ETS was interested and concerned about the gap between boys and girls in mathematics. Boys were outperforming girls in mathematics for many years, and ETS became concerned about why this was the case. One conclusion from research was that boys were being given more support and encouragement to learn mathematics in school. And they were also being encouraged to take more advanced courses in mathematics in school. This was leading the country to have more boys in mathematics and scientific fields than girls. After bringing this to the public’s attention, there was a concerted effort in the US to close that gap by encouraging girls from an early age to study mathematics. What we also learned was that the consequence of that gap was that more boys were becoming life scientists, and physical scientists, than girls.
So that’s an example of an achievement gap where we’ve made substantial progress in the US. But we’ve also learned that this gap exists not just in the US, it’s all over the world. We have been examining gaps by race in the US, and we have an extraordinary gap between Blacks and Whites and Asian Americans in the US. Huge gaps, not just in mathematics, but also in literacy, in reading and writing, and English. So the problem that ETS is trying to lead the country in changing is to try to figure out how to close that gap.
It’s a gap by race and also a gap by socioeconomic class. People who are poor tend to do poorly relative to people who are well off. We are concerned about those gaps in the US but also around the world. That’s why Lily’s work in Taitung is really important to us. We are convening Salzburg Global Seminars to examine this issue throughout the world, not just in the US. We also have a website, an ETS micro-website that is devoted to our work on closing achievement gaps.
Q: Ok, so Michael, in your research, you found this problem, these achievement gaps, worldwide achievement gaps. Is there any way to narrow the gaps? Can you share what you are doing right now to narrow the gap?
Dr. Nettles: Yes, I’d like to offer three things that we are focusing on. One thing is that we are focusing on raising expectations. The point is that we have to make sure that the people who are performing on the lower end of the spectrum have the expectation that they can actually perform at the higher level. So we are taking a look at efforts under way, that are being developed in cities around the country in the US, that are designed to raise expectations. One of these initiatives is called the Kalamazoo Promise, in Kalamazoo, Michigan, where the city and the local community has promised every child in the community, whether they are poor or rich or middle-class, that if they graduate from high school, their college tuition will be paid for. This helps to raises the expectation of the children that they should prepare themselves to pursue the highest levels of education possible. So that’s one thing that we are focusing on. The second thing that we are focusing our attention on is the importance of having high quality teachers. What we have learned is that there is a relationship, a correlation, between the income of the students and the quality of the teachers they have Let me give you an example. We have two projects , one in the state of Tennessee and the other in the state of New York, where we are trying to help the teachers of economically disadvantaged and minority children in the field of mathematics. We have learned that the teachers that disadvantaged populations have are very often very good people, but their least favorite subject is mathematics.
Many of them don’t like mathematics, they fear it, they don’t embrace it. So we are trying to help them overcome the fear, and then to ensure that they have the content knowledge and the ability to teach their students mathematics. This is a very important project, because the process we are undertaking to improve their mathematics is something we will be working on to expand beyond the states of New York and Tennessee into other states and cities , and we think that the result of this will be an improvement in the performance, the knowledge, and achievement of the students.
So improving the teachers’ mathematics skills is a way to help the students?
Exactly, yes. The third area that we are launching our work is in early childhood education. Because this one is an emerging area for us at ETS, and we think it is extremely important. In order to close achievement gaps, we have to start the process very early. So, one of the things that we have not done historically at ETS is to develop assessments for children at really early ages. We have conducted research in the past on children at young ages, meaning age three to nine, in other words, pre-school and the earliest years of school. We have a history of conducting research and writing papers that are really substantial, very important papers. But we have not developed assessments yet. And one of the things that we think is important, is that there is an role for assessment to play in closing these gaps at the earliest ages. You see, the first time that we really observe these gaps at the national level in the US is through NAEP at fourth grade, and in international assessments, like PISA[i] and TIMSS[ii].We think it’s important to measure at that point, but it’s important to measure earlier and to intervene earlier, to address the learning differences that are occurring in pre-school and the early years of elementary school. So ETS has a new initiative to enter the early childhood education area.
Q: In your opinion, what do you think one needs to be equipped with to be considered a international talent in the future?
Yes, well, I think that it is important for people in Taiwan, in China, in the rest of the world, including the US, to have experience studying in other countries. Because we are now really very much a global society. We have more to gain internally and internationally by having greater exposure to languages, to learn more languages, not just English, but other languages as well, especially for people in the US, we are encouraging the acquisition of languages. I also think that, in the US, we have benefited greatly by having international students from Taiwan, from China, from Korea, and from other Asian countries to come to the US to study at the graduate level in particular. We think that both the development of science and scientific methodologies and of culture at the post-college level, beyond the bachelor’s degree, has benefited the US by having people return to their countries with greater familiarity with US society and culture and also the intellectual growth that they acquire from their experiences in the US. It helps international relationships. At the same time, the US has benefited by having the enormous talent of people from Taiwan, from China, from other countries to come and participate in the advancement, at the graduate level, especially in of science and technology. I like to promote the global society in higher education, and we appreciate and would like to continue to promote the participation by students from Taiwan, from China, from Asia to continue to study in the US. And we hope that we’ll be increasing opportunities for US students at the post-graduate level as well as undergraduate and post-doctoral level to engage in international collaboration with scientists in Taiwan and China and other countries, as we continue to develop as a global society.
That’s very inspiring for students in Taiwan, I believe. OK, Michael, I’ve finished all my questions. Thank you very much for your time.
[i] Programme for International Student Assessment
[ii] Trends in International Mathematics and Science Study