Important note: The views and opinions expressed in this material are solely those of the author based on the academic freedom to express thoughts and challenge the status quo, and do not necessarily reflect the official policy or position of the Technical University of Cluj-Napoca. This material is the sole responsibility of the author, prof. Stelian Brad.

Introduction

In the past few decades, the landscape of higher education has undergone significant changes. Universities are no longer just institutions of higher learning, but also have become economic entities that operate in a globalized and highly competitive environment. To succeed in this environment, universities have adopted various models to operate efficiently and effectively. These models vary widely depending on the university’s size, mission, funding sources, and geographic location. Today’s universities operate in a complex web of relationships with various stakeholders, including students, faculty, staff, alumni, donors, governments, and industry partners. In this context, universities must balance their academic mission with the need to generate revenue, maintain their reputation, and stay competitive in a rapidly changing world.

Despite these trends, many universities are still trapped in the so called Goodhart’s law. This is a concept in economics and social sciences that states that “when a measure becomes a target, it ceases to be a good measure.” In other words, when a particular metric is used as a target to achieve a goal, people may start to manipulate or game the system to achieve the target, rather than actually accomplishing the underlying goal. This can lead to unintended consequences and distortions in behavior. Goodhart’s law is named after economist Charles Goodhart, who first proposed the concept in the context of monetary policy. He argued that using a single metric, such as the money supply, to guide monetary policy could lead to unintended consequences and distortions in the economy.

One example of Goodhart’s law in action is in education. If a school district sets a goal of increasing student test scores, teachers may focus exclusively on teaching to the test, rather than providing a more well-rounded education. This can lead to a narrowing of the curriculum and a focus on rote memorization, rather than critical thinking skills. Another example is in the workplace, where setting targets for sales or productivity can lead to employees cutting corners or engaging in unethical behavior to achieve the targets. Goodhart’s law highlights the importance of carefully selecting metrics and targets, and being aware of unintended consequences that may arise when using them to guide decision-making.

In the next sections, various practices from universities are analyzed from the lens of Goodhart’s law. After that, some ideas are emmitted for reflections in relation to improve the conditions and move away from the negative consequences of the Goodhart’s law.

Admission of students without an exam, only based on the baccalaureate grade and the general high school average

In the context of admission to a university, if the baccalaureate grade and general high school average become the only factors considered, they may become the sole targets and lose their effectiveness as measures of a student’s potential for success. This can have several unintended consequences, such as students solely focusing on achieving high grades rather than developing a well-rounded skill set or pursuing their passions. Additionally, this approach can lead to a lack of diversity in the student body, as students who may have valuable skills or experiences outside of academic achievement may be overlooked. The domino effect of this approach can also be seen in the larger education system, where a heavy emphasis on standardized testing and grades can lead to a narrowing of the curriculum and a focus on teaching to the test, rather than promoting critical thinking and creativity.

Funding of universities from the Ministry per equivalent student

When funding is allocated per equivalent student, Goodhart’s law suggests that institutions may prioritize increasing the number of equivalent students rather than focusing on the quality of education. This can lead to a focus on admitting students who are more likely to succeed academically or increasing enrollment in programs that require less resources, rather than investing in programs or initiatives that may improve the overall quality of education. In some cases, this can also lead to a decrease in academic rigor or standards in order to ensure that more students pass and can be counted towards the funding allocation. Additionally, institutions may also focus on retaining students rather than ensuring they receive a high-quality education, as retention rates also contribute to the calculation of equivalent students. Ultimately, this can lead to a distortion of incentives and priorities within educational institutions.

The criteria for awarding doctorate qualifications on the basis of scientific publications

Goodhart’s law can be applied to the criteria for awarding doctorate qualifications on the basis of scientific publications. When publications become the sole focus and criteria for earning a doctorate, researchers may prioritize quantity over quality or manipulate their research to produce more publications. This can result in a decrease in the overall quality and integrity of scientific research.

Furthermore, this can lead to a “publish or perish” culture, where researchers are incentivized to publish as many papers as possible in order to secure funding or advance their careers. This can lead to a situation where researchers prioritize quantity over quality, leading to an increase in low-quality or even fraudulent research.

In conclusion, while using scientific publications as a measure of research productivity and quality can be useful, it is important to be aware of the potential for Goodhart’s law to take effect and to ensure that quality is not sacrificed for quantity.

Advancement in academic career is mainly based on criteria related to publication in ISI journals

In the case of academic careers, a common measure of success and progress is publication in highly-ranked journals, such as those indexed by the ISI (Institute for Scientific Information). However, this can lead to the unintended consequences of researchers focusing on publishing in these journals at the expense of other important aspects of research, such as interdisciplinary collaboration, mentoring students, or addressing socially relevant topics that may not be seen as “hot” or “trendy” in the field. This can also lead to a bias in favor of certain research topics or methodologies that are more likely to be accepted by these journals, even if they do not have significant real-world impact or value. As a result, Goodhart’s law is applicable to the criteria for advancement in academic careers based on publication in ISI journals, as it can create unintended consequences and distort incentives in the academic research community.

International recognition of researchers based on the number of citations of their papers

When a metric is used to incentivize certain behaviors or outcomes, people may start manipulating the metric to achieve the desired outcome, leading to unintended consequences. In the case of international recognition of researchers based on the number of citations, this metric has become a target for many researchers who want to advance their careers. As a result, some researchers may engage in practices such as self-citation or citation rings to artificially boost their citation count, rather than focusing on producing high-quality research that contributes to the field.

Furthermore, this metric may also create a bias towards certain types of research that are more likely to be cited, such as studies in popular or high-impact journals, and state-of-the-art analyses, rather than research that may be equally valuable but does not receive as much attention. Overall, while citation count can be a useful metric for measuring the impact of research, it is important to be aware of the potential unintended consequences of using it as the sole criterion for international recognition of researchers. It is important to consider other factors such as the quality and relevance of the research, as well as the researcher’s contributions to their field beyond just their publication record.

The criteria for awarding additional funding to universities is mainly based on publications in Q1, and Q2 journals

Goodhart’s law phenomenon can be also observed where various metrics are used to evaluate the performance of researchers and universities. For example, in the case of awarding additional funding to universities, the criteria for selection may be based on the number of publications in Q1 and Q2 journals. However, this can lead to the unintended consequence of universities prioritizing the quantity of publications over the quality of research. Researchers may be incentivized to publish their work in these specific journals, rather than pursuing more innovative or impactful research. This can create a situation where universities may have a high number of publications, but the quality of their research may not be proportional to the funding they receive. Therefore, Goodhart’s law is a crucial consideration when designing evaluation systems in academia, as it is important to consider the potential unintended consequences of using certain metrics as targets.

Accreditation of specializations – the criteria used and strictness of the percentages allocated to various disciplines

Goodhart’s law  is also observed in educational systems where certain metrics are used to evaluate the quality of institutions, programs, or individuals, and these metrics can become the sole focus of attention, rather than the broader goals of education. In the case of accreditation of specializations, the criteria used and the strictness of the percentages allocated to various disciplines (fundamentals, specialization, complementary, domain) can lead to unintended consequences. For example, if the accreditation process focuses too heavily on specific metrics, such as the percentage of students in a particular specialization or the number of research publications in a specific field, then institutions may focus their resources on those metrics at the expense of other important aspects of education, such as interdisciplinary collaboration or community engagement.

Furthermore, if institutions are incentivized to prioritize certain metrics, this may lead to gaming the system, where institutions manipulate the data to appear more successful than they actually are. This can create a situation where the metrics no longer accurately reflect the quality of education, but rather the ability of the institution to meet the specific criteria.

Therefore, it is important to be aware of Goodhart’s law when designing evaluation metrics and accreditation criteria in education, to ensure that the metrics are used appropriately and do not become the sole focus of attention at the expense of broader goals of education.

Evaluation of research projects on the Hirch results of the project director

Goodhart’s law is particularly relevant when it comes to evaluating research projects based on the Hirsch index (h-index) of the project director. The h-index is a measure of a researcher’s productivity and impact in their field, based on the number of publications they have produced and the number of citations those publications have received. However, when this becomes the primary measure for evaluating research projects, it can incentivize researchers to prioritize quantity over quality and to focus on topics that are more likely to generate citations, rather than pursuing more innovative or risky research directions. This can lead to a lack of diversity and creativity in research, and can also put pressure on researchers to engage in practices such as self-citation or citation manipulation in order to boost their h-index. Ultimately, this can lead to a reduction in the quality and impact of research overall. Therefore, it is important to use multiple measures and criteria for evaluating research projects, rather than relying solely on a single metric such as the h-index.

The academic performance of a country is based on the number of students / total inhabitants and number of PhD students / total inhabitants

In the case of evaluating the academic performance of a country based on the number of students per total inhabitants, this can be seen as an example of Goodhart’s law. When the number of students becomes the target, the quality of education and the actual knowledge gained by students can be neglected. This could lead to a decrease in the quality of education as institutions focus more on enrolling more students rather than ensuring they receive a high-quality education. Additionally, using the total number of inhabitants as the denominator does not take into account the actual number of students who are eligible and interested in pursuing higher education. It also does not consider factors such as the availability of educational resources and funding, which can significantly impact the quality of education provided. Thus, solely relying on this metric can lead to a misrepresentation of a country’s actual academic performance and may not accurately reflect the quality of education provided.

Involvement of students when they have gratuity of higher education

In the context of a gratuity of higher education, the measure of success may be the number of students who complete their degree program. However, if this measure becomes the sole focus, it may lead to unintended consequences. For example, students may be encouraged to take easier courses or to drop challenging classes to maintain their Grade Point Average (GPA) and ensure that they receive the gratuity. GPA is a numeric representation of a student’s academic performance in a particular period, usually a semester or academic year. This can result in a decrease in the quality of education and knowledge gained by the students.

Furthermore, if the measure of success is solely based on the number of students who complete their degree program, it may also lead to neglecting other important factors such as the quality of education, student engagement, and the overall student experience. The gratuity may incentivize students to complete their degree program, but it may not necessarily incentivize them to become more involved in the learning process or to pursue excellence in their studies.

Therefore, while the gratuity of higher education can be a motivating factor for students to complete their degree program, it should not be the sole focus. Other important factors such as the quality of education and student engagement should also be taken into account to ensure that students are receiving a well-rounded education that prepares them for success in their future careers.

Scholarships associated with the grades, but not considering that specializations of students are not comparable

In this lens, the measure is the grades and the target is the scholarship. When scholarships are associated with grades, it creates an incentive for students to focus solely on achieving high grades instead of learning and understanding the material. This can lead to the phenomenon of grade inflation, where students receive higher grades than they actually deserve, and it can also create a culture of competition among students. Additionally, this approach does not take into account the fact that different specializations have different levels of difficulty and should not be compared solely based on grades. Therefore, while scholarships can be an effective way to motivate students, they need to be designed in a way that does not promote unintended consequences such as grade inflation and a lack of focus on actual learning.

The international ranking of universities using different models

Goodhart’s law can also be applied to the international ranking of universities. International rankings play a critical role in shaping the perception of the quality and prestige of universities, influencing not only student enrollment decisions but also funding decisions by governments and other organizations. However, rankings are typically based on various criteria, such as research output, teaching quality, student-to-faculty ratio, and internationalization, which can be gamed or manipulated. As a result, universities may focus on optimizing their rankings by selectively allocating resources and efforts towards the ranking criteria, often at the expense of other areas. For example, a university may prioritize research output to improve its ranking, leading to a decrease in teaching quality or neglect of other important areas such as community engagement or diversity. This can create a situation where universities are incentivized to optimize their ranking rather than pursuing excellence in all areas. Therefore, Goodhart’s law applies to university rankings, as universities may end up manipulating their performance on the criteria used for the ranking, rather than striving for excellence across all dimensions of their mission.

How do universities have to act to escape from the trap of Goodhart’s law?

Unfortunately, most of the universties nowadays operate with Cartezian, linear value-added models, where linear measures are designed to gauge performances. When such measures become the focus of attention, they push the actors from the system to manipulation and distortion of the measures themselves. This can be a challenge for universities, which often rely on measures such as publications, citations, and rankings to assess performance and attract funding. To escape the trap of Goodhart’s law, universities might take several actions, such as:

• Diversify the measures and ensure equivalence of measures to capture diversity: Universities can use a variety of measures to assess performance, including measures that go beyond traditional metrics such as publications and citations. This can include measures such as student satisfaction, community engagement, and impact on society. Also they can include measures of student success beyond just graduation rates, such as job placement or graduate school acceptance rates.
• Contextualize the measures: Universities can take into account the context in which measures are being used. For example, if a university has a strong focus on teaching and community engagement, measures such as student satisfaction and community impact may be more relevant than research publications.
• Use measures as a guide, not a target: Universities can use measures as a guide to performance, rather than a target to be met. This can help ensure that measures are not distorted or manipulated to achieve a desired outcome.
• Foster a culture of transparency and integrity: Universities can foster a culture of transparency and integrity in which all stakeholders, including faculty, administrators, and students, are encouraged to act with honesty and integrity in all aspects of university life. This can help reduce the likelihood of measures being manipulated or distorted.
• Foster a culture of innovation and continuous improvement: Finally, universities can seek to foster a culture of innovation and continuous improvement, rather than simply striving to meet predetermined metrics. This can involve encouraging experimentation and risk-taking, and promoting a growth mindset among students, faculty, and staff. By focusing on innovation and improvement rather than simply meeting specific targets, universities can avoid the negative effects of Goodhart’s law and instead create a more dynamic and resilient educational environment.

By taking these actions, universities can escape the trap of Goodhart’s Law and ensure that measures are used in a responsible and effective manner to assess performance and guide decision-making.

But how universities could escape from Goodhart’s law as long as most of these rules are imposed by central or international bodies?

While it is true that many of these rules and metrics are imposed by central or international bodies, universities can still take action to mitigate the negative effects of Goodhart’s law. One approach is to diversify the metrics and criteria used to evaluate success, instead of relying solely on the metrics or criteria imposed by central bodies. For example, instead of solely using publication metrics to evaluate the success of a research project or academic program, universities could also consider other factors such as impact on society, collaborations with industry or community partners, and teaching effectiveness.

Another approach is to encourage transparency and accountability in the use of metrics and criteria by the central bodies. This includes regularly reviewing and updating the metrics and criteria used, as well as communicating to stakeholders how these metrics are being used and any limitations or potential biases associated with them.

Ultimately, it is important for universities to remember that while metrics and criteria are useful tools for evaluation and decision-making, they should not be the sole determinants of success or failure. Universities should strive to maintain a holistic view of their goals and objectives, and use a variety of approaches to measure progress and success.

Leaders of universities still have some more means to mitigate the negative effects of Goodhart’s law on the institution, even within the constraints of rules imposed by central bodies, such as:

• Promote a culture of transparency and accountability: Ensure that all stakeholders understand the limitations of the metrics and rankings used to evaluate the institution and the potential unintended consequences of solely focusing on these measures. Encourage an open dialogue about the goals and values of the university, and regularly report on progress towards these goals.
• Focus on the quality of education: Emphasize the importance of providing students with a good, balanced and comprehensive education that prepares them for a variety of careers and life paths, rather than simply chasing high rankings or metrics. This can involve investing in innovative teaching methods, personalized mentorship, and experiential learning opportunities.
• Encourage interdisciplinary research and collaboration: Foster a culture of collaboration and cross-disciplinary research, which can lead to innovative solutions and a more holistic understanding of complex problems. This can also help to counteract the tendency for researchers to focus on narrow, easily measurable topics in order to boost their rankings.
• Engage with stakeholders and the wider community: Focus on building strong partnerships with local businesses, non-profits, and other organizations, and seek to understand and address their needs and priorities. This can help to demonstrate the value of the university beyond traditional metrics, and can also lead to new opportunities for collaboration and innovation.
• Advocate for change at the central level: Work with other university leaders to advocate for changes in the metrics and rankings used to evaluate institutions, and to promote a more nuanced and holistic understanding of the role and value of higher education. This can involve engaging with policymakers, lobbying for regulatory reform, and collaborating with other universities and stakeholders to advance a shared agenda.

Conclusion

Goodhart’s law presents a major challenge to universities, as it can lead to unintended consequences and negative outcomes when metrics become the focus rather than the actual goals of education and research. This is particularly true when external bodies impose rules and regulations that may not align with the mission and values of individual institutions. As seen in our discussion, universities must be proactive in addressing Goodhart’s law by carefully selecting and monitoring metrics, considering unintended consequences, and placing a stronger emphasis on qualitative measures of success. This will require a shift in mindset and a commitment to prioritizing the true goals of education and research over the pressure to conform to external measures of success. Ultimately, by taking a more thoughtful and holistic approach to measuring and evaluating performance, universities can help to mitigate the negative impact of Goodhart’s law and create a more effective and sustainable system of education and research.

The emphasis on metrics and quantifiable measures of success leads to unintended negative impacts, such as the narrowing of research focus, the prioritization of certain types of publications, and the neglect of other important aspects of academia. This has not only affects the quality of education and research but also results in a lack of innovation and progress in certain areas.

Furthermore, the Goodhart’s law has the potential to create a culture of gaming the system and incentivize unethical behavior, as individuals and institutions may prioritize achieving certain metrics at the expense of ethical standards and values. Who is responsible for the long-term negative impact on society and possible unethical use of public funds when we simply conform to poor regulations without critically examining and intervening for necessary corrections?