1. What is the WJP Rule of Law Index®?
The WJP Rule of Law Index is an innovative quantitative assessment tool designed by the World Justice Project to offer a detailed and comprehensive picture of the extent to which countries adhere to the rule of law in practice. The Index provides detailed information and original data regarding a variety of dimensions of the rule of law, which enables stakeholders to assess a nation’s adherence to the rule of law in practice, identify a nation’s strengths and weaknesses in comparison to similarly situated countries, and track changes over time.
2. Which elements of the rule of law are measured by the Index?
The WJP Rule of Law Index provides data on nine dimensions of the rule of law:
- Limited Government Powers
- Absence of Corruption
- Order and Security
- Fundamental Rights
- Open Government
- Effective Regulatory Enforcement
- Effective Civil Justice
- Effective Criminal Justice
- Informal Justice
These nine factors are further disaggregated into 52 sub-factors. The Index aims to provide a picture of where countries stand with regard to a number of widely accepted outcomes that rule of law societies seek to achieve, as opposed to the institutional means, such as legal and regulatory frameworks, to attain them.
3. What are the Index’s data sources?
The WJP Rule of Law Index methodology utilizes two main sources of new data: (i) a general population poll (GPP), designed by the World Justice Project and conducted by leading local polling companies using a representative sample of a total of 1,000 respondents in three cities per country; and (ii) a qualified respondents’ questionnaire (QRQ) consisting of closed-ended questions completed by in-country practitioners and academics with expertise in civil and commercial law, criminal justice, labor law, and public health.
To date, over 2,000 experts from 66 countries and jurisdictions have contributed their knowledge and expertise to the Index. In addition over 66,000 individuals from these countries have participated in the general population poll.
4. How is this index different from other indices?
The WJP Rule of Law Index offers a reliable and independent data source for policy makers, businesses, non-governmental organizations and other constituencies to: (1) assess a nation’s adherence to the rule of law in practice, rather than in theory; (2) identify a nation’s strengths and weaknesses in comparison to similarly situated countries; and (3) track changes over time.
While the WJP Rule of Law Index enters a crowded field of indicators on different aspects of the rule of law, it has new features that set it apart:
Comprehensive. While existing indices cover aspects of the rule of law, they do not yield a full picture of rule of law compliance.
New data. The Index findings are based entirely on new data collected by the WJP from independent sources. This contrasts it with other indices based solely on data aggregated from third party sources, or on sources that are self-reported by governments or other interested parties.
Rule of law in practice. The Index measures adherence to the rule of law by looking not to the laws as written but at how they are actually applied in practice.
Anchored in actual experiences. The Index combines expert opinion with rigorous polling of the general public to ensure that the findings reflect the conditions experienced by the population, including marginalized sectors of society.
Action oriented. Findings are presented in disaggregated form, identifying strong and weak performers across the nine rule of law dimensions examined in each country.
5. How are countries selected?
Countries are selected to ensure diversity and representation of all regions, income levels, population sizes and legal traditions of the world. Methodological challenges are also taken into consideration.
6. What is the standard error of the Index scores?
This question can be rephrased in terms of the sources of uncertainty associated with the Index estimates. There are several sources of uncertainty associated with the Index estimates, including sampling error, missing data, weighting, normalization, or aggregation rules, to mention just a few. The standard error is related to one of these sources (sampling error) and derives from the fact that we select one sample out of the many possible samples available in the population.
To assess the impact of all these uncertainties on our estimates the Joint Research Centre of the European Commission has performed a sensitivity analysis based on a combination of a Monte Carlo experiments and a multi-modeling approach. The basic idea is to create hundreds of data sets, each one obtained under different assumptions, so as to create a confidence interval for the simulated ranks for all nine factors. The results show that country classifications across the nine factors are robust to methodological changes (90 percent of the countries shift less than ± 1 position).
The study authored by Michaela Saisana and Andrea Saltelli, as well as additional information about the composite indicators methodology developed by the EU-JRC and the OECD, is available here. Currently, we are working with the EU Joint Research Centre to generate thousands of samples using bootstrapping techniques, in order to incorporate the uncertainty associated with sampling (sampling error).
A related frequently asked question is about the margin of error of the general population polls conducted by the WJP Rule of Law Index in countries around the world. For most countries, with a confidence level of 95%, the margin of error is +/- 3.1% for most questions.
7. Does the Index reflect the true adherence to the rule of law even though it was only implemented in three cities?
The accuracy of the Index estimates depends on a number of variables, including the factor and the country in question. We should recall the national/local scope of the factors introduced by the Index is broad. Some factors, for instance, concern matters that are national in nature (such as factor 1.2 – government powers are effectively limited by the legislature), while others address issues that can vary from one neighborhood to another (such as factor 3.1 - crime rates).
In a similar fashion, the rule of law might differ between urban and rural environments. If the differences between the three largest cities and other areas (either urban or rural) are large, our results are less generalizable. With that said, in general, we believe our results can be generalized to other urban environments —where 65% of the population in our sampled countries lives. Moreover, we believe that there is much to learn from cross-country comparisons among urban populations as, in most cases, these variations tend to be more pronounced than in-country disparities.
The situation of rural areas is more complex and varies widely among countries. In the future, the WJP intends to conduct pilot tests of the Index in rural environments.
8. Do opinions of experts match the opinions of ordinary people?
The general population polls and the expert questionnaires are complementary sources, but they also provide different views of the same concepts. For those questions asked to both groups (and about which both are knowledgeable), the correlation is very high (above 0.8 in most cases).
9. How are experts selected?
The QRQs are completed by in-country practitioners and academics with expertise in civil and commercial law, criminal justice, labor law, and public health. Experts are selected by two methods: The first method -by which most of the experts are selected- involves a two-stage procedure. In the first stage, a large number of organizations are selected from a set of directories of law firms, universities/colleges, research organizations, and non-governmental organizations (NGOs). In the second stage, a random sample of experts is drawn from within the selected organizations. Once a sufficient number of potential respondents are identified, the questionnaires are sent to the selected individuals, with whom the team follows up. The second method builds on the WJP network of practitioners and academics, and people who have provided significant input to the development of the WJP Rule of Law Index.
10. What is the growth plan?
The WJP Rule of Law Index 2010 report, released to the public in October 2010, covers 35 countries. The WJP Rule of Law Index 2011, released to the public in June 2011, covers 66 countries and jurisdictions. It is expected that the Index will attain global coverage of 100 countries by the end of 2012.
11. Are women’s rights incorporated in the Index?
Yes, women’s rights are incorporated in two ways. First, by including questions to measure the concept of discrimination and equal treatment, which is a fundamental right; and second, by including a quota in the general population poll of 50% male and 50% female.
12. What does the Index assume about the functions of the state in different countries?
The Index assumes very little about the functions of the state. When it does so, it incorporates functions that are recognized by all societies, such as access to justice or order and security. However, it does not say anything about the institutional means to achieve these ends.
The WJP Rule of Law Index 2011 is the culmination of over four years of development, intensive consultation, and vetting with academics, practitioners, and community leaders from over 100 countries and 17 professional disciplines. The Index team regularly maintains dialogue with scholars, leaders, and practitioners from multiple disciplines to ensure that the Index is methodologically sound and applicable to societies with diverse social, political, and legal systems.
13. What is the purpose of the Index?
The Index is a tool to advance the rule of law. It is the WJP’s hope that by providing a comprehensive picture of each country’s situation with regard to the components of the WJP Rule of Law Index, we deliver a tool that can help policy makers, businesses, and civil society to identify trends, make arguments for action regarding important public policy issues, and place their country’s performance relative to others at the center of the policy discourse.
The Index, however, is not a tool for policy design. It is simply a tool to monitor the health of a country’s institutional environment—such as whether government officials are accountable under the law, and whether legal institutions protect fundamental rights and allow ordinary people access to justice. Its value lies in providing standardized indicators across countries, which allows cross-country comparisons and macro-level analysis.
14. What are some limitations of the Index?
Despite the methodological strengths of the Index, the findings should be interpreted in light of certain inherent limitations. While the Index is helpful in tracking the “temperature” of the rule of law situation in countries under study, it does not provide a full diagnosis or dictate concrete priorities for action. No single index can convey a full picture of a country’s situation. Rule of law analysis requires a careful consideration of multiple dimensions – which may vary from country to country – and a combination of sources, instruments, and methods. A more detailed account of limitations is available in the Data Notes section of the 2011 report.