Cours Methodology of the RGI, tutoriel & guide de travaux pratiques en pdf.

Calculating the RGI

The 45 indicators are given a score on a scale from 0 to 100 and are grouped thematically into three so-called components of the final index (Institutional and Legal Setting, Reporting Practices, and Safeguards and Quality Control). However, the amount of questions and indicators per component is not equally distributed with the Reporting Practices component, which is highly transparency focused, containing the majority of questions. There is also a fourth component (Enabling Environment) that is constructed using over 30 external measures and indices created by third parties3 that are combined into 5 indicators of the broader governance environment. It is worth noting that Michener (2015) warns against “index cannibalism” where indexes incorporate other indexes into their scores. However, the RGI report does contain a section arguing for and against the inclusion of the Enabling Environment external component indicating that the authors are aware of the risks present. Finally, the RGI data tool allows for custom weighting of the 4 components so users can exclude this component from the score.
If an indicator contains more than one question the score is a simple average of the question scores. The component score is also a simple average of the indicator scores. The values of the external
indicators for the Enabling Environment component are first normalized to a 0-100 scale. Finally, the RGI score is a weighted average of the four components’ scores and is also presented in the form of a single figure on the 0-100 scale that is divided into four performance ranges: 71-100 is satisfactory, 51-70
is partial, 41-50 is weak, and 0-40 is failing.
Table 1: The four components of the RGI
Finally, margins of error are also calculated as part of the RGI methodology. Margins of error are
estimated based on the extent of disagreement across indicators and components. First the simple average
of the standard deviation is calculated within and across components providing an average standard
deviation across the sample of 8. The implied margin of error around a country’s point estimate is
calculated to be +/- 13 with a 90 per cent confidence interval. There is also variance across individual countries with a range of 6.5–9.5 which creates different confidence intervals; the top and bottom
performers have lower standard deviations in general while those in the middle have higher ones.
It is important to point out that this is the second edition of the RGI index. However, the first edition was a pilot index in 2010 developed solely by the Resource Watch Institute, included only 41 countries, used less questions, and did not estimate margins of error. Given these changes, in the words of the authors themselves, the 2010 index is not comparable with the current edition. This is rather unfortunate because being able to evaluate the evolution of scores through time is a rather useful feature and would create a more robust measure. It will be interesting to see whether the results of the 2016 RGI will be comparable with the current edition and how this feature will be used.

Collecting the data

The primary data for the questionnaires, answering all 191 questions for the 58 countries, was collected by a team of researchers composed of 20 local civil society experts, 12 independent consultants based in-country, and 14 experts based outside the country they assessed. However, it is not clear whether these numbers refer to the experts per country or they refer to the totality of the human resources working on all countries. It is not specified in the methodological section at any point which is unusual since there is a big difference between the two cases.
Michener (2015) also warns about the pitfalls of relying on in-country experts that have been documented in the literature, which can create risks of feedback loops and biased information based on perceptions. It is also not clear whether the team of experts filling out the questionnaires is permanent or the team is composed every time a new edition of the RGI is being made and therefore different experts can work on different editions.

INCORPORATING GIS TOOLS INTO THE RGI METHODOLOGY

Critical review of the RGI

While not aiming to discredit the work of the NRGI and the success of the RGI as a resource governance measure several areas for improvement have been identified in the scholarly literature and some interesting geospatial applications in this field have motivated the following proposals for an improved index.
One important recommendation stemming from the literature, directly applicable to the RGI, is the fact that most transparency initiatives and measures focus on resource revenues and not expenditures. The argument is that if we want to measure corruption, development effects, and equitable distribution of resource wealth it is much more important to focus on expenditures rather than revenues (Bleischwitz 2009, Mejia Acosta 2010, Kolstad and Wiig 2009). “There is thus something of a disconnect between prominent current transparency initiatives, and the literature on the resource curse.” (Kolstad and Wiig p527, 2009)
Michener (2015) provides a salient analysis of international transparency policy indexes (including the RGI) and looks at their validity from several perspectives. Two of the more important aspects discussed are content validity4 and variable substitutability5. The RGI receives a positive evaluation of its content validity because it measures several dimensions of transparency as well as de jure and de facto elements. However, it does not go far enough on the question of substitutability since when looking at the production of transparent information it uses categories6 that do not sufficiently characterize the transparency of information: “The problem here is not only the concept of “distributed to the public”, which should be specified, but also the difficulty of differentiating between the first two – “not produced” from “produced for internal purposes” (Michener 2015). An important positive aspect of the RGI is the fact that it disaggregates the data used on the sub-national level. This feature makes it particularly interesting as a candidate for a deeper and potentially spatial use as a governance measure.
Finally, one could argue that the RGI focuses too heavily on transparency as a measure of good governance since 122 out of the 170 questions focus on transparency. It is as much about transparency as it is about governance or rather transparency is put forth as the main measure of good governance known as governance by disclosure or regulation by revelation in the literature (Michener 2015). Furthermore, a significant amount of questions are not applicable to all countries. Because of different institutional arrangements some questions simply do not correspond to the context of certain countries. A section on state-owned companies, for example, cannot be applied to a country like the United Kingdom which does not have state-owned companies and so the score for that country will be based on a smaller number of indicators. Another example are the 20 “reporting practices” questions which are not applicable to Canada (Alberta) because of the institutional arrangements in that case meaning that “Canada’s reporting score is based on nearly half as many evaluative queries as other countries” (Michener 2015). This type of mismatch dampens the legitimacy of the measure, creates possible biases toward certain institutional settings, possibly spreads policy isomorphism, and creates inflexible indexes that undermine comparability and applicability across countries (Michener 2015).
As mentioned before, the question of scale has been identified as one of high importance within the context of governance of natural resources. Mechanisms of cause and effect, stakeholder interactions, historic, national and regional contexts, and many other aspects of the extractive industry exist across temporal and spatial scales. In order to evaluate the governance of the sector and potentially improve development outcomes a multi-scale analysis is needed (Gilberthorpe and Papyrakis 2015, Ostendorf 2011). The RGI focuses on the national scale and for the moment is a score of a single year. It would therefore seem that this score does not take into account spatial and temporal scale complexities which makes it vulnerable to missing the whole picture.
The RGI therefore does not have major gaps in its methodology and seems to adequately capture the evaluation of transparency in the extractive industry. However, transparency is only one measure of governance and the RGI focuses perhaps too much on this proxy. There is definitely potential for a wider scope and improvements can be made to capture more dimensions of governance. Furthermore, the RGI report does not give any room to a theoretical framework of the concepts it purports to measure and it would surely benefit from an in-depth theoretical analysis of these concepts. It is nevertheless a relatively recent project with limited resources which can explain some of these shortfalls and it will be interesting to see its evolution in the future.

GIS and the RGI

Figure 9 Countries tab on the RGI website
Currently, the RGI website includes a “Countries” tab that presents a visualization of the RGI scores on a global scale and provides links to individual country and regional fact sheets and reports. Figure 9 presents a screenshot of the “Countries” tab on the RGI website. These sections are summaries of the country score broken down by component with graphs visualizing the data, links to download data and to view or download individual questionnaires. There is also a basic online data tool application that allows for interactive data visualization and manipulation. Figure 10 provides a screenshot of an individual country fact sheet and score analysis.
Figure 10 Example of an individual country fact sheet on the RGI website
In parallel, many countries already have or are putting in place online cadaster registries or portals that map data on concessions, licenses, individual mines, geological information, protected areas, and others. This is a very positive development but ideally all of this information on the extractive industry could be centralized. The Open Government Guide7 already has the creation of a public online registry of all natural resources concessions as an advanced commitment in the transparency of the extractive industry. Their website is a good example of centralization of existing information from multiple sources. As a first step, the RGI could include a question on the existence of such an online tool in their questionnaire or even a whole section on spatial data and tools available. The argument being that spatial data and tools to use it are very important in the context of governance of natural resources (extractive industries and resources are first of all physical, spatial phenomena). Therefore, the existence of an online mining registry or cadaster can surely be seen as a positive point for natural resources governance. The fact that such a tool exists and is usable by the public also attests to the commitment to transparency of the government providing it. The RGI could, therefore, first of all evaluate whether such tools exist, their quality, and also point to their location so that RGI users can quickly find them.
FlexiCadastre is a company that provides the service of development and management of online mining cadaster portals to governments and has already produced a number of such tools (Botswana, DRC, Kenya, Namibia, Mozambique, Papua New Guinea, Rwanda, South Sudan, Tanzania, Uganda, and Zambia)8. Their website even acknowledges the link between the EITI and their work arguing that their tools support the EITI goals. Western Australia’s (WA) Department of Mines and Petroleum has a very advanced website with lots of information and spatial data available for their extractive industry. According to McHenry et al. (2015) this is due to policy reform and the resulting Mining Securities Fund. Unfortunately their online map application requires the installation of Microsoft Silverlight which may prevent some users from accessing this tool. However, the main limiting factor of such applications is data availability; figure 11 illustrates this point. “Too often critical spatial information layers are difficult to come by at the appropriate spatial resolution and extent, hence reducing the overall strength of the combined evidence” (Ostendorf 2011). It seems somewhat strange then that the RGI website does not point users to such existing resources that could support the evaluation of resource governance and complement the numerical score. Collaboration between such efforts would greatly improve overall evaluation.