| Popis: |
Recent high-profile statements, criticisms, and boycotts organized against certain quantitative indicators (e.g. the DORA declaration) have brought misuses of performance metrics to the center of attention. A key concern captured in these movements is that the metrics appear to carry authority even where established agents of quality control have explicitly outlined limits to their validity and reliability as measurement tools. This raises a number of challenging questions for those readers of this journal that are implicated in questions of indicator ‘production’ and, by extension, ‘effects’. In this opinion piece we wish to critically engage the question of how producers of indicators can come to terms with their role as (partly-) responsible parties in the current age of ‘evaluative bibliometrics’. We do so through the illuminating case of the professional scientometrics community. The article first describes how the landscape in which scientometrics operates has rapidly changed, becoming much more polyvocal. Inevitably this means more open competition for the attention space of users, and as a worst-case scenario the expertise of scientometrics risks being relegated to but one voice in a crowded marketplace. We then problematize how the community has often managed this relationship. We argue that the recourse taken thus far is towards an upstream solution, framed in terms that reinforce one’s own epistemic capabilities and professional position. This kind of boundary drawing between professional scientometrics and that of its intended audiences has contradictory effects, because it reproduces the gap between scientometric expertise and the practice level that the community is trying to bridge.Improving validity and reliability and standards development are useful endeavors, but if scientometricians draw the line of intervention here they risk - at best - maintaining the status quo in terms of how bibliometric products get (mis-) used in practice.The community can potentially draw on a large technical and social-scientific knowledge base, making them well placed to help analyze and change the ranges of conceivable types of actions and norms in current practices of research evaluation. The role of ‘objective outsiders’ who produce measures and standards but take no part in their intervention is no longer credible as a normative stance. In that particular arrangement the field situates itself almost entirely outside the practices of which scientometricians are also inherently part. |
