|
|
|||||||||
|
|
|
|
|
|||||
|
Budapest Open Access Initiative: BOAI Forum Archive [BOAI] [Forum Home] [index] [prev] [next] [options] [help]boaiforum messages[BOAI] Re: OA policies and their "weight"From: Stevan Harnad <harnad AT ecs.soton.ac.uk>
Dear Reme, if I may also make an intervention in your exchange with Steve ↵ Hitchcock about the MELIBEA OA policy evaluator: http://www.accesoabierto.net/politicas/ The MELIBEA service is extremely timely and promising, and could be potentially ↵ useful and even influential in shaping OA mandates -- but that makes it all the ↵ more important to get it right, rather than releasing MELIBEA prematurely, when ↵ it still risks increasing confusion rather than providing clarity and ↵ direction. You are right to point out that -- unlike the CSIC's University Ranking and the ↵ Repository Ranking -- the policy evaluator is not really a ranking. But you ↵ have set up the composite algorithm and the graphics to make it a ranking just ↵ the same. You are also point out, correctly, that the policy criteria for institutions ↵ and funders are not (and should not be) the same. Yet, with the MELIBEA coding ↵ as well as the algorithm, they are treated the same way. You also point out, rightly, that gold OA publishing policy is not central to ↵ institutional OA policy making, yet there it is, as part of the MELIBEA ↵ algorithm. You also point out that the color code has nothing to do with the ↵ "green" OA coding -- yet there it is, competing with the widespread ↵ use of green to designate self-archiving, and thereby causing confusion, both ↵ overt and covert. I would be more than happy to give you feedback on every aspect of MELIBEA -- ↵ it could be a useful and natural complement to the ROARMAP registry of OA ↵ policies. But as it is designed now, I can only agree with Steve Hitchcock's points and ↵ conclude that consulting MELIBEA today would be likely to induce confusion and ↵ would not help in bringing the all-important focus and direction to OA ↵ policy-making that I am sure CSIC, too, seeks, and seeks to help bring about. Here are just a few prima facie points: (1) Since MELIBEA is not, and should not be construed as a ranking of OA ↵ policies -- especially because it includes both institutional and funder ↵ policies -- it is important NOT to plug it into an algorithm until and unless ↵ the algorithm has first been carefully tested, with consultation, to make sure ↵ it weights policy criteria in a way that optimizes OA progress and guides ↵ policy-makers in the right direction. (2) For this reason, it is more important to allow users to generate separate ↵ flat lists of institutions or funders on the various policy criteria, ↵ considered and compared independently, rather than on the basis of a ↵ prematurely and arbitrarily weighted joint algorithm. (3) This is all the more important since the data are based on less then 200 ↵ institutions, whereas the CSIC University Rankings are based on thousands. ↵ Since the population is still so small, MELIBEA risks having a disproportionate ↵ effect on initial conditions and hence direction-setting; all the more reason ↵ NOT to amplify noise and indirection by assigning untested initial weights ↵ without carefully thinking through and weighing the consequences. (4) A potential internal cross-validator of some of the criteria would be a ↵ reliable measure of outcome -- but that requires much more attention to ↵ estimating the annual size and growth-rate of each repository (in terms of OA's ↵ target contents, which are full-text articles), normalized for institution size ↵ and annual total target output. Policy criteria (such as request/require or ↵ immediate/delayed) should be cross-validated against these outcome measures ↵ (such as percentage and growth rate of annual target output). (5) The MELIBEA color coding needs to be revised, and revised quickly, if there ↵ is to be an algorithm at all. All those arbitrary colors in the display of ↵ single repositories as ranked by the algorithm are both unnecessary and ↵ confusing. The objective should be to order and focus clearly and intuitively. ↵ Whatever is correlated with more green OA output (such as a higher level or ↵ faster growth rate in OA's target content) should be coded as darker or bigger ↵ shades of green. The same should be true for the policy criteria, separately ↵ and jointly: in each case, request/require, delayed/immediate, etc., the ↵ greenward polarity is obvious and intuitive. This should be reflected in the ↵ graphics as well as in any comparative rankings. (6) If you include repositories with no OA policy at all (i.e., just a ↵ repository and an open invitation to deposit) then all you are doing is ↵ duplicating ROAR and ROARMAP, whereas the purpose, presumably, of MELIBEA, is ↵ to highlight, weigh and compare specific policy differences among (the very ↵ few) repositories that DO have policies. (7) The sign-up data -- ↵ http://www.accesoabierto.net/politicas/nueva.php?directorio=politicas -- are ↵ also rather confusing; the criteria are not always consistent, relevant or ↵ applicable. The sign-up seems to be designed to make a funder mandate the ↵ generic option, whereas this is quite the opposite of reality. There are far ↵ more institutions and institutional repositories and policies than funders. ↵ There should be separate criterial lists for institutional policies and for ↵ funder policies; they are not the same. There is also far too much focus on ↵ gold OA policy and payment. If included at all, this should only be at the end, ↵ as an addendum, not the focus at the beginning, and on a par with green OA ↵ policy. (8) There is also potential confusion on the matter of "waivers": ↵ There are two aspects of a mandate. One concerns whether or not deposit is ↵ required (and if so, whether that requirement can be waived) and the other ↵ concerns whether or not rights-reservation is required (and if so, whether that ↵ requirement can be waived). These two distinct and independent ↵ requirements/waivers are completely conflated in the current version of ↵ MELIBEA. I hope there will be substantive consultation and conscientious redesign of ↵ these and other aspects of MELIBEA before it is can recommended for serious ↵ consideration and use. Stevan Harnad On 2010-07-19, at 5:18 AM, Remedios Melero wrote: > Dear Steve, > > I apologize for the delay in my response, but I will try to give some ↵ explanations to make clear some issues you raised in your message (my comments ↵ are in capital letters, to distinguish them from yours) > > > > El 15/07/2010 11:22, Steve Hitchcock escribió: >> Reme, Thank you for bringing this new service to our attention. OA ↵ policies are vitally important to the development of institutional ↵ repositories, and services that can highlight and bring attention to this ↵ development can be valuable. >> >> There are a few aspects of the validation aspects of the new MELIBEA ↵ service that confuse, and possibly trouble, me. The first is the main ↵ indicator, %OAval, which is the most visible result for a policy. What do you ↵ expect this will tell people about a given policy? I randomly selected a couple ↵ of policies, one of which was for my own school, to find they each scored about ↵ 50%. I would expect these to be among the leaders in terms of OA policies, so ↵ this seems a surprisingly unhelpful score. >> >> So what's the explanation? Note that the objects being evaluated are ↵ institutional OA policies; they are effectively being presented in relation to ↵ institutional repositories when the policy specifies where to archive is an IR ↵ with a URL. It seems that the scores include ratings for OA publication policy, ↵ libre vs gratis OA, publisher pdf, sanctions (score if Yes), incentives (score ↵ if Yes), etc., some of which an institution might specify but which might not ↵ apply to an IR >> http://www.accesoabierto.net/politicas/politicas_estructura.php >> . However you weight these factors they are still contributors to the ↵ overall score, so a policy that is specific to an IR is immediately ↵ handicapped, or appears to be unless there is more context to understand the ↵ scores. >> >> >> > AS I WROTE BEFORE THIS IS NOT A RANKING, IT IS NOT THE AIM OF MELIBEA BUT ↵ TO HAVE A KIND OF REFERENCE ON WHAT TOPICS, ISSUES OR MATTERS TO BE INCLUDED IN ↵ AN OA POLICY. WE ARE TALKING ABOUT INSTITUTIONAL POLICIES OF DIFFERENTE NATURE, ↵ NOT ABOUT REPOSITORIES POLICIES. IF THE POLICY ONLY TALKS ABOUT THE REQUIREMENT ↵ TO DEPOSIT IN A REPOSITORY, IT SHOULD SPECIFY WHAT, WHEN AND UNDER WHAT ↵ CONDITIONS, IF ANY. IT IS NOT THE SAME TO SAY WHAT DOCUMENTS AND WHAT VERSIONS ↵ AND WHEN THAN SIMPLY SAY " ANY" OR "AS SOON AS POSSIBLE" ↵ (this could be a month after publication or years after publication, depending ↵ on one's criteria). GOLD ROUTE, NEVER IS REQUIRED ACCORDING OUR APPROACH ↵ ("Gold (Recommended in OA journals") AND NOT ALL OA JOURNALS ARE ↵ SUPPORTED BY SAME ECONOMIC MODEL. > > >> Which leads me to another question on the visualisation of the ↵ validator, and its use of green, gold (and red) in the meter. Do the green and ↵ gold refer the the classic OA colours? This would be quite convenient, since it ↵ would appear that the green repository policies I mentioned above are achieving ↵ almost full scores in the green zone of the meter. However, I suspect this ↵ cannot be the case, because it would assume that institutions must have a green ↵ AND gold policy, but not simply gold (whatever argument could be put for that). ↵ >> >> > COLORS DO NOT MEAN THAT, WE WANTED JUST TO DISTINGUISH ZONES LIKE IT WERE ↵ A SPECTRA. > >> It is important that new services should help reveal and promote OA ↵ policies, as you seek to do, but at the same time not to prejudice the ↵ development of such policies by mixing and not fairly separating the ↵ contributing factors, especially where these relate to different types of OA. >> >> > I DO NOT THINK WE ARE MIXING, IN FACT THERE TWO MODELS, ONE FOR UNIV. AND ↵ RESEARCH CNETRES AND ANOTHER FOR FUNDERS AND GOV. INSTITUTIONS AND THE ↵ QUESTIONS FOR THEM ARE DIFFERENT, for instance, FOR A FUNDER THE QUESTION ABOUT ↵ DEPOSIT O THESIS IS NOT APPLICABLE. > IN SUMMARY, OUR MODEL COULD NOT BE "PERFECT" BUT I IS ONE, WHICH ↵ COULD DETECT DIFFERENCES BETWEEN REQUEST AND REQUIRE, WHO, WHAT , WHEN IF THERE ↵ ARE ANY INCENTIVES OR SANCTIONS ( this has not to be a negative point but to ↵ remember we should assume reponsible attitudes). > > However we will revise the model to see if we can make any improvement to ↵ make it clear, we are working also in a graph interface to show some data in ↵ graphical form. > Best wishes > Reme > >> >>> R. Melero >>> IATA, CSIC >>> Avda Agustín Escardino 7, 46980 Paterna (Valencia), Spain >>> TEl +34 96 390 00 22. Fax 96 363 63 01 >>> E-mail >>> rmelero AT iata.csic.es >>> >>> >>> http://www.accesoabierto.net >>> >>> >>> -- >>> >>> > > > -- > To unsubscribe from the BOAI Forum, use the form on this page: > http://www.soros.org/openaccess/forum.shtml?f -- To unsubscribe from the BOAI Forum, use the form on this page: http://www.soros.org/openaccess/forum.shtml?f [BOAI] [Forum Home] [index] [prev] [next] [options] [help]
|
E-mail: