Reme, Thank you for bringing this new service to our attention. OA policies are vitally important to the development of institutional repositories, and services that can highlight and bring attention to this development can be valuable. There are a few aspects of the validation aspects of the new MELIBEA service that confuse, and possibly trouble, me. The first is the main indicator, %OAval, which is the most visible result for a policy. What do you expect this will tell people about a given policy? I randomly selected a couple of policies, one of which was for my own school, to find they each scored about 50%. I would expect these to be among the leaders in terms of OA policies, so this seems a surprisingly unhelpful score. So what's the explanation? Note that the objects being evaluated are institutional OA policies; they are effectively being presented in relation to institutional repositories when the policy specifies where to archive is an IR with a URL. It seems that the scores include ratings for OA publication policy, libre vs gratis OA, publisher pdf, sanctions (score if Yes), incentives (score if Yes), etc., some of which an institution might specify but which might not apply to an IR http://www.accesoabierto.net/politicas/politicas_estructura.php. However you weight these factors they are still contributors to the overall score, so a policy that is specific to an IR is immediately handicapped, or appears to be unless there is more context to understand the scores.
AS I WROTE BEFORE THIS IS NOT A RANKING, IT IS NOT THE AIM OF MELIBEA BUT TO HAVE A KIND OF REFERENCE ON WHAT TOPICS, ISSUES OR MATTERS TO BE INCLUDED IN AN OA POLICY. WE ARE TALKING ABOUT INSTITUTIONAL POLICIES OF DIFFERENTE NATURE, NOT ABOUT REPOSITORIES POLICIES. IF THE POLICY ONLY TALKS ABOUT THE REQUIREMENT TO DEPOSIT IN A REPOSITORY, IT SHOULD SPECIFY WHAT, WHEN AND UNDER WHAT CONDITIONS, IF ANY. IT IS NOT THE SAME TO SAY WHAT DOCUMENTS AND WHAT VERSIONS AND WHEN THAN SIMPLY SAY " ANY" OR "AS SOON AS POSSIBLE" (this could be a month after publication or years after publication, depending on one's criteria). GOLD ROUTE, NEVER IS REQUIRED ACCORDING OUR APPROACH ("Gold (Recommended in OA journals") AND NOT ALL OA JOURNALS ARE SUPPORTED BY SAME ECONOMIC MODEL.
COLORS DO NOT MEAN THAT, WE WANTED JUST TO DISTINGUISH ZONES LIKE IT WERE A SPECTRA.Which leads me to another question on the visualisation of the validator, and its use of green, gold (and red) in the meter. Do the green and gold refer the the classic OA colours? This would be quite convenient, since it would appear that the green repository policies I mentioned above are achieving almost full scores in the green zone of the meter. However, I suspect this cannot be the case, because it would assume that institutions must have a green AND gold policy, but not simply gold (whatever argument could be put for that).
I DO NOT THINK WE ARE MIXING, IN FACT THERE TWO MODELS, ONE FOR UNIV. AND RESEARCH CNETRES AND ANOTHER FOR FUNDERS AND GOV. INSTITUTIONS AND THE QUESTIONS FOR THEM ARE DIFFERENT, for instance, FOR A FUNDER THE QUESTION ABOUT DEPOSIT O THESIS IS NOT APPLICABLE.It is important that new services should help reveal and promote OA policies, as you seek to do, but at the same time not to prejudice the development of such policies by mixing and not fairly separating the contributing factors, especially where these relate to different types of OA.
R. Melero IATA, CSIC Avda Agustín Escardino 7, 46980 Paterna (Valencia), Spain TEl +34 96 390 00 22. Fax 96 363 63 01 E-mail firstname.lastname@example.org http://www.accesoabierto.net --