Dea Steve,

I apologize for the delay in my response, but I will try to give some explanations to make clear some issues you raised in your message (my comments are in capital letters, to distinguish them from yours)



El 15/07/2010 11:22, Steve Hitchcock escribió:
Reme,    Thank you for bringing this new service to our attention. OA policies are vitally important to the development of institutional repositories, and services that can highlight and bring attention to this development can be valuable.

There are a few aspects of the validation aspects of the new MELIBEA service that confuse, and possibly trouble, me. The first is the main indicator, %OAval, which is the most visible result for a policy. What do you expect this will tell people about a given policy? I randomly selected a couple of policies, one of which was for my own school, to find they each scored about 50%. I would expect these to be among the leaders in terms of OA policies, so this seems a surprisingly unhelpful score.

So what's the explanation? Note that the objects being evaluated are institutional OA policies; they are effectively being presented in relation to institutional repositories when the policy specifies where to archive is an IR with a URL. It seems that the scores include ratings for OA publication policy, libre vs gratis OA, publisher pdf, sanctions (score if Yes), incentives (score if Yes), etc., some of which an institution might specify but which might not apply to an IR http://www.accesoabierto.net/politicas/politicas_estructura.php. However you weight these factors they are still contributors to the overall score, so a policy that is specific to an IR is immediately handicapped, or appears to be unless there is more context to understand the scores.

  

AS I WROTE BEFORE THIS IS NOT A RANKING, IT IS NOT THE AIM OF MELIBEA  BUT TO HAVE A KIND OF REFERENCE ON WHAT TOPICS, ISSUES OR MATTERS TO BE INCLUDED IN AN OA POLICY. WE ARE TALKING ABOUT INSTITUTIONAL POLICIES OF DIFFERENTE NATURE, NOT ABOUT REPOSITORIES POLICIES. IF THE POLICY ONLY TALKS ABOUT THE REQUIREMENT TO DEPOSIT IN A REPOSITORY, IT SHOULD  SPECIFY WHAT, WHEN AND UNDER WHAT CONDITIONS, IF ANY. IT IS NOT THE SAME TO SAY WHAT DOCUMENTS  AND WHAT VERSIONS AND WHEN THAN SIMPLY SAY " ANY" OR "AS SOON AS POSSIBLE" (this could be a month after publication or years after publication, depending on one's criteria). GOLD ROUTE, NEVER IS REQUIRED ACCORDING OUR APPROACH ("Gold  (Recommended in OA journals") AND NOT ALL OA JOURNALS ARE SUPPORTED BY  SAME ECONOMIC MODEL.


Which leads me to another question on the visualisation of the validator, and its use of green, gold (and red) in the meter. Do the green and gold refer the the classic OA colours? This would be quite convenient, since it would appear that the green repository policies I mentioned above are achieving almost full scores in the green zone of the meter. However, I suspect this cannot be the case, because it would assume that institutions must have a green AND gold policy, but not simply gold (whatever argument could be put for that). 
  
COLORS DO NOT MEAN THAT, WE WANTED JUST TO DISTINGUISH ZONES LIKE IT WERE A SPECTRA.

It is important that new services should help reveal and promote OA policies, as you seek to do, but at the same time not to prejudice the development of such policies by mixing and not fairly separating the contributing factors, especially where these relate to different types of OA.
  
I DO NOT THINK WE ARE MIXING, IN FACT THERE TWO MODELS, ONE FOR UNIV. AND RESEARCH CNETRES AND ANOTHER FOR FUNDERS AND GOV. INSTITUTIONS AND THE QUESTIONS FOR THEM ARE DIFFERENT, for instance, FOR A FUNDER THE QUESTION ABOUT DEPOSIT O THESIS IS NOT APPLICABLE.
IN SUMMARY, OUR MODEL COULD NOT BE "PERFECT" BUT I IS ONE, WHICH COULD DETECT DIFFERENCES BETWEEN REQUEST AND REQUIRE, WHO, WHAT , WHEN IF THERE ARE ANY INCENTIVES OR SANCTIONS (  this has not to be a negative point but to remember we should  assume  reponsible attitudes).

However we will revise the model to see if we can make any improvement to make it clear, we are working also in a graph interface to show some data in graphical form.
Best wishes
Reme


R. Melero 
IATA, CSIC 
Avda Agustín Escardino 7, 46980 Paterna (Valencia), Spain 
TEl +34 96 390 00 22. Fax 96 363 63 01 
E-mail rmelero@iata.csic.es 
http://www.accesoabierto.net

--