Budapest Open Access Initiative      

Budapest Open Access Initiative: BOAI Forum Archive

[BOAI] [Forum Home] [index] [options] [help]

boaiforum messages

[BOAI] OA policies and their "weight"

From: Remedios Melero <rmelero AT iata.csic.es>
Date: Thu, 15 Jul 2010 09:14:19 +0200


Threading:      • This Message
             [BOAI] Re: OA policies and their "weight" from sh94r AT ecs.soton.ac.uk
             [BOAI] Re: OA policies and their "weight" from rmelero AT iata.csic.es
             [BOAI] Re: OA policies and their "weight" from rmelero AT iata.csic.es
             [BOAI] Re: OA policies and their "weight" from harnad AT ecs.soton.ac.uk


Good mornig!

In the last Open Repositories Conference which was held last week in 
Madrid (http://or2010.fecyt.es/publico/Home/index.aspx ) was presented 
in the poster session the project called MELIBEA.

MELIBEA (http://www.accesoabierto.net/politicas/)  is a directory and  a 
validator of institutional open-access (OA) policies regarding 
scientific and academic work. As a directory, it describes the existing 
policies. As a validator, it subjects them to qualitative and 
quantitative analysis based on fulfilment of a set of indicators ( 
http://www.accesoabierto.net/politicas/politicas_estructura.php) that 
reflect the bases of an institutional policy.

Based on the values assigned to a set of indicators, weighted according 
to their importance, the validator indicates a score and a percentage of 
fulfilment for each policy analyzed. The sum of weighted values of each 
indicator is converted to a percentage scale to give what we have called 
the "validated open-access percentage" (see how i t is calculated:  
http://www.accesoabierto.net/politicas/default.php?contenido=acerca ).

The types of institution analyzed include universities, research 
centres, funding agencies and governmental organizations.

MELIBEA has three main objectives:

    * 1. To establish indicators that reveal the strong and weak points
      of institutional OA polices.
    * 2. To propose a methodology to guide institutions when they are
      drawing up an institutional OA policy.
    * 3. To offer a tool for comparing the contents of policies between
      institutions.

The aim is not to be a ranking, but to offer a tool where to aanlyse and 
visualize the weaknesses or strenghts of an institutional OA policy 
based on its wording. It seems something trivial  but accomplishment of 
a policy is based on its terms.
Please if you detect any mistake or you would like to make a comment, 
contact me. I will be pleased if you could check your policy, if any, to 
analyse our approach.
Best wishes
Reme


R. Melero
IATA, CSIC
Avda Agustín Escardino 7, 46980 Paterna (Valencia), Spain
TEl +34 96 390 00 22. Fax 96 363 63 01
E-mail rmelero AT iata.csic.es
http://www.accesoabierto.net

ATTACHMENT: message.html!

-- To unsubscribe from the BOAI Forum, use the form on this page: http://www.soros.org/openaccess/forum.shtml?f

[BOAI] Re: OA policies and their "weight"

From: Steve Hitchcock <sh94r AT ecs.soton.ac.uk>
Date: Thu, 15 Jul 2010 10:22:30 +0100


Threading: [BOAI] OA policies and their "weight" from rmelero AT iata.csic.es
      • This Message

Reme,    Thank you for bringing this new service to our attention. OA policies 
are vitally important to the development of institutional repositories, and 
services that can highlight and bring attention to this development can be 
valuable.

There are a few aspects of the validation aspects of the new MELIBEA service 
that confuse, and possibly trouble, me. The first is the main indicator, 
%OAval, which is the most visible result for a policy. What do you expect this 
will tell people about a given policy? I randomly selected a couple of 
policies, one of which was for my own school, to find they each scored about 
50%. I would expect these to be among the leaders in terms of OA policies, so 
this seems a surprisingly unhelpful score.

So what's the explanation? Note that the objects being evaluated are 
institutional OA policies; they are effectively being presented in relation to 
institutional repositories when the policy specifies where to archive is an IR 
with a URL. It seems that the scores include ratings for OA publication policy, 
libre vs gratis OA, publisher pdf, sanctions (score if Yes), incentives (score 
if Yes), etc., some of which an institution might specify but which might not 
apply to an IR http://www.accesoabierto.net/politicas/politicas_estructura.php. 
However you weight these factors they are still contributors to the overall 
score, so a policy that is specific to an IR is immediately handicapped, or 
appears to be unless there is more context to understand the scores.

Which leads me to another question on the visualisation of the validator, and 
its use of green, gold (and red) in the meter. Do the green and gold refer the 
the classic OA colours? This would be quite convenient, since it would appear 
that the green repository policies I mentioned above are achieving almost full 
scores in the green zone of the meter. However, I suspect this cannot be the 
case, because it would assume that institutions must have a green AND gold 
policy, but not simply gold (whatever argument could be put for that). 

It is important that new services should help reveal and promote OA policies, 
as you seek to do, but at the same time not to prejudice the development of 
such policies by mixing and not fairly separating the contributing factors, 
especially where these relate to different types of OA.

Steve Hitchcock
IAM Group, Building 32
School of Electronics and Computer Science
University of Southampton, SO17 1BJ, UK
Email: sh94r AT ecs.soton.ac.uk
Twitter: http://twitter.com/stevehit
Connotea: http://www.connotea.org/user/stevehit
Tel: +44 (0)23 8059 7698    Fax: +44 (0)23 8059 2865

On 15 Jul 2010, at 08:14, Remedios Melero wrote:

> Good mornig!
> In the last Open Repositories Conference which was held last week in 
Madrid (http://or2010.fecyt.es/publico/Home/index.aspx ) was presented in the 
poster session the project called MELIBEA.
> MELIBEA (http://www.accesoabierto.net/politicas/)  is a directory and  a 
validator of institutional open-access (OA) policies regarding scientific and 
academic work. As a directory, it describes the existing policies. As a 
validator, it subjects them to qualitative and quantitative analysis based on 
fulfilment of a set of indicators ( 
http://www.accesoabierto.net/politicas/politicas_estructura.php) that reflect 
the bases of an institutional policy.
> 
> Based on the values assigned to a set of indicators, weighted according to 
their importance, the validator indicates a score and a percentage of 
fulfilment for each policy analyzed. The sum of weighted values of each 
indicator is converted to a percentage scale to give what we have called the 
“validated open-access percentage” (see how i t is calculated:  
http://www.accesoabierto.net/politicas/default.php?contenido=acerca ).
> 
> The types of institution analyzed include universities, research centres, 
funding agencies and governmental organizations.
> 
> MELIBEA has three main objectives:
> 
> 	• 1. To establish indicators that reveal the strong and weak points of 
institutional OA polices.
> 	• 2. To propose a methodology to guide institutions when they are drawing 
up an institutional OA policy.
> 	• 3. To offer a tool for comparing the contents of policies between 
institutions.
> The aim is not to be a ranking, but to offer a tool where to aanlyse and 
visualize the weaknesses or strenghts of an institutional OA policy based on 
its wording. It seems something trivial  but accomplishment of a policy is 
based on its terms.
> Please if you detect any mistake or you would like to make a comment, 
contact me. I will be pleased if you could check your policy, if any, to 
analyse our approach.
> Best wishes
> Reme
> 
> 
> R. Melero 
> IATA, CSIC 
> Avda Agustín Escardino 7, 46980 Paterna (Valencia), Spain 
> TEl +34 96 390 00 22. Fax 96 363 63 01 
> E-mail rmelero AT iata.csic.es 
> http://www.accesoabierto.net
> 
> --      
> To unsubscribe from the BOAI Forum, use the form on this page:
> http://www.soros.org/openaccess/forum.shtml?f



Steve Hitchcock
IAM Group, Building 32
School of Electronics and Computer Science
University of Southampton, SO17 1BJ, UK
Email: sh94r AT ecs.soton.ac.uk
Twitter: http://twitter.com/stevehit
Connotea: http://www.connotea.org/user/stevehit
Tel: +44 (0)23 8059 7698    Fax: +44 (0)23 8059 2865








        
--      
To unsubscribe from the BOAI Forum, use the form on this page:
http://www.soros.org/openaccess/forum.shtml?f


[BOAI] Re: OA policies and their "weight"

From: Remedios Melero <rmelero AT iata.csic.es>
Date: Thu, 15 Jul 2010 11:59:30 +0200


Threading: [BOAI] OA policies and their "weight" from rmelero AT iata.csic.es
      • This Message

Thanks Steve for your comments,

I will be very briefly now because I have to leave, however if you read 
how the weight have been calculated, and the variables within the model, 
you will  realise that this approach  has been prepared for any kind of 
instituttion, funder or univ... that means some questions are valid not 
for any of them and that has been taken into account in the formula to 
calculate the percentage.

About colors, we could modify them, but that is not the issue, I will  
make clear any other question early tomorrow. In any case the aim of  
MELIBEA is to foster OA policies not to confuse people, therefore any 
suggestion to improve it will be considered, analysed and applied .
Reme







El 15/07/2010 11:22, Steve Hitchcock escribió:
> Reme,    Thank you for bringing this new service to our attention. OA 
policies are vitally important to the development of institutional 
repositories, and services that can highlight and bring attention to this 
development can be valuable.
>
> There are a few aspects of the validation aspects of the new MELIBEA 
service that confuse, and possibly trouble, me. The first is the main 
indicator, %OAval, which is the most visible result for a policy. What do you 
expect this will tell people about a given policy? I randomly selected a couple 
of policies, one of which was for my own school, to find they each scored about 
50%. I would expect these to be among the leaders in terms of OA policies, so 
this seems a surprisingly unhelpful score.
>
> So what's the explanation? Note that the objects being evaluated are 
institutional OA policies; they are effectively being presented in relation to 
institutional repositories when the policy specifies where to archive is an IR 
with a URL. It seems that the scores include ratings for OA publication policy, 
libre vs gratis OA, publisher pdf, sanctions (score if Yes), incentives (score 
if Yes), etc., some of which an institution might specify but which might not 
apply to an IR http://www.accesoabierto.net/politicas/politicas_estructura.php. 
However you weight these factors they are still contributors to the overall 
score, so a policy that is specific to an IR is immediately handicapped, or 
appears to be unless there is more context to understand the scores.
>
> Which leads me to another question on the visualisation of the validator, 
and its use of green, gold (and red) in the meter. Do the green and gold refer 
the the classic OA colours? This would be quite convenient, since it would 
appear that the green repository policies I mentioned above are achieving 
almost full scores in the green zone of the meter. However, I suspect this 
cannot be the case, because it would assume that institutions must have a green 
AND gold policy, but not simply gold (whatever argument could be put for that).
>
> It is important that new services should help reveal and promote OA 
policies, as you seek to do, but at the same time not to prejudice the 
development of such policies by mixing and not fairly separating the 
contributing factors, especially where these relate to different types of OA.
>
> Steve Hitchcock
> IAM Group, Building 32
> School of Electronics and Computer Science
> University of Southampton, SO17 1BJ, UK
> Email: sh94r AT ecs.soton.ac.uk
> Twitter: http://twitter.com/stevehit
> Connotea: http://www.connotea.org/user/stevehit
> Tel: +44 (0)23 8059 7698    Fax: +44 (0)23 8059 2865
>
> On 15 Jul 2010, at 08:14, Remedios Melero wrote:
>
>    
>> Good mornig!
>> In the last Open Repositories Conference which was held last week in 
Madrid (http://or2010.fecyt.es/publico/Home/index.aspx ) was presented in the 
poster session the project called MELIBEA.
>> MELIBEA (http://www.accesoabierto.net/politicas/)  is a directory and  
a validator of institutional open-access (OA) policies regarding scientific and 
academic work. As a directory, it describes the existing policies. As a 
validator, it subjects them to qualitative and quantitative analysis based on 
fulfilment of a set of indicators ( 
http://www.accesoabierto.net/politicas/politicas_estructura.php) that reflect 
the bases of an institutional policy.
>>
>> Based on the values assigned to a set of indicators, weighted 
according to their importance, the validator indicates a score and a percentage 
of fulfilment for each policy analyzed. The sum of weighted values of each 
indicator is converted to a percentage scale to give what we have called the 
“validated open-access percentage” (see how i t is calculated:  
http://www.accesoabierto.net/politicas/default.php?contenido=acerca ).
>>
>> The types of institution analyzed include universities, research 
centres, funding agencies and governmental organizations.
>>
>> MELIBEA has three main objectives:
>>
>> 	• 1. To establish indicators that reveal the strong and weak points 
of institutional OA polices.
>> 	• 2. To propose a methodology to guide institutions when they are 
drawing up an institutional OA policy.
>> 	• 3. To offer a tool for comparing the contents of policies between 
institutions.
>> The aim is not to be a ranking, but to offer a tool where to aanlyse 
and visualize the weaknesses or strenghts of an institutional OA policy based 
on its wording. It seems something trivial  but accomplishment of a policy is 
based on its terms.
>> Please if you detect any mistake or you would like to make a comment, 
contact me. I will be pleased if you could check your policy, if any, to 
analyse our approach.
>> Best wishes
>> Reme
>>
>>
>> R. Melero
>> IATA, CSIC
>> Avda Agustín Escardino 7, 46980 Paterna (Valencia), Spain
>> TEl +34 96 390 00 22. Fax 96 363 63 01
>> E-mail rmelero AT iata.csic.es
>> http://www.accesoabierto.net
>>
>> --
>> To unsubscribe from the BOAI Forum, use the form on this page:
>> http://www.soros.org/openaccess/forum.shtml?f
>>      
>
>
> Steve Hitchcock
> IAM Group, Building 32
> School of Electronics and Computer Science
> University of Southampton, SO17 1BJ, UK
> Email: sh94r AT ecs.soton.ac.uk
> Twitter: http://twitter.com/stevehit
> Connotea: http://www.connotea.org/user/stevehit
> Tel: +44 (0)23 8059 7698    Fax: +44 (0)23 8059 2865
>
>
>
>
>
>
>
>
>
> --
> To unsubscribe from the BOAI Forum, use the form on this page:
> http://www.soros.org/openaccess/forum.shtml?f
>    

        
--      
To unsubscribe from the BOAI Forum, use the form on this page:
http://www.soros.org/openaccess/forum.shtml?f


[BOAI] Re: OA policies and their "weight"

From: Remedios Melero <rmelero AT iata.csic.es>
Date: Mon, 19 Jul 2010 11:18:23 +0200


Threading: [BOAI] OA policies and their "weight" from rmelero AT iata.csic.es
      • This Message
             [BOAI] Re: OA policies and their "weight" from rmelero AT iata.csic.es


Dea Steve,

I apologize for the delay in my response, but I will try to give some 
explanations to make clear some issues you raised in your message (my 
comments are in capital letters, to distinguish them from yours)



El 15/07/2010 11:22, Steve Hitchcock escribió:
> Reme,    Thank you for bringing this new service to our attention. OA 
policies are vitally important to the development of institutional 
repositories, and services that can highlight and bring attention to this 
development can be valuable.
>
> There are a few aspects of the validation aspects of the new MELIBEA 
service that confuse, and possibly trouble, me. The first is the main 
indicator, %OAval, which is the most visible result for a policy. What do you 
expect this will tell people about a given policy? I randomly selected a couple 
of policies, one of which was for my own school, to find they each scored about 
50%. I would expect these to be among the leaders in terms of OA policies, so 
this seems a surprisingly unhelpful score.
>
> So what's the explanation? Note that the objects being evaluated are 
institutional OA policies; they are effectively being presented in relation to 
institutional repositories when the policy specifies where to archive is an IR 
with a URL. It seems that the scores include ratings for OA publication policy, 
libre vs gratis OA, publisher pdf, sanctions (score if Yes), incentives (score 
if Yes), etc., some of which an institution might specify but which might not 
apply to an IR http://www.accesoabierto.net/politicas/politicas_estructura.php. 
However you weight these factors they are still contributors to the overall 
score, so a policy that is specific to an IR is immediately handicapped, or 
appears to be unless there is more context to understand the scores.
>
>    

AS I WROTE BEFORE THIS IS NOT A RANKING, IT IS NOT THE AIM OF MELIBEA  
BUT TO HAVE A KIND OF REFERENCE ON WHAT TOPICS, ISSUES OR MATTERS TO BE 
INCLUDED IN AN OA POLICY. WE ARE TALKING ABOUT INSTITUTIONAL POLICIES OF 
DIFFERENTE NATURE, NOT ABOUT REPOSITORIES POLICIES. IF THE POLICY ONLY 
TALKS ABOUT THE REQUIREMENT TO DEPOSIT IN A REPOSITORY, IT SHOULD  
SPECIFY WHAT, WHEN AND UNDER WHAT CONDITIONS, IF ANY. IT IS NOT THE SAME 
TO SAY WHAT DOCUMENTS  AND WHAT VERSIONS AND WHEN THAN SIMPLY SAY " 
ANY" 
OR "AS SOON AS POSSIBLE" (this could be a month after publication or 
years after publication, depending on one's criteria). GOLD ROUTE, NEVER 
IS REQUIRED ACCORDING OUR APPROACH ("Gold  (Recommended in OA 
journals") 
AND NOT ALL OA JOURNALS ARE SUPPORTED BY  SAME ECONOMIC MODEL.


> Which leads me to another question on the visualisation of the validator, 
and its use of green, gold (and red) in the meter. Do the green and gold refer 
the the classic OA colours? This would be quite convenient, since it would 
appear that the green repository policies I mentioned above are achieving 
almost full scores in the green zone of the meter. However, I suspect this 
cannot be the case, because it would assume that institutions must have a green 
AND gold policy, but not simply gold (whatever argument could be put for that).
>    
COLORS DO NOT MEAN THAT, WE WANTED JUST TO DISTINGUISH ZONES LIKE IT 
WERE A SPECTRA.

> It is important that new services should help reveal and promote OA 
policies, as you seek to do, but at the same time not to prejudice the 
development of such policies by mixing and not fairly separating the 
contributing factors, especially where these relate to different types of OA.
>    
I DO NOT THINK WE ARE MIXING, IN FACT THERE TWO MODELS, ONE FOR UNIV. 
AND RESEARCH CNETRES AND ANOTHER FOR FUNDERS AND GOV. INSTITUTIONS AND 
THE QUESTIONS FOR THEM ARE DIFFERENT, for instance, FOR A FUNDER THE 
QUESTION ABOUT DEPOSIT O THESIS IS NOT APPLICABLE.
IN SUMMARY, OUR MODEL COULD NOT BE "PERFECT" BUT I IS ONE, WHICH 
COULD 
DETECT DIFFERENCES BETWEEN REQUEST AND REQUIRE, WHO, WHAT , WHEN IF 
THERE ARE ANY INCENTIVES OR SANCTIONS (  this has not to be a negative 
point but to remember we should  assume  reponsible attitudes).

However we will revise the model to see if we can make any improvement 
to make it clear, we are working also in a graph interface to show some 
data in graphical form.
Best wishes
Reme

>
>> R. Melero
>> IATA, CSIC
>> Avda Agustín Escardino 7, 46980 Paterna (Valencia), Spain
>> TEl +34 96 390 00 22. Fax 96 363 63 01
>> E-mail rmelero AT iata.csic.es
>> http://www.accesoabierto.net
>>
>> --
>>      


ATTACHMENT: message.html!

-- To unsubscribe from the BOAI Forum, use the form on this page: http://www.soros.org/openaccess/forum.shtml?f

[BOAI] Re: OA policies and their "weight"

From: Stevan Harnad <harnad AT ecs.soton.ac.uk>
Date: Mon, 19 Jul 2010 09:56:09 -0400


Threading: [BOAI] OA policies and their "weight" from rmelero AT iata.csic.es
      • This Message

Dear Reme, if I may also make an intervention in your exchange with Steve 
Hitchcock about the MELIBEA OA policy evaluator:
http://www.accesoabierto.net/politicas/ 

The MELIBEA service is extremely timely and promising, and could be potentially 
useful and even influential in shaping OA mandates -- but that makes it all the 
more important to get it right, rather than releasing MELIBEA prematurely, when 
it still risks increasing confusion rather than providing clarity and 
direction.

You are right to point out that -- unlike the CSIC's University Ranking and the 
Repository Ranking -- the policy evaluator is not really a ranking. But you 
have set up the composite algorithm and the graphics to make it a ranking just 
the same. 

You are also point out, correctly, that the policy criteria for institutions 
and funders are not (and should not be) the same. Yet, with the MELIBEA coding 
as well as the algorithm, they are treated the same way. 

You also point out, rightly, that gold OA publishing policy is not central to 
institutional OA policy making, yet there it is, as part of the MELIBEA 
algorithm.

You also point out that the color code has nothing to do with the 
"green" OA coding -- yet there it is, competing with the widespread 
use of green to designate self-archiving, and thereby causing confusion, both 
overt and covert.

I would be more than happy to give you feedback on every aspect of MELIBEA -- 
it could be a useful and natural complement to the ROARMAP registry of OA 
policies. 

But as it is designed now, I can only agree with Steve Hitchcock's points and 
conclude that consulting MELIBEA today would be likely to induce confusion and 
would not help in bringing the all-important focus and direction to OA 
policy-making that I am sure CSIC, too, seeks, and seeks to help bring about.

Here are just a few prima facie points:

(1) Since MELIBEA is not, and should not be construed as a ranking of OA 
policies -- especially because it includes both institutional and funder 
policies -- it is important NOT to plug it into an algorithm until and unless 
the algorithm has first been carefully tested, with consultation, to make sure 
it weights policy criteria in a way that optimizes OA progress and guides 
policy-makers in the right direction.

(2) For this reason, it is more important to allow users to generate separate 
flat lists of institutions or funders on the various policy criteria, 
considered and compared independently, rather than on the basis of a 
prematurely and arbitrarily weighted joint algorithm.

(3) This is all the more important since the data are based on less then 200 
institutions, whereas the CSIC University Rankings are based on thousands. 
Since the population is still so small, MELIBEA risks having a disproportionate 
effect on initial conditions and hence direction-setting; all the more reason 
NOT to amplify noise and indirection by assigning untested initial weights 
without carefully thinking through and weighing the consequences.

(4) A potential internal cross-validator of some of the criteria would be a 
reliable measure of outcome -- but that requires much more attention to 
estimating the annual size and growth-rate of each repository (in terms of OA's 
target contents, which are full-text articles), normalized for institution size 
and annual total target output. Policy criteria (such as request/require or 
immediate/delayed) should be cross-validated against these outcome measures 
(such as percentage and growth rate of annual target output).

(5) The MELIBEA color coding needs to be revised, and revised quickly, if there 
is to be an algorithm at all. All those arbitrary colors in the display of 
single repositories as ranked by the algorithm are both unnecessary and 
confusing. The objective should be to order and focus clearly and intuitively. 
Whatever is correlated with more green OA output (such as a higher level or 
faster growth rate in OA's target content) should be coded as darker or bigger 
shades of green. The same should be true for the policy criteria, separately 
and jointly: in each case, request/require, delayed/immediate, etc., the 
greenward polarity is obvious and intuitive. This should be reflected in the 
graphics as well as in any comparative rankings.

(6) If you include repositories with no OA policy at all (i.e., just a 
repository and an open invitation to deposit) then all you are doing is 
duplicating ROAR and ROARMAP, whereas the purpose, presumably, of MELIBEA, is 
to highlight, weigh and compare specific policy differences among (the very 
few) repositories that DO have policies.

(7) The sign-up data -- 
http://www.accesoabierto.net/politicas/nueva.php?directorio=politicas -- are 
also rather confusing; the criteria are not always consistent, relevant or 
applicable. The sign-up seems to be designed to make a funder mandate the 
generic option, whereas this is quite the opposite of reality. There are far 
more institutions and institutional repositories and policies than funders. 
There should be separate criterial lists for institutional policies and for 
funder policies; they are not the same. There is also far too much focus on 
gold OA policy and payment. If included at all, this should only be at the end, 
as an addendum, not the focus at the beginning, and on a par with green OA 
policy. 

(8) There is also potential confusion on the matter of "waivers": 
There are two aspects of a mandate. One concerns whether or not deposit is 
required (and if so, whether that requirement can be waived) and the other 
concerns whether or not rights-reservation is required (and if so, whether that 
requirement can be waived). These two distinct and independent 
requirements/waivers are completely conflated in the current version of 
MELIBEA.

I hope there will be substantive consultation and conscientious redesign of 
these and other aspects of MELIBEA before it is can recommended for serious 
consideration and use.

Stevan Harnad


On 2010-07-19, at 5:18 AM, Remedios Melero wrote:

> Dear Steve,
> 
> I apologize for the delay in my response, but I will try to give some 
explanations to make clear some issues you raised in your message (my comments 
are in capital letters, to distinguish them from yours)
> 
> 
> 
> El 15/07/2010 11:22, Steve Hitchcock escribió:
>> Reme,    Thank you for bringing this new service to our attention. OA 
policies are vitally important to the development of institutional 
repositories, and services that can highlight and bring attention to this 
development can be valuable.
>> 
>> There are a few aspects of the validation aspects of the new MELIBEA 
service that confuse, and possibly trouble, me. The first is the main 
indicator, %OAval, which is the most visible result for a policy. What do you 
expect this will tell people about a given policy? I randomly selected a couple 
of policies, one of which was for my own school, to find they each scored about 
50%. I would expect these to be among the leaders in terms of OA policies, so 
this seems a surprisingly unhelpful score.
>> 
>> So what's the explanation? Note that the objects being evaluated are 
institutional OA policies; they are effectively being presented in relation to 
institutional repositories when the policy specifies where to archive is an IR 
with a URL. It seems that the scores include ratings for OA publication policy, 
libre vs gratis OA, publisher pdf, sanctions (score if Yes), incentives (score 
if Yes), etc., some of which an institution might specify but which might not 
apply to an IR 
>> http://www.accesoabierto.net/politicas/politicas_estructura.php
>> . However you weight these factors they are still contributors to the 
overall score, so a policy that is specific to an IR is immediately 
handicapped, or appears to be unless there is more context to understand the 
scores.
>> 
>>   
>> 
> AS I WROTE BEFORE THIS IS NOT A RANKING, IT IS NOT THE AIM OF MELIBEA  BUT 
TO HAVE A KIND OF REFERENCE ON WHAT TOPICS, ISSUES OR MATTERS TO BE INCLUDED IN 
AN OA POLICY. WE ARE TALKING ABOUT INSTITUTIONAL POLICIES OF DIFFERENTE NATURE, 
NOT ABOUT REPOSITORIES POLICIES. IF THE POLICY ONLY TALKS ABOUT THE REQUIREMENT 
TO DEPOSIT IN A REPOSITORY, IT SHOULD  SPECIFY WHAT, WHEN AND UNDER WHAT 
CONDITIONS, IF ANY. IT IS NOT THE SAME TO SAY WHAT DOCUMENTS  AND WHAT VERSIONS 
AND WHEN THAN SIMPLY SAY " ANY" OR "AS SOON AS POSSIBLE" 
(this could be a month after publication or years after publication, depending 
on one's criteria). GOLD ROUTE, NEVER IS REQUIRED ACCORDING OUR APPROACH 
("Gold  (Recommended in OA journals") AND NOT ALL OA JOURNALS ARE 
SUPPORTED BY  SAME ECONOMIC MODEL.
> 
> 
>> Which leads me to another question on the visualisation of the 
validator, and its use of green, gold (and red) in the meter. Do the green and 
gold refer the the classic OA colours? This would be quite convenient, since it 
would appear that the green repository policies I mentioned above are achieving 
almost full scores in the green zone of the meter. However, I suspect this 
cannot be the case, because it would assume that institutions must have a green 
AND gold policy, but not simply gold (whatever argument could be put for that). 

>>   
>> 
> COLORS DO NOT MEAN THAT, WE WANTED JUST TO DISTINGUISH ZONES LIKE IT WERE 
A SPECTRA. 
> 
>> It is important that new services should help reveal and promote OA 
policies, as you seek to do, but at the same time not to prejudice the 
development of such policies by mixing and not fairly separating the 
contributing factors, especially where these relate to different types of OA.
>>   
>> 
> I DO NOT THINK WE ARE MIXING, IN FACT THERE TWO MODELS, ONE FOR UNIV. AND 
RESEARCH CNETRES AND ANOTHER FOR FUNDERS AND GOV. INSTITUTIONS AND THE 
QUESTIONS FOR THEM ARE DIFFERENT, for instance, FOR A FUNDER THE QUESTION ABOUT 
DEPOSIT O THESIS IS NOT APPLICABLE.
> IN SUMMARY, OUR MODEL COULD NOT BE "PERFECT" BUT I IS ONE, WHICH 
COULD DETECT DIFFERENCES BETWEEN REQUEST AND REQUIRE, WHO, WHAT , WHEN IF THERE 
ARE ANY INCENTIVES OR SANCTIONS (  this has not to be a negative point but to 
remember we should  assume  reponsible attitudes).
> 
> However we will revise the model to see if we can make any improvement to 
make it clear, we are working also in a graph interface to show some data in 
graphical form.
> Best wishes
> Reme
> 
>> 
>>> R. Melero 
>>> IATA, CSIC 
>>> Avda Agustín Escardino 7, 46980 Paterna (Valencia), Spain 
>>> TEl +34 96 390 00 22. Fax 96 363 63 01 
>>> E-mail 
>>> rmelero AT iata.csic.es
>>>  
>>> 
>>> http://www.accesoabierto.net
>>> 
>>> 
>>> --      
>>>     
>>> 
> 
> 
> --      
> To unsubscribe from the BOAI Forum, use the form on this page:
> http://www.soros.org/openaccess/forum.shtml?f


        
--      
To unsubscribe from the BOAI Forum, use the form on this page:
http://www.soros.org/openaccess/forum.shtml?f


[BOAI] Re: OA policies and their "weight"

From: Remedios Melero <rmelero AT iata.csic.es>
Date: Tue, 20 Jul 2010 09:22:24 +0200


Threading: [BOAI] Re: OA policies and their "weight" from rmelero AT iata.csic.es
      • This Message
             [BOAI] Re: OA policies and their "weight" from isabel.bernal AT bib.csic.es

Good morning!

Thanks Stevan for your comments, in fact I did not expect so much 
controversial discussion, but I prefer this since it serves to think 
about some aspects that one has not considered or has misssed. I will 
revise your comments and see how we can implement some improvements to 
avoid any missunderstanding. However, something I would like to make 
clear is:

1. The CSIC ranking of repositories and universities is nothing to 
compare, that is a ranking build on "visibility" of an  institution 
in 
the web. A graphic representation has not to mean necesarily a ranking 
but a measurement
2. MELIBEA is not a directory of repositories, but OA policies, there 
ara a lot of repositories but few OA policies, obviously most of the 
policies are associated with the deposit in an insituttional or subject 
repository.
3. The model  is based on the weights of some variables, you are right, 
they could be different because it  will be always an approach, however 
they have been tested, changed and modified before MELIBEA was publicly 
announced.

I will take the advantage  of summer holidays to think about all these  
issues and hopefully after them I can show some changes aimed to improve 
the tool. I wil start with the colors which I see were more important 
than thought!


Reme


El 19/07/2010 15:56, Stevan Harnad escribió:
> Dear Reme, if I may also make an intervention in your exchange with Steve 
Hitchcock about the MELIBEA OA policy evaluator:
> http://www.accesoabierto.net/politicas/
>
> The MELIBEA service is extremely timely and promising, and could be 
potentially useful and even influential in shaping OA mandates -- but that 
makes it all the more important to get it right, rather than releasing MELIBEA 
prematurely, when it still risks increasing confusion rather than providing 
clarity and direction.
>
> You are right to point out that -- unlike the CSIC's University Ranking 
and the Repository Ranking -- the policy evaluator is not really a ranking. But 
you have set up the composite algorithm and the graphics to make it a ranking 
just the same.
>
> You are also point out, correctly, that the policy criteria for 
institutions and funders are not (and should not be) the same. Yet, with the 
MELIBEA coding as well as the algorithm, they are treated the same way.
>
> You also point out, rightly, that gold OA publishing policy is not central 
to institutional OA policy making, yet there it is, as part of the MELIBEA 
algorithm.
>
> You also point out that the color code has nothing to do with the 
"green" OA coding -- yet there it is, competing with the widespread 
use of green to designate self-archiving, and thereby causing confusion, both 
overt and covert.
>
> I would be more than happy to give you feedback on every aspect of MELIBEA 
-- it could be a useful and natural complement to the ROARMAP registry of OA 
policies.
>
> But as it is designed now, I can only agree with Steve Hitchcock's points 
and conclude that consulting MELIBEA today would be likely to induce confusion 
and would not help in bringing the all-important focus and direction to OA 
policy-making that I am sure CSIC, too, seeks, and seeks to help bring about.
>
> Here are just a few prima facie points:
>
> (1) Since MELIBEA is not, and should not be construed as a ranking of OA 
policies -- especially because it includes both institutional and funder 
policies -- it is important NOT to plug it into an algorithm until and unless 
the algorithm has first been carefully tested, with consultation, to make sure 
it weights policy criteria in a way that optimizes OA progress and guides 
policy-makers in the right direction.
>
> (2) For this reason, it is more important to allow users to generate 
separate flat lists of institutions or funders on the various policy criteria, 
considered and compared independently, rather than on the basis of a 
prematurely and arbitrarily weighted joint algorithm.
>
> (3) This is all the more important since the data are based on less then 
200 institutions, whereas the CSIC University Rankings are based on thousands. 
Since the population is still so small, MELIBEA risks having a disproportionate 
effect on initial conditions and hence direction-setting; all the more reason 
NOT to amplify noise and indirection by assigning untested initial weights 
without carefully thinking through and weighing the consequences.
>
> (4) A potential internal cross-validator of some of the criteria would be 
a reliable measure of outcome -- but that requires much more attention to 
estimating the annual size and growth-rate of each repository (in terms of OA's 
target contents, which are full-text articles), normalized for institution size 
and annual total target output. Policy criteria (such as request/require or 
immediate/delayed) should be cross-validated against these outcome measures 
(such as percentage and growth rate of annual target output).
>
> (5) The MELIBEA color coding needs to be revised, and revised quickly, if 
there is to be an algorithm at all. All those arbitrary colors in the display 
of single repositories as ranked by the algorithm are both unnecessary and 
confusing. The objective should be to order and focus clearly and intuitively. 
Whatever is correlated with more green OA output (such as a higher level or 
faster growth rate in OA's target content) should be coded as darker or bigger 
shades of green. The same should be true for the policy criteria, separately 
and jointly: in each case, request/require, delayed/immediate, etc., the 
greenward polarity is obvious and intuitive. This should be reflected in the 
graphics as well as in any comparative rankings.
>
> (6) If you include repositories with no OA policy at all (i.e., just a 
repository and an open invitation to deposit) then all you are doing is 
duplicating ROAR and ROARMAP, whereas the purpose, presumably, of MELIBEA, is 
to highlight, weigh and compare specific policy differences among (the very 
few) repositories that DO have policies.
>
> (7) The sign-up data -- 
http://www.accesoabierto.net/politicas/nueva.php?directorio=politicas -- are 
also rather confusing; the criteria are not always consistent, relevant or 
applicable. The sign-up seems to be designed to make a funder mandate the 
generic option, whereas this is quite the opposite of reality. There are far 
more institutions and institutional repositories and policies than funders. 
There should be separate criterial lists for institutional policies and for 
funder policies; they are not the same. There is also far too much focus on 
gold OA policy and payment. If included at all, this should only be at the end, 
as an addendum, not the focus at the beginning, and on a par with green OA 
policy.
>
> (8) There is also potential confusion on the matter of 
"waivers": There are two aspects of a mandate. One concerns whether 
or not deposit is required (and if so, whether that requirement can be waived) 
and the other concerns whether or not rights-reservation is required (and if 
so, whether that requirement can be waived). These two distinct and independent 
requirements/waivers are completely conflated in the current version of 
MELIBEA.
>
> I hope there will be substantive consultation and conscientious redesign 
of these and other aspects of MELIBEA before it is can recommended for serious 
consideration and use.
>
> Stevan Harnad
>
>
> On 2010-07-19, at 5:18 AM, Remedios Melero wrote:
>
>    
>> Dear Steve,
>>
>> I apologize for the delay in my response, but I will try to give some 
explanations to make clear some issues you raised in your message (my comments 
are in capital letters, to distinguish them from yours)
>>
>>
>>
>> El 15/07/2010 11:22, Steve Hitchcock escribió:
>>      
>>> Reme,    Thank you for bringing this new service to our attention. 
OA policies are vitally important to the development of institutional 
repositories, and services that can highlight and bring attention to this 
development can be valuable.
>>>
>>> There are a few aspects of the validation aspects of the new 
MELIBEA service that confuse, and possibly trouble, me. The first is the main 
indicator, %OAval, which is the most visible result for a policy. What do you 
expect this will tell people about a given policy? I randomly selected a couple 
of policies, one of which was for my own school, to find they each scored about 
50%. I would expect these to be among the leaders in terms of OA policies, so 
this seems a surprisingly unhelpful score.
>>>
>>> So what's the explanation? Note that the objects being evaluated 
are institutional OA policies; they are effectively being presented in relation 
to institutional repositories when the policy specifies where to archive is an 
IR with a URL. It seems that the scores include ratings for OA publication 
policy, libre vs gratis OA, publisher pdf, sanctions (score if Yes), incentives 
(score if Yes), etc., some of which an institution might specify but which 
might not apply to an IR
>>> http://www.accesoabierto.net/politicas/politicas_estructura.php
>>> . However you weight these factors they are still contributors to 
the overall score, so a policy that is specific to an IR is immediately 
handicapped, or appears to be unless there is more context to understand the 
scores.
>>>
>>>
>>>
>>>        
>> AS I WROTE BEFORE THIS IS NOT A RANKING, IT IS NOT THE AIM OF MELIBEA  
BUT TO HAVE A KIND OF REFERENCE ON WHAT TOPICS, ISSUES OR MATTERS TO BE 
INCLUDED IN AN OA POLICY. WE ARE TALKING ABOUT INSTITUTIONAL POLICIES OF 
DIFFERENTE NATURE, NOT ABOUT REPOSITORIES POLICIES. IF THE POLICY ONLY TALKS 
ABOUT THE REQUIREMENT TO DEPOSIT IN A REPOSITORY, IT SHOULD  SPECIFY WHAT, WHEN 
AND UNDER WHAT CONDITIONS, IF ANY. IT IS NOT THE SAME TO SAY WHAT DOCUMENTS  
AND WHAT VERSIONS AND WHEN THAN SIMPLY SAY " ANY" OR "AS SOON AS 
POSSIBLE" (this could be a month after publication or years after 
publication, depending on one's criteria). GOLD ROUTE, NEVER IS REQUIRED 
ACCORDING OUR APPROACH ("Gold  (Recommended in OA journals") AND NOT 
ALL OA JOURNALS ARE SUPPORTED BY  SAME ECONOMIC MODEL.
>>
>>
>>      
>>> Which leads me to another question on the visualisation of the 
validator, and its use of green, gold (and red) in the meter. Do the green and 
gold refer the the classic OA colours? This would be quite convenient, since it 
would appear that the green repository policies I mentioned above are achieving 
almost full scores in the green zone of the meter. However, I suspect this 
cannot be the case, because it would assume that institutions must have a green 
AND gold policy, but not simply gold (whatever argument could be put for that).
>>>
>>>
>>>        
>> COLORS DO NOT MEAN THAT, WE WANTED JUST TO DISTINGUISH ZONES LIKE IT 
WERE A SPECTRA.
>>
>>      
>>> It is important that new services should help reveal and promote 
OA policies, as you seek to do, but at the same time not to prejudice the 
development of such policies by mixing and not fairly separating the 
contributing factors, especially where these relate to different types of OA.
>>>
>>>
>>>        
>> I DO NOT THINK WE ARE MIXING, IN FACT THERE TWO MODELS, ONE FOR UNIV. 
AND RESEARCH CNETRES AND ANOTHER FOR FUNDERS AND GOV. INSTITUTIONS AND THE 
QUESTIONS FOR THEM ARE DIFFERENT, for instance, FOR A FUNDER THE QUESTION ABOUT 
DEPOSIT O THESIS IS NOT APPLICABLE.
>> IN SUMMARY, OUR MODEL COULD NOT BE "PERFECT" BUT I IS ONE, 
WHICH COULD DETECT DIFFERENCES BETWEEN REQUEST AND REQUIRE, WHO, WHAT , WHEN IF 
THERE ARE ANY INCENTIVES OR SANCTIONS (  this has not to be a negative point 
but to remember we should  assume  reponsible attitudes).
>>
>> However we will revise the model to see if we can make any improvement 
to make it clear, we are working also in a graph interface to show some data in 
graphical form.
>> Best wishes
>> Reme
>>
>>      
>>>        
>>>> R. Melero
>>>> IATA, CSIC
>>>> Avda Agustín Escardino 7, 46980 Paterna (Valencia), Spain
>>>> TEl +34 96 390 00 22. Fax 96 363 63 01
>>>> E-mail
>>>> rmelero AT iata.csic.es
>>>>
>>>>
>>>> http://www.accesoabierto.net
>>>>
>>>>
>>>> --
>>>>
>>>>
>>>>          
>>
>> --
>> To unsubscribe from the BOAI Forum, use the form on this page:
>> http://www.soros.org/openaccess/forum.shtml?f
>>      
>
>
> --
> To unsubscribe from the BOAI Forum, use the form on this page:
> http://www.soros.org/openaccess/forum.shtml?f
>    

        
--      
To unsubscribe from the BOAI Forum, use the form on this page:
http://www.soros.org/openaccess/forum.shtml?f


[BOAI] Re: OA policies and their "weight"

From: "Isabel Bernal" <isabel.bernal AT bib.csic.es>
Date: Tue, 20 Jul 2010 18:27:52 +0200


Threading: [BOAI] Re: OA policies and their "weight" from rmelero AT iata.csic.es
      • This Message
             Re: [BOAI] Re: OA policies and their "weight" from harnad AT ecs.soton.ac.uk
             [BOAI] Re: OA policies and their "weight" from harnad AT ecs.soton.ac.uk


Good afternoon everybody,
 
Following the ongoing discussion in which the CSIC (Spanish National
Research Council) and its open access efforts have been mentioned, we would
like to clarify the following:
 
The CSIC Presidency signed the Berlin Declaration on Open Access in January
2006, and as a result of it the Spanish National Research Council is driving
and implementing open access principles through 2 institutional initiatives:

-Digital.CSIC (https://digital.csic.es/) is the institutional repository
that provides open access to, organises and preserves the scientific output
resulting from the research activities by CSIC 147 institutes and centers.
The repository is a project by CSIC Libraries Coordination Unit.

-Revistas-CSIC (http://revistas.csic.es/) provides open access to the 35
scientific Journals published by the institution, covering a wide variety of
scientific disciplines. To date, 14 Journals provide immediate open access,
while 22 apply an embargo period of six months. Before the end of this year,
at least 4 more Journals are planned to move to full OA. Revistas-CSIC is a
project run by the CSIC Publication Department, and is a member of OASPA
under the category of OA Professional Publishing Organization.

These 2 initiatives fall within the CSIC Vice-presidency of Organization and
Institutional Relations.

To date, CSIC does not have an open access institutional mandate. In the
absence of a nation-wide open access related law yet, there are regional
laws in favour of open access that have a direct effect on CSIC, such as
that of the Government of the Community of Madrid.

Thus, MELIBEA should not be considered a CSIC institutional project.

Best wishes,

Agnès Ponsati, Director of CSIC Libraries Coordination Unit 

Ramón Rodríguez, Coordinator of Revistas-CSIC



-----Mensaje original-----
De: boai-forum-bounces AT ecs.soton.ac.uk
[mailto:boai-forum-bounces AT ecs.soton.ac.uk] En nombre de Stevan Harnad
Enviado el: lunes, 19 de julio de 2010 15:56
Para: boai-forum AT ecs.soton.ac.uk
CC: SPARC Open Access Forum; JISC-REPOSITORIES AT JISCMAIL.AC.UK;
AMERICAN-SCIENTIST-OPEN-ACCESS-FORUM AT LISTSERVER.SIGMAXI.ORG
Asunto: [BOAI] Re: OA policies and their "weight"

Dear Reme, if I may also make an intervention in your exchange with Steve
Hitchcock about the MELIBEA OA policy evaluator:
http://www.accesoabierto.net/politicas/ 

The MELIBEA service is extremely timely and promising, and could be
potentially useful and even influential in shaping OA mandates -- but that
makes it all the more important to get it right, rather than releasing
MELIBEA prematurely, when it still risks increasing confusion rather than
providing clarity and direction.

You are right to point out that -- unlike the CSIC's University Ranking and
the Repository Ranking -- the policy evaluator is not really a ranking. But
you have set up the composite algorithm and the graphics to make it a
ranking just the same. 

You are also point out, correctly, that the policy criteria for institutions
and funders are not (and should not be) the same. Yet, with the MELIBEA
coding as well as the algorithm, they are treated the same way. 

You also point out, rightly, that gold OA publishing policy is not central
to institutional OA policy making, yet there it is, as part of the MELIBEA
algorithm.

You also point out that the color code has nothing to do with the 
"green" OA
coding -- yet there it is, competing with the widespread use of green to
designate self-archiving, and thereby causing confusion, both overt and
covert.

I would be more than happy to give you feedback on every aspect of MELIBEA
-- it could be a useful and natural complement to the ROARMAP registry of OA
policies. 

But as it is designed now, I can only agree with Steve Hitchcock's points
and conclude that consulting MELIBEA today would be likely to induce
confusion and would not help in bringing the all-important focus and
direction to OA policy-making that I am sure CSIC, too, seeks, and seeks to
help bring about.

Here are just a few prima facie points:

(1) Since MELIBEA is not, and should not be construed as a ranking of OA
policies -- especially because it includes both institutional and funder
policies -- it is important NOT to plug it into an algorithm until and
unless the algorithm has first been carefully tested, with consultation, to
make sure it weights policy criteria in a way that optimizes OA progress and
guides policy-makers in the right direction.

(2) For this reason, it is more important to allow users to generate
separate flat lists of institutions or funders on the various policy
criteria, considered and compared independently, rather than on the basis of
a prematurely and arbitrarily weighted joint algorithm.

(3) This is all the more important since the data are based on less then 200
institutions, whereas the CSIC University Rankings are based on thousands.
Since the population is still so small, MELIBEA risks having a
disproportionate effect on initial conditions and hence direction-setting;
all the more reason NOT to amplify noise and indirection by assigning
untested initial weights without carefully thinking through and weighing the
consequences.

(4) A potential internal cross-validator of some of the criteria would be a
reliable measure of outcome -- but that requires much more attention to
estimating the annual size and growth-rate of each repository (in terms of
OA's target contents, which are full-text articles), normalized for
institution size and annual total target output. Policy criteria (such as
request/require or immediate/delayed) should be cross-validated against
these outcome measures (such as percentage and growth rate of annual target
output).

(5) The MELIBEA color coding needs to be revised, and revised quickly, if
there is to be an algorithm at all. All those arbitrary colors in the
display of single repositories as ranked by the algorithm are both
unnecessary and confusing. The objective should be to order and focus
clearly and intuitively. Whatever is correlated with more green OA output
(such as a higher level or faster growth rate in OA's target content) should
be coded as darker or bigger shades of green. The same should be true for
the policy criteria, separately and jointly: in each case, request/require,
delayed/immediate, etc., the greenward polarity is obvious and intuitive.
This should be reflected in the graphics as well as in any comparative
rankings.

(6) If you include repositories with no OA policy at all (i.e., just a
repository and an open invitation to deposit) then all you are doing is
duplicating ROAR and ROARMAP, whereas the purpose, presumably, of MELIBEA,
is to highlight, weigh and compare specific policy differences among (the
very few) repositories that DO have policies.

(7) The sign-up data --
http://www.accesoabierto.net/politicas/nueva.php?directorio=politicas -- are
also rather confusing; the criteria are not always consistent, relevant or
applicable. The sign-up seems to be designed to make a funder mandate the
generic option, whereas this is quite the opposite of reality. There are far
more institutions and institutional repositories and policies than funders.
There should be separate criterial lists for institutional policies and for
funder policies; they are not the same. There is also far too much focus on
gold OA policy and payment. If included at all, this should only be at the
end, as an addendum, not the focus at the beginning, and on a par with green
OA policy. 

(8) There is also potential confusion on the matter of "waivers": 
There are
two aspects of a mandate. One concerns whether or not deposit is required
(and if so, whether that requirement can be waived) and the other concerns
whether or not rights-reservation is required (and if so, whether that
requirement can be waived). These two distinct and independent
requirements/waivers are completely conflated in the current version of
MELIBEA.

I hope there will be substantive consultation and conscientious redesign of
these and other aspects of MELIBEA before it is can recommended for serious
consideration and use.

Stevan Harnad


On 2010-07-19, at 5:18 AM, Remedios Melero wrote:

> Dear Steve,
> 
> I apologize for the delay in my response, but I will try to give some
explanations to make clear some issues you raised in your message (my
comments are in capital letters, to distinguish them from yours)
> 
> 
> 
> El 15/07/2010 11:22, Steve Hitchcock escribió:
>> Reme,    Thank you for bringing this new service to our attention. OA
policies are vitally important to the development of institutional
repositories, and services that can highlight and bring attention to this
development can be valuable.
>> 
>> There are a few aspects of the validation aspects of the new MELIBEA
service that confuse, and possibly trouble, me. The first is the main
indicator, %OAval, which is the most visible result for a policy. What do
you expect this will tell people about a given policy? I randomly selected a
couple of policies, one of which was for my own school, to find they each
scored about 50%. I would expect these to be among the leaders in terms of
OA policies, so this seems a surprisingly unhelpful score.
>> 
>> So what's the explanation? Note that the objects being evaluated are
institutional OA policies; they are effectively being presented in relation
to institutional repositories when the policy specifies where to archive is
an IR with a URL. It seems that the scores include ratings for OA
publication policy, libre vs gratis OA, publisher pdf, sanctions (score if
Yes), incentives (score if Yes), etc., some of which an institution might
specify but which might not apply to an IR 
>> http://www.accesoabierto.net/politicas/politicas_estructura.php
>> . However you weight these factors they are still contributors to the
overall score, so a policy that is specific to an IR is immediately
handicapped, or appears to be unless there is more context to understand the
scores.
>> 
>>   
>> 
> AS I WROTE BEFORE THIS IS NOT A RANKING, IT IS NOT THE AIM OF MELIBEA  BUT
TO HAVE A KIND OF REFERENCE ON WHAT TOPICS, ISSUES OR MATTERS TO BE INCLUDED
IN AN OA POLICY. WE ARE TALKING ABOUT INSTITUTIONAL POLICIES OF DIFFERENTE
NATURE, NOT ABOUT REPOSITORIES POLICIES. IF THE POLICY ONLY TALKS ABOUT THE
REQUIREMENT TO DEPOSIT IN A REPOSITORY, IT SHOULD  SPECIFY WHAT, WHEN AND
UNDER WHAT CONDITIONS, IF ANY. IT IS NOT THE SAME TO SAY WHAT DOCUMENTS  AND
WHAT VERSIONS AND WHEN THAN SIMPLY SAY " ANY" OR "AS SOON AS 
POSSIBLE" (this
could be a month after publication or years after publication, depending on
one's criteria). GOLD ROUTE, NEVER IS REQUIRED ACCORDING OUR APPROACH 
("Gold
(Recommended in OA journals") AND NOT ALL OA JOURNALS ARE SUPPORTED BY  
SAME
ECONOMIC MODEL.
> 
> 
>> Which leads me to another question on the visualisation of the 
validator,
and its use of green, gold (and red) in the meter. Do the green and gold
refer the the classic OA colours? This would be quite convenient, since it
would appear that the green repository policies I mentioned above are
achieving almost full scores in the green zone of the meter. However, I
suspect this cannot be the case, because it would assume that institutions
must have a green AND gold policy, but not simply gold (whatever argument
could be put for that). 
>>   
>> 
> COLORS DO NOT MEAN THAT, WE WANTED JUST TO DISTINGUISH ZONES LIKE IT WERE
A SPECTRA. 
> 
>> It is important that new services should help reveal and promote OA
policies, as you seek to do, but at the same time not to prejudice the
development of such policies by mixing and not fairly separating the
contributing factors, especially where these relate to different types of
OA.
>>   
>> 
> I DO NOT THINK WE ARE MIXING, IN FACT THERE TWO MODELS, ONE FOR UNIV. AND
RESEARCH CNETRES AND ANOTHER FOR FUNDERS AND GOV. INSTITUTIONS AND THE
QUESTIONS FOR THEM ARE DIFFERENT, for instance, FOR A FUNDER THE QUESTION
ABOUT DEPOSIT O THESIS IS NOT APPLICABLE.
> IN SUMMARY, OUR MODEL COULD NOT BE "PERFECT" BUT I IS ONE, WHICH 
COULD
DETECT DIFFERENCES BETWEEN REQUEST AND REQUIRE, WHO, WHAT , WHEN IF THERE
ARE ANY INCENTIVES OR SANCTIONS (  this has not to be a negative point but
to remember we should  assume  reponsible attitudes).
> 
> However we will revise the model to see if we can make any improvement to
make it clear, we are working also in a graph interface to show some data in
graphical form.
> Best wishes
> Reme
> 
>> 
>>> R. Melero 
>>> IATA, CSIC 
>>> Avda Agustín Escardino 7, 46980 Paterna (Valencia), Spain 
>>> TEl +34 96 390 00 22. Fax 96 363 63 01 
>>> E-mail 
>>> rmelero AT iata.csic.es
>>>  
>>> 
>>> http://www.accesoabierto.net
>>> 
>>> 
>>> --      
>>>     
>>> 
> 
> 
> --      
> To unsubscribe from the BOAI Forum, use the form on this page:
> http://www.soros.org/openaccess/forum.shtml?f


        
--      
To unsubscribe from the BOAI Forum, use the form on this page:
http://www.soros.org/openaccess/forum.shtml?f


        
--      
To unsubscribe from the BOAI Forum, use the form on this page:
http://www.soros.org/openaccess/forum.shtml?f


Re: [BOAI] Re: OA policies and their "weight"

From: Stevan Harnad <harnad AT ecs.soton.ac.uk>
Date: Tue, 20 Jul 2010 21:06:26 +0100


Threading: [BOAI] Re: OA policies and their "weight" from isabel.bernal AT bib.csic.es
      • This Message
             Re: [BOAI] Re: OA policies and their "weight" from isabel.bernal AT BIB.CSIC.ES

On Tue, 20 Jul 2010, Frederick Friend wrote:

> I am glad that Isabel Bernal has sent this message about the excellent OA 
> developments in Spain. I was becoming concerned that comments on the 
MELIBEA 
> service - whether justified or unjustified - were giving the impression 
that 
> our Spanish colleagues are adopting a faulty approach to OA

(1) The (friendly) critique was of MELIBEA, not of CSIC or Spain.

(2) The critique was not of an approach to OA but of an approach to
evaluating approaches to OA.

(3) MELIBEA is a (potentially important) project of a CSIC lab (IATA) 
that was presented at OR2010 http://accesoabierto.net/node/61

> (can any way of bringing OA be faulty?).

Yes it can (e.g., if it is unsuccessful, or needlessly less successful
than it could be).

> I have looked at both the Digital.CSIC and 
> Revistas.CSIC sites, and I found them attractive and very easy to use 
(which 
> unfortunately cannot be said about all repositories).

There are more attractive repositories and less attractive repositories.
Their appearance matters infinitely less than their success in capturing
their total target OA contents (which can only be determined by
comparing their annual deposits to their total annual target output).

OA contents are not searched and retrieved at the repository level, but
at the harvester level. Hence the attractiveness of an individual
repository, be it institutional or central, is of rather minor
importance.

(Neither the attractiveness nor even the deposit rate of the CSIC
repository is relevant to the point under discussion, which is about
how to go about evaluating institutional and funder OA policies, e.g.,
as MELIBEA is being designed to do.)

> The policies underpinning the dual approach make a lot of sense 
> as national policies.

I am not sure what "dual approach" policy is the intended referent 
here,
or whose, but I assume it is (1) green OA self-archiving mandates and 
(2) gold OA publishing payment commitments.

The latter (2) only makes sense after the former (1) has been adopted.

As Isabel Bernal indicates below, CSIC has not yet mandated green OA
(1).

None of this has anything to do with evaluating MELIBEA, the evaluator
of OA policies.

> OA supporters in every country have to encourage OA in whatever 
> way suits their political and cultural environment, and it seems 
> to me that this is what our Spanish colleagues are doing very effectively.

Until a country or funder or institution mandates green OA
self-archiving, it has not yet been effective in promoting or providing
OA.

the substantive is wondering about what might be the substantive issue
underlying this exchange with Fred, it is the usual one, about putting
the gold cart before the green horse.)

Stevan Harnad

> Fred Friend
> JISC Scholarly Communication Consultant
> Honorary Director Scholarly Communication UCL
>
>
> ----- Original Message ----- From: "Isabel Bernal" 
> <isabel.bernal AT BIB.CSIC.ES>
> To: <JISC-REPOSITORIES AT JISCMAIL.AC.UK>
> Sent: Tuesday, July 20, 2010 5:27 PM
> Subject: Re: [BOAI] Re: OA policies and their "weight"
>
>
> Good afternoon everybody,
>
> Following the ongoing discussion in which the CSIC (Spanish National
> Research Council) and its open access efforts have been mentioned, we 
would
> like to clarify the following:
>
> The CSIC Presidency signed the Berlin Declaration on Open Access in 
January
> 2006, and as a result of it the Spanish National Research Council is 
driving
> and implementing open access principles through 2 institutional 
initiatives:
>
> -Digital.CSIC (https://digital.csic.es/) is the institutional repository
> that provides open access to, organises and preserves the scientific 
output
> resulting from the research activities by CSIC 147 institutes and centers.
> The repository is a project by CSIC Libraries Coordination Unit.
>
> -Revistas-CSIC (http://revistas.csic.es/) provides open access to the 35
> scientific Journals published by the institution, covering a wide variety 
of
> scientific disciplines. To date, 14 Journals provide immediate open 
access,
> while 22 apply an embargo period of six months. Before the end of this 
year,
> at least 4 more Journals are planned to move to full OA. Revistas-CSIC is 
a
> project run by the CSIC Publication Department, and is a member of OASPA
> under the category of OA Professional Publishing Organization.
>
> These 2 initiatives fall within the CSIC Vice-presidency of Organization 
and
> Institutional Relations.
>
> To date, CSIC does not have an open access institutional mandate. In the
> absence of a nation-wide open access related law yet, there are regional
> laws in favour of open access that have a direct effect on CSIC, such as
> that of the Government of the Community of Madrid.
>
> Thus, MELIBEA should not be considered a CSIC institutional project.
>
> Best wishes,
>
> Agnès Ponsati, Director of CSIC Libraries Coordination Unit
>
> Ramón Rodríguez, Coordinator of Revistas-CSIC
>
>
>
> -----Mensaje original-----
> De: boai-forum-bounces AT ecs.soton.ac.uk
> [mailto:boai-forum-bounces AT ecs.soton.ac.uk] En nombre de Stevan Harnad
> Enviado el: lunes, 19 de julio de 2010 15:56
> Para: boai-forum AT ecs.soton.ac.uk
> CC: SPARC Open Access Forum; JISC-REPOSITORIES AT JISCMAIL.AC.UK;
> AMERICAN-SCIENTIST-OPEN-ACCESS-FORUM AT LISTSERVER.SIGMAXI.ORG
> Asunto: [BOAI] Re: OA policies and their "weight"
>
> Dear Reme, if I may also make an intervention in your exchange with Steve
> Hitchcock about the MELIBEA OA policy evaluator:
> http://www.accesoabierto.net/politicas/
>
> The MELIBEA service is extremely timely and promising, and could be
> potentially useful and even influential in shaping OA mandates -- but that
> makes it all the more important to get it right, rather than releasing
> MELIBEA prematurely, when it still risks increasing confusion rather than
> providing clarity and direction.
>
> You are right to point out that -- unlike the CSIC's University Ranking 
and
> the Repository Ranking -- the policy evaluator is not really a ranking. 
But
> you have set up the composite algorithm and the graphics to make it a
> ranking just the same.
>
> You are also point out, correctly, that the policy criteria for 
institutions
> and funders are not (and should not be) the same. Yet, with the MELIBEA
> coding as well as the algorithm, they are treated the same way.
>
> You also point out, rightly, that gold OA publishing policy is not central
> to institutional OA policy making, yet there it is, as part of the MELIBEA
> algorithm.
>
> You also point out that the color code has nothing to do with the 
"green" OA
> coding -- yet there it is, competing with the widespread use of green to
> designate self-archiving, and thereby causing confusion, both overt and
> covert.
>
> I would be more than happy to give you feedback on every aspect of MELIBEA
> -- it could be a useful and natural complement to the ROARMAP registry of 
OA
> policies.
>
> But as it is designed now, I can only agree with Steve Hitchcock's points
> and conclude that consulting MELIBEA today would be likely to induce
> confusion and would not help in bringing the all-important focus and
> direction to OA policy-making that I am sure CSIC, too, seeks, and seeks 
to
> help bring about.
>
> Here are just a few prima facie points:
>
> (1) Since MELIBEA is not, and should not be construed as a ranking of OA
> policies -- especially because it includes both institutional and funder
> policies -- it is important NOT to plug it into an algorithm until and
> unless the algorithm has first been carefully tested, with consultation, 
to
> make sure it weights policy criteria in a way that optimizes OA progress 
and
> guides policy-makers in the right direction.
>
> (2) For this reason, it is more important to allow users to generate
> separate flat lists of institutions or funders on the various policy
> criteria, considered and compared independently, rather than on the basis 
of
> a prematurely and arbitrarily weighted joint algorithm.
>
> (3) This is all the more important since the data are based on less then 
200
> institutions, whereas the CSIC University Rankings are based on thousands.
> Since the population is still so small, MELIBEA risks having a
> disproportionate effect on initial conditions and hence direction-setting;
> all the more reason NOT to amplify noise and indirection by assigning
> untested initial weights without carefully thinking through and weighing 
the
> consequences.
>
> (4) A potential internal cross-validator of some of the criteria would be 
a
> reliable measure of outcome -- but that requires much more attention to
> estimating the annual size and growth-rate of each repository (in terms of
> OA's target contents, which are full-text articles), normalized for
> institution size and annual total target output. Policy criteria (such as
> request/require or immediate/delayed) should be cross-validated against
> these outcome measures (such as percentage and growth rate of annual 
target
> output).
>
> (5) The MELIBEA color coding needs to be revised, and revised quickly, if
> there is to be an algorithm at all. All those arbitrary colors in the
> display of single repositories as ranked by the algorithm are both
> unnecessary and confusing. The objective should be to order and focus
> clearly and intuitively. Whatever is correlated with more green OA output
> (such as a higher level or faster growth rate in OA's target content) 
should
> be coded as darker or bigger shades of green. The same should be true for
> the policy criteria, separately and jointly: in each case, 
request/require,
> delayed/immediate, etc., the greenward polarity is obvious and intuitive.
> This should be reflected in the graphics as well as in any comparative
> rankings.
>
> (6) If you include repositories with no OA policy at all (i.e., just a
> repository and an open invitation to deposit) then all you are doing is
> duplicating ROAR and ROARMAP, whereas the purpose, presumably, of MELIBEA,
> is to highlight, weigh and compare specific policy differences among (the
> very few) repositories that DO have policies.
>
> (7) The sign-up data --
> http://www.accesoabierto.net/politicas/nueva.php?directorio=politicas -- 
are
> also rather confusing; the criteria are not always consistent, relevant or
> applicable. The sign-up seems to be designed to make a funder mandate the
> generic option, whereas this is quite the opposite of reality. There are 
far
> more institutions and institutional repositories and policies than 
funders.
> There should be separate criterial lists for institutional policies and 
for
> funder policies; they are not the same. There is also far too much focus 
on
> gold OA policy and payment. If included at all, this should only be at the
> end, as an addendum, not the focus at the beginning, and on a par with 
green
> OA policy.
>
> (8) There is also potential confusion on the matter of 
"waivers": There are
> two aspects of a mandate. One concerns whether or not deposit is required
> (and if so, whether that requirement can be waived) and the other concerns
> whether or not rights-reservation is required (and if so, whether that
> requirement can be waived). These two distinct and independent
> requirements/waivers are completely conflated in the current version of
> MELIBEA.
>
> I hope there will be substantive consultation and conscientious redesign 
of
> these and other aspects of MELIBEA before it is can recommended for 
serious
> consideration and use.
>
> Stevan Harnad
>
>
> On 2010-07-19, at 5:18 AM, Remedios Melero wrote:
>
>>  Dear Steve,
>>
>>  I apologize for the delay in my response, but I will try to give some
> explanations to make clear some issues you raised in your message (my
> comments are in capital letters, to distinguish them from yours)
>> 
>> 
>>
>>  El 15/07/2010 11:22, Steve Hitchcock escribió:
>> >  Reme,    Thank you for bringing this new service to our 
attention. OA
> policies are vitally important to the development of institutional
> repositories, and services that can highlight and bring attention to this
> development can be valuable.
>> > 
>> >  There are a few aspects of the validation aspects of the new 
MELIBEA
> service that confuse, and possibly trouble, me. The first is the main
> indicator, %OAval, which is the most visible result for a policy. What do
> you expect this will tell people about a given policy? I randomly selected 
a
> couple of policies, one of which was for my own school, to find they each
> scored about 50%. I would expect these to be among the leaders in terms of
> OA policies, so this seems a surprisingly unhelpful score.
>> > 
>> >  So what's the explanation? Note that the objects being evaluated 
are
> institutional OA policies; they are effectively being presented in 
relation
> to institutional repositories when the policy specifies where to archive 
is
> an IR with a URL. It seems that the scores include ratings for OA
> publication policy, libre vs gratis OA, publisher pdf, sanctions (score if
> Yes), incentives (score if Yes), etc., some of which an institution might
> specify but which might not apply to an IR
>> >  http://www.accesoabierto.net/politicas/politicas_estructura.php
>> >  . However you weight these factors they are still contributors 
to the
> overall score, so a policy that is specific to an IR is immediately
> handicapped, or appears to be unless there is more context to understand 
the
> scores.
>> > 
>> > 
>> >
>>  AS I WROTE BEFORE THIS IS NOT A RANKING, IT IS NOT THE AIM OF MELIBEA 
 BUT
> TO HAVE A KIND OF REFERENCE ON WHAT TOPICS, ISSUES OR MATTERS TO BE 
INCLUDED
> IN AN OA POLICY. WE ARE TALKING ABOUT INSTITUTIONAL POLICIES OF DIFFERENTE
> NATURE, NOT ABOUT REPOSITORIES POLICIES. IF THE POLICY ONLY TALKS ABOUT 
THE
> REQUIREMENT TO DEPOSIT IN A REPOSITORY, IT SHOULD  SPECIFY WHAT, WHEN AND
> UNDER WHAT CONDITIONS, IF ANY. IT IS NOT THE SAME TO SAY WHAT DOCUMENTS  
AND
> WHAT VERSIONS AND WHEN THAN SIMPLY SAY " ANY" OR "AS SOON 
AS POSSIBLE" (this
> could be a month after publication or years after publication, depending 
on
> one's criteria). GOLD ROUTE, NEVER IS REQUIRED ACCORDING OUR APPROACH 
("Gold
> (Recommended in OA journals") AND NOT ALL OA JOURNALS ARE SUPPORTED 
BY  SAME
> ECONOMIC MODEL.
>> 
>> 
>> >  Which leads me to another question on the visualisation of the 
>> >  validator,
> and its use of green, gold (and red) in the meter. Do the green and gold
> refer the the classic OA colours? This would be quite convenient, since it
> would appear that the green repository policies I mentioned above are
> achieving almost full scores in the green zone of the meter. However, I
> suspect this cannot be the case, because it would assume that institutions
> must have a green AND gold policy, but not simply gold (whatever argument
> could be put for that).
>> > 
>> >
>>  COLORS DO NOT MEAN THAT, WE WANTED JUST TO DISTINGUISH ZONES LIKE IT 
WERE
> A SPECTRA.
>> 
>> >  It is important that new services should help reveal and promote 
OA
> policies, as you seek to do, but at the same time not to prejudice the
> development of such policies by mixing and not fairly separating the
> contributing factors, especially where these relate to different types of
> OA.
>> > 
>> >
>>  I DO NOT THINK WE ARE MIXING, IN FACT THERE TWO MODELS, ONE FOR UNIV. 
AND
> RESEARCH CNETRES AND ANOTHER FOR FUNDERS AND GOV. INSTITUTIONS AND THE
> QUESTIONS FOR THEM ARE DIFFERENT, for instance, FOR A FUNDER THE QUESTION
> ABOUT DEPOSIT O THESIS IS NOT APPLICABLE.
>>  IN SUMMARY, OUR MODEL COULD NOT BE "PERFECT" BUT I IS ONE, 
WHICH COULD
> DETECT DIFFERENCES BETWEEN REQUEST AND REQUIRE, WHO, WHAT , WHEN IF THERE
> ARE ANY INCENTIVES OR SANCTIONS (  this has not to be a negative point but
> to remember we should  assume  reponsible attitudes).
>>
>>  However we will revise the model to see if we can make any 
improvement to
> make it clear, we are working also in a graph interface to show some data 
in
> graphical form.
>>  Best wishes
>>  Reme
>> 
>> > 
>> > >  R. Melero
>> > >  IATA, CSIC
>> > >  Avda Agustín Escardino 7, 46980 Paterna (Valencia), Spain
>> > >  TEl +34 96 390 00 22. Fax 96 363 63 01
>> > >  E-mail
>> > >  rmelero AT iata.csic.es
>> > > 
>> > > 
>> > >  http://www.accesoabierto.net
>> > > 
>> > > 
>> > >  --
>> > > 
>> > > 
>> 
>>
>>  --
>>  To unsubscribe from the BOAI Forum, use the form on this page:
>>  http://www.soros.org/openaccess/forum.shtml?f
>
>
>
> --
> To unsubscribe from the BOAI Forum, use the form on this page:
> http://www.soros.org/openaccess/forum.shtml?f
>

[BOAI] Re: OA policies and their "weight"

From: Stevan Harnad <harnad AT ecs.soton.ac.uk>
Date: Tue, 20 Jul 2010 21:06:26 +0100 (BST)


Threading: [BOAI] Re: OA policies and their "weight" from isabel.bernal AT bib.csic.es
      • This Message

On Tue, 20 Jul 2010, Frederick Friend wrote:

> I am glad that Isabel Bernal has sent this message about the excellent OA 
> developments in Spain. I was becoming concerned that comments on the 
MELIBEA 
> service - whether justified or unjustified - were giving the impression 
that 
> our Spanish colleagues are adopting a faulty approach to OA

(1) The (friendly) critique was of MELIBEA, not of CSIC or Spain.

(2) The critique was not of an approach to OA but of an approach to
evaluating approaches to OA.

(3) MELIBEA is a (potentially important) project of a CSIC lab (IATA) 
that was presented at OR2010 http://accesoabierto.net/node/61

> (can any way of bringing OA be faulty?).

Yes it can (e.g., if it is unsuccessful, or needlessly less successful
than it could be).

> I have looked at both the Digital.CSIC and 
> Revistas.CSIC sites, and I found them attractive and very easy to use 
(which 
> unfortunately cannot be said about all repositories).

There are more attractive repositories and less attractive repositories.
Their appearance matters infinitely less than their success in capturing
their total target OA contents (which can only be determined by
comparing their annual deposits to their total annual target output).

OA contents are not searched and retrieved at the repository level, but
at the harvester level. Hence the attractiveness of an individual
repository, be it institutional or central, is of rather minor
importance.

(Neither the attractiveness nor even the deposit rate of the CSIC
repository is relevant to the point under discussion, which is about
how to go about evaluating institutional and funder OA policies, e.g.,
as MELIBEA is being designed to do.)

> The policies underpinning the dual approach make a lot of sense 
> as national policies.

I am not sure what "dual approach" policy is the intended referent 
here,
or whose, but I assume it is (1) green OA self-archiving mandates and 
(2) gold OA publishing payment commitments.

The latter (2) only makes sense after the former (1) has been adopted.

As Isabel Bernal indicates below, CSIC has not yet mandated green OA
(1).

None of this has anything to do with evaluating MELIBEA, the evaluator
of OA policies.

> OA supporters in every country have to encourage OA in whatever 
> way suits their political and cultural environment, and it seems 
> to me that this is what our Spanish colleagues are doing very effectively.

Until a country or funder or institution mandates green OA
self-archiving, it has not yet been effective in promoting or providing
OA.

the substantive is wondering about what might be the substantive issue
underlying this exchange with Fred, it is the usual one, about putting
the gold cart before the green horse.)

Stevan Harnad

> Fred Friend
> JISC Scholarly Communication Consultant
> Honorary Director Scholarly Communication UCL
>
>
> ----- Original Message ----- From: "Isabel Bernal" 
> <isabel.bernal AT BIB.CSIC.ES>
> To: <JISC-REPOSITORIES AT JISCMAIL.AC.UK>
> Sent: Tuesday, July 20, 2010 5:27 PM
> Subject: Re: [BOAI] Re: OA policies and their "weight"
>
>
> Good afternoon everybody,
>
> Following the ongoing discussion in which the CSIC (Spanish National
> Research Council) and its open access efforts have been mentioned, we 
would
> like to clarify the following:
>
> The CSIC Presidency signed the Berlin Declaration on Open Access in 
January
> 2006, and as a result of it the Spanish National Research Council is 
driving
> and implementing open access principles through 2 institutional 
initiatives:
>
> -Digital.CSIC (https://digital.csic.es/) is the institutional repository
> that provides open access to, organises and preserves the scientific 
output
> resulting from the research activities by CSIC 147 institutes and centers.
> The repository is a project by CSIC Libraries Coordination Unit.
>
> -Revistas-CSIC (http://revistas.csic.es/) provides open access to the 35
> scientific Journals published by the institution, covering a wide variety 
of
> scientific disciplines. To date, 14 Journals provide immediate open 
access,
> while 22 apply an embargo period of six months. Before the end of this 
year,
> at least 4 more Journals are planned to move to full OA. Revistas-CSIC is 
a
> project run by the CSIC Publication Department, and is a member of OASPA
> under the category of OA Professional Publishing Organization.
>
> These 2 initiatives fall within the CSIC Vice-presidency of Organization 
and
> Institutional Relations.
>
> To date, CSIC does not have an open access institutional mandate. In the
> absence of a nation-wide open access related law yet, there are regional
> laws in favour of open access that have a direct effect on CSIC, such as
> that of the Government of the Community of Madrid.
>
> Thus, MELIBEA should not be considered a CSIC institutional project.
>
> Best wishes,
>
> Agnès Ponsati, Director of CSIC Libraries Coordination Unit
>
> Ramón Rodríguez, Coordinator of Revistas-CSIC
>
>
>
> -----Mensaje original-----
> De: boai-forum-bounces AT ecs.soton.ac.uk
> [mailto:boai-forum-bounces AT ecs.soton.ac.uk] En nombre de Stevan Harnad
> Enviado el: lunes, 19 de julio de 2010 15:56
> Para: boai-forum AT ecs.soton.ac.uk
> CC: SPARC Open Access Forum; JISC-REPOSITORIES AT JISCMAIL.AC.UK;
> AMERICAN-SCIENTIST-OPEN-ACCESS-FORUM AT LISTSERVER.SIGMAXI.ORG
> Asunto: [BOAI] Re: OA policies and their "weight"
>
> Dear Reme, if I may also make an intervention in your exchange with Steve
> Hitchcock about the MELIBEA OA policy evaluator:
> http://www.accesoabierto.net/politicas/
>
> The MELIBEA service is extremely timely and promising, and could be
> potentially useful and even influential in shaping OA mandates -- but that
> makes it all the more important to get it right, rather than releasing
> MELIBEA prematurely, when it still risks increasing confusion rather than
> providing clarity and direction.
>
> You are right to point out that -- unlike the CSIC's University Ranking 
and
> the Repository Ranking -- the policy evaluator is not really a ranking. 
But
> you have set up the composite algorithm and the graphics to make it a
> ranking just the same.
>
> You are also point out, correctly, that the policy criteria for 
institutions
> and funders are not (and should not be) the same. Yet, with the MELIBEA
> coding as well as the algorithm, they are treated the same way.
>
> You also point out, rightly, that gold OA publishing policy is not central
> to institutional OA policy making, yet there it is, as part of the MELIBEA
> algorithm.
>
> You also point out that the color code has nothing to do with the 
"green" OA
> coding -- yet there it is, competing with the widespread use of green to
> designate self-archiving, and thereby causing confusion, both overt and
> covert.
>
> I would be more than happy to give you feedback on every aspect of MELIBEA
> -- it could be a useful and natural complement to the ROARMAP registry of 
OA
> policies.
>
> But as it is designed now, I can only agree with Steve Hitchcock's points
> and conclude that consulting MELIBEA today would be likely to induce
> confusion and would not help in bringing the all-important focus and
> direction to OA policy-making that I am sure CSIC, too, seeks, and seeks 
to
> help bring about.
>
> Here are just a few prima facie points:
>
> (1) Since MELIBEA is not, and should not be construed as a ranking of OA
> policies -- especially because it includes both institutional and funder
> policies -- it is important NOT to plug it into an algorithm until and
> unless the algorithm has first been carefully tested, with consultation, 
to
> make sure it weights policy criteria in a way that optimizes OA progress 
and
> guides policy-makers in the right direction.
>
> (2) For this reason, it is more important to allow users to generate
> separate flat lists of institutions or funders on the various policy
> criteria, considered and compared independently, rather than on the basis 
of
> a prematurely and arbitrarily weighted joint algorithm.
>
> (3) This is all the more important since the data are based on less then 
200
> institutions, whereas the CSIC University Rankings are based on thousands.
> Since the population is still so small, MELIBEA risks having a
> disproportionate effect on initial conditions and hence direction-setting;
> all the more reason NOT to amplify noise and indirection by assigning
> untested initial weights without carefully thinking through and weighing 
the
> consequences.
>
> (4) A potential internal cross-validator of some of the criteria would be 
a
> reliable measure of outcome -- but that requires much more attention to
> estimating the annual size and growth-rate of each repository (in terms of
> OA's target contents, which are full-text articles), normalized for
> institution size and annual total target output. Policy criteria (such as
> request/require or immediate/delayed) should be cross-validated against
> these outcome measures (such as percentage and growth rate of annual 
target
> output).
>
> (5) The MELIBEA color coding needs to be revised, and revised quickly, if
> there is to be an algorithm at all. All those arbitrary colors in the
> display of single repositories as ranked by the algorithm are both
> unnecessary and confusing. The objective should be to order and focus
> clearly and intuitively. Whatever is correlated with more green OA output
> (such as a higher level or faster growth rate in OA's target content) 
should
> be coded as darker or bigger shades of green. The same should be true for
> the policy criteria, separately and jointly: in each case, 
request/require,
> delayed/immediate, etc., the greenward polarity is obvious and intuitive.
> This should be reflected in the graphics as well as in any comparative
> rankings.
>
> (6) If you include repositories with no OA policy at all (i.e., just a
> repository and an open invitation to deposit) then all you are doing is
> duplicating ROAR and ROARMAP, whereas the purpose, presumably, of MELIBEA,
> is to highlight, weigh and compare specific policy differences among (the
> very few) repositories that DO have policies.
>
> (7) The sign-up data --
> http://www.accesoabierto.net/politicas/nueva.php?directorio=politicas -- 
are
> also rather confusing; the criteria are not always consistent, relevant or
> applicable. The sign-up seems to be designed to make a funder mandate the
> generic option, whereas this is quite the opposite of reality. There are 
far
> more institutions and institutional repositories and policies than 
funders.
> There should be separate criterial lists for institutional policies and 
for
> funder policies; they are not the same. There is also far too much focus 
on
> gold OA policy and payment. If included at all, this should only be at the
> end, as an addendum, not the focus at the beginning, and on a par with 
green
> OA policy.
>
> (8) There is also potential confusion on the matter of 
"waivers": There are
> two aspects of a mandate. One concerns whether or not deposit is required
> (and if so, whether that requirement can be waived) and the other concerns
> whether or not rights-reservation is required (and if so, whether that
> requirement can be waived). These two distinct and independent
> requirements/waivers are completely conflated in the current version of
> MELIBEA.
>
> I hope there will be substantive consultation and conscientious redesign 
of
> these and other aspects of MELIBEA before it is can recommended for 
serious
> consideration and use.
>
> Stevan Harnad
>
>
> On 2010-07-19, at 5:18 AM, Remedios Melero wrote:
>
>>  Dear Steve,
>>
>>  I apologize for the delay in my response, but I will try to give some
> explanations to make clear some issues you raised in your message (my
> comments are in capital letters, to distinguish them from yours)
>> 
>> 
>>
>>  El 15/07/2010 11:22, Steve Hitchcock escribió:
>> >  Reme,    Thank you for bringing this new service to our 
attention. OA
> policies are vitally important to the development of institutional
> repositories, and services that can highlight and bring attention to this
> development can be valuable.
>> > 
>> >  There are a few aspects of the validation aspects of the new 
MELIBEA
> service that confuse, and possibly trouble, me. The first is the main
> indicator, %OAval, which is the most visible result for a policy. What do
> you expect this will tell people about a given policy? I randomly selected 
a
> couple of policies, one of which was for my own school, to find they each
> scored about 50%. I would expect these to be among the leaders in terms of
> OA policies, so this seems a surprisingly unhelpful score.
>> > 
>> >  So what's the explanation? Note that the objects being evaluated 
are
> institutional OA policies; they are effectively being presented in 
relation
> to institutional repositories when the policy specifies where to archive 
is
> an IR with a URL. It seems that the scores include ratings for OA
> publication policy, libre vs gratis OA, publisher pdf, sanctions (score if
> Yes), incentives (score if Yes), etc., some of which an institution might
> specify but which might not apply to an IR
>> >  http://www.accesoabierto.net/politicas/politicas_estructura.php
>> >  . However you weight these factors they are still contributors 
to the
> overall score, so a policy that is specific to an IR is immediately
> handicapped, or appears to be unless there is more context to understand 
the
> scores.
>> > 
>> > 
>> >
>>  AS I WROTE BEFORE THIS IS NOT A RANKING, IT IS NOT THE AIM OF MELIBEA 
 BUT
> TO HAVE A KIND OF REFERENCE ON WHAT TOPICS, ISSUES OR MATTERS TO BE 
INCLUDED
> IN AN OA POLICY. WE ARE TALKING ABOUT INSTITUTIONAL POLICIES OF DIFFERENTE
> NATURE, NOT ABOUT REPOSITORIES POLICIES. IF THE POLICY ONLY TALKS ABOUT 
THE
> REQUIREMENT TO DEPOSIT IN A REPOSITORY, IT SHOULD  SPECIFY WHAT, WHEN AND
> UNDER WHAT CONDITIONS, IF ANY. IT IS NOT THE SAME TO SAY WHAT DOCUMENTS  
AND
> WHAT VERSIONS AND WHEN THAN SIMPLY SAY " ANY" OR "AS SOON 
AS POSSIBLE" (this
> could be a month after publication or years after publication, depending 
on
> one's criteria). GOLD ROUTE, NEVER IS REQUIRED ACCORDING OUR APPROACH 
("Gold
> (Recommended in OA journals") AND NOT ALL OA JOURNALS ARE SUPPORTED 
BY  SAME
> ECONOMIC MODEL.
>> 
>> 
>> >  Which leads me to another question on the visualisation of the 
>> >  validator,
> and its use of green, gold (and red) in the meter. Do the green and gold
> refer the the classic OA colours? This would be quite convenient, since it
> would appear that the green repository policies I mentioned above are
> achieving almost full scores in the green zone of the meter. However, I
> suspect this cannot be the case, because it would assume that institutions
> must have a green AND gold policy, but not simply gold (whatever argument
> could be put for that).
>> > 
>> >
>>  COLORS DO NOT MEAN THAT, WE WANTED JUST TO DISTINGUISH ZONES LIKE IT 
WERE
> A SPECTRA.
>> 
>> >  It is important that new services should help reveal and promote 
OA
> policies, as you seek to do, but at the same time not to prejudice the
> development of such policies by mixing and not fairly separating the
> contributing factors, especially where these relate to different types of
> OA.
>> > 
>> >
>>  I DO NOT THINK WE ARE MIXING, IN FACT THERE TWO MODELS, ONE FOR UNIV. 
AND
> RESEARCH CNETRES AND ANOTHER FOR FUNDERS AND GOV. INSTITUTIONS AND THE
> QUESTIONS FOR THEM ARE DIFFERENT, for instance, FOR A FUNDER THE QUESTION
> ABOUT DEPOSIT O THESIS IS NOT APPLICABLE.
>>  IN SUMMARY, OUR MODEL COULD NOT BE "PERFECT" BUT I IS ONE, 
WHICH COULD
> DETECT DIFFERENCES BETWEEN REQUEST AND REQUIRE, WHO, WHAT , WHEN IF THERE
> ARE ANY INCENTIVES OR SANCTIONS (  this has not to be a negative point but
> to remember we should  assume  reponsible attitudes).
>>
>>  However we will revise the model to see if we can make any 
improvement to
> make it clear, we are working also in a graph interface to show some data 
in
> graphical form.
>>  Best wishes
>>  Reme
>> 
>> > 
>> > >  R. Melero
>> > >  IATA, CSIC
>> > >  Avda Agustín Escardino 7, 46980 Paterna (Valencia), Spain
>> > >  TEl +34 96 390 00 22. Fax 96 363 63 01
>> > >  E-mail
>> > >  rmelero AT iata.csic.es
>> > > 
>> > > 
>> > >  http://www.accesoabierto.net
>> > > 
>> > > 
>> > >  --
>> > > 
>> > > 
>> 
>>
>>  --
>>  To unsubscribe from the BOAI Forum, use the form on this page:
>>  http://www.soros.org/openaccess/forum.shtml?f
>
>
>
> --
> To unsubscribe from the BOAI Forum, use the form on this page:
> http://www.soros.org/openaccess/forum.shtml?f
>
        
--      
To unsubscribe from the BOAI Forum, use the form on this page:
http://www.soros.org/openaccess/forum.shtml?f


Re: [BOAI] Re: OA policies and their "weight"

From: Isabel Bernal <isabel.bernal AT BIB.CSIC.ES>
Date: Tue, 20 Jul 2010 18:27:52 +0200


Threading: Re: [BOAI] Re: OA policies and their "weight" from harnad AT ecs.soton.ac.uk
      • This Message

Good afternoon everybody,
 
Following the ongoing discussion in which the CSIC (Spanish National
Research Council) and its open access efforts have been mentioned, we would
like to clarify the following:
 
The CSIC Presidency signed the Berlin Declaration on Open Access in January
2006, and as a result of it the Spanish National Research Council is driving
and implementing open access principles through 2 institutional initiatives:

-Digital.CSIC (https://digital.csic.es/) is the institutional repository
that provides open access to, organises and preserves the scientific output
resulting from the research activities by CSIC 147 institutes and centers.
The repository is a project by CSIC Libraries Coordination Unit.

-Revistas-CSIC (http://revistas.csic.es/) provides open access to the 35
scientific Journals published by the institution, covering a wide variety of
scientific disciplines. To date, 14 Journals provide immediate open access,
while 22 apply an embargo period of six months. Before the end of this year,
at least 4 more Journals are planned to move to full OA. Revistas-CSIC is a
project run by the CSIC Publication Department, and is a member of OASPA
under the category of OA Professional Publishing Organization.

These 2 initiatives fall within the CSIC Vice-presidency of Organization and
Institutional Relations.

To date, CSIC does not have an open access institutional mandate. In the
absence of a nation-wide open access related law yet, there are regional
laws in favour of open access that have a direct effect on CSIC, such as
that of the Government of the Community of Madrid.

Thus, MELIBEA should not be considered a CSIC institutional project.

Best wishes,

Agnès Ponsati, Director of CSIC Libraries Coordination Unit 

Ramón Rodríguez, Coordinator of Revistas-CSIC



-----Mensaje original-----
De: boai-forum-bounces AT ecs.soton.ac.uk
[mailto:boai-forum-bounces AT ecs.soton.ac.uk] En nombre de Stevan Harnad
Enviado el: lunes, 19 de julio de 2010 15:56
Para: boai-forum AT ecs.soton.ac.uk
CC: SPARC Open Access Forum; JISC-REPOSITORIES AT JISCMAIL.AC.UK;
AMERICAN-SCIENTIST-OPEN-ACCESS-FORUM AT LISTSERVER.SIGMAXI.ORG
Asunto: [BOAI] Re: OA policies and their "weight"

Dear Reme, if I may also make an intervention in your exchange with Steve
Hitchcock about the MELIBEA OA policy evaluator:
http://www.accesoabierto.net/politicas/ 

The MELIBEA service is extremely timely and promising, and could be
potentially useful and even influential in shaping OA mandates -- but that
makes it all the more important to get it right, rather than releasing
MELIBEA prematurely, when it still risks increasing confusion rather than
providing clarity and direction.

You are right to point out that -- unlike the CSIC's University Ranking and
the Repository Ranking -- the policy evaluator is not really a ranking. But
you have set up the composite algorithm and the graphics to make it a
ranking just the same. 

You are also point out, correctly, that the policy criteria for institutions
and funders are not (and should not be) the same. Yet, with the MELIBEA
coding as well as the algorithm, they are treated the same way. 

You also point out, rightly, that gold OA publishing policy is not central
to institutional OA policy making, yet there it is, as part of the MELIBEA
algorithm.

You also point out that the color code has nothing to do with the 
"green" OA
coding -- yet there it is, competing with the widespread use of green to
designate self-archiving, and thereby causing confusion, both overt and
covert.

I would be more than happy to give you feedback on every aspect of MELIBEA
-- it could be a useful and natural complement to the ROARMAP registry of OA
policies. 

But as it is designed now, I can only agree with Steve Hitchcock's points
and conclude that consulting MELIBEA today would be likely to induce
confusion and would not help in bringing the all-important focus and
direction to OA policy-making that I am sure CSIC, too, seeks, and seeks to
help bring about.

Here are just a few prima facie points:

(1) Since MELIBEA is not, and should not be construed as a ranking of OA
policies -- especially because it includes both institutional and funder
policies -- it is important NOT to plug it into an algorithm until and
unless the algorithm has first been carefully tested, with consultation, to
make sure it weights policy criteria in a way that optimizes OA progress and
guides policy-makers in the right direction.

(2) For this reason, it is more important to allow users to generate
separate flat lists of institutions or funders on the various policy
criteria, considered and compared independently, rather than on the basis of
a prematurely and arbitrarily weighted joint algorithm.

(3) This is all the more important since the data are based on less then 200
institutions, whereas the CSIC University Rankings are based on thousands.
Since the population is still so small, MELIBEA risks having a
disproportionate effect on initial conditions and hence direction-setting;
all the more reason NOT to amplify noise and indirection by assigning
untested initial weights without carefully thinking through and weighing the
consequences.

(4) A potential internal cross-validator of some of the criteria would be a
reliable measure of outcome -- but that requires much more attention to
estimating the annual size and growth-rate of each repository (in terms of
OA's target contents, which are full-text articles), normalized for
institution size and annual total target output. Policy criteria (such as
request/require or immediate/delayed) should be cross-validated against
these outcome measures (such as percentage and growth rate of annual target
output).

(5) The MELIBEA color coding needs to be revised, and revised quickly, if
there is to be an algorithm at all. All those arbitrary colors in the
display of single repositories as ranked by the algorithm are both
unnecessary and confusing. The objective should be to order and focus
clearly and intuitively. Whatever is correlated with more green OA output
(such as a higher level or faster growth rate in OA's target content) should
be coded as darker or bigger shades of green. The same should be true for
the policy criteria, separately and jointly: in each case, request/require,
delayed/immediate, etc., the greenward polarity is obvious and intuitive.
This should be reflected in the graphics as well as in any comparative
rankings.

(6) If you include repositories with no OA policy at all (i.e., just a
repository and an open invitation to deposit) then all you are doing is
duplicating ROAR and ROARMAP, whereas the purpose, presumably, of MELIBEA,
is to highlight, weigh and compare specific policy differences among (the
very few) repositories that DO have policies.

(7) The sign-up data --
http://www.accesoabierto.net/politicas/nueva.php?directorio=politicas -- are
also rather confusing; the criteria are not always consistent, relevant or
applicable. The sign-up seems to be designed to make a funder mandate the
generic option, whereas this is quite the opposite of reality. There are far
more institutions and institutional repositories and policies than funders.
There should be separate criterial lists for institutional policies and for
funder policies; they are not the same. There is also far too much focus on
gold OA policy and payment. If included at all, this should only be at the
end, as an addendum, not the focus at the beginning, and on a par with green
OA policy. 

(8) There is also potential confusion on the matter of "waivers": 
There are
two aspects of a mandate. One concerns whether or not deposit is required
(and if so, whether that requirement can be waived) and the other concerns
whether or not rights-reservation is required (and if so, whether that
requirement can be waived). These two distinct and independent
requirements/waivers are completely conflated in the current version of
MELIBEA.

I hope there will be substantive consultation and conscientious redesign of
these and other aspects of MELIBEA before it is can recommended for serious
consideration and use.

Stevan Harnad


On 2010-07-19, at 5:18 AM, Remedios Melero wrote:

> Dear Steve,
> 
> I apologize for the delay in my response, but I will try to give some
explanations to make clear some issues you raised in your message (my
comments are in capital letters, to distinguish them from yours)
> 
> 
> 
> El 15/07/2010 11:22, Steve Hitchcock escribió:
>> Reme,    Thank you for bringing this new service to our attention. OA
policies are vitally important to the development of institutional
repositories, and services that can highlight and bring attention to this
development can be valuable.
>> 
>> There are a few aspects of the validation aspects of the new MELIBEA
service that confuse, and possibly trouble, me. The first is the main
indicator, %OAval, which is the most visible result for a policy. What do
you expect this will tell people about a given policy? I randomly selected a
couple of policies, one of which was for my own school, to find they each
scored about 50%. I would expect these to be among the leaders in terms of
OA policies, so this seems a surprisingly unhelpful score.
>> 
>> So what's the explanation? Note that the objects being evaluated are
institutional OA policies; they are effectively being presented in relation
to institutional repositories when the policy specifies where to archive is
an IR with a URL. It seems that the scores include ratings for OA
publication policy, libre vs gratis OA, publisher pdf, sanctions (score if
Yes), incentives (score if Yes), etc., some of which an institution might
specify but which might not apply to an IR 
>> http://www.accesoabierto.net/politicas/politicas_estructura.php
>> . However you weight these factors they are still contributors to the
overall score, so a policy that is specific to an IR is immediately
handicapped, or appears to be unless there is more context to understand the
scores.
>> 
>>   
>> 
> AS I WROTE BEFORE THIS IS NOT A RANKING, IT IS NOT THE AIM OF MELIBEA  BUT
TO HAVE A KIND OF REFERENCE ON WHAT TOPICS, ISSUES OR MATTERS TO BE INCLUDED
IN AN OA POLICY. WE ARE TALKING ABOUT INSTITUTIONAL POLICIES OF DIFFERENTE
NATURE, NOT ABOUT REPOSITORIES POLICIES. IF THE POLICY ONLY TALKS ABOUT THE
REQUIREMENT TO DEPOSIT IN A REPOSITORY, IT SHOULD  SPECIFY WHAT, WHEN AND
UNDER WHAT CONDITIONS, IF ANY. IT IS NOT THE SAME TO SAY WHAT DOCUMENTS  AND
WHAT VERSIONS AND WHEN THAN SIMPLY SAY " ANY" OR "AS SOON AS 
POSSIBLE" (this
could be a month after publication or years after publication, depending on
one's criteria). GOLD ROUTE, NEVER IS REQUIRED ACCORDING OUR APPROACH 
("Gold
(Recommended in OA journals") AND NOT ALL OA JOURNALS ARE SUPPORTED BY  
SAME
ECONOMIC MODEL.
> 
> 
>> Which leads me to another question on the visualisation of the 
validator,
and its use of green, gold (and red) in the meter. Do the green and gold
refer the the classic OA colours? This would be quite convenient, since it
would appear that the green repository policies I mentioned above are
achieving almost full scores in the green zone of the meter. However, I
suspect this cannot be the case, because it would assume that institutions
must have a green AND gold policy, but not simply gold (whatever argument
could be put for that). 
>>   
>> 
> COLORS DO NOT MEAN THAT, WE WANTED JUST TO DISTINGUISH ZONES LIKE IT WERE
A SPECTRA. 
> 
>> It is important that new services should help reveal and promote OA
policies, as you seek to do, but at the same time not to prejudice the
development of such policies by mixing and not fairly separating the
contributing factors, especially where these relate to different types of
OA.
>>   
>> 
> I DO NOT THINK WE ARE MIXING, IN FACT THERE TWO MODELS, ONE FOR UNIV. AND
RESEARCH CNETRES AND ANOTHER FOR FUNDERS AND GOV. INSTITUTIONS AND THE
QUESTIONS FOR THEM ARE DIFFERENT, for instance, FOR A FUNDER THE QUESTION
ABOUT DEPOSIT O THESIS IS NOT APPLICABLE.
> IN SUMMARY, OUR MODEL COULD NOT BE "PERFECT" BUT I IS ONE, WHICH 
COULD
DETECT DIFFERENCES BETWEEN REQUEST AND REQUIRE, WHO, WHAT , WHEN IF THERE
ARE ANY INCENTIVES OR SANCTIONS (  this has not to be a negative point but
to remember we should  assume  reponsible attitudes).
> 
> However we will revise the model to see if we can make any improvement to
make it clear, we are working also in a graph interface to show some data in
graphical form.
> Best wishes
> Reme
> 
>> 
>>> R. Melero 
>>> IATA, CSIC 
>>> Avda Agustín Escardino 7, 46980 Paterna (Valencia), Spain 
>>> TEl +34 96 390 00 22. Fax 96 363 63 01 
>>> E-mail 
>>> rmelero AT iata.csic.es
>>>  
>>> 
>>> http://www.accesoabierto.net
>>> 
>>> 
>>> --      
>>>     
>>> 
> 
> 
> --      
> To unsubscribe from the BOAI Forum, use the form on this page:
> http://www.soros.org/openaccess/forum.shtml?f


        
--      
To unsubscribe from the BOAI Forum, use the form on this page:
http://www.soros.org/openaccess/forum.shtml?f


[BOAI] [Forum Home] [index] [options] [help]

 E-mail:  openaccess@soros.org .