Evaluation of the Academy of Finland

150 Pages·2013·4.19 MB·English

Evaluation of the Academy of Finland
Reports of the Ministry of Education and Culture, Finland 2013:14

ISBN 978-952-263-224-1 (pbk)
ISBN 978-952-263-225-8 (PDF)
ISSN-L 1799-0343
ISSN 1799-0343 (Print)
ISSN 1799-0351 (Online) Helsinki 2013

Evaluation of the
Academy of Finland
Reports of the Ministry of Education and Culture, Finland 2013:14
Erik Arnold
Terttu Luukkonen
Patries Boekholt
Anke Nooijen
Zsuzsa Jávorka
Frank Zuijdam
With a Panel Review of the Academy conducted by
Susan Cozzens
Jos van der Meer
Jens Nielsen
John O’Reilly
Ministry of Education and Culture • Department for Higher Education and Science Policy • 2013
Opetus- ja kulttuuriministeriö • Korkeakoulu- ja tiedepolitiikan osasto • 2013
Ministry of Education and Culture
Department for Higher Education and Science Policy, Science Policy Division
P.O. Box 29
00023 Government, Finland
http://www.minedu.fi/OPM/Julkaisut/julkaisulistaus?lang=en
Layout: Jussi Ekberg, Kopijyvä Oy
Cover image: Rodeo
Kopijyvä Oy, 2013
ISBN 978-952-263-224-1 (pbk)
ISBN 978-952-263-225-8 (PDF)
ISSN-L 1799-0343
ISSN 1799-0343 (Print)
ISSN 1799-0351 (Online)
Reports of the Ministry of Education and Culture, Finland 2013:14
Table of Contents
Summary 10
1 Introduction 12
1.1 Background and objectives of the study 12
1.2 Objectives and evaluation questions 14
1.3 Approach and methodology 15
1.4 Results of the previous evaluation 16
2 The Academy of Finland 19
2.1 History of the Academy of Finland 19
2.2 Mission and goals 21
2.3 Key functions of a research council 21
2.4 Governance 22
2.4.1 Respondents’ views on the organisational structure 25
2.5 Strategy of the Academy 26
2.5.1 The strategy of 2006 26
2.5.2 The current strategy 27
2.6 Budget 28
2.6.1 Additional funding for research programmes 29
2.6.2 Budget of the Research Councils 29
2.6.3 Views of respondents on budget allocation 30
2.6.4 Administration costs 31
2.7 International comparison 32
2.8 Conclusions 33
3 The role of the Academy in the Finnish research and innovation system 35
3.1 Policy and governance context 35
3.2 Recent changes in the Finnish research and innovation system 36
3.3 The Academy’s role in the Finnish R&I system 39
3.3.1 Complementarity of funding and functions 39
3.3.2 Cooperation with other stakeholders (e.g. Tekes) 40
3.3.3 Relation with universities and research institutes 40
3.3.4 Relationship with the Ministry of Education and Culture 41
3.3.5 The Academy as a policy advisor 43
3.3.6 Science communications 44
3.4 Stakeholder comments on the role of the Academy in a systems perspective 45
3.5 Conclusions 46
4 The Academy as a funding organisation 49
4.1 Funding instruments 49
4.1.1 Views of stakeholders on the portfolio of instruments 49
4.2 Designing research programmes 50
4.2.1 Views of the stakeholders on research programmes 51
4.3 Description of funding processes 51
4.3.1 Announcement and submission of calls 51
4.3.2 Views of respondents on motivations for funding applications 53
4.4 Review process 55
4.4.1 Stakeholders’ views on the funding process 57
4.5 Peer review 58
4.5.1 Stakeholders’ views on the peer review system 60
4.5.2 Nationality of the peers 61
4.6 Appraisal time 61
4.7 Confidentiality and conflict of interest 64
4.8 Reporting 64
4.9 Full cost model 65
4.9.1 Stakeholders’ view on the full cost model 67
4.10 Size of the grants 68
4.11 Success rates 68
4.11.1 Analysis at the level of funding schemes 69
4.11.2 Analysis at the level of the universities 69
4.11.3 Gender balance 70
4.11.4 Analysis at the level of the research councils 73
4.12 Share of funding per research field 73
4.13 Age of applicants 73
4.14 Outcomes and impacts 73
4.15 Does the Academy select the best researchers to fund? 79
4.16 Evaluation at the Academy of Finland 79
4.17 International comparison 81
4.18 Conclusions 84
5 Internationalisation 86
5.1 Introduction 86
5.2 International strategy 87
5.3 The position of internationalisation in the Academy 88
5.4 Portfolio of activities and instruments 89
5.4.1 Opening up Academy Programmes to foreign nationals 89
5.4.2 Bilateral agreements 89
5.4.3 FiDiPro 90
5.4.4 ERA-Nets and Joint Programming 92
5.4.5 Nordic cooperation 92
5.5 The views from the stakeholders 92
5.6 International comparison 94
5.7 Achievement of objectives for internationalisation 95
5.8 Conclusions on internationalisation in the Academy 97
6 Conclusions and recommendations 101
6.1 The context 101
6.2 The Academy 102
6.3 Recommendations 104
APPENDIX A – Evaluation of the Academy of Finland:
Report of the Peer Review Panel 106
APPENDIX B – The Academy’s portfolio 114
B.1 Personal grants 114
B.2 Non thematic projects and programmes 115
B.3 Thematic programmes 116
B.4 Grants for infrastructure 120
B.5 International programmes 121
B.6 Monodisciplinary programmes 122
APPENDIX C – Data input-output analysis 123
C.1 Appraisal time 123
C.2 Funding schemes without competition 123
C.3 Comparison between research organisations 125
C.4 Comparison satisfaction funding process 125
APPENDIX D – Online Survey 126
D.1 Characteristics of the survey respondents 126
D.2 Survey results 128
APPENDIX E – List of Interviewees 141
APPENDIX F – Approach and Methodology 144
Table of Figures
Figure 1 Evaluation approach ................................................15 Figure 15 Steps in processing applications ..........................52
Figure 2 The Academy in the governance of state Figure 16 Reasons to apply for funding ..............................53
institutions in the National Research and Innovation
Figure 17 Ranking of the importance of
System ...........................................................................................20
various motivations for application to the
Figure 3 Organisation chart of the Academy of Finland .24 Academy’s funding schemes by the different
groups of stakeholders ..............................................................54
Figure 4 Staff Refreshment at AKA .....................................25
Figure 18 Proposal assessment scales
Figure 5 Current (2010) Strategy of the Academy .............27
– peer review ................................................................................56
Figure 6 Survey respondents’ views on the
Figure 19 Based on your most recent experience
strategy of the Academy ............................................................28
of applying for Academy funding, to what extent were the
Figure 7 Development of Academy of Finland budget following characteristics of the funding process satisfactory
and Finnish GBAORD 2004–2011..........................................29 – Non-successful applicant survey ...........................................59
Figure 8 Research funding decisions made Figure 20 Based on your most recent experience of
from 2004-2011 (x €1000) .........................................................30 applying for Academy funding, to what extent were the
following characteristics of the funding process satisfactory
Figure 9 Shares of the research-funding
– Grant holder survey ................................................................59
budget 2004-2011 per council domain ....................................31
Figure 21 How responsive do you find the Academy
Figure 10 Overhead of AKA 2004-2011 .............................31
to inputs from the research community about: Needs for
Figure 11 Administrative Costs of Research change in research funding rules or schemes? .......................21
Funders .........................................................................................32
Figure 22 How do you see the role of the
Figure 12 How well does the Academy communicate Academy in the future? Should there be a shift
about its work and the impacts in its activities (basic research versus focus on strategic
of its funding? .............................................................................44 research)? ......................................................................................62
Figure 13 To what extent do you agree that Academy Figure 23 Ratio among number of reviews by
funding schemes support the creation nationality of peers (2007-2011) ..............................................63
of a balanced portfolio across the various
Figure 24 Number of reviewers per country
research domains .........................................................................50
(total 2007-2011) .........................................................................63
Figure 14 How responsive do you find the
Figure 25 Change of share in number of reviewers per
Academy to inputs from the research community about
country between 2007/2008 and 2011/2012 (%-points) .....63
changes in thematic priorities, for example
by starting new research programmes?? ..................................52 Figure 26 Appraisal time (in months) of applications
received between 2007-2011 .....................................................65
Figure 27 How far do you feel that the amount of time Figure 49 Applications and grants from 10 most
spent on the application procedure taking frequent nationalities of foreign grant holders (2007-2011) 91
into account the success rate of the funding schemes is: ....65
Figure 50 Participation of the survey respondents in
Figure 28 Full cost model: type of costs .............................67 international research projects ..................................................95
Figure 29 Number of applications and success rate Figure 51 Types of international projects in which
(2007-2011) ..................................................................................67 survey participants are / have been recently engaged ..........96
Figure 30 Distribution size of the funding Figure 52 In how many different types of international
(2010-2011) (x €1000) ................................................................68 project are / have survey respondents been engaged? .........96
Figure 31 Gross and net success rate of Figure 53 Role of participants in the international
competitive funding schemes (2007-2011) .............................71 projects, normalised against the number of respondents ....96
Figure 32 Applications and success rate by institution Figure 54 For Finland, and your unit in particular, what
(2007-2011) ..................................................................................72 are the challenges in the internationalisation of research? ..98
Figure 33 Shares of institutions in funding granted Figure 55 To what extent do you agree that Academy
(2007-2011) ..................................................................................72 funds facilitate the international networking needed for
your research institution? ...........................................................98
Figure 34 Number of (granted) applications by gender
(2007-2011) ..................................................................................74 Figure 56 Survey respondents’ views on AKA support
for the internationalisation of research ...................................99
Figure 35 Success rates in applications
(2007-2011) ..................................................................................74 Figure 57 Timetable and Interviewees of the Peer
Review Panel ..............................................................................113
Figure 36 Proportion of female applicants (2006-2010) ..74
Figure 58 Average appraisal time
Figure 37 Success rates for the different Research
2007-2011 (in months) .............................................................123
Councils between 2007-2012 ....................................................76
Figure 59 Ranks of research organisations .......................124
Figure 38 Research funding by field of science
(2007-2011) ..................................................................................76 Figure 60 Percentage of respondents who agreed that
the following characteristics of the funding process were
Figure 39 Number of applicants by age (2007-2011) .......77
completely satisfactory .............................................................125
Figure 40 Number of grantees per age of grantee
Figure 61 Basic information on the respondents .............127
(2007-2012) ..................................................................................77
Figure 62 Organisational affiliation of the survey
Figure 41 To what extent do you agree with the
respondents ................................................................................127
following statements regarding the selection of the
Academy grants? .........................................................................78 Figure 63 Positions held by the survey respondents in
different country groups: Finland, other EU27 and
Figure 42 How far do you feel that Academy funding
extra EU27 .................................................................................128
schemes contribute to the followings? Reponses presented
by age group distribution of respondents ..............................78 Figure 64 Profile of the survey respondents ....................128
Figure 43 Grant holders’ views on the contribution of Figure 65 Are you a member of a specific research
Academy funded projects to different results ........................80 centre or group? ........................................................................129
Figure 44 Grant holders’ views on the contribution of Figure 66 Q1. Please provide us with the following
the Academy’s funded projects to different achievements basic information: .....................................................................129
by funding instrument ................................................................81
Figure 67 Q2. What is your age range? ..............................129
Figure 45 Percentage of respondents who agreed
Figure 68 Q3. Could you please indicate your gender? ...129
completely or to a large extent with the following
statements regarding the types of impact of the AKA Figure 69 Q4. What is your current (main) position? .....129
funding schemes ..........................................................................82
Figure 70 Q5. What type of organisation do you
Figure 46 Grant holders’ views on continuation of work at?.......................................................................................130
their project in the future ..........................................................82
Figure 71 Q6. What is your area of research? ..................130
Figure 47 Evaluations reviewed ............................................84
Figure 72 Q7. Are you a member of a specific
Figure 48 Origins of foreign applicants for Academy research centre or group: .........................................................130
Funding 2007-2011 .....................................................................91
Figure 73 Q8. Are you or have you in the past: ...............130
Figure 74 Q9. Please would you estimate the Figure 92 Q26. How responsive do you find the
breakdown of your external research funding (that is Academy to inputs from the research community about: ..139
not institutional funding) in 2012 that comes from: ...........131
Figure 93 Q27. How well does the Academy communicate
Figure 75 Q10. How attractive do you regard the about its work and the impacts of its funding? ...................139
following Academy of Finland, Tekes and European
Figure 94 Q28. How do you see the role of the
funding schemes as a funding source for your research
Academy in the future? Should there be a shift in its
activities? .....................................................................................131
activities (basic research versus focus on strategic
Figure 76 Q11. Please indicate, how the schemes of research)? ....................................................................................140
the Academy compare with other national or
Figure 95 Q29. Suggestions, recommendations and
international funding sources, concerning: ...........................132
other comments ........................................................................140
Figure 77 Q12. Based on your most recent experience
Figure 96 List of interviewees .............................................140
of applying for Academy funding, to what extent were
the following characteristics of the funding process Figure 97 Evaluation approach............................................145
satisfactory? (Results of the Applicant Survey) ...................132
Figure 78 Q12. Based on your most recent experience
of applying for Academy funding, to what extent were
the following characteristics of the funding process
satisfactory? (Results of the Grant Holder Survey) ............133
Figure 79 Q13. How far do you feel that the amount
of time spent on the application procedure taking into
account the success rate of the schemes is: .........................133
Figure 80 Q14. To what extent do you agree with the
following statements? ...............................................................134
Figure 81 Q15. How far do you feel that Academy
funding schemes contribute to: ..............................................134
Figure 82 Q16. What is the impact of the Academy’s
funding schemes? Do they contribute to: .............................135
Figure 83 Q17. Please indicate the kind of grant you
answer for and the first year of funding: (only GHS) ........135
Figure 84 Q18. To what extent were the following
motives important to you when you applied for this
particular project?......................................................................136
Figure 85 Q19. To what extent were the following
motives important to you when you applied for this
particular project? (only GHS) ................................................137
Figure 86 Q20. Beyond the grant period (only GHS) .....137
Figure 87 Q21. Have you engaged in any international
research project collaboration since 2004? ...........................137
Figure 88 Q22. Could you please describe the type of
international project(s) in which you are / were engaged?
Please select all that apply. .......................................................138
Figure 89 Q23. Could you please describe your role in
the international project(s) in which you are / were
engaged? Please select all that apply ......................................138
Figure 90 Q24. To what extent do you agree with the
following statements about the Academy’s support for
the internationalisation of research? ......................................138
Figure 91 Q25. For Finland, and your unit in particular,
what are the challenges in the internationalisation of
research? .....................................................................................139