Secrecy and Open Government: Why Governments Want You to Know

202 Pages·1999·1.748 MB·English

by  K. G. Robertson (auth.)

6(&5(&<�$1'�23(1�*29(510(17
$OVR�E\�.��*��5REHUWVRQ
%5,7,6+�$1'�$0(5,&$1�$3352$&+(6�72�,17(//,*(1&(
38%/,&�6(&5(76��$�6WXG\�LQ�WKH�'HYHORSPHQW�RI�*RYHUQPHQW�
6HFUHF\
6HFUHF\�DQG�2SHQ�
*RYHUQPHQW
:K\�*RYHUQPHQWV�:DQW�<RX�WR�.QRZ
.��*��5REHUWVRQ
,QWHOOLJHQFH�0DQDJHPHQW�&RQVXOWDQW
‹�.��*��5REHUWVRQ�����
Softcover reprint of the hardcover 1st edition 1999 978-0-333-75367-5
$OO�ULJKWV�UHVHUYHG��1R�UHSURGXFWLRQ��FRS\�RU�WUDQVPLVVLRQ�RI
WKLV�SXEOLFDWLRQ�PD\�EH�PDGH�ZLWKRXW�ZULWWHQ�SHUPLVVLRQ�
1R�SDUDJUDSK�RI�WKLV�SXEOLFDWLRQ�PD\�EH�UHSURGXFHG��FRSLHG�RU
WUDQVPLWWHG�VDYH�ZLWK�ZULWWHQ�SHUPLVVLRQ�RU�LQ�DFFRUGDQFH�ZLWK
WKH�SURYLVLRQV�RI�WKH�&RS\ULJKW��'HVLJQV�DQG�3DWHQWV�$FW������
RU�XQGHU�WKH�WHUPV�RI�DQ\�OLFHQFH�SHUPLWWLQJ�OLPLWHG�FRS\LQJ
LVVXHG�E\�WKH�&RS\ULJKW�/LFHQVLQJ�$JHQF\�����7RWWHQKDP�&RXUW
5RDG��/RQGRQ�:�3��+(�
$Q\�SHUVRQ�ZKR�GRHV�DQ\�XQDXWKRULVHG�DFW�LQ�UHODWLRQ�WR�WKLV
SXEOLFDWLRQ�PD\�EH�OLDEOH�WR�FULPLQDO�SURVHFXWLRQ�DQG�FLYLO
FODLPV�IRU�GDPDJHV�
7KH�DXWKRU�KDV�DVVHUWHG�KLV�ULJKW�WR�EH�LGHQWLILHG
DV�WKH�DXWKRU�RI�WKLV�ZRUN�LQ�DFFRUGDQFH�ZLWK�WKH�
&RS\ULJKW��'HVLJQV�DQG�3DWHQWV�$FW������
)LUVW�SXEOLVKHG������E\
0$&0,//$1�35(66�/7'
+RXQGPLOOV��%DVLQJVWRNH��+DPSVKLUH�5*����;6
DQG�/RQGRQ
&RPSDQLHV�DQG�UHSUHVHQWDWLYHV�
WKURXJKRXW�WKH�ZRUOG
ISBN 978-1-349-41300-3 ISBN 978-0-230-51302-0 (eBook)

DOI 10.1057/9780230513020
$�FDWDORJXH�UHFRUG�IRU�WKLV�ERRN�LV�DYDLODEOH
IURP�WKH�%ULWLVK�/LEUDU\�
7KLV�ERRN�LV�SULQWHG�RQ�SDSHU�VXLWDEOH�IRU�UHF\FOLQJ�DQG
PDGH�IURP�IXOO\�PDQDJHG�DQG�VXVWDLQHG�IRUHVW�VRXUFHV�
��� ��� �� ��� �� ��� �� � ���� ��� �� ������ ��� �� ��� �� � ���
��� � ��������� ����� � ��� � ��� � ��� � ��� � ��� � ���
This page intentionally left blank
To my wife Ellen, with all my love
Contents

1 Introduction 1

2 The Morality of Secrecy: the Philosophy and Politics

of State Secrecy 9

3 Surveillance and Secrecy 25

4 The End of Whitehall? Reform of the British Civil Service 41

5 Accountable Government? 57

6 Data Protection, Privacy and Limited Rights of Access 67

7 Listening In 79

8 Official Secrecy 85

9 Defending the Realm: MI5 97

10 The Democratisation of UK Intelligence? 111

11 Overseas Experience: FOI in Canada, Australia and

New Zealand 125

12 Freedom of Information in the UK 141

13 Conclusion 157

Notes 161

Index 191

vii
1 Introduction

This book is about democracy, power and secrets. It is also the story of
how the UK came to abandon its secretive ways and adopt freedom of
information. However, it is not a romantic tale but one of self-interest
and struggle; it is the story of how power and the control of informa-
tion are intertwined. Although it is full of ideals and references to
democracy, values, liberties and rights, it does not accept that such
ideals are the `cause' of change and the introduction of `open govern-
ment'. Rather I argue that the nature of governance, the methods by
which governments govern, has altered and changed the balance of
power between different groups within the British state. Such changes
in interests affected the incentives to control and release information.
I argue that the recent decision to introduce freedom of information is
not merely a reflection of the views of a particular political party or
ideology but reflects a changed political environment with new incen-
tives to manage information. Such changes range from new manage-
ment structures in the National Health Service to a new crime-fighting
role for the Security Service (MI5).
This book challenges the conventional wisdom that secrecy is best
understood as a matter of values, of right and wrong. The relationship
between values and interests is a complex one but all too often writers
on secrecy refer only to values and never to power and interests. Such
an approach is convenient for those who wish to believe in progress or
`inevitability' but it is of limited relevance to understanding the real
causes of change.
In my book Public Secrets I argued that:
It is the extent to which information is seen as part of the political
struggle for office which determines the level of control of informa-
tion. There are then two variables; the motive to exercise control of
information and the ability to implement the degree of control
desired.1
It is the structure of power, and the interests which it generates, which
explains the incentives to compel others to keep secrets or to allow
access. The more centralised and clearly hierarchical the structure of
power, the more will be the incentive to compel others to keep secrets.
The more decentralised and `flat' the structure of power, the more will
be the incentive to compel others to allow access. The balance of
1
K. G. Robertson, Secrecy and Open Government
© K. G. Robertson 1999
2 Secrecy and Open Government
interest is the extent to which scrutiny by others ± that is, some form
of public access ± enhances or diminishes politicians' sense of author-
ity, of being in charge. The argument can be clarified by being
expressed negatively: commitment to the value of openness or the
legitimacy of secrecy has very little part to play in explaining variations
between democracies; why one has more or less secrecy than another.
This is not to say that values are irrelevant to democracy or to secrecy,
as Tant believes I claim,2 but that secrecy or openness can both be
justified on the grounds that they contribute to `good government',
which is the principal justification of democracy.3 They cannot both
do so at the same time or in the same set of circumstances but they
can contribute to democracy, depending upon how democracy is
organised. In some forms of democracy it can at least be argued,
without this being a contradiction in terms, that secrecy contributes
to democracy, while it is equally appropriate to argue that in other
forms of democracy openness greatly contributes to democracy. The
value of secrecy and openness are secondary in that both are instru-
mental and must be judged in the light of their contribution to
democracy. As I argued in Public Secrets, the creation of the official
secret was legitimately seen as one solution to the problem of demo-
cratic government.4 It is not necessary to my argument to assert that
the British form of democracy is more democratic than others.
Indeed, it is perfectly possible to argue that, even in the nineteenth
century, it took a form that was a less perfect expression of democracy
than in other nations. The point is this: given its form of democracy,
secrecy was both comprehensible and legitimate. What legitimated the
secrecy was not some notion that secrecy was a `good thing' but that it
contributed to the process by which elected representatives, ministers,
and the institution of Parliament controlled the process of decision-
making. They were the guardians of the popular will and if secrecy
helped them to carry out the popular will then democracy actually
legitimated the official secret. In some other nations, where demo-
cracy took a different expression, this was clearly not the case. If the
way in which the popular will was expressed, the way in which demo-
cracy was organised was different, then in those nations secrecy would
not contribute to democracy and would therefore be neither compre-
hensible nor legitimate.
The key to understanding the evolution of official secrecy in Britain
was, and it will be argued remains, the way in which pressures to
control or release information are mediated through a structure of
power. This structure of power crucially involves relationships of