OSINF: the lifeblood of decision-making

Information is collected, collated and organised data. In combination with analysis, it is the prerequisite for creating 'knowledge', a term often confused with 'intelligence'. Intelligence is a specialised form of knowledge that is simply a more appropriate and accepted term in government and public sector circles (by dint of tradition and history more than anything else), whereas knowledge is an equivalent, more modern term increasingly in use in the private and non-public sectors. Whichever variant of knowledge/intelligence the reader is more comfortable with (for the purposes of this article I will use the term 'knowledge') they both have two distinguishing features. First, people (analysts) create knowledge. Second, knowledge is the prerequisite for decision-making and then action. Knowledge that only informs the analyst who creates it is a waste of time and money. Thus communication and dissemination is implicit in decision-making and action.

'Open source' information (OSINF) is that which is in the public domain and is legally obtainable. Being in the public domain, however, should not be confused with being available to the public. There are barriers to entry - notably money and effort. In the absence of a knowledge- or ability-based economy, the exchange of information - for money and/or endeavour - remains a potent validation of the worth of that information.

OSINF and espionage

OSINF can also be discreetly acquired. Information that is neither in the public domain nor legally obtainable is gathered by espionage, which is the clandestine acquisition of information illegally without the knowledge of the target, and is characterised by the variety of specialties bearing the suffix '-INT'. It is conceivable that OSINF might be clandestinely or illegally acquired (for example, stealing an academic journal from a library), but this seems rather pointless when discretion or anonymity will suffice. Information obtained clandestinely or openly, whose disclosure creates vulnerabilities for sources, methods or intentions, may become 'closed' by classification or commercial sensitivity procedures. OSINF need not remain open once obtained, but it seems unlikely, given its origin, that it will become classified due to its source or because of the methods by which it was obtained. If information is classified without justification, it can no longer be integrated into decision-making process. Regrettably, the 'need to know' has become a debate complicated more by issues of organisational culture and personal vested interests than by operational security. It was ever thus. Time, the mounting dilemmas of a postmodern risk society and the recognition of the value of OSINF will change this. Mercifully, however, that is another debate not for these pages.

It has been estimated by many senior representatives of the intelligence community that approximately 80% of knowledge, upon which decisions are made and action is taken, is derived from OSINF1. My own research indicates that this anecdotal figure may now be nearer 90% for all-source collecting agencies. But is this 90% of a final intelligence report or 90% of 'action outcomes' such as an arrest or a threat interdiction? Whatever the percentage represents, it is presently a subjective estimate. Given the combination of extraordinary technological and geopolitical changes witnessed over the last 15 years, that have propelled the Western world into postmodernism and have made open source possible, OSINF's perceived efficacy by practitioners - and, more importantly, customers - is likely only to increase. What seems unquestionable is that open source provides the matrix in which all the other clandestine -INTs can set their nuggets of closed information, as well as the foundation upon which they can be used more effectively and efficiently.

Reasons for using OSINF

Why is OSINF so good when held against its traditional alternative, information obtained through espionage? The more perceptive organisations that require knowledge to function are beginning to appreciate that the two are not in competition but are in fact mutually supportive. OSINF:

· Is fast, flexible, dynamic and cheap;

· Is communicable, sharable, trust-creating and partner-forming;

· Identifies and mitigates risk at the strategic, operational, tactical and technical levels;

· Contextualises the requirement both historically and currently (for example, by taking into account the political, economic, social, technical, legal and economic factors);

· Contributes to the all-source collection process and makes available other -INT resources for more concentrated espionage;

· Provides 'cover' and risk communication possibilities for the other -INTs; and

· Provides a 'horizon scan' or 'heads-up', and thus focuses other -INT resources.

With the exception of the last statement (which applies to all methods of intelligence collection), it is irresistible to compare OSINF with clandestine methods in each of the categories above. Enlightened organisations have undertaken this comparison and been persuaded by the relative benefits.2

Range of sources

What open sources are available? Newspaper cuttings and now the Internet are no longer the stereotypical sources of OSINF. Indeed the Internet is not of itself a source but merely the means by which sources are accessed, stored and supported. However, the power and potential of the Internet to facilitate open source access increases daily and it seems inevitable, with increased digitisation, that it will become the first port of call for open source collection. Furthermore, the Internet is not merely the facilitator of access to information and databases. It is also a communications backbone in support of open 'systems': for example, telephonic communication (IP telephony); systems monitoring (closed-circuit television security or simple doorway usage monitors); temperature monitors in mobile and static refrigeration units; and entity tracking (such as in container shipments). Open sources can broadly be split into the following categories (examples are given in brackets):

· Traditional media broadcast:

(BBC Monitoring Service, Foreign Broadcast Monitoring Service (FBIS));

· Commercial online premium:

(Factiva, Lexis-Nexis, Dialog for global media coverage; Jane's, Oxford Analytica, Economist Intelligence Unit for specialist technical/tactical coverage);

·'Grey' literature - information obtained through libraries and other specialised channels including academic and private information brokers:

(Ebsco Host, Swetswise for peer-reviewed articles);

· Overt human experts and observers - the most valuable means of ascertaining the 'ground truth'

(Social Science Citation Index, International Directory of Associations [Gale Research or through Factiva], International Alert, Amnesty International and others);

· Commercial imagery3 - 11 private (commercial) high-resolution (1m) remote sensing satellites are available to credit card holders, including:

(Terraserver, Spot, The Journalists' Guide to Remote Sensing Resources on the Internet); and

· Mapping:

(Maps.com, Mapquest, World Ports).

Language is a significant and critical issue implicit to each of the categories above; one that remains to be addressed by open source as well as the other -INTs. The fact seems to be ignored that 29 languages are considered minimum entry for a complete intelligence picture.4 We ignore this challenge at our peril.

Online open sources

It is worth understanding a little more about the workings of the Internet. The Internet is divided into the surface web (the 'superficial' web) and deep web (the 'invisible' web). The deep web comprises those sites that are on the Internet but are inaccessible to conventional search engines (Google, for example). There are several websites dedicated to searching the deep web. The most comprehensive of these is Direct Search, maintained by one Gary Price, a reference librarian at George Washington University. The deep web contains approximately 7,500 terabytes of information (550 billion documents), compared to 19 terabytes (1 billion documents) on the surface web (a terabyte is 1,000 gigabytes). If one realises that the Internet is growing at approximately a million documents a day and that one's sole recourse to gathering information is Google, combined with the estimate that the Internet facilitates access currently to only 10-20% of all OSINF, the implications of the requirements for sophisticated, trained and properly resourced open source analysis are stark.

Checking OSINF's reliability

Knowledge, regardless of the origin of its precursor information (open or clandestine), must be timely, accurate, relevant and verifiable. It must answer a question and it must engender a proactive actionable decision even if that decision is not to act. One of the criticisms of OSINF is that it is not easily verifiable or evaluated. This is particularly true of information derived freely from the Internet. It is a less valid criticism of information derived from premium content sites, academic peer-reviewed grey literature or ground truth experience. Like all sources of information, trust, the passage of time and analyst expertise become the defining arbiters of value. The perception that the degree of covert collection, as represented by degree of classification, is the defining value is at best misguided. Closed information is a mark of the source's sensitivity, the method or methods by which it was obtained or the intention for which it is being used - not the value it affords the creation of knowledge, decision-making and action. The open source convention (for example in evaluating a website) is to consider and review the following checklist for each and every open source:5

· Authority - does the website clearly identify itself and is it 'route-traceable'? [use www.samspade.org]. Is the website cited by others? Does it demonstrate influence? Has it been attacked electronically? Check its hit-rate.

· Accuracy - has the material on the website been corroborated and bench-marked against other validated all-source material?

· Objectivity - does it advocate or balance views? To whom does it link? Who or what does it represent?

· Relevance - check site and page updates.

· Coverage - is it relevant: does it add understanding or is it just 'interesting'?

Information is the front-end ingredient for the process of analysis by which knowledge is created. OSINF is a growing contributor to decision-making, particularly in security sector operations that recognise the critical importance of timely, accurate and verifiable knowledge as the starting point of the risk management process. OSINF is not a replacement for or competitor to closed information derived clandestinely but a complement, with significant benefits in its own right and advantages and disadvantages by comparison with the other -INTs. It is becoming accepted practice in both the public and private sector, where it is described as 'knowledge management', 'competitive intelligence' or 'open source intelligence'. It is becoming more sophisticated, with specifically developed techniques, tools, evaluation procedures and expert training. It would seem sensible to conclude that if OSINF contributes such a significant and growing input to decision-making, the security world should sit up and take note. The issue for OSINF is no longer its validity or usefulness but how it can be developed, institutionalised and rolled out as a discipline common to government intelligence analysts and commercial knowledge workers alike.

Steve Gibson is a PhD research student at Cranfield University and the UK Defence Academy. He is researching the role of open source information in support of security decision-making

NOTES

1. There are numerous credible sources for this estimate, for example: Lieutenant General Sam Wilson (former Director, Defense Intelligence Agency), Washington Times 17 November 1997; Ward Elcock (former Director, Canadian Security Intelligence Service), cited in Government Information Quarterly Vol 13 No2, p161; Professor Arthur S Hulnick in International Journal of Intelligence and Counter-Intelligence Vol 15, No4, p565.

2. NATO; EUROPOL [the European Police Force]; UK Ministry of Defence (MoD); Swedish MoD; Dutch MoD; US Defense Intelligence Agency; CIA; UK HM Customs & Excise - to list a few in the public sector.

3. For further information, see YA Dehqanzada & AM Florini, Secrets for Sale: How Commercial Satellite Imagery will Change the World (2000).

4. RD Steele, Peacekeeping Intelligence: Emerging Concepts for the Future (2001).

5. NATO Open Source Intelligence Handbook, November 2001.