The 434th Military Intelligence Detachment (Strategic) 200 Wintergreen Avenue New Haven, CT 06515
As Americans, we live in an information age. As soldiers, our success on the battlefield will depend on our ability to manage information in war. As military intelligence officers, our ability to access, process, analyze and disseminate intelligence information quickly and effectively can determine the outcome of future wars. The purpose of this handbook is to enhance the capabilities of MI Officers by incorporating Open Source Information (OSI) into their thinking, training and war-fighting agenda.
While the acronym OSI is used throughout this handbook to represent open source information, it is important to distinguish OSI from the other INT's (HUMINT, SIGINT, IMINT and MASINT) that are collection activities that generally produce classified products. Open source information is acquired from a wide range of sources in the information marketplace.
OSI is information drawn from the vast collections of information in the open, unclassified literature. Researchers in industry and the academic world, journalists, and governments produce a continuous flood of this information. It deals with every part of the world and every aspect of human existence. It is collected, published or broadcast in unending torrents.
Dramatic increases in the power and availability of computers support the tools required to store and retrieve useful information from the vast open source literature. This manual presents the basic concepts of identifying, accessing and searching computerized databases. The effective use of OSI may require change both in individual and organizational perspectives and skills. These changes and the reasons for them are discussed in the handbook.
Finally, a section describes the costs and benefits of OSI to the MI officer.
This handbook stresses the importance of OSI to the MI officer faced with shifting and unanticipated threats, shortened response times, the need to share intelligence with coalition partners, and diminishing budgets. This manual would not have been possible without the assistance of Mr. Robert D. Steele, President of OPEN SOURCE SOLUTIONS, Inc., a non-profit educational corporation based in Oakton, Virginia. He has provided advice, criticism and a rich store of material on the theory and practice of OSI.
Mr. Steele, a major in the Reserve of the United States Marine Corps, served as a GM-14 and the founding Special Assistant and Deputy Director of the Marine Corps Intelligence Center. It was there that he discovered, after a lifetime spent collecting secrets, that he could meet 80% of his consumers' requirements, at 20% of the cost of classified information, using only open sources of intelligence.
The handbook was prepared by the 434th Military Intelligence Detachment(Strategic), an Army Reserve Unit, as part of its 1993-94 training project in support of the Department of Tactics, Intelligence and Military Science at the U.S. Army Intelligence Center and Fort Huachuca, Arizona. Members of the MID study team were: LTC Robert R. Simmons, Commander; CW4 Alan D. Tompkins, Project Officer; LTC Martin J. Foncello, Jr., Military Intelligence Officer and SGT Eliot A. Jardines, Senior Analyst.
Questions or comments may be directed to: Commander 434th Military Intelligence Detachment (Strategic) 200 Wintergreen Avenue New Haven, CT 06515.
The Project Officer may be contacted by electronic mail at email@example.com or by U.S. mail at: P.O. Box 928 Williston, VT 05495.
Robert R. Simmons COL, MI Commanding
Pity the hapless intelligence analyst! After four decades of studying almost exclusively, anything with a Red Star painted on its side, the intelligence community is now tasked with a mission impossible: to boldly go into the unknown to monitor and track potential threats to Western interests in any one of 80 countries.
This handbook is designed to acquaint intelligence professionals with the wide range of sources and tools currently available to collect and utilize intelligence information from open sources. These sources and tools offer the intelligence professional unparalleled opportunities to provide commanders and policy makers the timely, accurate and balanced information they require. As such, they solve the dilemma of the "hapless intelligence analyst" who can no longer rely simply on classified sources to answer the myriad questions presented to him.
This handbook describes the characteristics of these information resources and presents the technology employed to access, search and store electronic information.
In order to make full and effective use of the information resources described in this handbook, you may have to re-think some of your ideas about the duties and responsibilities of an intelligence professional. The organization in which you work may also have to change. Both personal and organizational change can be difficult and even painful.
However, the U.S. role in a rapidly-changing world compels intelligence professionals at all levels to examine new opportunities and to exploit those which enhance the responsiveness and quality of the products which they produce.
While the dramatic explosion of information technology will change the profession and its organizations, the need for reasoned, balanced and professional human judgment will not change. In fact, the skills of the intelligence analyst are even more important in the current information-rich environment.
This handbook discusses the wide range of open information resources and the techniques used to explore and use them. It also discusses the benefits of utilizing them, both in the quality and timeliness of the intelligence products produced and in cost reduction.
Welcome to the Information Age.
2. Information - a professional necessity Information - grist for the mill and the final product
Intelligence is the collection, analysis and dissemination of information For the intelligence officer, this process is driven by the commander's needs. This focus was present in Caesar's legions, in the wars that preceded his time and in all of the time since he crossed the Rubicon into Gaul.
The technology has changed, the urgency has increased and the volume of information has exploded. However, the single, unchanging goal of all of these efforts has been, is, and will continue to be to provide the commander or policy maker with the focused and balanced intelligence upon which sound decisions are made.
The intelligence professional provides the knowledge, judgment and skills in these activities. Just as any professional must understand the available tools, the intelligence officer must understand the tools available for collecting, analyzing and disseminating information.
Some jobs are characterized by very clear task definition and measurement. If you are told that you must stack up a pile of boxes, each box added to the pile is a clear indication of your progress toward completing the job.
The job of an intelligence professional, however, is different. It has some of the characteristics of that recurring nightmare many of us have; you know, that bad dream about going into a large hall to take an exam on a subject about which you know nothing.
You can never know all there is to know about the capabilities or intentions of an opposing commander or another country or group. You can never anticipate all of the questions you will be asked in the future.
The only viable professional response to this situation is to think continually about the areas in which you work. Seek out sources of information, and try improved methods for analysis and dissemination. In short, carry out your professional responsibilities with the same kind of entrepreneurial drive that characterizes some private enterprises. The stakes are certainly higher than those associated with an end-of-year balance sheet.
The only constant is change
There is always resistance to change. Of course, we all try to bring initiative and a creative spirit to our work, but organizations and individuals do not always welcome or reward change. Make no mistake about it, making extensive use of the vast array of open information resources is different from "reading the classified traffic".
We have all sorts of sayings to describe what happens to those who attempt to try something new or different. Sayings like The early Christians get the best lions or You can tell the pioneers because they are the ones with the arrows sticking out describe common attitudes and, unfortunately, experiences.
The changed world situation, the information explosion and shrinking budgets will not allow the continuation of a business-as-usual approach. One expert described the current situation by saying:
"The Defense Department is confronted with a phenomena which is encountered in private business only rarely. For the past forty years the Defense Department designed its information systems to deal with a single customer. With the demise of our wholesale customer, we suddenly find ourselves in the retail business. It is a retail business which is totally unpredictable, where we don't know where the next customer will come from, and how that customer has to be served."
New Technology and New Organization
Dissemination of intelligence
The maximum payback from the effective use of information technology may require changes in the dissemination and organization of intelligence. One report on intelligence in Desert Shield/Storm stated that:
The inability to reliably disseminate intelligence, particularly imagery, within the theater was one of the major intelligence failures.. Component headquarters staff often failed to pass available intelligence downward..
A Marine officer writes:
Much of DESERT SHIELD was spent figuring out how to get multipage reports out to widely dispersed units. It took extraordinary effort, and many reports were delivered late...
An Army officer comments:
I believe intelligence products are better "tailored" to that echelon when it is "produced" by that echelon. Give me information, I'll do my part to create the intelligence to what I am told the commander wants to know. While I thoroughly support and encourage talk up and down the MI ladder, I feel out of the loop if I can't automate my query process. Without access to national level systems here on my personally owned computer in my humble Bn S-2 shop, my hands are tied, and I must trudge up to the SCIF where often the focus is on classified material. National-level material is great, but (no offense intended) can be too ethereal for folks at this level.
Report on the results of an Army war game:
The key to Airland Battle Future (ALBF) is the efficient delivery of information...Current approaches focus on automating existing manual processes, and linking those processes that have always communicated. ALBF may require the Army to alter this paradigm.
Push instead of Pull
Planners and research organizations are now beginning to talk about designing intelligence systems based on the concept of pull rather than push. That is, instead of producing a wide range of intelligence products and attempting to distribute (push) them to all possible users, the users themselves will query collections of information, retrieving (pulling) only what is relevant to them.
Let's start a restaurant
Some of the deficiencies in intelligence dissemination revealed in Desert Storm can be illustrated by an example. Suppose that we start a restaurant run in an unusual manner. As each customer enters, he or she is seated at a table. The kitchen staff then begins to prepare everything on the menu, and once prepared, the waiters attempt to deliver every dish.
Of course, we soon learn that some customers are never served and go hungry. Other customers have more food than they can eat. They sample only a few of the dishes and often miss the best ones. Much of their food is discarded. The waiters and kitchen staff are constantly rushed and overworked. There is no time for the cooks to think about the dishes they prepare so that they might improve them.
The solution to the problem in our restaurant and perhaps in the distribution system would be to allow the user to select only the products he really wants.
Let the customer choose
The techniques employed to select information sources and retrieve relevant reports and other information are what this manual describes. The shape of future intelligence systems in not clear at present. However, many of the techniques and skills presented in this manual will be relevant as new systems are developed and deployed.
Reorganization of military intelligence is not the topic of this manual. However, it is important to keep in mind some of the lessons learned in recent years by U.S. business. As information infrastructures have changed and information has been widely available throughout the organization, structural changes followed and the skills and attitudes necessary for success changed.
Those are the forces which mandate change. They are far more significant than the usual arguments against change which follow.
Why change anything?
Let's briefly examine some of the arguments against change. You can probably come up with a few more, but these will illustrate the major sources of resistance.
If it ain't broke, don't fix it
One of the professional rewards in any field is the feeling of competence derived from an understanding of the tools, practices and skills in that profession. Change can undermine that feeling. However, complacency can lead to an inability to function effectively in changed circumstances. Change in the field of electronic information processing is most accurately described using terms like explosive, or dramatic. A constant effort to understand the changing environment is required. The Army is different
An intelligence professional in military service has a set of responsibilities and demands different from those of his or her civilian counterpart.
Physical fitness, mastery of the skills of a soldier, and the development of leadership skills all take time and make unique demands on the military professional.
It is also true that the systems and procedures used in the military may have to function despite the physical and human challenges described as the "fog of war".
However, the responsibilities accepted by a military professional demand a constant search for the most effective methods to carry out assigned duties.
Just twenty years ago, only a small group of computer professionals programmed, designed and operated computers. The microcomputer explosion has changed that situation dramatically. However, the professional work force has not been able to change as rapidly.
In a study released in late 1993, more than 100 executives in both the U.S. and the UK. were asked In your opinion, what percentage of the nation's top executives is not computer literate?
The respondents believed that 55% of U.S. executives are not computer literate. How willing will a senior commander who does not understand the technology be to undertake a new initiative based on that technology?
We'll wait for 'them' to provide the tools
There are efforts within the intelligence community to set standards and design and implement open source information systems to support the intelligence process. Some day, these systems will be widely available. But, as a responsible professional, can you ignore the new technologies until that day comes?
There are simple, inexpensive ways in which you can begin to explore and utilize the new information technologies and resources. If you understand them, then you will be able to evaluate and perhaps shape the community-wide tools as they are developed.
Strategies to use OSI now
Ready, fire, aim!
Tom Peters is the co-author of the book In Search of Excellence, and the author of a number of subsequent books. Each deals with the changes in organizations and attitudes which the current competitive environment demands. He uses the expression above to describe an essential approach. He suggests that it is far more useful to try a number of small-scale, quickly deployed projects than it is to carry out a much larger, more comprehensive project only after a lengthy study, review and approval process.
Fortunately, very small investments will allow an exploration and evaluation of new information technologies in the intelligence process. The lessons learned from such small-scale efforts are invaluable in understanding the technology and designing answers to real rather than imagined problems.
Go Simple, Go Cheap and Go Now!
Audrey Sutherland is an author and lecturer who has traveled alone for weeks along the Alaskan coast in a small, inflatable kayak. She did not put off her travels until some future time when she could purchase the perfect, high tech equipment. She summarizes her approach by suggesting that you Go Simple, Go Cheap, and Go Now!
Ignoring new technology is dangerous
By the time of World War I, the technology of the machine gun had been well developed, but European military leaders had not grasped the effect of this weapon on tactics. The frontal assault by infantry on enemy defenses was attempted repeatedly. Consider this account:
"Ferrers was first out from 'B' Company, his monocle in his eye, his sword in his hand. As the guns stopped firing there was a moment of silence...Almost at the same moment came another noise, the whip and crack of the enemy machine guns opening up with deadly effect...As the attack progressed the German positions which did most damage were two machine gun posts in front of the Middlesex. Not only did they virtually wipe out the 2nd Middlesex with frontal fire, but they caused many of the losses in the 2nd Scottish Rifles with deadly enfilade, or flanking fire."
In short, two German machine guns, a dozen Germans at most, brought to a halt two battalions of British infantry, about 1500 men.
When historians write about the end of this century, the computer and the explosion of information resources which it generated may be viewed as having a revolutionary effect on military operations, perhaps as dramatic as those caused by the machine gun.
With only a personal computer, a modem and funds for on line and search charges, you can explore the strengths and weaknesses of open information sources now.
3. What is Open Source Information (OSI)?
OSI is Open Source Information; that is, intelligence collected from open sources such as newspapers, television and radio broadcasts, books, reports, journals, photographs and other images. If is based on information collected from open sources. It is officially defined as follows:
Open source information for purposes of this directive is publicly available information (i.e., any member of the public could lawfully obtain the information by request or observation), as well as other unclassified information that has limited public distribution or access. Open source information also includes any information that may be used in an unclassified context without compromising national security or intelligence sources and methods. If the information is not publicly available, certain legal requirements relating to collection, retention, and dissemination may apply.
Of course, these sources have been used in the intelligence production process in the past. What is different now is a growing recognition that a more organized and focused effort must be made to take full advantage of open source materials.
An important subset of open source information is called Gray Literature. It has been defined by the Interagency Gray Literature Working Group (IGLWG) as follows:
Gray literature, regardless of media, can include, but is not limited to, research reports, technical reports, economic reports, trip reports, working papers, discussion papers, unofficial government documents, proceedings, preprints, research reports, studies, dissertations and theses, trade literature, market surveys, and newsletters. This material cuts across scientifie, political, socio-economic, and military disciplines.
Organizations that typically generate the largest quantities of gray literature include: research establishments (laboratories and institutes); national governments; private publishers (pressure groups/political parties); corporations; trade associations/unions; think tanks; and academia.
The quantity and quality of open sources has increased dramatically. Writing in 1992, Admiral Studeman, Deputy Director of Central Intelligence reported:
" We have identified some 8,000 commercial data bases - and the vast majority have potential intelligence value."
The number of worldwide periodicals has grown from 70,000 in 1972 to 116,000 last year.
The explosion of open source information is most apparent in the Commonwealth of Independent States, where today, there are some 1,700 newspapers that were not published three years ago.
The sources of 'gray literature', (i.e., private or public symposia proceedings, and academic studies) around the world are also increasing dramatically.
... FBIS (Foreign Broadcast Information Service) monitors over 3,500 publications in 55 foreign languages. And each day it collects a half a million words from its field offices around the world and another half a million words from independent contractors in the U.S. - that's equivalent to processing several copies of War and Peace every day.
Will OSI do the whole job?
Dr. Joseph Nye, until recently Chairman of the National Intelligence Council and now Assistant Secretary of Defense for International Security Affairs, has on many occasions used the jig-saw puzzle analogy to describe the relationship between OSI and the other traditional intelligence disciplines of human intelligence (HUMINT), imagery intelligence (IMINT), and signals intelligence (SIGINT). In paraphrase, he has noted:
Open source intelligence provides the outer pieces of the jig-saw puzzle, without which can neither begin nor complete the puzzle. But they are not sufficient of themselves. The precious inner pieces of the puzzle, often the most difficult and most expensive to obtain, come from the traditional intelligence disciplines. Open source intelligence is the critical foundation for the all-source intelligence product, but it cannot ever replace the totality of the all-source effort. Even the most ardent enthusiasts claim that OSI can provide no more than 60%-80% of the total intelligence requirements. It can provide the overall context in which to view a topic. It can help to focus the search through classified material.
"Using open sources for orientation purposes, as a preliminary to tasking classified systems, may be the single smartest thing an operator can learn to do."
Appendix A to this manual contains a the paper "ACCESS: in the Age of Information" that describes the role and importance of OSI.
Why OSI now?
The end of the bi-polar world
For almost fifty years, the world was divided into two opposing blocs of nations centered around the United States and the Soviet Union. The military, political and nuclear threats posed by the Soviet Union dominated and shaped the security and intelligence resources of the United States. Intelligence collection and analysis assets were focused on the greatest perceived threat to our national security.
The world is very different now. Who would have guessed ten years ago that by the end of 1993, the U.S. and its coalition partners would have fought and won a war in the Middle East, placed 30,000 troops in Somalia and have other forces stationed in Macedonia; or by the end of 1994, have 20,000 troops in Haiti, and a major deployment of forces to Kuwait?
The challenges now facing the intelligence community are sometimes referred to as the "Gray Area Phenomena" or GAP.
The GAP are threats to the stability of nation-states by non-state sectors and non-governmental processes and organizations. The Gray Areas at once appear to be strikingly new and uncomfortably old.... Just beyond the horizon of current events lie two possible political futures - both bleak. The first is a re-tribalization of large swaths of humanity by war and bloodshed ... in which culture is pitted against culture, people against people, tribe against tribe. ... The second is being borne by the onrush of technical, economic and ecological forces that demand increased integration and uniformity. The planet, it appears, is both falling apart and coming together at the very same moment.
A pessimistic view of the future tasks facing the intelligence professional was expressed at a recent conference. The speaker stated that there was no indication that there would be a fundamental change in human nature and that, therefore, conflict was inevitable. Secondly, history shows that no technology has every been stopped from spreading. Efforts at non-proliferation may slow, but will not stop the spread of technology such as that needed to produce a nuclear device. Thirdly, as the world shrinks and becomes a single "Global Village", local conflicts will increasingly become global in scope.
How can intelligence collection be flexible enough to respond as quickly as the political or military situation demands? Open source material is collected and reported continuously around the world. It is current and readily available.
Speed : An element of national power
In their book War and Anti-War, Alvin and Heidi Toffler describe the changes which have accompanied the change from Second Wave (industrial) to Third Wave (information based) economies. One of the characteristics of the Third Wave is the importance of speed in information flow and decision making.
Writing about Desert Storm, they say
But the issue in battle is not necessarily absolute speed, but speed relative to the enemy's pace. And here, there was no doubt about the superiority of the victors . And quoting Forbes magazine they say America won the military war...the same way the Japanese are winning the high-technology trade and manufacturing war against us: by using a fast-cycle, time-based competitive strategy.
The military consequences of speed are significant.
A brigade in two days is worth a division five months later. And therefore, precision warfare, precision delivery of violence, precision delivery of deterrence becomes a currency which requires a different configuration of information management.
Because open source material is collected continuously on almost every imaginable subject, it can respond to new and unanticipated requests with no delay.
Dramatic increase in electronic capabilities In 1965, a typical small business computer had about 16 KB of internal memory. Punched cards provided data input and output. A typical disk drive stored only 7 MB (million characters). Such a machine was usually leased for about $10,000 per month.
Today, a personal computer (PC) with 20-50 times the capacity of that machine can be purchased for around $3000. A typical business PC has 8 to 16 MB of internal random access memory (RAM). A typical disk drive stores over 500 MB and for about $300 more, 1 GB (gigabyte, or 1 billion bytes) drives are available.
A modem which sends and receives data over phone lines at a rate of about 100 KB (roughly 50 single-spaced, typed pages) per minute costs less than $100.00. Newer modems which send and receive at twice this rate are now available. Compact disk read-only-memory (CD-ROM) drives are becoming a standard part of most new PC's. A CD-ROM disk holds about 650 MB of text, images or video.
The tools to collect, store and analyze large quantities of information are cheap, reliable and readily available.
Industry experts point out that, for more than the last decade, computer power had doubled every 18 months. They see this trend continuing for at least the next several years.
Both military and intelligence budgets have been reduced since the collapse of the Soviet Union. The concept of "doing more with less" characterizes the situation in both the military and civilian intelligence communities. If the same information can be collected from a foreign newspaper or television broadcast or from an expensive space-borne sensor, which is the more cost-effective source?
Open vs. classified sources
If it's really valuable, it must be classified
Classified information, with its access and handling restrictions, appears to have an authoritative quality, to represent the "real" story. However, that is not always the case. It may contains subtle biases or omissions just as does any other source. For example, history has revealed that some past estimates of Soviet military and economic strength were incorrect.
One commentator describes this preference for classified material as a psychological problem:
By this I mean the mind-set which stems from the Cold War itself. The secrecy which surrounded that period has produced in many analysts a belief in infallibility and a pride in exclusivity. The application of codewords and security classifications gives an air of accuracy and authority... However, the conflicts of today do not lend themselves to the old methods of closed shop analysis. While having breakfast, you hear on the news that a revolution has broken out in some country. You can't wait to get to work so that you can go to the SCIF and see what is "really" happening.
Did you ever think that "better" information might be available in open sources? Perhaps a quick review of newspaper accounts of the last election in the country or an article on the nature of long-standing ethnic tensions in the area might provide a far clearer understanding than will a highly classified report which details the number of rounds of small-arms ammunition stored in a hidden bunker and the fact that only 57 out of a possible 64 helicopters in the area are operational.
What is the value of an unclassified report?
Almost 15 years ago, the U.S. Defense Attach in Beijing, China wrote a letter to a small, Army Reserve detachment thanking it for an unclassified report which it had produced. The report described the geography of the Sino-Soviet border and analyzed likely avenues for a Soviet attack on China.
The attach wrote that although he had access to a great deal of classified material dealing with the same topic, the unclassified report was particularly useful because he could give it freely to his contacts in the Peoples' Liberation Army and others. It served as the basis for a number of useful discussions.
Increasingly, U.S. military forces have operated as members of a coalition of allied nations. Intelligence information based on open sources is far easier to share with coalition partners than is that drawn from classified, sensitive sources. Reports based on OSI can be disseminated to uncleared U.S. personnel. For example, enlisted soldiers who need to know what is happening around them.
But I serve in a tactical unit
It is not difficult to make the case that OSI provides useful background for intelligence professionals working at the national or strategic level. However, tactical intelligence officers also rely on OSI.
The following comments from an officer at the battalion level in the XVIIIth Airborne Corps provide an interesting and useful view of how creative intelligence professionals use OSI. Read the comments which follow carefully. Although written late at night by a junior officer, they are worthy of careful study and consideration by all military intelligence professionals. After you have finished reading them, go back and read them again. In response to the comment that OSI is a resource of great value to an MI professional supporting a tactical commander, he responded:
I totally concur. Quite often, my commander sharpshoots me with OSI he's found. Battalion and Brigade commanders are smarter than the average bear, tend to be avid readers, and track the news/commentary shows closely. 2's need to support their 6's proactively with OSI, perhaps in the following situations:
* daily summaries of contingency or hot areas (I do this during our DRF-1.[Division Ready Force is the first battalion to be deployed] I come into the office at 0500, pull up the AP wire and scan it for 18th ABN Corps top 10 countries. Limiting my total printout for the boss to 2 pages, I highlight key info. If there is an event that seems like it could be of particular interest, I'll look for an in-depth story on it from a different source (US News, for example), and have that printout available.
* weekly summaries of the same for all company commanders in my battalion. I try to be broad in my scope, but key in on the high points. This style helps the company commanders tailor the information to the soldiers in their company. First Sergeants like to have current events boards for the troops, and this helps them compile intelligence. During the U.S.'s involvement in Somalia, there was a fair amount of information available on Aideed and his forces. This helped me paint a picture for our troops about what they might face if we deployed. (Particularly helpful, because it was generally a non-Warsaw Pact "conventional" force.) I pushed out information I collected on Yugoslavia to the companies also. One First Sergeant compiled this (essentially a glorified Target Folder) into a large "Did You Know?" bulletin board.
* OSI supports briefings that I build for Platoon Sergeants and above within the battalion. Although I will normally add to the unclassified core of the briefs with classified intel, it helps tremendously to have an unclassified base to revert to. Normally the IRC (Initial Reaction Company) within our DRF-1 Bn likes to have briefings for all soldiers before and during our alert cycle.
* If I had time prior to a deployment to prepare intelligence products for the troops, I would certainly include Target Folders, Handbooks (smartbooks), and INTSUMs for the companies. An example of where this would happen is prior to Sinai rotations, where these products can be constructed out of sophisticated OSI, without the worry of classified restrictions. Knowing where to find more OSI than I currently have would help me with these products. The best part about this is that it would enable a savvy user at Brigade or below to produce his own products. I already maintain both a classified contingency "database" supplemented with unclassified. This supports not only briefings while "standing by", but production of products upon alert...
Bottom-line: OSI has helped me translate information into intelligence for infantrymen based on what their leaders have asked me to focus on.
During every hour of every day, people are collecting information for business, academic, news reporting or research purposes. Much of that information eventually is printed, broadcast or stored in an electronic database.
Does the intelligence community care today about the work of a graduate student in anthropology who has spent the last few years studying the clan structure in some Third World country? Probably not. But how valuable will that research be when clan rivalries lead to the seizure of a busload of U.S. tourists and the dispatch of U.S. forces to attempt to rescue them?
A recent example illustrates the value of OSI. In preparation for the "left hook" maneuver in Operation Desert Storm, CENTCOM tasked all intelligence agencies for all information they could get on trafficability through the wastelands of southern Iraq. CENTCOM needed to know where the sands would be too soft to support tanks and where defiles would stop vehicles and require bridging equipment.
A great hunt was launched for data. One very helpful source turned out to be the Library of Congress. A crew of intelligence officers spent three days pouring over old archeological manuscripts and found trafficability data. Archeologists early in this century had recorded minutiae on the countryside in their diaries as they slowly made their way across the sands on camelback.
Recent events in Somalia and Rwanda demonstrate how quickly a country or area can become a focus of U.S. and international interest. If U.S. troops are sent on short notice to an area which in the past has not been a focus of intelligence collection efforts, what is the best means to provide the thorough intelligence products which the commander will require?
The intelligence community can neither anticipate nor collect all of the information which may prove valuable at some point in the future. However, it is quite likely that the relevant information has been collected and recorded by someone. An OSI system provides the tools to find that information.
Timeliness Because coverage of events in the world is continuous, somewhere, there is usually current information available. If a U.S. trade journal publishes an article this month on the level of technology in the aircraft industry in another country, isn't it likely that the information in that article may be more useful than that contained in a classified study done by a contract research organization five years ago?
Differing points of view
OSI sources are extremely diverse. Reports and articles on a situation may be produced by both sides in a dispute. A government may inflate figures to indicate great success for its programs. Opposition factions may bias their analyses of the same programs to support their own political or social agenda.
An intelligence analyst must be able to judge the relative value of reports on the same topic or event emanating from different sources. The contrasts and discrepancies between several different accounts may be just as valuable as the actual reports themselves.
The value of an open forum
By definition, OSI material is not classified, It can, therefore, be subjected to analyses and comments by a very wide range of experts and commentators.
During W.W.II, the U.S. mounted a secret effort to designed to collect thousands of bats from caves in the southwestern United States. The bats were to be fitted with small incendiary devices, and released over Japan to cause thousands of fires.
After some planning and numerous discussions, a small team was dispatched to collect bats. Unfortunately, the bats were unwilling to participate in the war effort and proved impossible to collect in any numbers. The project was dropped.
What do you think would have been the results of that initial idea if elements of the idea had been subjected to a broad review by researchers who studied bats, "spelunkers" who had explored caves containing bats, and other knowledgeable commentators? Probably, the bats would never have been disturbed in the first place.
Possible government/private partnerships
Virtually all open source material is collected by private industry, the news media and academic institutions. Private organizations index court decisions, newspaper and journal articles and other material and sell access for a fee.
If the interests of the intelligence community and of private industry are identical, then it may be possible to contract for at least some of the information services required. There are currently efforts in progress to define standards for the storage and retrieval of open source information.
Every day, thousands of newspapers, research reports, books, magazines, radio and TV broadcasts and other documents are produced. They contain valuable information, but, clearly, very powerful tools are required to store, index and retrieve useful information from this enormous quantity of facts, opinions and rumors. A later section of this handbook describes some of the available tools.
The formats, packaging, language and type of information produced each day range from a single typewritten sheet of paper containing the conclusions of a research study to 150 cassettes of video tapes containing the results of a scientific symposium conducted entirely in a language other than English. Without a standard format for this information, it is almost impossible to retrieve useful information from it.
Initiatives have been presented to create a National Open Source Architecture which would allow access to vast collections of open source material by business, government, military and civilian researchers.
As mentioned above, the sheer volume and variety of the information available in open sources demand skill on the part of the analyst to evaluate the quality and accuracy of a given document or report.
Clearly, the agricultural production figures published by a government might be quite different from the figures published as part of a research project by a U.S. graduate student who lived in the countryside in a peasant village for several years. Which is to be believed?
The Bottom Line
OSI expert Robert Steele quotes a Navy officer who told him that if it's 85% accurate, on time, and I can share it, then that is a lot more useful to me than a compendium of Top Secret Codeword information that is too much, too late, and requires a safe and three security officers to move it around the battlefield.
The current state of OSI in the intelligence community
Although there is a growing recognition within the intelligence community that more attention and resources should be devoted to developing open sources, the effort to manage and exploit these sources remains fragmented and overlapping.
There exists no single mechanism to task open sources and to specify requirements. The existing culture and systems within the intelligence community often make it far easier to task a classified source than an open one.
Also, there is no common information architecture within which open source material is stored and accessed. Until this has been developed and put in place, it will be difficult to make full use of the technology.
The growing recognition of the importance of OSI was indicated by the appointment in 1992 of Mr. Paul Wallner as the Director of Central Intelligence's (DCI's) Open Source Coordinator, and the subsequent appointment, in 1994, of Dr. Joseph Markowitz as the Director of a much-expanded Community Open Source Program Office (COSPO). Outside the intelligence community, under the leadership of the Administrator of the Defense Technical Information Center (Mr. Kurt Molholm), there has existed since 1988 a semi-official consortium called CENDI which brings together the managers of the major scientific and technical information (S&TI) collection programs.
COSPO has announced that the community now spends 1% of its budget on open source collection and processing, and that its objective is to increase this amount to 2% of the budget. Some outside observers, however, have expressed the view that this effort, while worthy, will not lead to a fundamental restructuring of the entire intelligence community, and may remain marginal for many years to come.
In 1992 and 1993, conferences have been held in Washington, D.C. to bring together those experts working to improve the use of OSI.
The most recent conference held in November, 1993 was entitled National Security and National Competitiveness: Open Source Solutions. A number of papers were presented by representatives of private industry, the academic world and government all dealing with various aspects of OSI. The proceedings of these conferences as well as information on future conferences is available from the organizer.
As the title of the conference indicates, the problem of organizing and focusing OSI products is not only of interest to the intelligence community. It is also a key component of national power and competitiveness in the Information Age.
Over time, a realignment of funding and new organizational relationships between intelligence community agencies and private sector OSI providers should emerge. Commanders rather than intelligence professionals will determine when OSI is not "good enough". Improved electronic connectivity between the community and the private sector will ultimately create a "virtual intelligence community" far more capable and responsive to the needs of the policy maker and commander than the existing kaleidoscope of intelligence, investigative, and information capabilities.
OSI is coming back into its own as the foundation for all-source intelligence. The intelligence community that evolved through the Cold War is in the process of a radical make-over. It is OSI that will tide the commander over when dealing with Somalia, Rwanda, and other Third World contingencies for which the traditional intelligence community is not well trained, equipped, or organized. Ultimately, however, OSI is but the beginning, for nothing is ever what it appears, and spies are still needed.
4. How to be a knowledgeable information consumer in the electronic age
In a presentation dealing with the use of electronically stored and retrieved information, you may wonder why it is important to mention reading. Quite simply, it remains the best and most efficient way to keep current in a rapidly changing field.
General Maxwell Taylor served in the U.S. Army in a variety of positions including commander of the 101st Airborne Division in W.W.II, Army Chief of Staff and military advisor to President Kennedy. A staff member noted that whenever he traveled, General Taylor always read. Most of his staff passed the time chatting, sleeping or playing cards, but General Taylor read.
In most assignments, it will be an atypical day when you have a chance to share ideas with an expert in your field. However, you can do that daily if you read and think about what those experts have written in their books and articles.
Electronic information - the basics
Information Storage How is information stored electronically?
Information can be stored in a number of different ways. A tape recording of a lecture, a word processing file on a computer diskette and a video tape all contain information. However, the information in these different formats differs in the ease with which it can be searched.
Suppose you were taken into a room containing 5000 cassette tapes and told that somewhere on one of those tapes was some important information. Given a tape player and a very large amount of time and patience, you would probably find what you were looking for, but it would be neither cheap nor quick. By contrast, documents stored digitally in a computer can be indexed and searched very rapidly.
Multimedia or unimedia
The digital revolution - not multimedia but "unimedia"
Recent ads for personal computers often feature the term multimedia. This indicates that the computer can reproduce sound and graphics. If your computer has multimedia capabilities, you will be able to hear the phaser fire when you battle the Klingons. Nicholas Negroponte at the MIT Media Lab points out that this term is misleading. Current technology is not leading us towards multiple means of storing information, but rather towards a single way. All forms of information; text, sound and images, can all be stored digitally.
Digital information storage basically reduces each of these forms of information to a series of numbers which can then be stored, searched and reproduced by a computer.
The challenge for the intelligence professional seeking to exploit open source information resources is to understand the types of information available, and how they can be accessed and searched.
How is electronic information accessed?
On-line systems - basic concepts
Because large databases are expensive to collect and store, they are often housed in a centralized, powerful computer system which users access over telephone lines. A brief description of the mechanics of accessing a remote computer is useful because it is the means used to access many open source databases.
The host computer
On the other end of the telephone line from your terminal is the host computer. This is usually a large minicomputer or a mainframe system with vast amounts of on-line storage and software which allows a large number of users to access stored information simultaneously.
Host computer software provides access control, for example requiring the use of a user id and password to gain access. The software may also collect the information required to charge for access time.
Who provides on-line databases?
Information has a commercial value. We all understand that when we purchase a newspaper. A number of years ago, businesses which produce information began to store collections of that information on computers and charge for access. For example, in the 1970's, the New York Times began a project which included storing every issue of the paper in computer format. Subsequently, other major newspapers, such as the Los Angeles Times, the Washington Post and the Christian Science Monitor were added. This data base, which is constantly being updated, is now made available by a company called Mead Data Central as part of the LEXIS/NEXIS service.
Many organizations produce commercially valuable data, but do not wish to get into the business of providing computer access to it. The DIALOG system, operated by a division of Lockheed, is a service which provides access to hundreds of databases ranging from information on chemical compounds to financial information and news. By having access to the DIALOG system, a user can access information in any of the databases which it provides.
Many large corporations and government agencies operate on-line databases. For example, more than 25 years ago, IBM ran an internal system called ITIRC (IBM Technical Information Retrieval Center). The system was accessible from any IBM location. It contained all IBM manuals and publications, internal documents and reports and articles from the news media and journals which dealt with various aspects of the computer field. Similar systems have been in use in government agencies and the intelligence community for a number of years.
What kind of information is provided?
Most of the on-line information available today is text, that is words rather than images. A later section in this handbook describes how textual information is searched and retrieved. However, at this point, it is important to understand that the results of accessing an on-line database will be textual material, that is, documents. When the desired documents have been transmitted to your local terminal, they may be read on the display, stored for later use or printed.
Some on-line services also provide images, such as those taken from satellites. A number of commercial services provide weather maps, satellite imagery and maps. At high resolution (fine detail) the files containing these images can be too large for practical transmission over standard telephone lines.
How do you get access?
Commercial services typically have a sign-up or enrollment process. You provide information which will allow the service to bill you for access. The service provides you with telephone numbers and a user code.
These services also provide instructional material or classes to help you make efficient use of the service. If charges for a service are determined by the amount of time you are connected as well as the number of documents you retrieve, unstructured browsing can be very expensive.
What does it cost?
Before discussing typical costs, it is important to discuss the cost question more generally. In an organization, like the U.S. Army or any other large organization, the really important question is not How much does it cost? but rather How much does it cost me or my department?
Several years ago, the Army sponsored an on-line system which provided a number of conferences on which users with a common interest could exchange opinions and information.
Many of the conferences had useful exchanges of information. Then, a charge system was introduced which charged each user activity for time used. The network lost users and shut down in a few months.
The point is, while it may be very cost effective for the intelligence community as a whole to use OSI, the way in which individual activities are charged will be a critical issue.
5. Accessing OSI Sources
Open source intelligence comes in many forms. The diagram above depicts some of them. This chapter will describe the various types of information resources and providers.
Explore the sources
A visit to a research library is a useful place to start understanding what is available in open source material. Often, hundreds of newspapers and periodicals are available for browsing. In a very short time, you will be able to identify those which contain information of interest.
A short discussion with a capable professional librarian can be an extremely valuable investment of time. Librarians are trained to locate and retrieve information. In recent years, schools of library science have placed heavy emphasis on teaching the techniques of on-line searching. Many libraries have access to one or more on-line databases and are familiar with their contents.
Books, Journals and Newspapers
In most libraries, periodicals and newspapers from a wide variety of sources are available. You can very quickly scan these sources to see which ones seem to have material which covers your area of interest.
The major American newspapers include The New York Times, The Los Angeles Times, The Washington Post and The Christian Science Monitor. Other major, reliable English-language papers include The Manchester Guardian, The Times of London, The Jerusalem Post, The Toronto Globe and Mail and the Straits Times of Singapore. These newspapers all contain useful coverage of both local and world news.
Scores of journals contains articles dealing with technical, military, historical, economic and social issues. Among the best known are Foreign Policy, The Far Eastern Economic Review, and numerous other publications.
After you have determined which publications often contain articles of interest to you, you can then determine what electronic sources provide access to them.
Many research libraries have access to on-line information sources, such as DIALOG. The librarians are often skilled not only in using these resources, but in knowing which ones are most relevant for finding information on a particular topic.
Private Intelligence Providers
There are hundreds of skilled information brokers world-wide, most of whom speak English and are familiar with specialty databases as well as special libraries maintained by universities and corporations. The Burwell Directory of Information Brokers (Burwell Enterprises, Dallas, TX) is an annual directory which also indexes all brokers according to their specific areas of expertise (e.g. transportation, fuel ), their range of database access, and, new in their 1994 directory, their mastery of foreign languages and foreign databases. It is now possible, through this directory, to identify a skilled information broker who speaks Tamil or Urdu, and is familiar with databases in countries where those languages are spoken.
This firm provides several information products. One is the Oxford Analytica Daily Brief which is modelled on the daily briefings provided to senior government leaders around the world. Drawing on the expertise and experience of more than 500 scholars at Oxford and other universities, brief analytical commentaries on current and likely future events are prepared daily. These commentaries are then faxed or made available on-line to subscribers to the service.
Jane's was founded by John Frederick Thomas Jane who was born in 1865. As a child, he was fascinated by naval vessels. As a teenager, he started a sketchbook of warships which he called "Ironclads of the World". In 1898, his authoritative guide Jane's All the World's Fighting Ships was published. The company continues today to provide authoritative guides to military and commercial equipment and facilities worldwide.
All of the guides listed below are available in both printed form and on CD-ROM's. Jane's Fighting Ships contains data on more than 8,000 warships. Jane's Armour & Artillery is the world's only annual survey of armored fighting vehicles. Jane's Security & Counter-Insurgency Equipment provide information on more than 2,000 products from over 500 manufacturers. Jane's Battlefield Surveillance Systems contains details on all ground and air-based tracking systems in service. Other products describe ports and airports, railways and urban transportation systems. In addition, Jane's also publishes a number of periodicals.
Local Personal Computer(PC)
The basic components of a PC are familiar to most readers. A central processing unit (CPU) executes program instructions. Programs and some data are stored in random access memory (RAM) while the computer is working. A keyboard is used to enter data and commands and a display presents information. Programs and data are stored magnetically on a disk or on diskettes.
The CD-ROM drive
Many current PC's are equipped with a CD-ROM (compact disk read-only memory) disk drive. Each CD-ROM disk contains up to 650 MB of text, sounds and images. Encyclopedias, mapping programs, collections of books, newspaper articles and pictures are all available on CD-ROM's. Unlike on-line services which charge for each access, a CD-ROM can be used without additional costs beyond the initial purchase price. CD-ROM's can be used to build an enormous, compact, readily-available reference library.
Let us now examine the additional components needed to access remote information sources.
The Telephone LIne
Although the U.S. telephone system is converting to a digital network, at present, most of the system is designed to transmit sound, i.e. voice conversations. Computer information is typically stored in digital form. Therefore, at both ends of the communication link, a device called a modem (modulator/demodulator) is required. Simply put, its function is the change digital information into noises which are then transmitted on the phone system to a receiving modem where the process is reversed.
A telephone connection, often dedicated to computer use, is necessary to access remote information services. Typical PC modems are designed to connect to a standard telephone wall jack. These connectors are called RJ-11 by the telephone industry.
The function of the modem, as stated above, is to change strings of digital numbers into noises on one end of the communications link and to reverse the process on the other end. The speed of data transmissions are typically measured in thousands of bits per second or KB. Current modems transmit data at a nominal rate of 14.4 KB. A newly-accepted international standard will double that rate.
How long will it take to send or receive that document? An example will show why it is useful to know something about data transmission rates. A typical page of single-spaced typewritten text contains about 2000 characters. Each character requires about 10 bits for transmission.
Therefore, if we have a 100-page document it will contain about 200,000 characters, (100 pages x 2,000 characters/page). After adding some overhead for error checking, a 14.4 KB modem can transmit or receive about 100,000 characters per minute and it will take about 2 minutes to transmit our sample document. These times can be dramatically reduced using various forms of data compression which modern modems often employ automatically. But what about pictures? The size of a digitally stored image depends on the level of detail, amount of color and other factors. However, a detailed image of a photograph can be several million characters (megabytes, MB) in size. A video image with motion, like a movie clip, may be even larger. Standard modems will take about 10 minutes to transmit a megabyte of information. Again, various types of compression can dramatically reduce these times.
On a PC, communications software, such as Procomm transmits your keystrokes to the remote system and displays the responses which are returned.
The software also performs error checking to make sure that information sent on a noisy telephone line is correct. It may also allow you to store information received from a remote system on a local disk for later use.
Programs like Procomm provide "terminal emulation". That is, they make a PC "look" like some standard terminal widely supported by host computer software.
Increasingly, communications programs are available which work together with the remote system to simplify and speed communication with the remote software. For example, the on-line service America Online(AOL) provides each subscriber with a communications program designed to work with AOL's main computers. Compuserve offers similar programs. Even the Internet now is accessible using Mosaic, a communications program which presents a simple user interface and communicates with the Internet.
The documents retrieved from on-line searches are stored on the PC's local hard disk. To make full use of this information, it is important to be able to search these documents quickly.
A number of inexpensive (less than $500 retail price) programs are available which are designed to allow rapid searching of collections of documents stored on personal computers.
These programs generally employ one of two basic approaches to searching information on a computer. The simplest programs work like the search functions in a word processor. They scan each document looking for user-specified key words. These programs are very inexpensive (less than $50.00 retail price) but are slow when searching a large collection of documents.
More powerful programs begin by building an index containing a reference to every word in every document. New documents are added to the index when added to the database. Searches on an indexed database may require less than a few seconds even when thousands of documents are searched. These programs can be used to index locally generated correspondence and reports, documents retrieved from on-line systems and electronic mail messages.
On line Information Services
The services described below represent major offerings designed for both consumers and businesses. More specialized and extensive databases are offered by other vendors. Each has its own access arrangements, pricing schedule and support services. Many of these services are listed in an appendix to this manual. The two services described below are used as examples of what is available and how it is accessed.
AOL provides a communication program at no charge. Using the program, you can connect to AOL, register and explore the offerings for several hours before charges begin. Among the many services available on AOL are U.S. News & World Report, major newspapers and access to the Internet. The graphic interface provided by the AOL program makes access and navigation simple. Costs depend on the amount of usage but typically range in the $10-20/month range for daily usage.
Compuserve Information Service (CIS)
CIS is one of the oldest commercial on-line services. It contains hundreds of separate conferences dealing with almost every topic from cooking to current events. Of most interest from an OSI point of view are the several wire services and collections of journal articles available for searching. (User enters numbers of articles to be downloaded) Select and Download Last page, enter choice !7,8,12,22,26,34,36,41,42,56,57,72 Executive News Svc.($) (Now, first story, from the Washington Post, is downloaded to local computer and stored on the local hard disk.) WP 09/07 GIs Finding Friends Among Former Foes;`No ... GIs Finding Friends Among Former Foes;`No Secrets' in Maneuvers With Russians By Fred Hiatt Washington Post Foreign Service
TOTSKOYE TRAINING GROUND, Russia, Sept. 6 - The Russian officer approached the checkpoint after nightfall. By the rules of the joint U.S.-Russian military training exercise taking place here, he should Moscow. Opposition in the parliament... (rest of story not shown) (Other articles are downloaded..) Log Off Enter choice ! bye User enters command to end the session and log off Thank you for using CompuServe! Off at 11:18 EDT 7-Sep-94 Carrier was lost!
CIS also provides a service called the Executive News Service (ENS). This service allows you to setup an profile describing wire service articles in which you are interested. For example, you might specify a profile containing the search terms "(Soviet or Russian) and (armor or tank)". You then select the wire services you wish to scan, for example, the Associated Press, Reuters and the Washington Post.
As new stories come across the wire services, those which contain the search terms you have entered are marked. The next time you connect to CIS and look in the ENS area, you will get a message stating how many stories are waiting you review.
After scanning the selected stories, you can download those of interest to your local PC for reading, indexing and later retrieval locally.
Log On ATDT862-1575 Communications software dials local CIS access number CONNECT 2400/REL-MNP Modems connect at 2400 bits/second LOGIN is starting... (User ID and password entered by the communications software) CompuServe Information Service 11:06 EDT Wednesday 07-Sep-94 P (Executive Option) Last access: 11:01 07-Sep-94 CIS keeps track of the last time you accessed the service. Copyright © 1994 CompuServe Incorporated All Rights Reserved OK CIS ready for input Select a Section or Database go ens User command to go to the Executive News Service You have left basic services Notification of a higher rate structure One moment please... Executive News Svc.($)
1 Introduction to ENS 2 Review Current News 3 Search by Ticker 4 Create/Change/Delete a Personal Folder (E) 5 Review folder RUSSIA (121 stories) Articles in personal folder 6 Review BOSNIA News Clips (53 stories) Enter choice !5 User select menu option 5 - Review folder Executive News Svc.($) 121 stories selected 1 Scan by story titles 2 Scan by story leads 3 Read all stories Last page, enter choice !1 User selects Scan by Titles
Executive News Svc.($) List of 121 selected stories Browse Documents (First screen of article titles is displayed.) 1 APn 09/07 0955 Gulf-Navy 2 APn 09/07 0920 Russia-US 3 RTw 09/07 0825 Russian, German firms to set up space venture 4 RTw 09/07 0722 Bosnia contact group starts second day of talks 5 RTw 09/07 0602 Lifting U.N. arms embargo would be disaster - Rose 6 UPn 09/07 0526 Russia to repay S. Korean loans in kind 7 WP 09/07 GIs Finding Friends Among Former Foes;`No ... 8 RTw 09/07 0102 Japan military seeking own spy satellites 9 RTw 09/07 0059 Japan military seeking own spy satellites 10 RTw 09/06 1449 Sarajevans disappointed as Pope calls off visit Enter choice or <CR> for more ! User requests next screen (user scans remaining stories, noting numbers of interesting articles...) 121 APn 08/31 1243 Russia-Radioactive-Theft A Typical On-line Session Shown below is the actual transcript of a session on CIS. Comments are shown to the right in italics.
Using the capabilities shown above, you could scan all the major wire services each day in just a few minutes. The stories which you downloaded to your local hard disk would be indexed using one of the indexing programs described above. In a short period of time, you will build a database of thousands of pages of information about the topics in which you are interested.
While the transcript show above was from CIS, the interaction with other on-line services will be similar.
Weather Maps CIS also offers various types of weather maps and forecasts. These primarily provide coverage of the U.S. However, some satellite coverage of weather in the rest of the world is also available.
The detail level in this map is quite low, but temperatures and storm activity for Northeast Asia are displayed quite clearly. These maps are available for most areas of the world and can be downloaded and printed in just a few minutes. Other sources of weather information include the Air Force Global Weather Central Dial-In Subsystem (AFGWC). The National Oceanic and Atmospheric Administration (NOAA) provides on-line access to 15 different charts showing ocean current speed and direction, sea surface temperatures and other ocean data.
Open Source Service Agent (OSSA)
OSSA is an on-line service sponsored as a project by COSPO at the National Air Intelligence Center (NAIC). It is described as a Gateway to the World of Open Sources. The prototype system went on-line on 1 October 1994. OSSA provides the user with access to a storefront with access to electronic bulletin boards, electronic mail, file transfer mechanisms and gateways to the open source requirements systems and open source databases.
One component of the system is the Database Recommendation System (DBRS). Based on user-defined parameters like subject or region of interest, a list of recommended libraries which are likely to have the information needed is produced.
A Gray Literature Online Catalog (GLOC) is also available. Again, based on user interests, a recommended list of resources is produced. To assist information retrieval from a selected source, a variety of information retrieval mechanisms are provided. These include direct connections to government databases as well as single query, Internet database searches via WAIS, and the NAIC Library Automation System for commercial and some government databases. An On-line Machine Translation facility is also available to translate small segments of foreign language text to English in seconds. This is particularly useful for translating short titles or abstracts to determine if the full text translated later.A Tool Recommendation System (TRS) is also provided which includes an on-line catalog of software that is in use or currently under development within the government. Also, OSSA provides on-line access to a variety of analysis software ranging from personal data management to linkage and relational analysis packages.
Further information on OSSA is available from the National Air Intelligence Center at Wright-Patterson Air Force Base in Dayton, Ohio.
What is it?
The Internet is often mentioned in discussions of the Information Superhighway, the proposal to create a widely available means to access global information resources. The two terms, however, are not synonomous. A recent poll indicated that 66% of the U.S. population had never heard of the Information Superhighway. The percentage who believe that the Information Superhighway is a pretty good or excellent idea is 61%. However, the percentage who understand what the Information Superhighway is, is only 17%.
The Internet grew out of early work by the Defense Department to build a distributed network of computers which could continue to function even if some nodes were damaged or destroyed. Parts of the Internet, the so-called backbone are currently funded by the National Science Foundation.
The Internet basically consists of an agreement by all participants to use a standard format when transmitting information. Any system on the network can determine the destination of a message from information stored in the header of the message . How big is it?
What makes the Internet interesting from an OSI point of view is its size. An estimated 20 million users in thousands of universities, research establishments, governments and businesses around the world are connected through the Internet. Thousands of new users gain access each month. Research libraries and experts in almost every imaginable field communicate on the Internet.
The Internet currently has several weaknesses. One is that it has no security. Any message can potentially be read by some other user. That means that even business messages or information which is not classified in the defense meaning of that term may not be suitable for transmission on the Internet.
A second weakness is that using the Internet effectively can be a rather difficult process. However, several new programs, such as Mosaic, are coming on the market. These programs provide a user-friendly front end to the resources of the Internet.
Capacity limitations are becoming an increasing problem as the number of users continues to increase. The increasing use of programs such as Mosaic accelerate the increases in Internet usage.
What is available on the Internet?
The answer to that question is basically, almost anything you can imagine. For example, each month, an Internet user at the University of California posts an Internet quiz which can be read by any interested user. The quiz poses several questions and indicates a time limit for the submission of answers which must be found by searching resources available on the Internet. Recent questions have included: In 1990, how may students were living in dormitories in Ann Arbor, Michigan? When is the next scheduled launch of the Space Shuttle and what is its mission?. A typical quiz consists of 8-10 questions and usually several individuals or teams are able to successfully answer all of them within the allotted time.
Internet newsgroups are a particularly useful part of the Internet. These are conferences (more than 3,000 at last count) which deal with almost any imaginable topic. Some are filled with messages exchanging trivial banter about Monty Python or Madonna. Others discuss oriental cooking or nuclear physics or military science.. Some are moderated. That means that all exchanges are reviewed by a moderator who rejects trivial or off-topic submissions.
One interesting and useful newsgroup is SCI.MILITARY. A typical message exchange might involve an incident in which an aircraft had been shot down in some part of the world. One participant would submit a message asking what sort of missile system was responsible for the shoot-down. The answer might come from someone who studied the industrial capabilities of the country or knew exactly what systems they had. In most cases, the information provided is very authoritative.
In addition to the newsgroups, there are hundreds of databases available on the Internet. For example, at the University of New Mexico, there is a database of news items and commercial information on Latin America. Each day, newspapers and news broadcasts are translated and analyzed by people knowledgeable in the area. These are then entered into the database. Another database is available which contains information on arms control. Some of these databases have no charge for government use or very low charges.
The Internet continues to grow at a very rapid rate. Commercial vendors are bringing programs to market which simplify the use of the Internet. A program called Mosaic provides a graphical "point and shoot" interface. Until quite recently, the Internet was used principally by university and research institutions. Now, however, more and more commercial organizations are beginning to explore the use of the Internet for commercial purposes. A number of newspapers are now placing all of their printed material on the Internet. As an example, a long-established paper, The News & Observers published in Raleigh, NC has embarked on an ambitious program to provide Internet access, the entire paper on-line, electronic mail and other services to subscribers at a cost of about $20.00 per month. Their service, called NandO.net also provides Internet access and other services to North Carolina schools. The revenue to support these services comes mostly from advertising sold to local businesses. Innovative use of the medium results in advertising much different from the passive ads seen in magazines and television. A local computer store advertisement, for example, provides a program which asks questions about computer use and then produces a recommended configuration of hardware and software complete with prices.
The Globalink company in Fairfax, VA has recently begun offering a translation service on the Internet called the Message Translation Service (MTS). The service allows registered users to send messages and text files on the Internet and rapidly receive draft translations of their documents from English into Spanish, French, or German, or vice versa. The service is available 24 hours a day and cost is 5 cents per word with a $5.00 minimum charge.
A Cambridge, Massachusetts company named Individual has begun offering a new product called First for Mosaic. This product allows a user to define an interest profile which is then used to scan more than 400 different news sources each day. Users can then browse the titles of articles selected, reading those of most interest. The price of the service depends on the number of news sources to be scanned.
DIALOG and LEXIS/NEXIS
Commercial systems such as DIALOG and LEXIS/NEXIS and scores of others typically charge both for connect time and for each search and document retrieved. An active session which retrieves 20-30 documents can easily cost several hundred dollars.Each database offered by DIALOG has charges set by the provider of the database. For some very specialized and valuable databases, the costs may be several hundred dollars for even a short session.
Compuserve, Delphi, American on Line
Less specialized commercial services such as CompuServe offer access to many of the same databases provided by DIALOG at a slightly lower cost. Basic CompuServe membership is $8.95 per month. Searching newspaper archives runs about $25 per hour. CompuServe offers selective dissemination of information (SDI) as part of their Executive New Service. This service allows a user to specify an interest profile by indicating some key words. Then the information sources to be scanned are specified. The service will then mark each news item which contains the specified key words for review. Any of these can then be copied to your terminal.
An SDI example
Several years ago, an SDI profile was set up on CompuServe which used the key words Russian or Soviet and military or army. The system was instructed to scan every article from the Washington Post, and the Reuters and UPI news services. Approximately 200 articles a week have been marked by the system for review. A total database of about 2000-3000 pages has been created on the receiving terminal by downloading selected articles. The total cost to scan and download these articles has been less than $1000.
Additional providers of on-line databases are listed in Appendix C.
Bulletin Board Systems (BBS)
The BBS world
According to Jack Rickard, editor and publisher of Boardwatch magazine, there are 57,000 BBS (bulletin board systems) in the U.S. counting only those that welcome calls from strangers. That number doubles every 18 months. In North America, an estimated 15 million people use BBS systems. As a group, BBS users tend to be frugal, sending short messages, and using off-line readers to minimize costs. A typical BBS is a PC with a large hard disk and a dozen modems is owned or managed by a person called a sysop or systems operator. On average, each sysop has 300 customers. Paid BBS subscriptions average $45.00/year. Like rock musicians, 300-400 sysops make a great deal of money; another 5000-6000 make a small profit after expenses. The major change in the BBS world now is to offer full Internet access to BBS users. Many BBS systems now offer the ability to exchange messages with Internet news groups. Increasingly, full access is being provided to the Internet to allow file transfer and searching.
Focused and Specialized BBS Systems
Setting up and administering a BBS system is quick and inexpensive. Many organizations use BBS systems to allow public access to information. For example, a company may provide technical support for its products on a BBS. Government agencies post the text of legislation and regulations on BBS systems so that interested parties may read and comment on it. These systems also offer a forum for discussions between interested individuals and organizations. Perhaps the quickest way to get a feeling for the scope and variety of information available on BBS systems is to scan Appendix A in this manual. The multi-page listing there describes some of the BBS systems operated by various departments and agencies of the U.S. government.
7.The retrieval process The retrieval process
The most complete and accurate database in the world is of no use if it cannot be searched quickly and simply. How would you like to do research in a major library if there were no catalog available? To understand the basic concepts of searching a database, it is important first to understand some basic facts about human communication.
How do humans communicate?
When two people communicate, the process is simplified because of the enormous amount of shared experience both parties have just by virtue of being human. For example, if you say to a fellow worker, "This office is a pig pen, police it up before the close of business today!", what will happen? Would you expect to return in the morning and find the telephone, computers, maps on the wall and the safes all discarded in the trash? No, the meaning of what you requested is understood by another human being.
How do you talk to a computer?
Computers do not understand meaning. Computers have no "common sense". Their circuits are made from silicon, basically sand. They have no experiences in common with human beings. So, how can we ask a computer to search a huge body of information to extract items useful to us?
Keywords and phrases
In some databases, all material is categorized when entered, often by the use of keywords as descriptors. For example, a newspaper article describing a training exercise by Soviet naval infantry might have keywords such as "Soviet","naval","military" and "exercise". The value of assigning keywords when a document is added to the database is that it makes later searches faster. If the indexer entering the article knows the subject material, the additional keyword "marine" might also be entered. In the U.S. military, we do not use the term "naval infantry" as the usual descriptor for U.S. Marine Corps forces. A limitation in using keywords arises, often, with the passage of time. It may be that at the time of the exercise mentioned above, it seemed of interest only as a routine exercise. Later events may reveal that the exercise was conducted by key cadre designated to form a new type of unit for special operations. Now, different aspects of the account are of interest. It may also be too expensive to analyze and assign keywords to the thousands of articles and reports produced by the world's news services, journals, government agencies, businesses and research organizations.
Full text indexing
The usual solution to this problem is to use "full text indexing". In these systems, the computer builds an index which contains a reference to every word in every document. A list of 30-35 "stop words" such as "of, but, the, and", is excluded from indexing. In a system indexed using a full text index, a document can be retrieved using any word appearing anywhere in the document . Is one search word enough?
Searches using only a single word, however, are usually not very useful. Suppose that you wished to search a very large document database for information on the use of land mine warfare in Cambodia. A search on "mine" would select documents dealing with coal and iron mining, lost treasure found in an old gold mine and references to a Valentine's Day message "Be forever mine". Not too useful.
Enter George Boole
Several techniques are employed to solve this problem. One is the use of Boolean logic. Boolean logic takes its name from George Boole, a British logician who lived in the nineteenth century. Boolean logic deals with logical, that is true/false or yes/no values. The "connectors" in Boolean logic are "and", "or" and "not". So, you can query a database by asking for a list of all documents which contain the words "Soviet" AND "naval" AND "exercise" AND "winter". Only those documents which contain all of those words will be selected. Another search might be for documents which contain the words ("MILITARY" OR "ARMY") AND ("POLAR" OR "ARCTIC") AND ("RADIO" OR "COMMUNICATIONS"). In this search, we have specified words which have similar meanings which are connected with "OR". These phrases are then connected with "AND".
In some systems, you can define lists of synonyms to be used in searching. This provides at least a limited means to define a concept. For example, you might define the following words as synonyms: "Abrams", "M1A1", "armor", "armour", "T72", "Leopard", "Chieftain", and "MBT". Then, when you searched for any of these words, articles containing any of the other words would also be selected.
What about endings?
Most searching systems will also allow you to leave off suffixes. The notation varies, but often the $ character indicates that you want all suffixes. For example, the search term "COMPUT$" would retrieve documents containing the words "COMPUTER","COMPUTING", "COMPUTATIONAL" and "COMPUTE".
Its in the same neighborhood
"Proximity" searching is also provided in some systems as an aid to retrieving documents containing references to a phrase. For example, you can ask for all documents containing the word "armed" within three words of "gangs". This request would retrieve documents containing the phrases, "..gangs which are armed with machetes...", "...armed gangs roving the streets...", "...armed and dangerous gangs..."
What if its not text?
All of the indexing techniques described above are useful when the information being searched is text, that is written documents. How can we search a database which contains images such as photographs?
Suppose you have the job of categorizing photographs to be included in a database. You have a photograph taken in Vietnam 25 years ago. It shows a group of U.S. soldiers cooking eggs over a fire. How would you index that photo? Under "3rd Division?", "field rations?", "foraging?". Suppose that faintly visible in the background of the photo was the wreckage of a U.S. aircraft. Would you also index the photo under "MIA?" Would it be simple to index a file containing imagery so that you could later retrieve all images which contained "fortified areas?" There are indexing systems for imagery. The purpose of the discussion above is to show that the problems of indexing images is different from those associated with indexing text.
Information management tools
If you have read this handbook up to this point, you now have an understanding that there are numerous on-line databases containing information on almost any topic. You also have an understanding on how that information can be accessed and searched. You may now be thinking, "All that is wonderful, but, suppose I start to download 100 pages of documents each week. What will I do with them? If I print them out, I'll either pile them on the floor or attempt to file them? How can I manage all that information?
Just as databases stored on mainframe or minicomputers can be indexed, so can collections of documents stored on a personal computer. Programs such as ZyIndex( ZyLab (800) 544-6339) or Isys (Odyssey Development (303) 394-0091) both build full text indexes and even on very large collections of documents, search results are available instantaneously. List prices range from several hundred dollars to a thousand or more depending on the number of simultaneous users. Both companies have available free or very low cost demonstration versions of their programs which will allow you to work with a small database to become familiar with the software.
A growing number of programs are available which allow you to develop a query for information, read or prepare electronic mail or perform other tasks all before connecting to an on-line database. The goal of these programs is to minimize on-line connect time and therefore reduce charges.
Research is currently underway to develop programs using the techniques of artificial intelligence, a branch of computer science and linguistics. These programs will "learn" to understand the kind of information you require and the best sources to find that information. Some of the complexities involved in developing programs which can extract meaning from text are described in the previous section which discusses indexing. Some of this research is driven by the need to develop a way for consumers to find programs on the 500-channel cable systems soon to be available. If you try to "channel surf" through that many channels, you will inevitably miss the programs you wanted to watch.
A smart agent may learn that you always seem to download and spend a lot of time with documents and images which deal with narcotics smuggling using commercial aircraft. It would then seek out and provide you with additional information on that topic on a continuing basis. Many mail reader programs currently provide "twit" filters. These allow the user to specify topics or individuals whose messages are to be ignored. The program Eudora, described in Appendix C, provides powerful filter capabilities. A useful description of existing and future tools to aid the analyst is presented in a report produced by the P1000 Committee of the Advanced Information Processing and Analysis Steering Group. In a Vision Statement this report says:
The quality of human performance is a function of both the tools that are used and the innate abilities. The rapid evolution of electronic technologies will continue for the foreseeable future, but it is the human abilities that will remain static. Therefore, it is important to understand and to capitalize on existing human strengths. Perceptual and cognitive psychology are requisite disciplines to advance the understanding of human capabilities. In the partnership between the computer and the human that visualization technologies promise, knowledge of perception and cognition will contribute substantially to the design of productive tools. They will lay the groundwork upon which other advances are based.
8. Cost/benefits methods for information technology.
Placing a value in monetary terms on a piece of information is usually difficult or impossible to do. Some work has been done in the corporate world in an attempt to provide cost justification for investments in information technology, but the results are not always conclusive. Of course, after the fact, it is often easy to identify some particular information which would have been very valuable. For example, the Son Tay raid mounted to free U.S. POW's from a camp in North Vietnam would not have been attempted if it had been know that the prisoners had been moved out of the camp.
Robert D. Steele in his paper ACCESS: Theory and Practice of Intelligence in the Age of Information provides guidelines for evaluating information. He points out that the U.S. spends about $20-30 billion a year on "intelligence"; that is the collection of human and technical intelligence and its processing. However, the total spending by research organizations, universities, news gathering organizations and businesses for the collection of information is estimated to be about $100 billion per year. Of that information, only a small fraction is systematically exploited by the intelligence community.
He points out that the value of information is derived from three characteristics:
1v. Its substance or content 2v. The context in which is it being considered by others 3v. The timing with which it is received.
Of these, the last, timing, is the most important. Because OSI is by definition unclassified, all of the political, legal and economic constraints on the sharing of classified information are removed. It can be shared and transmitted quickly and easily. The Third Wave Information Age described by Toffler is here now. As an intelligence professional, you have a responsibility to understand it implications on your profession and the organization within which you work. The history of intelligence contains a number of accounts of military intelligence produced only from open sources. In the 1930's, as the German Army began to expand, a book was published in Switzerland which described the army's order of battle in great detail. The author, questioned by the Gestapo, demonstrated that all of the information in the book had been derived from open sources. A wedding announcement containing the rank and unit of the groom, a concert by a military band, all were printed in some newspaper.
The computer allows the same kind of painstaking search which that author performed to be done quickly and effectively. OSI is directly relevant to the needs of the military intelligence professional. The cost to establish and operate an OSI project is very low. Currently, a personal computer with printer, CD-ROM drive and modem can be purchased for less than $3000. An additional $500 will purchase indexing software. An on-line search budget of $200-$300 per month will support daily scanning of the newswires and a number of more detailed literature searches. The ability of the intelligence professional to understand and use open source resources may spell the difference between success or failure in future conflicts or policies. The opportunity is great, the costs are low and the potential return is enormous.
Appendix A ACCESS:Theory and Practice of Intelligence in the Age of Information
Robert D. Steele, President OPEN SOURCE SOLUTIONS, Inc.
95% de l'information dont une entreprise a besoin peut s'acquirir par des moyens honorables. Henry Stiller, Director General Histen Riller, Societe Civile
Point #1: In the Age of Information, "intelligence" is less a matter of penetrating secrets and more a matter of separating useful information from the flood of open information that is available legally and cheaply; electronic sources are especially useful.
Point #2: In combination, the economic and political cost of industrial espionage, or penetrations of other governments to divine "plans and intentions", are insupportable when contrasted with the benefits of open source intelligence (OSINT).
Point #3: The concept of "central" intelligence cannot survive in the Age of Information. By focusing on OSINT, a Nation can mobilize each of its knowledge sectors, and turn the entire Nation into a "virtual" intelligence agency with far greater collection, processing, and action capabilities than are provided by the existing bureaucracies dedicated to national and defense intelligence.
Point #4: Comprehensive national knowledge strategies must provide for connectivity, content, culture, coin, and C4 security: the "Five C's".
Table of Contents
1 Background. What is the issue; changed "rules of the game"; national information continuum; four information categories; three characteristics of value; speed as the foundation of security; hard copy versus electronic information.
2. Discussion. Role of intelligence and "virtual" intelligence; information as a substitute for capital and labor; possible investment strategy for information; political and economic cost of espionage; pre-publication/pre-secret windows of opportunity; speed advantages of open source exploitation.
3. Information Requirements and Player Identification. Priority versus gaps-driven collection; four major consumer groups of intelligence; refining the gaps-driven requirements process; model for consumer-oriented production; four major target groups for intelligence; four kinds of players in the open source arena.
4. Sources of Information and Methodology. Five distinguishing aspects of information sources; essential reorientation of intelligence toward open sources; privatization of intelligence; five elements of a national knowledge strategy.
5. Industrial Espionage, Sanctions, and Proscribed Information. U.S. views of Japanese and French; general attitudes about industrial espionage; sanctions; proscribed (proprietary) information.
6. Analysis. Rules of the game have changed; competitive advantage has shifted from secrecy to openness; new "order of battle" needed for national intelligence; national knowledge strategy is a critical initiative; strategic opportunity for competitive advantage exists.
7. Action Requirements. Reinvent national intelligence; realign resources; establish a national information requirements council; establish open source focal points within United States and other countries.
8. References. Resume and selected publications; other works of importance; date of information.
What is the issue? The issue of access has enormous importance for both the national security and the national competitiveness of any Nation.
The issue for the client is: how does a Nation achieve national security and national competitiveness in the Age of Information, and what does this mean to existing national policies on intelligence organization and the expenditure of public and private funds for information collection, processing, and dissemination activities.
Changed "Rules of the Game. An understanding of the "sources and methods" which comprise "access" in the Age of Information is absolutely vital for top-level decision makers in both government and the private sector. Top-level decision-makers must understand that the "rules of the game" have changed, and that competitive advantage in the Age of Information is dependent on the laws of cybernetics, not the laws of physics. Under the laws of physics, secrecy and the restriction of knowledge provided a temporary advantage. In cybernetics, openness and flexibility win.
Most great nations spend on the order of $20 billion to $30 billion a year on "intelligence", which is traditionally comprised of clandestine human intelligence, and technical collection of imagery and signals.
National Information Continuum. At the same time, most great nations have an "information continuum" (illustrated below) whose endeavors and products are going to waste...the capabilities of these elements of the national information continuum are not being exploited! This continuum represents, in a typical great nation, a $100 billion per year capability that is lying fallow.
K-12 Libraries PI/IB Government Intelligence ------------------------------------------- Universities Business Media Defense
Figure 1A. The National Information Continuum-Nine Sectors
NOTE: The above is an original representation that has been current in the literature for over a year. It is different from the more practical Sector Breakout because it reflects a focus on elements of the information continuum which do not-at this time-contribute significantly to national and defense research endeavors.
Four Information Categories. There are four "information categories" in the access arena. They are:
a. Open source or public information; within intelligence communities, this is known as open source intelligence or OSINT. "Grey literature", literature which is unclassified and not proprietary, but produced in limited quantities for limited purposes, is included as an element of OSINT. Open (unclassified) electronic information, such as that available through the INTERNET and related file servers and newsgroups, is also included in OSINT. The vast majority of scientific & technical intelligence is available through OSINT, to include 600 scientific & technical journals that appear only in electronic form.
b. Open proprietary information, discernible through open source investigation. This includes the reverse engineering of legitimately acquired products, and legally conducted "competitor intelligence". (Note: competitor intelligence is the globally accepted term for legal research efforts by businesses studying their competitor's products, organizations, and related matters.)
c. Closed proprietary information, available only through industrial espionage or clandestine and technical penetrations of regulatory agencies.
d. Classified information, available only through clandestine human intelligence or technical (imagery or signals) intelligence.
Open Sources Proprietary (Open) Classified ------------------------------------------- Grey Literature Proprietary (Closed)
Figure 1B. The Four Information Categories
Three Characteristics of Value. The value of information is derived from three characteristics of the information: its substance or content; the context within which it is being considered by others; and the timing with which it is received.
The single most significant step an organization can take to increase the value of the information it is acquiring is to increase the speed with which the information is acquired and acted upon. This is also the most inexpensive step-but only if top management is willing to accept significant changes in doctrine and procedure.
Speed as the Foundation for Security. The speed with which information moves and achieves value depends less on the information itself (the externality) and more on the degree to which the participating organizations are organized and aware of their requirements (the internality). Organizations are drowning in information because they have not learned to swim. They are not trained, equipped, or organized to collect, process, disseminate, and act upon information. The most important employees, the ones with the contextual understanding of the situation, are normally not empowered to act on information, and normally do not receive intelligence products.
Imbuing a national infrastructure with "speed" really means that a complete change is required in the way in which organizations relate to one another, and in the way in which managers relate to front line workers or action officers. The beauty of working with open sources is that it eliminates, in a single stroke, all of the political and legal, as well as the economic, constraints that characterize the sharing of classified information. We have been trying to water the desert with oil, instead of water.
Hard Copy versus Electronic Information. Finally, in discussing issues of access, it is important to understand the relative value of hard copy versus electronic information. Although hard-copy information far outweighs electronic sources in quantity, and particularly in relation to Third World sources, the electronic world is where "up and coming" technologists as well as "up and coming" leaders are communicating their most significant thoughts.
The electronic world is especially useful because it allows an enormous amount of research to be conducted from a single location, and also allows relatively anonymous browsing through other computers or commercially available databases. Hard-copy is an important secondary source, especially when investigating Third World and non-technical issues.
-------------------------------------------- First/Second World Hard Copy INTERNET
Figure 1C. Hard Copy versus Electronic Information
Role of Intelligence and "Virtual" Intelligence. There is no issue more important to a Nation in the Age of Information than that of the role of intelligence-not only of the role of the traditional national intelligence services, but also of the non-traditional "virtual" intelligence services which are represented by the other elements of the national information continuum.
A proper perspective on this matter, at the highest levels of both the government and the private sector, represents at least a $100 billion a year value. This is enormously important, not only because it will be the major policy area affecting the future of the Nation, but because it is relatively easy to achieve by realigning and coordinating existing capabilities and funds.
Information as a Substitute for Capital and Labor. In the Age of Information, when information is the "first order" commodity, and information is a substitute for time, space, capital, and labor, the implications of this discussion are enormous. The fate of the Nation depends on a proper appreciation of this issue, and on adequate coordination between government and private sector leaders responsible for elements of the information continuum.
Possible Investment Strategy for Information. The four "information categories" in the access arena can be evaluated as follows:
- Open source: 80% of what is required for sound decision-making, at 20% of the cost, in 20% of the time (relative to industrial espionage or classified collection). The value of open source information cannot be exaggerated.
- Proprietary (open): 5% of what is required, for an additional 10% cost increment.
- Proprietary (closed): 5% of what is required, for an additional 20% cost increment.
- Classified: 10% of what is required, for an additional 50% cost increment.
[-------------OSINT (80%)------------] ------------------------------------------ PROPRIETARY (10%) CLASSIFIED (10%)
Figure 2A. Possible Investment Strategy for Information
Political and Economic Cost of Espionage. It merits comment that industrial espionage in particular, but clandestine and technical intelligence as well, reflects a political risk, or a potential political cost, that is easily triple the economic cost-industrial espionage and classified penetrations are not only twice as costly as open source exploitation, they are also twice as likely to "explode" in the face of their sponsor.
Pre-Publication/Pre-Secret Windows of Opportunity. In the area of open source information, or open source intelligence (OSINT), it is very important that decision-makers understand the levels of access in terms of time -time is the vital aspect of cybernetics, and is the critical factor in national competitiveness:
Published sources are available to mass audiences at the same time-books are generally provided to the public years after they were actually written; articles generally months after they were written; and newspaper reporting days or weeks after being drafted..
"Grey literature" is available to specialized audiences at the same time, and to non-members after a period of time.
"Work in progress" in available to peer review groups and to specially equipped "outsiders".
"Pre-publication intelligence" is available to specially equipped outsiders who take the trouble to identify and cultivate selected sources of public information. This area is the "center of gravity" for those who seek to "reinvent" national intelligence.
Speed Advantages of Open Source Exploitation. There are two reasons why open source intelligence (OSINT) is a vital area of concentration: the first is that most applied technology, including proprietary or classified technology, begins with open publications about its sub-elements, and it is often possible to piece together very good intelligence reports "at the source" while avoiding the risk of industrial espionage or clandestine operations against foreign targets. The second reason is that every "dual-use" technology to which export controls are eventually applied spends at least two years, and sometimes up to ten years, in "beta" development. The Department of Commerce, which has the lead in classifying dual-use technology, is generally two years behind actual market developments; this is particularly true in the software arena. Thus, the best time to capture a "secret" or a restricted technology, is during the two year "beta" window before it becomes a secret or is restricted.
Real-Time Mind Link Beta Testing Traditional Sources ------------------------------------------- Work in Progress Gray Literature
Figure 2B. Speed Advantages in Open Source Exploitation
Refining one's open source intelligence (OSINT) intelligence process to collect information in the pre-publication stage, by identifying and keeping in touch with key experts who provide advance looks at "works in progress", adds a further six months to a year of competitive advantage. Electronic searching is the single most vital tool in identifying these experts "in time".
3. Information Requirements and Player Identification
There are three ways of looking at the information playing field: by focusing on the four consumer groups for national intelligence; by focusing on the four warrior classes of the future; and by focusing on the sources of information. Each will be discussed in turn, together with a means of executing gaps-driven collection and consumer-oriented production.
Priority versus Gaps-Driven Collection. The information requirements arena is traditionally one which intelligence communities have not mastered. Too often they collect what is collectable, or obviously protected, and they rarely produce intelligence that is tailored to a specific customer or delivered "just in time". Information requirements are typically driven by gross priorities (e.g. the Soviets are priority one, the Chinese priority two), rather than by "gaps" or real requirements. This often means that Third World encyclopedic intelligence (most of which is unclassified), and economic or demographic intelligence vital to penetrating foreign markets, does not receive the attention it requires.
Four Major Consumer Groups of Intelligence. There are four specific groups of information customers that every intelligence organization should be serving, but generally does not, because it focuses on the very highest levels of government rather than on the subordinate levels where policy is actually created and actions are taken on a day to day basis.
a. Departmental planners and programmers, in every Department of government, not only in the national security arena, require both strategic generalizations (rather than a flood of detailed reports about tiny parts of many problems), and political-military information heavily laden with information about "plans and intentions".
b. Regional planners and programmers, including Ambassadors and Assistant Secretaries of every Department of government, require regional generalizations and very detailed mobility and market information. This is the customer group most likely to take advantage of intelligence which focuses on opportunities for advantage, opportunities to prevent disaster or establish commercial gains before anyone else realizes there is a threat or an opportunity to be contested.
c. Ambassadors and corporate general managers in specific countries require both detail about the physical capabilities of their opponents or competitors, and very detailed evaluations of sustainability, availability, reliability, and or accuracy of competitor products. In the economic arena, intelligence about demographics and culture is more easily obtained, and more valuable, than internal corporate information about competing products. If you understand the BUYER's requirements, you do not need to collect every detail about competing SELLER's capabilities. This point merits elaboration: competitive advantage comes from satisfying the buyer, not from beating the opposing seller. It is far more important to understand every detail about what the buyer wants to buy, than attempting to understand opposing solutions.
d. System designers and project managers, and those at the most senior levels who make acquisition and investment decisions, generally receive adequate intelligence about technical details, but do not receive good intelligence (intelligence which is generally unclassified) about whether the system is really worth acquiring in terms of cost-value, competing means of meeting the requirement, cost of sustainability, and so on. For instance, most advanced nations have invested billions of dollars in fast-moving sophisticated systems and failed to establish the necessary communications and computer support to actually make those systems effective in the field.
System Designers Regional Planners ------------------------------------------- General Managers Department Planners
Figure 3A. Four Major Consumer Groups for Intelligence
Refining the Gaps-Driven Requirements Process. The greatest flaw in a priority-driven requirements process is that it is divorced from the day to day needs of the policy and action-level consumer. Priority-driven collection tends to err on the side of repetitive and "vacuum cleaner" collection against the highest priorities, and to completely disregard both encyclopedic and current intelligence requirements for targets which may be of a lesser priority in the "grand strategy" arena, but of vital interest at the operational and tactical levels. A gap-driven information requirements process will take its requirements each day (rather than through monthly or quarterly "priorities validation" meetings), both from the consumer of intelligence ("here is what I need to know tomorrow") and from the analyst ("here are the things I did not know in producing this report").
Model for Consumer-Oriented Production. The existing production model, at least in the United States, is based on "stove-pipe" production which is rarely as "all-source" as it could be (e.g. the National Security Agency produces reports drawn largely from signals intelligence), and is also severely deficient because it focuses on specific countries, topics, or weapons systems-i.e. the production is defined by the target, not by the needs of the consumer. Below is illustrated a superior line of consumer-oriented products, most of which would be unclassified and whose contents would be primarily " A brigade in two days is worth a division five months later. And therefore, precision warfare, precision delivery of violence, precision delivery of deterrence becomes a currency which requires a different configuration of information management." drawn from open sources of intelligence.
BASIC PRODUCTION: COUNTRY PROFILES. Integrate executive summaries for each of the four consumer groups, with brief encyclopedic intelligence summaries of each of the key industrial, geographic, and civil factors.
STRATEGIC PRODUCTION. Tailored products that focus on establishing strategic generalizations pertinent to specific mission areas (e.g. aircraft, automobiles, textiles), specific regions, and specific timeframes (generally long-term).
OPERATIONAL PRODUCTION. Tailored products focused on regional generalizations with a special emphasis on industrial areas within the region, on seasonal differences (whether terrain or trade), and on leadership character and demographics.
TACTICAL PRODUCTION.. For each industrial area (or industry area) both generalizations and specifics country by country, with emphasis on terrain, climatic, and civil constraints. It is only at this level that a consumer should have to deal with the level of detail which now characterizes most intelligence community products.
TECHNICAL PRODUCTION. Get away from system-specific production, and move instead toward industry-area production with regional and timeframe sub-sets. Focus on support to cost-benefit and trade-off decisions, not on the system in isolation.
Figure 3B. Model for Consumer-Oriented Production
Four Major Target Groups for Intelligence. There are four specific targets for information and intelligence activities; each of these target groups must be completely understood if a Nation is to maintain both its national security and its national competitiveness. Each target group has a different source of power, and a different way of training, organizing, and equipping itself for battle. Each requires a different intelligence approach and in essence a different kind of intelligence community. The traditional intelligence officer will not be competent against all four of the target sets-four different kinds of intelligence organizations must be trained, equipped, and organized for their specific target set.
a. The High-Tech Brute, similar to the United States of America, is that group which relies on expensive technical capabilities and hugh logistics trains. In industrial terms, this is the capital-intensive player.
b. The Low-Tech Brute, such as the narcotics trafficker or the Italian crime family, represents a "needle in the haystack" problem. In industrial terms, this is the labor-intensive player.
c. The High-Tech Seer, such as highly skilled and knowledgeable computer engineers, is comprised of both conglomerations of skilled individuals engaging in economic warfare, and single individuals, "hackers", able to penetrate advanced computer and telecommunications networks. In industrial terms, this is the brain-intensive player.
d. The Low-Tech Seer, such as the Islamic Fundamentalists, or Asian gangs in the United States, are those whose "weapons" are of a cultural or demographic kind, whose "command and control" system is comprised of the television and the pulpit-very difficult for a Western intelligence service to understand and address. In industrial terms, this is the labor union.
In each of these cases, using electronic sources of news and information provides a significant competitive advantage in terms of time, scope of review, and depth of understanding.
High-Tech Brute High-Tech Seer ------------------------------------------- High-Tech Brute Low-Tech Seer
Figure 3C. Four Major Target Groups for Intelligence
Four Kinds of Players in the Open Source Arena. The diagram in the "Background" section clearly identified the nine constituencies in the access arena. Naturally there are sub-constituencies (e.g. government is divided at the federal level into legislative, executive, and judicial, and also into federal, state, and local; media is divided into mainstream national papers, regional papers, niche journals, and technical newsletters).
In general terms, there are four kinds of players:
- Those who talk to one another but are not influential, are divorced from practical military or commercial applications: the "ivory tower" academics. Spend 10% of your resources on this group.
- Those who are influential but do not quote one another and contribute nothing substantial: the "bandwagon" journalists. Spend 10% of your resources on this group.
- Those who are both connected to one another and influential-this constitutes the "mainstream" of current thinking. The downside of the mainstream is that is tends to reflect conventional wisdom rather than innovative or revolutionary thinking. Spend 20% on this group.
- Those-and they are a small group-that are neither connected nor influential, but who are in fact the "up and coming" leaders in their disciplines. It is this group which not only represents the greatest potential value for a Nation, but which is the least protected! The only obstacle to exploiting this group is internal, it is bureaucratic! Spend 60% on this group. Although intelligence organizations are accustomed to thinking of this group as a source of "sleeper" agents, in fact it should be thought of as a source of "avant garde" thinking which is not only of enormous importance to international competitiveness and domestic security, but predominantly unclassified.
Mainstream (20%) Up and Coming (60%) ------------------------------------------- Bandwagon (10%) Ivory Tower (10%)
Figure 3D. Four Kinds of Players in the Open Source Arena
4. Sources of Information and Methodology
Five Distinguishing Aspects of Information Sources. Sources of information can be classified by medium, location, discipline, language, and level of classification.
The most pervasive medium is hard-copy information. However, most of the hard-copy information is of relatively low grade, and often not worth the expense of acquisition. Never-the-less, it cannot be ignored. Spend 20% of your resources on the hard-copy medium.
The next major medium is micro-fiche; although many organizations are phasing out their micro-fiche holdings, this remains an important medium, particularly in the patent and archival worlds. Spend 10% on this medium.
Electronic information is available through on-line services as well as off-line products. It is important to emphasize that electronic information includes imagery (SPOT, LANDSAT) as well as signals (foreign radio and television broadcasts, unencrypted cellular telephones, facsimiles, and telex transmissions). It is also important to note that, despite the fact that electronic information is a relatively small arena of interest in relation to hard-copy and microfiche, it is "exploding" and already dominates many of the most advanced disciplines as the "medium of choice. For instance, this medium contains over 600 scientific journals on-line that do not appear in hard-copy at all. The electronic medium is the battleground where strategic advantage can be gained at relatively low cost and with no political risk-it is open, it is pervasive, and it is not being exploited by other countries as well as it could be-this is a very important area. Spend 40% of your resources on this medium.
The most subtle storage medium is the human brain. No intelligence service will ever master the data entry or data collection problem. The most important capability any intelligence service can develop is that of establishing real-time mind-links between the customer and the best available source. The "intelligence minuteman" concept, first articulated in December 1992 at the First International Symposium on "National Security & National Competitiveness: Open Source Solutions", is the wave of the future. Spend 30% of your resources on this medium. Note that this medium provides real-time access to the other mediums-the human expert responsive to tasking can rapidly collect, process, and disseminate essential information from the other mediums, on demand.
It is important to note that in the Age of Information technology has made possible a radical shift in why and when one acquires information. It is now possible to train, equip, and organize collectors of information for "just in time" collection instead of "just in case" collection. Hard copy and microfiche were the dominant storage mediums under the old "just in case" paradigm. Electronic information, and direct access to an enormous global pool of overt human assets are the dominant access mechanisms under the new paradigm.
The location of the information is both geographic and physical. Some information cannot be obtained without a personal visit to its location. Most information can be obtained remotely, through a telephone call or correspondence, and payment if necessary to the appropriate person.
A single researcher skilled at remote data acquisition for a particular region is more valuable than 100 clandestine case officers spread over ten countries. Physically the information might be part of a central filing system or a personal filing system. 80% of the time the information will be part of a personal filing system, which reiterates the critical importance of developing human paths to the information.
Professional disciplines (e.g. physics, electro-optics) are the most global and well-organized structures through which to acquire information. Penetrating a professional association with global links is far more useful than penetrating a single government's nuclear facility, to take one example. Again, professional associations provide the human links and paths toward virtually any information-they are also the most likely to publish information before it becomes classified. It is futile to train intelligence professionals to pretend to be scientists. It is much more useful and cost-effective to provide existing scientists with the finest communication tools and travel budgets possible-they will absorb far more, and be able to report far more, than a few case officers working with a few agents of limited access.
Language skills cannot be ignored. English, French, Spanish, Japanese, Chinese, and Arabic are the most important languages, followed by Russian, Punjabi, Hindu, and Hebrew. Any international intelligence or information service which does not have at least 100 people fluent in each of the most important languages, and at least 20 people fluent in each of the minor languages listed above, is not serious. Language translation programs (e.g. Global Link) are important aids, and can be used to reduce the time of translation for a typical document from eight hours to three. The importance of linguistic and cultural nuances should not be underestimated. The language of the consumer should be the language of production, even if this involves extraordinary cost.
Level of classification (in governments generally Confidential, Secret, Top Secret, and Codeword; in business generally Proprietary, Trade Secret, Executive Only) is the most mis-understood and over-rated basis for selecting information. Enormous amounts of money are wasted, and significant political risk is undertaken, to obtain information that is classified "secret" or "proprietary". This is a fundamental mistake which reflects a lack of understanding about how knowledge works in society, a lack of understanding of current trends in information sources and methods.
MEDIUM DISCIPLINE CLASSIFICATION ------------------------------------------- LOCATION LANGUAGE
Figure 4A. Five Distinguishing Aspects of Information Sources
Essential Reorientation of Intelligence Toward Open Sources. The methods used to obtain information should be appropriate to the sector of information generation that is being studied. Each of the nine sectors has information producers. Each nation has representatives of their own in sectors of their own. For instance, your own journalists are frequently the best means of approaching foreign journalists, and of monitoring the production and the human sources being used by foreign journalists.
The fundamental flaw in the "methods" of most intelligence agencies is that they attempt to acquire all information using their own personnel and clandestine or technical methods. Instead, they should leverage the other sectors of the information continuum, and establish a "virtual" intelligence community that is comprehensive. By focusing on open information and the open exploitation of individuals in all sectors, a Nation with a $10 billion intelligence budget can leverage another $90 billion in existing independent but exploitable "virtual" intelligence capabilities whose "overhead" costs are not being paid by the government. The electronic medium is the "lever that can move the world" and give the intelligence professional enormous access.
Privatization of Intelligence
The most fundamental change in "methods", besides moving away from investments in clandestine and technical intelligence capabilities, and toward investments in open source intelligence, is that of the privatization of intelligence. It is more efficient, most cost-effective, indeed, more discreet, to utilize ad hoc private collectors and processors of information, than it is to use existing bureaucracies, including existing intelligence agencies and existing Embassies or local corporate offices overseas.
Five Elements of a National Knowledge Strategy. This focus on openness is extremely important because it means that the fruits of this open source intelligence effort can be shared directly with industry, the press, and the legislature, without the slightest political risk. For those who do not understand how "open" information can provide a competitive advantage, consider only how disorganized everyone else is-the first Nation to establish a national knowledge strategy which harnesses the full power of their information continuum will achieve an enormous competitive advantage as we enter the Age of Information. A national knowledge strategy is comprised for five elements:
a. CONNECTIVITY. Provide leading representatives within each sector with the telecomputing tools they require to keep in touch with their counterparts in all other countries, and with one another. At the same time, provide government and corporate intelligence analysts with the same tools, and provide everyone with incentives to communicate with one another.
b. CONTENT. Provide incentives to all parties to maximize the amount of information that they put "on-line"; the government can increase on-line information by testing new economic models (for instance, the "compound interest" model instead of the "single sale" model for compensating authors) and making appropriate adjustments to copyright and patent law. This is a two-way concept. It is not only important to increase the amount of foreign language material that is captured, translated, and placed on-line, but it is equally important that great emphasis be placed on exporting national intellectual products which have been fully and accurately translated into major foreign languages such as English, Japanese, and Chinese. In the Age of Information, "gunboat diplomacy" has been replaced by intellectual influence. The quality of a nation's intellectual output, and the degree to which it is made usable by others, will have a dramatic impact on the strategic position of the Nation.
c. CULTURE. Recognize the importance of accelerating the integration of ethnic populations and economically impoverished citizens. In particular, those ethnic groups which speak and read fluently the language of their adopted country, and the language of their former country, are priceless national assets which can be used as interpreters and translators.
d. COIN. A typical major nation wastes on the order of $2 billion a year just on end-user fiddling with personal computer hardware and software combinations. If all redundant and contradictory research & development were brought under control, just in the information technology arena, a typical major nation would probably save on the order of $10-20 billion a year.
e. C4 SECURITY. Every nation, and particularly the advanced nations of Europe and the Anglo-Saxon world, is extremely vulnerable to the interruption of command & control, communications, and computing services. Virtually every major antenna system, including the downlinks from the satellites, is completely unprotected. No nation has established a serious C4 security posture that recognized the degree to which government, military, and economic communications depend on a very vulnerable civil infrastructure. No nation has established an effective "cyberspace order of battle". This is the "Pearl Harbor" of the Age of Information, waiting to happen.
CONNECTIVITY CULTURE C4 SECURITY ------------------------------------------- CONTENT COIN (R&D)
Figure 4B. The Five Elements of a National Knowledge Strategy
5. Industrial Espionage, Sanctions, and Proscribed Information
U.S. Views of Japanese and French. There is a very strong perception within the United States that the Japanese view economic competition as "war", and that the French, while not necessarily viewing economic competition as war, are willing to engage in "unethical" measures including direct support from government intelligence agencies to specific French companies. The book Friendly Spies: How America's Allies are Using Economic Espionage to Steal Our Secrets, while discredited in some circles, has done enormous damage to French interests in the U.S. The running joke about Air France seats being "bugged", very common in U.S. business circles, is a good sign of how deeply this book has penetrated the "psyche" of the U.S. business community.
General Attitudes About Industrial Espionage. Most U.S. companies do not have a full appreciation of how easily others can penetrate their organizations, and most do not engage in significant industrial espionage. For instance, most U.S. companies have absolutely no computer security and no measures to protect their computers from external penetration. They are completely unaware of the ease with which computer screens and computer emissions can be captured from a van parked outside their building. There is a general sense among U.S. firmsthat industrial espionage is "not worth it". There are two exceptions to this: the first is the hirig of executives and key employees from competitor firms, and the second is bribery, but only oversea. Sanctions. The Toshiba case is a good example of the sanctions that might be imposed is specific instances when there is a public knowledge of violation, but in general the U.S. government will not publicize or act on cases of industrial espionage.
Proscribed (Proprietary) Information. There is a strong trend in the United States toward openness. This is true of the government, where many "secrets" and many secret technologies are about to be declassified, and also of industry, where there is a growing understanding that the restriction of information imposes internal costs that may not be warranted. In the case of specific chemical formulas or other "protectable" secrets, this may not be so, but in the case of general engineering practices, targeted markets, and so on, the general focus is on openness and staying ahead of the competition, rather than on protecting secrets.
The client is better able to evaluate the applicability of this new theory and practice of intelligence to their needs, but on balance one must conclude:
"Rules of the Game" Have Changed. The Age of Information has redefined our concepts of war and peace, of national security and national competitiveness. The "rules of the game" have changed, and there has been a reordering of both power and the sources of power. Information is now a commodity, and the most important resource to any Nation.
Competitive Advantage Has Shifted From Secrecy to Openness. The Age of Information has destroyed the ability of individuals, organizations, and governments to control information or restrict the dissemination of information. It is virtually impossible to keep a "secret" in this day and age. Hence, the competitive advantage has shifted from those able to conduct research in secret, to those able to RAPIDLY exploit the efforts of others through openly available information collected "just in time".
New "Order of Battle" Needed for National Intelligence. The Age of Information requires a new "order of battle" philosophy within national governments and their major corporate sectors. The degree to which individual minds can be linked across sectors now becomes more important than the number of tanks one has-existing conventional forces can be immobilized, and existing industrial processes can be superseded, by relatively modest applications of knowledge. The rapid development of competent electronic search & retrieval specialists, and particularly specialists in scientific & technical databases and newsgroups, as well as cultural matters, should be a national priority.
National Knowledge Strategy is a Critical Initiative. Although there are several nations, including Japan, Sweden, Israel, and Taiwan (and their tribal villages world-wide) which are generally ahead of all others in their national knowledge activities, no nation has actually developed a national knowledge strategy nor harnessed the potential of its nine sectors in the information continuum-a continuum that constitute a "virtual" intelligence community of enormous power.
Strategic Opportunity for Competitive Advantage Exists. A strategic opportunity for competitive advantage exists. In my judgment there is about a two to five year window within which an organized national effort can reap enormous dividends. After that time many organizations will both realize the power of open information and start developing their own collection capabilities more fully, and public encryption will be widespread, introducing a "Tower of Babel" effect into the "electronic English" information commons that is just beginning to appear.
7. Action Requirements
a. Reinvent National Intelligence. Each nation has an enormous store of non-traditional intelligence and information capabilities outside of government that are rarely called upon. Consider the potential of these non-traditional sources, and develop plans for the total mobilization of the nation's intellectual power.
b. Realign Resources. Roughly 80% of the existing intelligence budget could be realigned to open source intelligence (OSINT) collection, processing, and dissemination, in close cooperation with the other elements of the national information continuum, who might be inspired to effect their own realignments once they see the government in a leadership role.
CLASSIFIED INTELLIGENCE 10% PROPRIETARY INTELLIGENCE 10% OPEN SOURCE INTELLIGENCE 80%
Human Sources 30% 24%
- "Up and Coming" 60% 24% - Mainstream 20% 08% - Bandwagon 10% 04% - Ivory Tower 10% 04%
Hard Copy 20% 16%
Microfiche 10% 08%
Electronic 40% 32%
NOTE: Column one percentages are 100% of subheading percentage in column two, while column two percentages are 100% of column three subheading. The percentages in column three are actual percentages of the total budget.
Figure 7A. Notional National Intelligence Resource Realignments
c. Establish a National Information Requirements Council. Identify an organization able to coordinate demands for information with global collection activities and impose the common sense guideline of exploiting open sources as "the source of first resort". This has the interesting ramification of also making much more of what we need to know obtainable through private sector capabilities rather than government capabilities.
d. Establish Open Source Focal Points Within United States and Other Countries. Within the United States, contract with two separate organizations: the first should serve as a complement to the Embassy and as a rapid-response local representative for the national intelligence council. It's role should be one of requirements management, oversight, and product packaging. The second organization should actually carry out all the research through sub-contracted collection and production, with value-added quality control. Similar arrangements can be made in other countries, perhaps by building on the established SVP network.
A resume and a list of selected personal publications are attached. The Proceedings of the First International Symposium on "National Security & National Competitiveness: Open Source Solutions", are the sole existing foundation for a national intelligence restructuring of this magnitude. A copy of both volumes is provided as part of this report, together with copies of more recent articles, speeches, and testimony bearing on this issue.
Other works of importance include: Jon Sigurdsom and Yael Tagerud (eds), The Intelligent Corporation: The Privatization of Intelligence (Taylor Graham, 1992); Alvin Toffler, War and Anti-War: Survival at the Dawn of the 21st Century (Little, Brown, forthcoming), see especially the chapters on knowledge warriors and the future of the spy. Winn Schwartau, Information Warfare: How to Wage and Win War in Cyberspace (Interpact, forthcoming).
James Holden-Rhodes, Sharing the Secrets: Open Source Intelligence and the War on Drugs (Sandia National Laboratory, forthcoming)
9. Date of Information. 17 September 1993
Government BBS Systems The U.S. government provides access to information through a number of bulletin board systems (BBS). Many of these are accessible through a single gateway which then allows access to the whole range of BBS. This gateway is known as FedWorld. To connect to FedWorld, all you need is a personal computer and a modem. Use your communications software package to dial FedWorld at 703/321-8020. Set your parity to NONE, Data Bits to 8 and Stop Bit to 1 (N,8,1). Set your terminal emulation to ANSI. You can also reach FedWorld via Internet using the telnet command. Telnet to:
If you have technical problems or questions, please call 703/487-4608 weekdays, 10:00 a.m. to 4:00 p.m.. You can also leave questions in the form of E-mail on FedWorld to SYSOP.
These Government Online Systems Can Be Accessed by Calling (703) 321-8020 or on the Internet using http://www.fedworld.gov
BBS systems accessible through FedWorld ALF (USDA) :National Agricultural Library BBS ALIX (LC) :Automated Library Information eXchange BOM-BBN (DOI) :Bureau of Mines-Bulletin Board Network ISM-SIS (IRS) :ISM Support Info System CIC-BBS (GSA) :Consumer Information Center CLU-IN (EPA) :Superfund Data and Information CPO-BBS (Census) :Lists open jobs at the Census Dept ADA-BBS (DOJ) :Amer. With Disabilities Act Info Computer Security (NIST) :Computer Sys Lab Computer Security BBS DCBBS (DC Govt) :DC Government Information DMIE (NIST/CSL) :NIST/CSL Data Management Information EBB (DOC) :Economic data and information ELISA System (DOD) :DoD Export License Tracking System GSA/IRM BBS (GSA) :Information Resources Management Issues EPUB (DOE) :Energy information and data FDA's BBS (FDA) :FDA info and policies FDA/DMMS (FDA) :PMA, IDE, 510k & guidance documents FERC-CIPS (DOE) :Fed Energy Regulatory Commission FEBBS (FHWA) :FHWA information and data FEDERAL BBS (GPO) :GPO and Govt Data (Fee Based) OSS-BBS (GSA) :GSA On-line Schedules System Eximbank BBS (EXIMBANK) :Export/Import Bank data and info. JAG-NET (USN) :Navy Judge Advocate General Labor News (DOL) :Dept of Labor information and files Megawatt 1 (DOE) :Information on energy and DoE NADAP (USN) :Navy Drug and Alcohol Abuse Prevention RIBBS (USPS) :US Postal Service Rapid Information EHSC-DDS :Army Engineering & Housing NDB-BBS(USDA) :Human Nutrition Information Service NSSDC\NASA\Goddard :The NASA NODIS Locator System. OIS (Bureau of Prisons) :US Bureau of Prison employees SBIR-BBS (NASA) :NASA Small Business Innovation Research WTIE-BBS (EPA) :Wastewater Treatment Info Exchange QED-BBS (USGS) :Quick Epicenter Determ and EQ data SALEMDUG-BBS (FEMA) :State and local FEMA user groups SBA On Line (SBA) :SBA Information & Data (Full Service) GAO WATCHDOG (GAO) :Identify Waste Fraud and Abuse Sample Weather Data (NWS):Sample data from Fee Based System TELENEWS (DOE) :Data and info on Fossil fuels USA-GPCS BBS (USA) :Army Info System Software USCS-BBS (USCS) :Customs and Exchange Rate Data & Info USGS-BBS (USGS) :Geological Survey BBS/CD-ROM Info NLPBB (USN) :CNO's Navy Leadership Policy BB FMS-BBS (Treasury) :Inventory management data & programs OASH-BBS (HHS) :Health & AIDS Information & Reports FEDIX :Links Fed Data to Higher Education PSIC (USCG) :GPS, Loran & Omega Info/status NGCR-BBS (USN) :Next Generation Computer Resources Stan FAA Safety Exchange (FAA):Small Plane Safety Reports & info NTIS QuikSERVICE (NTIS) :Order NTIS Documents Online STIS (NSF) :Science & Technology Information System Census-BEA (Census) :Census BEA Electronic Forum NOAA-ESDD (NOAA) :NOAA Environmental Services Data Direct Offshore-BBs (DOI) :Off Shore Oil & Gas Data TQM-BBS (T. Glenn) :Total Quality Management NIDR Online (NIH) :Nat. Institute of Dental Research NIHGL (NIH) :Nat. Inst. of Health Grant Line BBS MARlinspike BBS (DOT) :Maritime Admin. Press Releases, Etc. NCJRS-BBS (DOJ) :National Criminal Justice Reference Sy DRIPSS (EPA) :Drinking Water Info Processing Support PIM BBS (EPA) :Pesticide Information Network OEPC BBS (DOI) :Interior's Off of Environment. Affairs CABB (Dof State) :Passport Info/ Travel Alerts FCC-State Link (FCC) :FCC daily digest & carrier stats/report FREND #1 (NARA) :Fed. Register Electronic News Delivery FREND #2 (NARA) :Fed. Register Electronic News Delivery HSETC MD (USN) :Naval Health Sci Edu & Training Command PPCUG/RDAMIS (DOD) :Pentagon Users Group BBS ORDBBS (EPA) :EPA Office of Research & Development BB CBEE (USCG) :Coast Guard On-Line Magazine & News ATD BBS (FAA) :Air Transport Div. BBS ATOS-BBS (FAA) :Air Traffic Operations Service BBS AEE BBS (FAA) :FAA Office of Environment & Energy GEMI (GSA) :GSA Electronic Management Information Airports BBS #1 (FAA) :Airport operators and designers EnviroNET (NASA) :Space Environment Information Service FAA HQ BBS (FAA) :FAA Headquarters BBS IMA BBS (USA) :Integration & Analysis Center BBS NTIA-BBS (DOC) :Radio Freq. Management Issues ED Board (DOEdu) :Dept of Ed Grant & Contract Info BHPr-BBS (HHS) :Medical & Health Services Information Marine Data BBS (NOAA) :Marine Databases & Files Call-ERS BBS (USDA) :Agriculture Economic Research Info Call ERS (USDA) :Economic Research Line Service Line 2 PTO-BBS (PTO) :Patent and Trademark Office BBS PerManNet (Dof State) :US Agency for International Development Quick Facts! (NIAAA/HHS) :Alcohol Abuse & Alcoholism Information IITF-BBS (NTIA) :Info. Infrastructure Task Force BBS IBNS/OMPAT BBS (DOD) :Military Performance Assessment NIH Info Center (NIH) :NIH Information, Files, Pubs OECI-BBS (DOC&DOD) :Defense Conversion Information AVADS-BBS (DOI) :Dept of Interior Job Announcements NRCDR-BB (NRC) :NRC Decommisioning Rulemaking BBS
Open Source Resources This appendix lists providers of on-line databases, organizations which provide focused reporting and searches and other books, articles and products useful in dealing with OSI.
Associations American Society for Information Science (ASIS) 8720 Georgia Avenue, Suite 501, Silver Spring, Maryland 20910, Voice(301) 95-0900, Fax (301)495-0810 Association of Information and Dissemination Centers (ASIDIC) Post Office Box 8105, Athens, Georgia 30602-8105, Voice(706)542-6820 Association of Independent Information Professionals (AIIP) 3724 FM 1960 West, Suite 214, Houston, Texas 77068, Voice(713)537-9051, Fax (713)537-8332 Association of Former Intelligence Officers (AFIO) 6723 Whittier Avenue, Suite 303A, McLean, Virginia 22101, Voice(703)790-0320, Fax (703)790-0264 Center for Civic Networking (CCN) 91 Baldwin Street, Charlestown, Massachusetts 02129, Voice(617)241-9205, Fax (617)241-5064 Coalition for Networked Information (CNI) 21 Dupont Circle, Washington, D.C. 20036, Voice (202)872-0884, Fax (202)296-5098 Electronic Frontier Foundation (EFF) 1001 G Street, N.W., Suite 9506, Washington, D.C. 20001, Voice (202)347-5400, Fax (202)393-5509 Information Industry Association (IIA) 555 New Jersey Avenue, N.W., Suite 800, Washington, D.C. 20001, Voice (202)639-8262, Fax (202)638-4403 International Association of Law Enforcement Analysts (IALEA) Post Office Box 52-2924, Miami, Florida 33152, Voice (305)653-3010, Fax (305)716-3218 Internet Society (ISOC) 12020 Sunrise Valley Drive, Suite 270, Reston, Virginia 22091, Voice: (703)648-9888, Facsimile: (703)648-9887 Society of Professional Journalists 16 South Jackson, Greencastle, Indiana 46135,Voice (317)653-3333, Fax (317)653-4631 Society of Competitor Intelligence Professionals (SCIP) 1700 Diagonal Road, Suite 520, Alexandria, Virginia 22314, Voice (703)739-0696, Fax (703)739-2524
Databases - CD-ROM Earth Science CD-ROM's: A Collection Buyer's Guide Article in the March 1994 issue of CD-ROM Professional lists recurring sources of information on earth science CD-ROM's, as well as an earth science CD-ROM sampler. Sources include the SIGCAT Newsletter; Omnet ScienceNet bulletin boards, the Information Systems Newsletter from NASA, the EEZ News from USGS-NOAA, New Publications of the U.S. Geological Survey, the National Geophysical Data Center Data Announcement, the U.S. Army Topographic Engineering Center's Digital Data Digest, and the NODC Environment Information Bulletin and Earth System Monitor. Among the core CD-ROMs recommended are the Marine Climatic Atlas of the World, the Tropical Ocean Global Atmosphere CD-ROM, the Geophysics of North America CD-ROM, the Seafloor Imagery and Bathymetry for Continental Shelf of Washington, Oregon, and California, and the Global Ocean Temperature and Salinity Profiles CD-ROM.
Political Risk Services on CD-ROM. Contains 100 country-specific forecasts, the equivalent of their Political Risk Yearbook. Price is $1,000 per year plus $35 for express delivery world-wide, or $1500 when purchased with the hard-copy Political Risk Yearbook. For information call (315) 472-1224 or fax (315) 472-1235.
The Economic Literature Index This index is found in DIALOG file 139 and is also available as a CD-ROM product. Published by the American Economic Association, this is a world-wide index, not just a U.S. index. Price is around $1,600. For details on the CD-ROM, write SilverPlatter Information Services, Inc., 100 River Ridge Drive, Norwood, MA 02062-5026, or fax (617-769-8763. (U.S. voice is 800-343-0064).
Information Management Tools
A set of software programs developed at the National Ground Intelligence Center. Mr. Tim Hendrickson at the PATHFINDER Project Office, (804) 980-7242. PATHFINDER provides programs to query databases, and perform analysis on the results of queries. A number of utility and data adminstration functions are also provided.
Internet Organizations and Databases
Clearinghouse for Subject-Oriented Internet Resources Guides
This effort is jointly sponsored by the University Library and the School of Information and Library Studies at the University of Michigan. In contains among others:
Aerospace Engineering: A Guide to Internet Resources Archives on the Internet Government Sources of Business & Economic Information A Guide to Environmental Resources on the Internet Neurosciences Internet Resource Guide US Technology Public Policy
To reach the clearinghouse <gopher.lib.umich.edu>, look for "What's New and Featured Resources=>Clearinghouse..." or anonymous ftp to <una.hh.lib.umich.edu path:/inetdirsstaks>. If you have trouble, email to <firstname.lastname@example.org>.
Daily News - Free Internet Sources
Electronic document published 16 May 1994. For a copy of this 25 page document send email to <email@example.com> or <firstname.lastname@example.org>.
Internet at <http://rowan.lib.utexas.edu/map-collection.html>
Qualcomm Inc, 6455 Lusk Boulevard, San Diego, CA 92121, voice (800) 238-3672, or Internet <email@example.com>. $65. Program which is an electronic mail manager and filter. Messages whose author or subject match the requirements defined in a filter are filed or discarded.
Information Professionals List (InfoPro)
is a private, inter-disciplinary network of professional information gatherers, including private investigators, law librarians, legal investigators for law firms, fee-based information brokers, public record research specialists, investigative reporters, competitor intelligence professionals, and related professions. One can participate in InfoPro from CompuServ, America Online, Delphi, and Internet (or any other server with email, for that matter). If you wish to receive an application to join InfoPro, communicate with Mr. James Cook at <firstname.lastname@example.org> (from CompuServ, >INTERNET: email@example.com), or at voice: (415) 364-6121.
Library-Oriented Lists and Electronic Serials
List compiled by Charles W. Bailey, Jr., Assistant Director for Systems, University Libraries, University of Houston, Houston, Texas 77204-2091. He can be reached at voice: (713) 743-9804, fax: (713) 743-9748, email <LIB3@UHUPVM1.UH.EDU>.
Pacific Rim Internet Resources
Asia-Pacific Regional Information gopher <emailhost.alt.ac.th>; Georgia State University Asian Information gopher <gopher.gsu.edu>; Australia telnet <info.anu.edu.au> (ELISA); Japan gopher <ncc.go.jp>; Asian News Publications on the Internet gopher <gopher.cic.net>; newsgroups at <soc.culture.*> (replace "*" with one of the following: asian, australian, china, filipino, hongkong, india, indonesia, japan, korean, malaysia, new-zealand, singapore, sri-lanka, taiwan, tamil, thai, vietnamese); and ftp site <ftp rtfm.mit.edu ed/pub/usenet/news.answers/*, where "*" is equal to country identifiers in previous listing, and allows access to Frequently Asked Questions pertaining to the respective newsgroup.
Please Copy This Disk (PCTD)
This project, on demand, copies existing electronic texts from throughout the Internet, and mails you the diskette at a cost of $10 per document. Also publishes a monthly newsletter listing new sources, and can provide information about electronic texts available in various areas. Communicate with them at <firstname.lastname@example.org>.
Radio Free Europe daily transcripts
can be found via gopher under news at many locations, including the University of Michigan's Blue server at <gopher.itd.umich.edu>. To get the day's transcripts delivered to you personally, email <email@example.com>, writing in the body of your message <subscribe RFERL-L your-first-name your-last-name>. Scout Report
Describes new Internet resources and other related news. To receive the electronic mail version of the Scout Report, send e-mail to <firstname.lastname@example.org> and in the body of the message type "subscribe scout-report email@example.com>. Gopher access at <is.internic.net/Information Services>, WWW to <http://www internic.net>.
The Internet Society NEWS To receive membership information for the Internet Society, call (703) 648-9888, fax to (703) 648-9887, or send e-mail to <firstname.lastname@example.org>. An essential reference for all users of the Internet.
The Electronic Newsstand
Presents articles selected by The New Republic from major publications. Gopher to <enew.com> or telnet to <enews.com> signing in as "enews" with no password required.
WAIS Inc. Kevin Oliveau. (301)309-1280. Sells retrieval software for use on the Internet for $15,000 with support and upgrades. Allows creation of documents to be published on the Internet. Will soon be selling Internet In A Box, a complete package for Internet access.
Has material essential for understanding Wide Area Information Servers. For a complete copy of the package, communicate with Barbara Lincoln Brooks, WAIS, Inc.; 1040 Noel Drive, Menlo Park, CA 94025. Voice: (415) 327-9247, facsimile (415) 327-6513. INTERNET: email@example.com
Meetings and Conferences
Although there are hundreds of conferences offered by both the government and private sector on various aspects of open sources, systems, and services, the following stand out as the most useful for the military intelligence community.
Annual OSS Conference
This is the core conference, addressing OSI strategy, policy, collection, production, and hands-on applications. It takes place in November each year, always in Washington, D.C., and includes over 30 international speakers, 30 interactive interest groups, several pre-conference sessions, and 50-75 exhibits. Around 800-900 people from all over the world attend. For information call Ms. Sharon Becker at (703) 242-1700 or fax (703) 242-1711.
Annual S&TI Sources Conference
Sponsored by NASA, this conference focuses on International Document Acquisition and takes place in Washington, D.C. each September, generally attracting around 300 U.S. government professionals. For information call Mr. Tom Hermann at (301) 621-0148.
Annual Tools Conference Advanced Information Processing and Analysis Symposium. Sponsored by the Advanced Information Processing and Analysis Steering Group of the Intelligence Research & Development Council, this conference focuses on advanced software and hardware solutions for the multi-media all-source product. The next conference is scheduled for 28-30 March 1995 in the Washington, D.C. area. For information call Ms. Susan Parker at (703) 351-2635.
Planned Industry Day
The joint COSPO-CENDI Industry Day is in the planning stage and expected to be in the Spring of 1995, late April to early May. It will be a 2-day activity with 32 sessions and additional side-bars to facilitate information exchange. Some 60 exhibitors will be on hand for 150-200 invited guests. The purpose of the events will be to: (1) Identify gaps in the availability of The Community Open Source Program Office (COSPO) is planning to sponsor an annual industry day to brief vendors on its requirements. The first one is tentatively scheduled for March 1995. For information call Mr. Robert Korte at (703) 281-8912.
Outside of these focused OSINT-specific conferences, there are a variety of industrial conferences such as the U.S. and the London "OnLine" conferences, the annual meetings of the Special Libraries Association and the Association of Information and Dissemination Centers and the other associations listed in this appendix.
Sixth National OPSEC Conference "Focus on the Future: Managing Risk in a Dynamic World," 1-6 May 1994, Hilton Hotel, Alburquerque, NM (for information on future conferences, contact IOSS or OPS listed under organizations in this appendix. While not directly related to OSI, one reviewer of this handbook pointed out that open sources should be used in planning OPSEC measures because they can reveal what is already publically known about friendly activity so that resources are not wasted trying to protect information that is inherently unprotectable or is already known to potential adversaries. He suggests that the ability to retrieve and manipulate open source material would have to be considered a prerequisite for an effective examination of operations from an OPSEC perspective.
Organizations - Government
Advanced Information Processing and Analysis Steering Group (AIPASG) For information contact Mrs. Susan Parker at (703) 351-2635. This group of the U.S. Intelligence Community holds an annual conference focusing on analytic tools and the supporting technologies supporting the intelligence analyst.
British Broadcasting Service
Program listings and other information are available via World-Wide-Web at <http://www.bbcnc.org.uk//. CENDI(Commerce, Energy, National Aeronautics and Space Administration, National Library of Medicine, Defense, Interior)
U.S. government agencies interested in joining the CENDI consortium are encouraged to communicate with Ms. Bonnie Carroll, President, Information International Associates, Inc., Post Office Box 4219, Oak Ridge, TN 37831. Voice: (615) 481-0388, fax (615) 481-0390. Group exploring ways to exploit information for government use.
Conflict Studies Research Centre - RMS Sandhurst (CSRC)
Various reports can be retrieved on the Internet at URL //gopher.nato.int:70/1, under menu: Other International Organizations/Institutes of International/Strategic Affairs/Conflict Studies Research Centre - RMA Sandhurst (CSRC): Defense Technical Information Center (DTIC) (703) 274-3848. Has an on-line service. Environmental Defense Intelligence Defense Intelligence Agency, Mr. Constantine. Voice (202) 373-3455, fax (202) 373-8741. Heading a project to define the needs and requirements for producing environmental/disaster relief intelligence.
STIC Open Source Committee (OSS) Mr. Thomas Pedtke, Chair. (503) 257-6121. This committee was formed in October 1991. It mission is fourfold. First, to advocate open source information and the formation of DCI level advocacy committees and mechansims. Second, to advocate the development of analytical tools to cope with the volume of textual information. Third, to provide a forum for representatives of the technical analysis community to share information about open source information, access and analytical tools. Fourth, represent the interests of the technical analytic communities in the Open Source Steering Council.
Foreign Broadcast Information Service (FBIS) (703) 733-6315. Has an on-line service. Handbook of Latin American Studies Library of Congress' Hispanic Division. Voice: (415)691-2207, fax (415)964-0943, or via Internet at <firstname.lastname@example.org>. Contains 11,000 entries starting with 1990. Annual access for one to five simultaneous users costs $690 a year.
Interagency Gray Literature Working Group (IGLWG)
Chairman is Michael Pounder, FBIS (703) 733-6315. The IGLWG meets every third Wednesday of the month at FBIS, Reston, VA. All government and military users and suppliers of gray literature are welcome.
Latin American Data Base
A database consortium on Latin America formed with the University of Miami and the University of California at Los Angeles. Provides access to an electronic archive of publications and publishes several electronic publications.
NASA's Tech Scan For additional information send electronic mail to <email@example.com>. Voice line is (301) 621-0390. Free service alerts subscribers to the latest aerospace-related worldwide scientific and technical information that has been published. For instant access via Internet, <firstname.lastname@example.org>, or <gopher.sti.nasa.gov> or <ftp.sti.nasa.gov>.
National Military Intelligence Association (NMIA)
9200 Centerway Road, Gaithersburg, Maryland 20879, Voice: (301)840-6642, Facsimile: (301)840-8502
National Technical Information Center (703) 487-4080. Has an on-line service.
Open Source Intelligence proponent The U.S. intelligence community is represented in open source intelligence matters by Dr. Joseph Markowitz, Director of the Community Open Source Program Office (COSPO). Dr. Markowitz can be reached at (703) 482-8486, fax (703) 482-0684 (from 0800-1700 EST only). His address is: Dr. Joseph Markowitz, Director Community Open Source Program Office c/o Central Intelligence Agency Washington, D.C. 20505
Pathfinder Software Advisory Group (SAG) Pathfinder is the only Intelligence Community developed analytical tool. Mr. Timothy Hendrickson is the Program Manager. He is located at the National Ground Intelligence Center, Charlottesville, VA. (804) 980-7242. Electronic mail <email@example.com>. Pathfinder is available without charge to all military and government organizations. The SAG meets every two to three months at Presearch, Inc. in Fairfax, VA.
The Electronic Embassy Project of The American University
School of International Service, The American University, 4400 Massachusetts Ave. NW, Washington, DC 20016-8071. Voice: (202)885-1701, fax: (202)885-2494 (fax), Internet: <firstname.lastname@example.org>. Provides Internet access to resources at American University and communication with U.S. embassies to those seeking information about foreign countries.
U.S. Army Combined Arms Command, Foreign Military Studies Office (FMSO) Director, Foreign Military Studies Office, ATTN: ATZL-SAS, Ft. Leavenworth, KS 66027-5015. Voice: (913) 684-4333/4434, Fax: (913) 684-4701, Internet: <SANZT@LEAV-EMH.ARMY.MIL> . FMSO produces a wide range of products on military security matters. All FMSO products are listed in the annual FMSO Production Program, which is available on request.
Organizations and Business Services - Private
ABI/Inform Ondisc-Global Edition
For information call England, +(44 88) 374-4123, fax +(44 88) 374-4024. Provides a database including more than 350 sources from outside the U.S.
ACCESS, 1511 K Street, N.W., Suite 643, Washington, D.C. 20005-1401. Voice: (202) 783-6050 or (800) 888-6033, fax (202) 783-4767. A non-profit, non-advocacy information service on international security, peace, and world affairs issues.
Aerobureau Corporation Suite 115, 1350 Beverly Road, McLean, VA 22101, voice (703) SKY-NEWS. Provide several air-breathing surveillance aircraft including a remote-controlled drone.
Bookstacks Unlimited, Inc. is an on-line bookstore which carries more than 250,000 titles which can be searched by subject, author, title, keyword or ISBN. Connect through the INTERNET at telnet books.com or through modem connection to (216) 861-0469 (8/N/1).
Britton Lee Inc. P.O. Box 8, Los Gatos, CA 95031, voice (408) 370-1598, facsimile (408) 370-1400. For information contact Anne Wheeler. Developing tools to scan very large databases using "semantic network database technology".
Centre for Information Research, Moscow, Russia Kirill Tchaschchin or Valery Bardin at +7 095 195-4573 (Russia), email email@example.com or firstname.lastname@example.org; in the United States contact Steve Usdin, Global Press, Suite 630, National Press Building, Washington, D.C. 20045, voice: (202) 662-7431, fax (202) 662-7433, email email@example.com. Provide information for clients in the former Soviet Union and internationally.
Delphi Delphi Internet Services Corp., 1030 Massachusetts Ave., Cambridge, MA 02138. (800)695-4005. On-line service provider offering full access to the Internet. Over 150 services including document searches, news, weather, downloading of software, forums, e-mail, financial, travel, games.
Direct Information Access Corporation (DIAC) President is LtGen James Williams, USA (Ret), former director of the Defense Intelligence Agency. Voice (703) 978-9428 or (703) 978-9428. The company offers a wide range of political and economic risk analyses, overseas site selection surveys and a program to support corporate counterintelligence . Dow Jones Information Services (609) 452-1511. Offers a real-time news retrieval service scanning 60 different databases including financial information, world news, quotes, Standard & Poors, Dun & Bradstreet.
Drexel's College of Information Studies Professor Thomas A. Childers, PhD, Drexel University (Philadelphia, PA). For information contact Ms. Profit at Internet <firstname.lastname@example.org>. Developing courses in competitive intelligence. Also offers a course on the Internet.
Earth Observation Satellite Company (EOSAT) EOSAT, 4300 Forbes Boulevard, Lanham, MD 20706, voice (301) 552-0537, fax (301) 552-0507, TELEX 277685 LSAT UR. Providing products from Landsat and other satellite platforms.
Economist Intelligence Unit Economist Intelligence Unit, 111 West 57th Street, New York City, NY 10003. For information call Cathy Shelton at (212) 554-0600, fax (212) 586-1182. Produces numerous products including quarterly country forecasts, country risk assessments, and regional newsletters. Preparing to offer a CD-ROM version of these products.
Forensic & Investigative Accountants Paul McLaughlin, Lindquist Avey Macdonald Baskerville, One Financial Place, One Adelaide Street East 30th Floor, Toronto, Canada M5C 2V9. Voice: +1 (416) 777-2440, Fax +1 (416) 777-2441. Expertise incurrent and historical tracking of funds associated with the proliferation of weapons, terrorism, nacotics, and economic warfare. INDIVIDUAL, Inc. Jim Leightheiser, voice (800) 766-4224 x303, fax (617) 864-4066. Cambridge, MA company founded in 199. Uses the proprietary software developed at Cornell University, called SMART (System for Retrievaland Manipulation of Text). Scan 10,000 articles received daily and provide tailored reports to custoers by fax or e-mail. Information Access Mr. Winston Maike, email to <email@example.com> or fax to +61 7 368 4969. Offers document acquisitio services in Australia. Information Security Research Centre, Director, Information Security Research Centre, Queensland University of Technology, GPO Box 2434 Brsbane, Queensland 4001 Australia, voice +61 7 223-2752 for literature. Activities include cryptograhy and virus protection. Jane's Information Group Jane's Information Group, 1340 Braddock Place, Suite 300, Alexandria, VA 22314-1651. Voice (800)824-768, (703)683-3700, fax (703)836-1593. For information contact Sally Velthaus. Premier worldwide nformation source on defense, weaponry, civil aviation and transportation. Also publishes the Sentinl series of country reports containing military, geographic and civil information. Information provied in printed, CD-ROM formats and on-line. Klassic Concepts, Intelligence Collection Consultants Mr. Kenneth Cates, 4813 Lake Hurst Drive, Waco, TX 76710, voice/fax (817) 776-9171. Provides privatesector signal intelligence (SIGINT) and imagery intelligence (IMINT) services. Logicon Operating Systems Tybel Litwin, voice: (703) 486-3500 x 2353, fax (703) 920-7086. Company has developed a new tool forthe Logicon Message Dissemination System (LMDS) called Automated Transaction Generator (ATG). They ae working on an installation for America Online which will allow users to create personal profiles ad fine-tune the input they receive from the different newsgroups. At the same time, using ATG (compaible with Lotus Notes), they can create and update relational databases and impose uniform format onmultiple incoming sources. Mead Data Central (800) 426-7675, (513) 865-7981. Provide the LEXIS/NEXIS databases which contain thousands of newspapr articles. MITRE Corp Mr. William Ruh. (703) 883-6529. Provides support to U.S. government agencies designing and implemening open source systems. Monterey Institute of International Studies Mr. Christopher C. Fitz. (408)647-4193. Publishes databases on diskettes (ASCII format). Has three min database diskette packages. Two deal with international commerce (specifically, missile technolog and nuclear technology). The third deals with CIS nuclear issues. The CIS database contains primarysource information on nuclear proliferation. Currently developing a database on international organizations. Also have a special database on Iraq Cost: search engine software is $3,000. Annual subscriptions are $3,000 for one database or $5,000 fr 2 databases. NAMEBASE Mr. Daniel Brandt, Public Information Research, Post Office Box 680635, San Antonio, Texas 78268. Toorder full-text for specific names (you can ask him to do a search for you on the spot), call (210) 09-3160, or fax (210) 509-3161. Program on three diskettes containing 76,229 names, 165,275 citation, 560 sources, and 543 annotations compiled over ten years which identifies U.S. and foreign intellience personnel as well as international criminals. NewsNet (800) 952-0122, (215) 527-8030. Provides easy-to-use access to a database containing 700 different nwsletters focused on a wide range of industries as well as twelve international wire services. OPSEC Professionals Society (OPS) 120 West Church Street. Frederick, MD 21701-5411 (301) 663-1418
PSC, Inc. Mr. Larry Harvey. 12330 Pinecrest Road, Reston, VA 20191. (703) 716-5039. Research @pscusa.com. http://www.pscusa.com Provides support to US government agencies and the business sector designing and implementing open source systems. PSC specializes in open source research related to international telecommunications, competitive information and law enforcement issues as well as translation services.
Prentice Hall Online Prentice Hall Legal & Financial Services, 15 Columbus Circle, New York, NY 0023. (800)833-9848. Online service (800) 833-0431. Provides legal and financial information on corporations and individuals, both US and foreign. Offers training and on-line support.
Research Front Database David A. Pendlebury, Editor of Science Watch, 3501 Market Street, Philadelphia, PA 19104.(215) 386-0100. This database covers over ten years of the combined Science Sitation Index and the Social Science Citation Index as well as the SCI-Map Software Package which allows users to explore the structure of research in any nation or discipline.
ResearchBase FJV Company, fax (703) 271-5989 request for introductory package. This database identifies and tracks the latest research on international relations, international trade, and global economic issues.
Syracuse University, School of Information Science Mike Weiner, TextWise, voice (716) 787-0120, fax (716) 787-0111. Developing a software system called TextWise which uses sophisticated linguistic analysis techniquest to extract information from large databases.
Teldan Advanced Systems, Ltd 7, Derech Hashalom, Tel-Aviv 67892, Israel. Voice (972) 3 695-0073, fax (972) 2 695-6359. Publishes Global Defense Information on CD-ROM, consisting of three main elements: DefenseNet provides abstracts and indexes on the world's defense and aerospace publications from 1986 to date, including companies and contracts. Accompanying DefenseNet is the U.S. Naval Institute Military Database in two parts: "Nations/Armed Forces/Order of Battle", and "Weapons/ Systems/ Platforms".
TELTECH: Technical Knowledge Service Roger Anderson, Government Account Executive, TELTECH, Inc., at 2624 Woodley Place NW, Washington, DC 2008, voice (202) 667-8444, facsimile (202) 667-8399. Provides fast, confidential telephone access to leading specialists in virtually every area of science and technology.
The Information Professionals Institute 46 Hiller Drive, Oakland, CA 94618, voice (510) 649-9743, facsimile (510) 704-8646, CompuServe <76220,454>. Offers one-day courses ($225) on how to be an information broker; overview of on-line searching; and a public records seminar in a number of locations.
The Interagency OPSEC Support Staff (IOSS) 6411 Ivy Lane, Suite 400 Greenbelt, MD 20770-1405 (301) 982-0323
Tokyo Intelligence & Research College Director Kosei Tashiro, Collins Building, 3-1-22 Nishiochiai, Shinjuku-ku, Tokyo, Japan 161, voice +(81) 3-5982-3261, fax +(81) 3-5982-3439.
Toll-Free Access to Bulletin Board Services Concentric Research Corp.,400 41st Street, Bay City, Mich., 48708. (800)745-2747, or Ms. Kristine Loosley (517) 895-0500, Internet: KLOOSLEY@CRIS.COM. Concentric Research Corp. (CRC) and AT&T are combining the attributes of two services that will enable subscribers to gain access to hundreds of electronic bulletin boards (BBSs) through the convenience of a nationwide seven-digit toll-free number.
UnCover Company 3801 East Florida, Suite 200, Denver, CO 80210. Email address - firstname.lastname@example.org. (303) 758-3030. This company is a spin off of the Colorado Alliance of Research Libraries. It offers Internet access to a database covering over 5 million articles from 15,000 journals updated within 48 hours of receipt. For a fee, articles can be directly faxed.
United Press International's Photo Network Ms. Alyson Myers, Director of Business Development for UPI's Photo Network. Voice (202) 898-8248, fax (202) 371-8247. This bulletin board system contains current photographs pertaining to the news of the day, both foreign and domestic. Each downloaded image costs $25.00 plus time charges. UPI provides the software and user technical support.
American Intelligence Journal The Editor, AIJ, ISF Box 6712, Falls Church, Virginia 22040. Quarterly journal for intelligence professionals. $25/year(U.S.A), $34(others).
CD-ROM Professional Pemberton Press, Inc. 462 Danbury Road, Wilton, Connecticut 06897-2126. $98/year. March 1994 article on "Earth Science CD-ROMS: A Collection Buyer's Guide"
Central America Update Electronic publication available from the Latin American Database.
Chronicle of Latin American Economic Affairs
CIA Video TVCN-9355, "Open Source: First Choice" An unclassified but For Official Use Only videotape that is 15 minutes long, and would be a useful introduction to a class or discussion considering open source intelligence support to the community. CIA employees call (703) 482-7409. Other U.S. government employees call Mary Lewis at (703) 482-2216. Foreign governments and private sector, work through your appropriate U.S. government action officer.
Competitive Intelligence Review Kurt Molholm and Forrest Frank of the Defense Technical Information Center, article titled "CENDI-Improving the Flow of Scientific and Technical Information", Spring 1994
CompuServe Companion: Finding Newspapers and Magazines Online. This directory focuses on the 3,300 magazines, newsletter, and newspapers available in fulltext on CmpuServe. It also suggest the fastest, least expensive route to these sources. Price $29.95 plus posage. To order when paying by Visa or MasterCard, call (800) 247-6553, ask for Ruth Orenstein. To ordr by mail, call to obtain postage cost and mailing address. European Security Frank Cass and Co., Gainsorough House, Gainsborough Road, London E11 1RS. Subscription approximatel $50/year individual and $90/year institution depending on prevailing exchange rate. Examining the Intelligence Community's Infrastructure for Exploiting Gray Information Mason H. Soule, Hugh L. Shaffer and Shepard S. Kanarek. Final Report. Project HAVE SAM, dated 29 Sepember 1994. Battelle, 505 King Avenue, Columbus, OH 43201-2693. This report was prepared for the Natonal Air Intelligence Center. It contains listings of gray information databases maintained by both .S. and foreign governments and private organizations. Fulltext Sources Online By Ruth Orenstein. Bibliodata. For ordering details call (617-444-1154 or fax (617) 449-4584. Twice-early identifies periodicals and databases by specific areas and topics. $90 for a single issue, $16 for one year. Information Gatherer Newsletter Worldwide Consultants (Chicago), email <email@example.com. $20/year. A new electronic and hardcpy quarterly for investigators, information brokers, records researchers, intelligence analysts, libarians, and others. Intelligence and National Security Frank Cass and Co., Gainsorough House, Gainsborough Road, London E11 1RS. Subscription approximatel $50/year individual and $90/year institution depending on prevailing exchange rate. Intelligence Newsletter & MedNews 10, rue du Sentier, 75002 Paris, FRANCE. Voice: + 33 1 45 08 14 80, Facsimile +33 1 45 08 59 83. A b-weekly newsletter in English covering intelligence, technology, weaponry, proliferation and extremim. Approximate cost is $450/year with a 50% discount for government agencies. INTELSCOPE: The Professional Newsmagazine of the International Association of Law Enforcement Intellgence Analysts, Inc. : For a sample issue and/or an application form for membership which includes subscriptions, call (35) 716-3024. or (305) 597-2007. Examples of some short articles: "Military Intelligence Concepts on a New Battlefield: Domestic Drug Investigations", "Private Vendor Database Important Support for Law Enforcement", and "International Organized Crime: Emerging Threat to U.S. Security".
Intercepts P.O. Box 7176, Armarillo, Texas 79114. A monthly newsletter produced by Steven Douglas, author of The Comprehensive Guide to Military Monitoring, which provides updated signals intelligence information on U.S. military installations. $18.00/year.
International Journal of Intelligence and Counterintelligence Intel Publishing Group, Inc., Box 188, Stroudsburg, PA 18360. Subscription is $45/year for individuals, $75/year for institutions.
Journal of the American Society for Information Science Volume 45, (January 1994), article by Ruth C. T. Morris, "Toward a User-Centered Information Service". Discusses the role of libraries in providing decision-support services.
Monitoring Technology Proliferation: An Open Source Methodology for Generating Proliferation Intelligence Lt Daniel Green, USN, thesis written while a student at the Naval Postgraduate School.
netguide This 384-page book describes 60,000 bulletin boards, 9,000 networks, 500 libraries and all the commercial on-line services. Free updates are offered on-line. Price - $19.00. (800) 345-8112 to order.
North Korea: A Potential Time Bomb Mr. Robert Hall, Editor, Jane's Intelligence Review, Sentinel House, 163 Brighton Road, Coulsdon, Surrey, CR5 2NH, United Kingdom. Voice: +(44 81) 763-1030, Fax +(44 81) 763 1423. Supplement to the April 1994 issue of Jane's Intelligence Review. Provides coverage of North Korean chemical and biological capabilities, their ballistic missile program, and how they might attack across the demilitarized zone.
NotiSur-Latin American Political Affairs Electronic publication available from the Latin American Database. Open Source Quarterly Open Source Publishing, PO Box 3752, Reston, VA 20195-1752. Subscription is $40/year for individuals, $28/year for students and free to government libraries. The OSQ serves as the open source community's professional journal. For more information visit the Open Source Quarterly Web Site at http://www.eajardines.com/osq.html
OSS NOTICES The monthly 20-page publication of OPEN SOURCE SOLUTIONS, Inc., the non-profit educational association which serves as the international clearinghouse for open sources, systems, and services. An annual subscription costs $500 for institutions, $250 for individuals. This is the single most authoritative and comprehensive vehicle for monitoring open source intelligence policy, practice, and sources, systems, and services. To subscribe call (703) 242-1700 or fax (703) 242-1711.
OSS '92 and OSS '93 - Video tapes Video tapes of presentations at these Open Source Solution conferences are available from OPEN SOURCE SOLUTIONS, Inc., 11005 Langton Arms Court, Oakton, VA 22124-1807. (703) 242-1701.
P1000 Strategic Plan for Information Visualization P1000 is an operating committee of the Advanced Information Processing and Analysis Steering Group. This report dated 21 September 1994 is described as a roadmap to provide information visualization technology broadly within the Intelligence Community. Information visualization is the two-way visual and interactive interface between humans and their information resources. Visualization technologies meld the human's capacity for visual thinking with the computational capacity for analytical computing. The goal of visualization is to enhance the process of discovery, understanding, and presentation of knowledge relevant to intelligence analysis. Visualization techniques are an important tool in the need for analysts to deal with dramatically increased volumes on information.
Powershift: Knowledge, Wealth, and Violence at the Edge of the 21st Century Alvin Toffler. Bantam 1990. This book is a must read for everyone in the information age. The world s being divided into the fast and the slow (relative to information handling). Power has three legs,wealth, violence, and knowledge. Wealth is the purview of the rich. Violence is the purview of the srong. But knowledge can be used by the poor and weak.
Preparing U.S. Intelligence for the Information Age - Coping With the Information Overload Scientific and Technical Intelligence Committee (STIC) 93-001 dated January 1993, (Unclassified). This report describes the problems faced by analysts confronted with enormous quantites of information. It suggests that new analytical tools are required to select and prioritize collected information.
Preparing U.S. Intelligence for the Information Age - Part II: AnalyticalTools to Cope With the Open-Source Explosion. STIC 93-0007, dated December 1993, (Unclassified). This report describes the need for tools to collect and organize information. Tools must also be developed which can help to understand the meaning of information. Lastly, tools used to build tools must be developed.
Preparing U.S. Intelligence for the Information Age - Part III: Tools Specifications STIC (in press). This report specifies the type of tools needed by the Intelligence Community to acquire information, process and analyze the information, and disseminate finished intelligence products.
Preparing U.S. Intelligence for the Information Age - Part IV: Training STIC (in press). The creation and delivery of analytical tools, along with training, will be the linchpin of the Intelligence Community's ability to convert information into intelligence. Training and a continued mentorship of Intelligence Community analysts will be required to master the use of tools, network across the Open Source Information System architecture, and apply these skills in new and artful ways of performing analysis.
Preparing U.S. Intelligence for the Information Age - Part V: Professional Intermediaries STIC (in press). The changing and new roles of the professional intermediary are examined in relation to the changing role of the Intelligence Community and the recent emphasis on open source information and the Community's Open Source Information System.
Secrets of the Super Searchers: The Accumulated Wisdom of 23 of the World's Top Online Searchers. by Reva Basch, an internationally recognized information broker. Available from Online Inc., 462 Danbury Road, Wilton CT 06897-2126. (203) 761-1466, fax (203) 761-1444. $34.95.
SHARING THE SECRETS: Open Source Intelligence and the War on Drugs by Dr. James Holden-Rhodes. Available in a special limited edition from OSS, Inc., and is expected to be published in 1995 by a commercial printing house. This book is the true story of how Department of Energy laboratories utilized their analysts and computers to exploit vast quantities of open source information to produce tactical intelligence in support of the drug interdiction effort under the operational command of the Commander in Chief, U.S. Southern Command. The book also documents problems with existing intelligence and operational practices which interfere with the commander's ability to act on intelligence.
SIGNAL Subscriptions Department, 4400 Fair Lakes Court, Fairfax, Virginia 22033-3899. $44/year(U.S.), $65/year(all others). Articles on imaging technologies, law enforcement applications.
SourceMex-Economic News & Analysis on Mexico Electronic publication available from the Latin American Database. The Comprehensive Guide to Military Monitoring Universal Electronics, Inc., Columbus, OH. (614) 866-4605. $23.95. Book by Steven Douglas describing signals intelligence information on U.S. military installations.
The Journal of Strategic Studies Frank Cass and Co., Gainsorough House, Gainsborough Road, London E11 1RS. Subscription approximately $50/year individual and $90/year institution depending on prevailing exchange rate.
War in the Information Age Monograph available from Director, Strategic Studies Institute, U.S. Army War College, Carlisle Barracks, Pennsylvania 17013-5050. (717) 245-3234.
Washington Post 2-3 August 1993, two-part series of articles describing the threat posed by organizations united by ideology. "Islamic Warriors: Radical Movements Thrive on Loose Structure, Strict Ideology" (insert article: "Mlitarized Hezbollah Follows Lead of Iran") "Islamic Warriors: Global Network Provides Money, Haven" (insert article: "A New Strain of Terrorism Groups are Fast, Loose, Hard to Find") 12 June 1994, "The Secret Korea Debate: Washington Teeters Between Mind-Reading, Miscalculation" by avid Ignatius in the Outlook section. WIRED WIRED, Post Office Box 191826, San Francisco, California 94119-9866. Magazine covering developments n cyberspace. On the Internet, e-mail to <firstname.lastname@example.org>, or call (800) 769-4733], or (415 904-0660. Worldwide Directory of Defense Authorities with International Defense Organizations and Treaties Jonathan Hixon, Publisher, 7979 Old Georgetown Road, Suite 900, Bethesda, MD. Voice (301) 718-8770, ax (301) 718-8494.
Appendix D Open Source Program
The full text of the directive establishing the intelligence community open source program is providd below.
DIRECTOR OF CENTRAL INTELLIGENCE DIRECTIVE COMMUNITY OPEN SOURCE PROGRAM (Effective 1 March 1994)
Pursuant to the National Security Act of 1947, as amended, and Executive Order 12333, policies and procedures are hereby established for the management of the Intelligence Community Open Source Program.
I. Purpose The Intelligence Community recognizes that more effective use of open sources in a variety of intelligence applications will lead to improved products and services for Intelligence Community consumers. To manage the use of open source information by the Intelligence Community, this directive establishes the Community Open Source Program and a Community Open Source Program Office (COSPO) within the Central Intelligence Agency to develop, coordinate, and oversee implementation of the Community Open Source Program. The Director of COSPO will be appointed by the Director of Central Intelligence, will be his Principal for all open source matters, and will serve as the Open Source Program Manager. This directive also establishes a Community Open Source Steering Committee comprised of senior managers appointed by the Director of Central Intelligence.
2. Definition Open source information for purposes of this directive is publicly available information (i.e., any member of the public could lawfully obtain the information by request or observation), as sell as other unclassified information that has limited public distribution or access. Open source information also includes any information that may be used in an unclassified context without compromising national security or intelligence sources and methods. If the information is not publicly available, certain legal requirements relating to collection, retention, and dissemination may apply.
3. Objectives The COSPO is responsible for the definition and defense of the Open Source Program in the planning cycle, and for providing guidance and oversight to the program in the execution cycle. The Office, with Community departmental open source program managers, develops an optimum allocation of resources across the Community in the execution year, subject to ratification by the Open Source Steering Committee. Changes in the scope and resources of the Open Source Program must be agreed to by the Steering Committee. Through this collaborative process, the objectives of the COSPO are to:
a. oversee a process for coordinating responsive actions to satisfy user needs; b. provide advocacy and defense of departmental development and operational efforts; c. ensure funds for critical open source activities; d. oversee a process for identifying and prioritizing open source substantive requirements.
4. Open Source Management
The Open Source Steering Committee provides top-level program and policy guidance to the Open Source Program. The Steering Committee is chaired by the Executive Director for Intelligence Community Affairs and includes the Executive Director of the Central Intelligence Agency and the Directors of the Defense Intelligence Agency and the National Security Agency, as well as the Deputy Director for Science and Technology of the Central Intelligence Agency, the Deputy Director for Support Services of the National Security Agency, and the Deputy Director for Production of the Defense Intelligence Agency. The Director of COSPO serves as Executive Secretary to the Steering Committee. The Departmental Program Council, comprised of customer and provider representatives from all interested Intelligence Community organizations, will be chaired by the Director of COSPO and will provide advice and counsel to the Community Open Source Program Office. It will serve the Director of COSPO as a vehicle for discussing proposed Community program initiatives, communicating customer and provider feedback, and ensuring implementation of the Open Source Program.
The COSPO will establish other standing or ad hoc advisory and coordinating boards, panels, and committees as necessary to carry out its program management responsibilities.
5. COSPO Functions Functions of COSPO shall include: a. Strategic Planning - The COSPO will oversee implementation of the Community Open Source Strategic Plan. As necessary, the COSPO will review and update the plan. b. Program Formulation and Representation - The COSPO will issue planning guidance and will coordinate preparation and execution of the Open Source Program. With departmental open source managers, the COSPO will review execution-year programs prior to the execution year in order to optimize the flexibility and responsiveness of the Open Source Program. The COSPO also will conduct ad hoc program analysis and evaluations as necessary. c. Initiative and Innovation Sponsorship - Using funds appropriated for the purpose, the COSPO will begin Community initiatives and innovations, which subsequently will be funded in departmental programs. d. Operational Services of Common Concern - The COSPO will ensure Community coordination of the collection and acquisition of open source information and will represent open source capabilities and interests in national processes designed to manage user information needs. e. Systems Architecture - The COSPO will coordinate the design and implementation of the Community open source architecture and associated standards. The COSPO will independently assess currently employed technologies and procedures and evaluate promising alternatives. f. Development of Services of Common Concern - The COSPO will coordinate the development of new processing and exploitation tools and promote the integration of automated data processing tools developed elsewhere. g. Open Source Advocacy and Representation - The COSPO will defend Open Source Program submissions before Community executive review bodies, the Office of Management and Budget, and the Congress, and will represent the Open Source Program in appropriate Government and public forums.
6. Structure and Administration
As an Intelligence Community entity, the COSPO, located in the Central Intelligence Agency and reporting to the Deputy Director for Science and Technology, is jointly staffed in roughly equal proportion by the Central Intelligence Agency and other Intelligence Community officers who are detailed to the COSPO. The Director of COSPO will normally, but not necessarily, be an officer of the Central Intelligence Agency. The serving Director and Principal Deputy will not be from the same department or agency. Managers of the General Defense Intelligence Program, the Consolidated Cryptologic Program, the Central Agency Intelligence Program, and other intelligence programs will provide appropriate personnel to staff the COSPO with required skills; officers normally will serve rotational assignments of at least two years' duration.