The Defense Advanced Research Projects Agency (DARPA) is an “innovation icon.” It has created many breakthrough technologies that have revolutionized defense, including stealth, unmanned aerial systems, and precision-guided munitions. Many DARPA-developed technologies have had sweeping societal and economic impacts, including portable GPS receivers, new types of computer chips, voice- recognition software, interactive and personal computers, and, most famously, the ARPANET and its successor, the internet. Current DARPA research also may have dramatic future impacts, including self- driving vehicles, robots and exoskeletons, and cognitive computing – computers that emulate brain-like processing. In addition, DARPA has fostered the creation of important technical communities, including the fields of materials science, computer networking, and engineering biology.

How and why has DARPA succeeded? Which features of its operation and environment contribute to this success?² How can it succeed today and in the future in a vastly changed technological and security environment?

This article was originally published in DARPA: Defense Advanced Research Projects Agency 1958-2018

This introduction focuses on DARPA’s key features – its mission, organization, linkages to other organizations, and “political design” – and how these have contributed to its success. It will review the evolving foci of DARPA’s research and conclude with some lessons for the agency as it strives to continue to deliver transformative technologies in the coming years.

Over its 60 years, DARPA has evolved and, on a couple of occasions, come close to being dissolved. Its organizational structure, as well as some important operational mechanisms, have changed. There is no singular accurate depiction of DARPA because it has adapted based on how the world has changed – especially the national security environment. At a given point in time, one can discern several DARPAs as different parts of the organization have embraced different technologies and development approaches. DARPA’s history has been perturbed by political dynamics as well as the dynamics of the technologies it has pursued. Perhaps the most important hallmark of DARPA has been its adaptability and flexibility to respond, sometimes extremely rapidly, to changing circumstances.

DARPA’s Origins – 1958-1970

In October 1957, the Soviet Union launched the first artificial satellite, Sputnik I, an event that shocked the United States. Many Americans worried that the country was losing technological leadership to its Cold War adversary. After the launch of the first Sputnik, President Dwight D. Eisenhower followed the advice of Secretary of Defense Neil McElroy and leading scientists to propose the creation of the Advanced Research Projects Agency (ARPA).³ Initially the agency focused on three assignments from the president: space, missile defense, and the detection of nuclear weapons tests. However, Eisenhower emphasized that space was to be the realm of a civilian agency. Toward that end, later in 1958, Congress and the president created the National Aeronautics and Space Administration (NASA), a civilian agency responsible for the country’s principal space programs. As such, NASA absorbed much of DARPA’s space program. The two other presidential assignments for ARPA, missile defense and nuclear test detection, continued as the dominant foci for about 15 years.

Soon after its founding, ARPA took on Project AGILE – a decade-long, highly classified program supporting U.S. efforts in Vietnam – proposed by its Deputy Director, William Godel. In retrospect, much of AGILE was naive, poorly managed, and rife with amateurism. AGILE was run with little oversight; it was hardly scientific and focused on near-term solutions. Thus, it serves now as a lesson of what DARPA should not be.

With the transfer of the space program to NASA, ARPA spent the 1960s focused on missile defense, nuclear test detection, and AGILE. In the early 1960s, however, ARPA began to pursue a set of smaller, technically focused programs to “prevent technological surprise.” Initial programs were in materials science, information technology, and behavioral science. Arguably, ARPA “invented” these as areas of technological pursuit. For example, in 1961 ARPA Director Jack Ruina hired J.C.R. Licklider as the first director of the Information Processing Techniques Office (IPTO), which played a vital role in the creation of personal computing and the ARPANET – the basis for the future internet.

DARPA’s Resuscitation in the 1970s

While there were successes in these early pursuits, by the mid- 1960s DARPA was a troubled agency – a victim of the Vietnam malaise, resource cutbacks that affected all of the Department of Defense (DOD), and the fact that its post-space program thrusts – missile defense (Defender) and nuclear test detection (Vela), as well as AGILE – had essentially run their respective courses. Indeed, in 1965, Deputy Secretary of Defense Cyrus Vance “came to advocate abolishing the agency.”⁴ However, DARPA evolved both organizationally and programmatically from this crisis under Director of Defense Research and Engineering (DDR&E) John Foster. By the mid-1970s, DARPA had jettisoned AGILE and transitioned Defender to the Army. DARPA sought new directions first under Director Eberhart Rechtin, who created a Strategic Technologies Office, and his successor Steven Lukasik, who closed AGILE down for the most part, though salvaging portions of it within a new Tactical Technology Office. Thus, by the mid-1970s, DARPA leadership had substantially restructured the agency around technology offices rather than mission-focused assignments.

DARPA was rejuvenated when it took on the job of seeking technological alternatives to nuclear weapons to respond to the Soviet Union’s force posture in Europe. This initiative was highly important to President Richard M. Nixon and his National Security Adviser Henry Kissinger, and continued under President Gerald R. Ford and his Secretary of Defense James Schlesinger. Under Lukasik, DARPA formulated a Long Range Research Planning Program in 1973 that guided DARPA research programs for the next 20 years. Among these were network-centric warfare, stealth technology, precision and autonomous systems, and theater missile defenses. In 1975, DARPA Director George Heilmeier – most famous for his “catechism” of questions he asked program managers to help determine the worthiness of the programs they proposed – scaled up major efforts in stealth aircraft, space-based lasers, space-based infrared technology, and artificial intelligence (AI).

DARPA in the 1980s – Transformative Technology Development and Transition

In the 1980s, DARPA developed and demonstrated new tactical capabilities through programs on stealth, standoff precision strike, and tactical surveillance via unmanned aerial vehicles (UAVs). Through persistent efforts, working with the DDR&E and the Secretary of Defense, DARPA worked to transition these capabilities to the military, creating what Under Secretary of Defense William Perry and Secretary of Defense Harold Brown (under President Jimmy Carter) would call the “Offset Strategy.” These DARPA programs are among its most important programs in terms of the agency’s impact on defense capabilities and are touted as DARPA’s impact in ushering in a “revolution in military affairs” – that offset Soviet military capabilities by transforming tactical warfare.⁵

Parallel to DARPA’s transformational military programs in the 1970s and 1980s were programs revolutionizing information technology, building on Licklider’s vision of “man-computer symbiosis.” DARPA’s research was foundational to computer science. ARPANET was one element of a much broader, increasingly coherent program based on the technological future that Licklider imagined. He and his IPTO colleagues conceived a multi-pronged development of the technologies underlying the transformation of information processing from clunky, room-filling, inaccessible mainframe machines to a ubiquitous network of interactive and personal computing capabilities.⁶ This transformation continues today in DARPA’s pursuit of artificial intelligence, cognitive (brain-like) computing, and robotics.

DARPA in the 1990s – The Cold War Ends

Early in the 1990s, DOD, and with it DARPA, adapted to the fact that the United States’ primary adversary, the USSR, had collapsed. With this momentous change in the world order, the focus of DARPA’s weapons research had disappeared. Moreover, the United States was in a budget crisis partly due to the vast defense spending of the 1980s. The Clinton administration employed the rubric “dual-use”– technologies that would have both defense and civilian economy payoffs – aiming to make the economy more competitive, while maintaining leadership in defense technologies. Under this approach, DOD sought to leverage the civilian sector to cut costs to develop new technologies. These dual-use programs were a major redirection of DARPA (renamed ARPA in 1993 to emphasize the dual-use focus), a shift that became highly contentious with elements in Congress. The Technology Reinvestment Program (TRP) was created to promote collaborations amongst defense technology developers, commercial firms, and universities. Despite the lack of a peer adversary, the Office of the Secretary of Defense (OSD) and DARPA worked with the White House to continue DARPA’s exploration and development of “breakthrough” technologies. Programs in unmanned systems and precision-strike capabilities continued with Predator and Global Hawk UAVs, and the agency fostered new technologies in information and electronics, advanced sensing, and battlefield surveillance. Programs in biotechnology were started. DARPA capped off the 1990s by taking on a program with the Army for a radical approach to ground combat using networked robotics – the Future Combat System – which proved unsuccessful. It was overly ambitious, rushed into acquisition by the Army, and eventually cancelled by the Secretary of Defense.

DARPA in the 2000s – War on Terror

At the outset of the 2000s, DARPA continued embracing major technology themes of precision strike, networking of operations, advanced sensors, and unmanned systems. However, within months of Anthony “Tony” Tether becoming director, the terrorist attacks of Sept. 11, 2001, occurred and DARPA became enmeshed in the “War on Terror.” The Total Information Awareness (TIA) program became the most notable DARPA response. This was a highly controversial program, as the use of information technologies to identify possible terrorists and terrorist attacks raised issues of privacy, and it was terminated amidst this controversy. DARPA also developed fast-response programs to support combat needs in Iraq and Afghanistan, including tactical robots and sensor systems. Secure, robust, self-healing sensor and communications networks also became a priority. Two new offices – the Information Awareness Office and the Information Exploitation Office – were created to address the terrorist threat. The agency pursued an array of robotics programs and related efforts in human-machine interaction, such as exoskeleton technology and advanced prosthetics. During this period, DARPA also emphasized programs in artificial intelligence, cognitive computing, quantum computing, and autonomous systems. The “DARPA Challenges” for self-driving cars in 2004, 2005, and 2007 were highly visible demonstrations of autonomous capabilities and were successful in creating interest and incentivizing teams of researchers. There was a renewed emphasis on “assured use of space” illustrated by the ambitious Orbital Express program for robotic on-orbit refueling and reconfiguring of satellites.

DARPA in the 2010s – Technology for Security in a Globalized World

In the current decade, DARPA has recognized that technology has become globalized. In the past, the United States and DOD led in technology development; now global competitors pursue many of the very technologies that DARPA pioneered. With growing peer competition in the security arena, and terrorism an ongoing concern, DARPA’s mission of avoiding technological surprise and creating technological surprise for adversaries is more daunting and pressing than ever.

One area of increased DARPA focus is the life sciences. While DARPA had been conducting biotechnology research for some time, in 2014 it created a new Biological Technologies Office with one of its thrusts centered on synthetic biology, a form of genetic engineering that opens new opportunities for such applications as sensing hazardous compounds and the efficient bio-production of novel coatings, fuels, and drugs. Another driver for biological technologies was DOD’s own massive health care concerns as injured soldiers from two wars required new medical responses. DARPA’s flexibility enabled it to pursue a “convergence” approach combining biology with engineering, physical, and computational sciences to seek new kinds of therapies, including advanced prosthetic technologies.

In the information domain, DARPA is focused on artificial intelligence, cyber threats, and approaches for advancing microelectronics, including efforts in quantum computing and neuro-synaptic processors based on how the brain processes information. With a foundation on previous research in aeronautics and propulsion, DARPA is embarking on a major thrust in hypersonic systems. Meanwhile, growing cyber threats spurred several ambitious DARPA programs in cybersecurity. This review shows that DARPA’s technical and security foci have changed with the times, although its mission – ensuring technological leadership for the United States – has remained largely the same. To carry out this mission today, however, the agency must focus on creating and demonstrating breakthrough technologies for national security even as many highly capable players enter the technology competition and as technology disseminates globally more swiftly than ever.

DARPA’s Organization and Approach

To achieve its mission, DARPA has been a highly adaptive, responsive organization. The hallmark of DARPA is agility. DARPA is organized into technology offices, the number and specific roles of which have changed over time. Today DARPA has six technical offices:

• Biological Technologies Office (BTO)
• Defense Sciences Office (DSO)
• Information Innovation Office (I2O)
• Microsystems Technology Office (MTO)
• Strategic Technology Office (STO)
• Tactical Technology Office (TTO)

DARPA is a funding agency; it has no laboratories or research staff of its own. At the heart of DARPA are approximately 100 program managers (PMs) and the office directors, deputy office directors, and agency director and deputy director who supervise them. These are all government employees, most hired using special hiring authorities on a term basis – usually of three to five years. Importantly, none of these is permanent staff – all are, in essence, temporary.⁷ The agency’s budget is approximately $3 billion a year.

Program managers have great autonomy: they propose their programs, seek approval and funding from senior DARPA officials, write the funding solicitations, select the R&D performers, and supervise and assist these performers. A typical program will have specific technical objectives, a budget of tens of millions of dollars, and will last three to five years. Often an individual program will fund multiple R&D projects run by different performers so as to compare and test different technical ideas.

Each DARPA technology office also can fund small “seed” programs, which provide a way for program managers to generate and test new ideas. In recent years, each office also has run an annual “open” competition in which applicants can propose work in areas of technology not covered in the office’s current programs.

DARPA uses a “portfolio” approach: it funds a wide range of R&D programs and often funds multiple projects within a single program. Since the focus of DARPA is high-risk programs to achieve “high payoff” results, the agency and its program managers invest in a range of promising technologies to increase the chances of success while accepting the inherent risk that some research may not succeed.

How DARPA Transfers Its Technologies

As an advanced research organization, DARPA does not fund the construction of actual operational systems; it succeeds by developing and demonstrating “proof-of-concept” prototypes that others in defense and/or the commercial world can further develop, modify, and implement – a process that DARPA calls “technology transition.”⁸ DARPA succeeds when other organizations in government and the corporate world further develop, and then commercialize and buy the new potentially transformative technologies that it has fostered.⁹

DARPA also succeeds when its efforts lead to an understanding of whether something can be done, how well it can be done, or if it cannot be done. What features of the DARPA model and the overall national innovation system help technology transition?¹⁰

DARPA’s willingness to challenge incumbent technologies. DARPA is willing to challenge existing technologies and the organizations that produce and use them – it sees its job as changing people’s minds about what is possible. For example, it showed that a computer network using open standards could replace proprietary networking systems. It created and then, with support from the Office of the Secretary of Defense (OSD), pushed for the adoption of stealth, unmanned aerial systems, precision strike, and night vision. It uses conferences, prize competitions, “technology insertion projects” (demonstrations of new technology in actual military systems), and other techniques to demonstrate and publicize new technical capabilities.

A community of technology advocates. DARPA and its performers have created new technical communities. Besides helping DARPA undertake new research, investigators in these new communities also often become knowledgeable, enthusiastic advocates. Some of these experts work in government, some in universities, some for large firms, and some start new entrepreneurial companies. They share an overall vision of what can be done, and they often become “communities of change-state advocates” – people who are willing and able to change the technology world. This is one reason that DARPA has been so influential.

Close ties to DOD leaders. DARPA works on problems important to high- level leadership. The agency’s close ties to the Secretary of Defense and other senior officials not only help DARPA maintain its independence, but these officials also can become “champions” who want to further develop the technologies and see them transitioned and deployed. For example, senior DOD officials pushed the U.S. Air Force to adopt both stealth aircraft and unmanned aerial vehicles, which DARPA programs had developed and demonstrated.

Connection to technically sophisticated customers. Turning radical new technologies into new operational capabilities is usually risky, difficult, and expensive. DARPA and the Department of Defense deal with this in two ways. First, the DOD itself can turn new prototype technologies into actual products. The department’s senior leaders may want advanced technologies, and through its laboratories, contractors, and acquisition system, it can refine and buy these new products. However, “transitioning” new technologies from DARPA to the military services is often difficult because DARPA-developed capabilities are frequently “disruptive,” challenging the current ways of conducting operations. Successful transition into a military capability, therefore, is usually preceded by considerable time and effort by DARPA managers.

A second pathway for transitioning is when the agency works with the private sector in commercializing DARPA-demonstrated technologies. DARPA repeatedly has worked with the semiconductor industry to adopt advanced chip technologies, for example, silicon-on-insulator technology and Monolithic Microwave Integrated Circuit signal processing chips. The new commercial frontier of self-driving vehicles is another example of an industry adopting and building upon DARPA-funded research.

Sustaining the DARPA Vision

DARPA’s higher-risk, longer-term R&D agenda distinguishes it from other defense R&D organizations. Perhaps the most important effect of DARPA’s work is to change people’s minds as to what is possible. DARPA’s 60-year history reveals an institution driven by a constant imperative to create novel, high-payoff capabilities by pushing the frontiers of knowledge. DARPA has many of the same features as its research. DARPA began as an experiment aimed at overcoming the usual incremental processes of technology development. Like the research it is chartered to develop, DARPA consistently has been purposively “disruptive” and “transformational.” Over the decades, there have been various efforts to tone down DARPA, make its research more compatible and integrated into the rest of DOD R&D, and have it focus more heavily on nearer-term, more incremental applications – that is, to shift its focus away from disruptive possibilities. Also, there have been efforts to broaden its charter into prototyping systems beyond the proof-of-concept demonstrations DARPA traditionally has carried out. However, with strong internal leadership, both within DARPA and in the OSD, as well as with support from Congress, DARPA has been able to perform a truly unique role for six decades. It has been and continues to be DOD’s “Chief Innovation Agency,” pushing the frontiers of what is possible for the benefit of national security and the nation.

DARPA remains an impressive “opportunity farm.” For example, DARPA helped move AI from an inchoate notion with almost no technological underpinnings into pervasive capabilities affecting our everyday lives and supporting real-time military operational decision-making. It is now pursuing similar advances in electronics and autonomous technologies. It is pursuing fundamental advances in materials, such as biomaterials, and accelerated materials development. The agency has revolutionized the realm of distributed sensing. Among many current DARPA research topics that populate the opportunity farm are heterogeneous electronics, engineering biology, agile access to space, and hypersonic systems.

Looking to the future, the question is not whether DARPA can still pursue new change-state prospects. The question is this: How can DARPA and the Department of Defense identify and focus on what these should be in the changing geopolitical and technological environments? DARPA has been adroit in addressing emerging technological prospects – but in today’s world, it has to be even more focused on where it can have leverage as others are investing, often massively, in the very technologies that DARPA initially championed.

With global investments in robotics, AI, synthetic biology, quantum computing, and advanced materials, on what should DARPA focus? From a military applications perspective, what should DARPA do to harness and promote the potential use of such emerging technologies into defense uses? To what needs, as opposed to today’s defined requirements, should DARPA seek to employ these technologies? Crucially, how should DOD go about achieving the fruition of these efforts?

Today DARPA faces new challenges that raise a key issue concerning its future success – the ability to draw upon extraordinary technical talent for program managers. The commercial high-tech sector, particularly in such areas as information technology, autonomous systems, advanced biology – areas in which DARPA is focused – is aggressively spending vast sums and hiring the very best. These firms attract this talent with high salaries and relatively unfettered work environments in locations far from Washington, D.C. Foreign nationals make up a growing proportion of the hires. Moreover, many leading tech companies are now outside of the United States, in Asia and Europe. Thus, there is greater competition for technical talent and greater competition worldwide in advanced technologies. Even as DARPA now must confront a tougher recruiting context than it has had to in the past, it still presents prospective program managers unique opportunities to affect the future that few other organizations can offer. These talent dynamics are crucial to understanding today’s DARPA and its ongoing mission to identify, demonstrate, and develop the technologies of the future.

These questions emphasize a crucial point – DARPA does not succeed by itself. Its success resides in the opportunities it creates that others bring to fruition. Thus, its success must build upon the larger U.S. innovation infrastructure. That innovation ecosystem has changed fundamentally over the past 25 years. For DARPA research to be successful, it must eventually culminate in transition, whether in an operational military capability or a new field of technology that expands frontiers for decades. DARPA itself is not responsible for executing transitions, but it depends on effective transition paths being there. These paths need to be better understood, and other stakeholders, beyond DARPA, need to support the measures that foster transition – whether within DOD or within industry. Some worry that military transition mechanisms within the DOD have eroded. In the broader commercial economy, transition paths have become more uncertain and diffuse. For DARPA to continue to have transformative impacts, it must exist within an economic and policy environment that encourages implementation. These are critical technology policy concerns that the United States must address to ensure that DARPA can continue to deliver breakthrough technologies in the decades to come.

This article was originally published in DARPA: Defense Advanced Research Projects Agency 1958-2018

1. This article draws upon the author’s collaboration over the years with several others, especially William Bonvillian, Seymour Deitchman, Michael Lippitz, Sidney Reed and Patrick Windham, as well as numerous discussions with many DARPA and Department of Defense officials.

2. The key features of the “DARPA model” were laid out in Richard Van Atta, “Fifty Years of Innovation and Discovery,” DARPA: 50 Years of Bridging the Gap, DARPA 2008, the introduction to DARPA’s 50th-anniversary publication.

3. ARPA was formed by Secretary McElroy on Feb. 7, 1958, through DOD Directive 5105.15. Congress, through an amendment by Senator Mike Mansfield, renamed “ARPA” as “DARPA” in 1972, adding the word “Defense.” Congress, through Senator Jeff Bingaman, renamed it “ARPA” again in February 1993 because of its “dual-use” role in creating technologies with commercial as well as military applications. The name reverted to “DARPA” in March 1996.

4. Richard J. Barber Associates, Inc., The Advanced Research Projects Agency, 1958-1974, December 1975. p. VII-3.

5. Richard H. Van Atta and Michael J. Lippitz, with Jasper C. Lupo, Rob Mahoney, and Jack H. Nunn, “Transformation and Transition: DARPA’s Role in Fostering an Emerging Revolution in Military Affairs, Volume 1 – Overall Assessment,” Alexandria, Virginia: Institute for DefenseAnalyses, IDA Paper P-3698, April 2003.

6. This transformation is detailed in M. Mitchell Waldrop, The Dream Machine, New York: Viking 2001.

7. The agency also has approximately 100 other government employees who provide important services, such as contracting, legal services, human resources, and security, and at any one time it also has several military liaisons. Additionally, contractors support these government employees. Some of these contractors are highly trained PhD scientists and engineers who provide valuable technical assistance to program managers, and others are support staff.

8. There was one significant exception. DARPA did develop operational technology for seismic detection of Soviet underground nuclear tests. DARPA was only able to transition this seismic detection network to the Air Force after running it for approximately 20 years.

9. Care should be taken to differentiate between transitioning such radical, “high-risk, high payoff” technologies and transitioning incremental improvements in existing technologies. DARPA explores many prospective advanced technologies and concepts and often its projects will demonstrate that many of these will not work given current knowledge and state-of-the-art. One of the most important aspects of DARPA management – from the program manager to the office director to the agency’s director – is determining whether the concept should be further pursued, and in what way, if a particular program is not bearing fruit. Another major management responsibility is determining how best to move projects into transition and use when they show promising results.

10. This section draws largely from Bonvillian and Van Atta, “ARPA-E and DARPA: Applying the DARPA Model to Energy Innovation,” published in 2011.