Memex

For the software company, see Memex Technology Limited.

The memex (originally coined "at random", though sometimes said to be a portmanteau of "memory" and "index") is the name of the hypothetical proto-hypertext system that Vannevar Bush described in his 1945 The Atlantic Monthly article "As We May Think". Bush envisioned the memex as a device in which individuals would compress and store all of their books, records, and communications, "mechanized so that it may be consulted with exceeding speed and flexibility". The memex would provide an "enlarged intimate supplement to one's memory". The concept of the memex influenced the development of early hypertext systems (eventually leading to the creation of the World Wide Web) and personal knowledge base software. The hypothetical implementation depicted by Bush for the purpose of concrete illustration was based upon a document bookmark list of static microfilm pages and lacked a true hypertext system, where parts of pages would have internal structure beyond the common textual format. Early electronic hypertext systems were thus inspired by memex rather than modeled directly upon it.

Details

A proto-hypertext system

In "As We May Think", Bush describes a memex as an electromechanical device enabling individuals to develop and read a large self-contained research library, create and follow associative trails of links and personal annotations, and recall these trails at any time to share them with other researchers. This device would closely mimic the associative processes of the human mind, but it would be gifted with permanent recollection. As Bush writes, "Thus science may implement the ways in which man produces, stores, and consults the record of the race".

The technology used would have been a combination of electromechanical controls, microfilm cameras and readers, all integrated into a large desk. Most of the microfilm library would have been contained within the desk, but the user could add or remove microfilm reels at will. A memex would hypothetically read and write content on these microfilm reels, using electric photocells to read coded symbols recorded next to individual microfilm frames while the reels spun at high speed, stopping on command. The coded symbols would enable the memex to index, search, and link content to create and follow associative trails.

The top of the desk would have slanting translucent screens on which material could be projected for convenient reading. The top of the memex would have a transparent platen. When a longhand note, photograph, memoranda, or other things were placed on the platen, the depression of a lever would cause the item to be photographed onto the next blank space in a section of the memex film.

The memex would become "'a sort of mechanized private file and library'. It would use microfilm storage, dry photography, and analog computing to give postwar scholars access to a huge, indexed repository of knowledge – any section of which could be called up with a few keystrokes."

The vision of the memex predates, and is credited as the inspiration for, the first practical hypertext systems of the 1960s. Bush describes the memex and other visions of As We May Think as projections of technology known in the 1930s and 1940s – in the spirit of Jules Verne's adventures, or Arthur C. Clarke's 1945 proposal to orbit geosynchronous satellites for global telecommunication. The memex proposed by Bush would create trails of links connecting sequences of microfilm frames, rather than links in the modern sense where a hyperlink connects a single word, phrase or picture within a document and a local or remote destination.

Associative trails

An associative trail as conceived by Bush would be a way to create a new linear sequence of microfilm frames across any arbitrary sequence of microfilm frames by creating a chained sequence of links in the way just described, along with personal comments and side trails. At the time Bush saw the current ways of indexing information as limiting and instead proposed a way to store information that was analogous to the mental association of the human brain: storing information with the capability of easy access at a later time using certain cues (in this case, a series of numbers as a code to retrieve data). The closest analogy with the modern Web browser would be to create a list of bookmarks to articles relevant to a topic, and then to have some mechanism for automatically scrolling through the articles (for example, use Google to search for a keyword, obtain a list of matches, repeatedly use the "open in new tab" feature of the Web browser, and then visit each tab sequentially). Modern hypertext systems with word and phrase-level linking offer more sophistication in connecting relevant information, but until the rise of wiki and other social software models, modern hypertext systems have rarely imitated Bush in providing individuals with the ability to create personal trails and share them with colleagues – or publish them widely.

Other features

The memex would have features other than linking. The user could record new information on microfilm, by taking photos from paper or from a touch-sensitive translucent screen. A user could "...insert a comment of his own, either linking it into the main trail or joining it by a side trail to a particular item. ...Thus he builds a trail of his interest through the maze of materials available to him." A user could also create a copy of an interesting trail (containing references and personal annotations) and "...pass it to his friend for insertion in his own memex, there to be linked into the more general trail." As observers like Tim Oren have pointed out, the memex could be considered to be a microfilm-based precursor to the personal computer. The September 10, 1945, Life magazine article showed the first illustrations of what the memex desk could look like, as well as illustrations of a head-mounted camera, which a scientist could wear while doing experiments, and a typewriter capable of voice recognition and of reading text by speech synthesis. Considered together, these memex machines were probably the earliest practical description of what we would call today the Office of the future.

"Given a memex, a scholar could create her own knowledge tools as connections within reams of information, share these tools, and use complexes of tools to create yet more sophisticated knowledge that could in turn be deployed toward this work. The memex has been envisioned as a means of turning an information explosion into a knowledge explosion. This remains one of the defining dreams of new media."

Extending, storing, and consulting the record of the species

Bush's idea for the memex extended far beyond a mechanism which might augment the research of one individual working in isolation. In Bush's idea, the ability to connect, annotate, and share both published works and personal trails would profoundly change the process by which the "world's record" is created and used:

Wholly new forms of encyclopedias will appear, ready made with a mesh of associative trails running through them, ready to be dropped into the memex and there amplified. The lawyer has at his touch the associated opinions and decisions of his whole experience, and of the experience of friends and authorities. The patent attorney has on call the millions of issued patents, with familiar trails to every point of his client's interest. The physician, puzzled by a patient's reactions, strikes the trail established in studying an earlier similar case, and runs rapidly through analogous case histories, with side references to the classics for the pertinent anatomy and histology. ...

The historian, with a vast chronological account of a people, parallels it with a skip trail which stops only on the salient items, and can follow at any time contemporary trails which lead him all over civilization at a particular epoch. There is a new profession of trail blazers, those who find delight in the task of establishing useful trails through the enormous mass of the common record. The inheritance from the master becomes, not only his additions to the world's record, but for his disciples the entire scaffolding by which they were erected. — As We May Think

Bush states that "technical difficulties of all sorts have been ignored," but that, "also ignored are means as yet unknown which may come any day to accelerate technical progress as violently as did the advent of the thermionic tube." Indeed, anyone who stops to consider the performance consequences of trail following – let alone link-directed pointer chasing – over a microfilm library of near universal scope should quickly come to the conclusion that microfilm is no more appropriate a technology for implementing AWMT's vision than Jules Verne's cannon is an appropriate technology for sending astronauts to the Moon. In both cases the vision may be more significant than the specific technology used to describe it. See Michael Buckland's conclusion: "Bush's contributions in this area were twofold: (i) A significant engineering achievement by the team under his leadership in building a truly rapid prototype microfilm selector, and (ii) a speculative article, 'As We May Think,' which, through its skillful writing and the social prestige of its author, has had an immediate and lasting effect in stimulating others."

In "Memex: Getting Back on the Trail", Tim Oren argues that Bush's original vision expressed in AWMT describes a "...private device into which public encyclopedias and colleague's trails might be inserted to be joined with the owner's own work."

However, in Bush's manuscript draft of "Memex II" of 1959, Bush says, "Professional societies will no longer print papers..." and states that individuals will either order sets of papers to come on tape – complete with photographs and diagrams – or download 'facsimiles' by telephone. Each society would maintain a 'master memex' containing all papers, references, tables "intimately interconnected by trails, so that one may follow a detailed matter from paper to paper, going back through the classics, recording criticism in the margins."

Missing features: search and metadata

The AWMT paper did not describe any automatic search, nor any universal metadata scheme such as a standard library classification or a hypertext element set like the Dublin core. Instead, when the user made an entry, such as a new or annotated manuscript, typescript or image, he was expected to index and describe it in his personal code book. By consulting his code book, the user could retrace annotated and generated entries.

Between 1990 and 1994, Paul Flaherty, a Stanford student who was looking for a project, was introduced by his wife to her supervisor. The supervisor had just seen a demonstration of the World Wide Web and suggested it could be improved and better conformed to the memex described by Vannevar Bush if links did not have to be manually inserted and instead one could follow a link simply by using the words themselves. Flaherty went on to create AltaVista, the first searchable, full-text database of a large part of the Web.

By 1999, many companies had created web annotation systems, where web site publishers or users could annotate web pages. Sentius Corporation, for one, developed technology that would automatically insert hyperlinks into web or text documents, from a dictionary of terms. This was used in particular to display Japanese translations of English medical terms when the mouse hovered over a term, while maintaining the look of a standard text document normally, and then extended for other purposes.

Legacy

This idea directly influenced computer pioneers J.C.R. Licklider (see his 1960 paper Man-Computer Symbiosis), Douglas Engelbart (see his 1962 report Augmenting Human Intellect), and also led to Ted Nelson's groundbreaking work in concepts of hypermedia and hypertext.

As We May Think also predicted many kinds of technology invented after its publication in addition to hypertext such as personal computers, the Internet, the World Wide Web, speech recognition, and CD-ROM encyclopedias such as Encarta and online encyclopedias such as Wikipedia: "Wholly new forms of encyclopedias will appear, ready-made with a mesh of associative trails running through them, ready to be dropped into the memex and there amplified."

Bush's influence is still evident in research laboratories of today in Gordon Bell's MyLifeBits (from Microsoft Research), which implements path-based systems reminiscent of the Memex, is especially impactful in the areas of information retrieval and information science.

A fictional implementation of the memex appears in The Laundry Files series by Charles Stross.

A high-performance computing cluster (HPC) at the Carnegie Institution for Science is named "Memex".

DARPA's Memex program

In early 2014, the Defense Advanced Research Projects Agency (DARPA) released a statement on their website outlining the preliminary details of the "Memex program", which aims at developing new search technologies overcoming some limitations of text-based search. DARPA wants the Memex technology developed in this research to be usable for search engines that can search for information on the Deep Web – the part of the Internet that is largely unreachable by commercial search engines like Google or Yahoo. DARPA's website describes that "The goal is to invent better methods for interacting with and sharing information, so users can quickly and thoroughly organize and search subsets of information relevant to their individual interests". As reported in a 2015 Wired article, the search technology being developed in the Memex program "aims to shine a light on the dark web and uncover patterns and relationships in online data to help law enforcement and others track illegal activity". DARPA intends for the program to replace the centralized procedures used by commercial search engines, stating that the "creation of a new domain-specific indexing and search paradigm will provide mechanisms for improved content discovery, information extraction, information retrieval, user collaboration, and extension of current search capabilities to the deep web, the dark web, and nontraditional (e.g. multimedia) content". In their description of the program, DARPA explains the program's name as a tribute to Bush's original Memex invention, which served as an inspiration.

In April 2015, it was announced parts of Memex would be open sourced. Modules were available for download.

Memex revisited

In 1967, Vannevar Bush published a retrospective article entitled "Memex Revisited" in his book Science Is Not Enough. Published 22 years after his initial conception of the Memex, Bush details the various technological advancements that have made his vision a possibility. Specifically, Bush cites photocells, transistors, cathode ray tubes, magnetic and video tape, "high-speed electric circuits", and "miniaturization of solid-state devices" such as the TV and radio. The article claims that magnetic tape would be central to the creation of a modern Memex device. The erasable quality of the tape is of special significance, as this would allow for modification of information stored in the proposed Memex.

In the article, Bush stresses the continued importance of supplementing "how creative men think" and relates that the systems for indexing data are still insufficient and rely too much on linear pathways rather than the association-based system of the human brain. Bush writes that a machine with the "speed and flexibility" of the brain is not attainable, but improvements could be made in regard to the capacity to obtain informational "permanence and clarity".

Bush also relates that, unlike digital technology, Memex would be of no significant aid to business or profitable ventures, and as a consequence its development would occur only long after the mechanization of libraries and the introduction of what he describes as the specialized "group machine", which would be useful for the sharing of ideas in fields such as medicine. Furthermore, although Bush discusses the compressional ability and rapidity so key to modern machines, he relates that speed will not be an integral part of Memex, stating that a tenth of a second would be an acceptable interval for its data retrieval, rather than the billionths of a second that modern computers are capable of. "For Memex," he writes, "the problem is not swift access, but selective access". Bush states that although the code-reading and potential linking capabilities of the rapid selector would be key to the creation of Memex, there is still an issue of enabling "moderately rapid access to really large memory storage". There is an issue concerning selection, Bush conveys, and despite the fact that improvements have been made in the speed of digital selection, according to Bush, "selection, in the broad sense, is still a stone adze in the hands of the cabinetmaker". Bush goes on to discuss the record-making process and how Memex could incorporate systems of voice-control and user-propagated learning. He proposes a machine that could respond to "simple remarks" as well as build trails based on its user's "habits of association," as Belinda Barnet described them in "The Technical Evolution of Vannevar Bush's Memex." Barnet also makes the distinction between the idea of a constructive Memex and the "permanent trails" described in As We May Think, and attributes Bush's machine learning concepts to Claude Shannon's mechanical mouse and work with "feedback and machine learning".

See also

People

• Andries van Dam
• Douglas Engelbart
• J. C. R. Licklider
• Paul Otlet, considered one of the fathers of information science
• Ted Nelson
• Tim Berners-Lee, creator of the World Wide Web
• Vannevar Bush

Ideas

• As We May Think
• Intelligence amplification
• Mundaneum, the organization created in 1910 that aimed to classify all knowledge
• Office of the future
• Victorian Internet, term to describe 19th century telecommunications technologies
• World Wide Web

Bibliography

• Bush, Vannevar (Jul 1945), "As We May Think", The Atlantic Monthly, 176 (1): 101–8.
• Bush, Vannevar (1967), "Memex Revisited" (PDF), Science is Not Enough.
• Wardrip-Fruin, Noah; Montfort, Nick, eds. (2003), The New Media Reader, MIT Press, ISBN 978-0-262-23227-2.

External links

• "As We May Think - A Celebration of Vannevar Bush's 1945 Vision", at Brown University
• Vannevar Bush and Memex – Living Internet