History of Tcl

People often ask me how I got the idea for Tcl and why it became popular. In this page I've attempted to reconstruct the thought processes that led to Tcl and Tk and trace the evolution of Tcl and Tk from the first idea in 1987 through its development at Berkeley, Sun, and Scriptics to the formation of the Tcl Core Team in 2000. Tcl's beginnings were very modest; the success of the system was quite a surprise to me, and it's only in retrospect that I've begun to understand the reasons for Tcl's popularity.

The birth of Tcl

The Tcl scripting language grew out of my work on design tools for integrated circuits at the University of California at Berkeley in the early 1980's. My students and I had written several interactive tools for IC design, such as Magic and Crystal. Each tool needed to have a command language (in those days people tended to invoke tools by typing commands; graphical user interfaces weren't yet in widespread use). However, our primary interest was in the tools, not their command languages. Thus we didn't invest much effort in the command languages and the languages ended up being weak and quirky. Furthermore, the language for one tool couldn't be carried over to the next, so each tool ended up with a different bad command language. After a while this became tiresome and embarrassing.

In the fall of 1987, while on sabbatical at DEC's Western Research Laboratory, I got the idea of building an embeddable command language. The idea was to spend extra effort to create a good interpreted language, and furthermore to build it as a library package that could be reused in many different applications. The language interpreter would provide a set of relatively generic facilities, such as variables, control structures, and procedures. Each application that used the language would add its own features into the language as extensions, so that the language could be used to control the application. The name Tcl (Tool Command Language) derived from this intended usage.

The notion of embeddability is one of the most unique aspects of Tcl, and it led me to the following three overall goals for the language:

• The language must be extensible: it must be very easy for each application to add its own features to the basic features of the language, and the application-specific features should appear natural, as if they had been designed into the language from the start.
• The language must be very simple and generic, so that it can work easily with many different applications and so that it doesn't restrict the features that applications can provide.
• Since most of the interesting functionality will come from the application, the primary purpose of the language is to integrate or "glue together" the extensions. Thus the language must have good facilities for integration.

I started work on Tcl when I returned from my sabbatical early in 1988, and I began using the first version of Tcl in a graphical text editor in the spring of 1988. By this time I had stopped working on IC design tools in my research, so the experiments with Tcl were mostly academic: I didn't think that anyone besides myself would be interested in an embeddable command language.

The birth of Tk

One of my other interests at that time was graphical user interfaces. As GUIs became more and more popular in the 1980s I had noticed that the complexity of interactive software was rising rapidly. The most interesting new developments seemed to require large projects with enormous investments. As a professor with modest resources, this worried me: would it become impossible for small teams like my research groups to build innovative interactive systems?

I concluded that the only hope for us was to reduce the resource requirements by building large systems out of reusable components. If most of the complexity of a system was in the components, and if we could carry the components forward from system to system, perhaps we could build large powerful systems with small teams. It would take quite a bit of work to develop the components, but this could be done gradually over time, perhaps by several smaller groups working together.

I also reasoned that component-based design would not work unless there was a powerful and flexible mechanism for integrating the components. These thoughts occurred shortly after I had begun thinking about Tcl, and it occurred to me that an embeddable command language such as Tcl might also be useful as a scripting language for integrating components. I decided to test this theory by creating a set of GUI components as a Tcl extension and using Tcl to assemble the components into graphical user interfaces. This extension became Tk.

I began work on Tk in late 1988, but it was a part-time project so it took about two years before there was enough functionality for Tk to be useful.

Early distributions

I don't remember exactly when I began giving Tcl away to other people. By 1989 I was using Tcl in a couple of projects, and I believe that I gave a talk on Tcl during the Berkeley Industrial Liaison Conference, where several hundred people from industry came to Berkeley to hear about research in our department. I believe that this led to invitations to give talks on Tcl at a few interested companies, and that some of the people at those talks asked if they could have copies of Tcl. In any case, I gave away a few copies of Tcl to early adopters in 1989.

In January of 1990 I presented a paper on Tcl at the USENIX Conference. The talk was attended by several hundred people and generated enough interest that many people began asking me for copies of Tcl. Around this time I decided to make the Tcl source code freely available on our FTP site at Berkeley. Once that happened, Tcl usage began to spread by word of mouth over the Internet.

One of the people at the USENIX presentation was Don Libes of the National Institute of Standards and Technology. Don had been wanting for some time to build a program to automate interactive Unix applications, but the project had stalled because it needed a good command language and Don didn't have the time to build one. After hearing my talk, Don rushed home, downloaded Tcl, built a wonderful application called Expect, and wrote a paper on it in time for the submission deadline for the next USENIX Conference. This whole process took just three weeks. Don made Expect freely available after presenting his paper in the summer of 1990, and it became an instant hit among system administrators. Expect was not only the first widely distributed Tcl application, but for many years it was the most popular. Many of the early Tcl users learned about it by using Expect.

The basic features of Tk became usable late in 1990, and I presented a paper on Tk at both the USENIX Conference and the X Conference in January 1991. Within a few weeks of these conferences I made the first Internet release of Tk. The first version of Tk lacked several important widgets (e.g., it had no multi-line text widget), but by the summer of 1992 I had added both a text widget and a powerful canvas widget. At this point usage of Tk began to increase dramatically.

Growing popularity

Usage of Tcl and Tk grew roughly an order of magnitude each year in the early 1990s, from a few users in 1989 to several tens of thousands in 1993. There were two primary reasons for this growth.

The first reason for the growth in Tcl/Tk usage was that it provided the easiest way to create graphical user interfaces under Unix (in those days Tcl only ran on Unix systems). Tk's GUI facilities were both very simple and very powerful; interesting applications could be written entirely in Tcl, without writing any C code at all. C-based alternatives such as the Motif toolkit were much more complicated, yet provided less functionality. It quickly became clear that GUIs could be created with 5-10x less effort in Tcl than Motif. It also turned out that many people in industry were just as concerned as I was about the high cost of developing interactive applications.

The second reason for the popularity of Tcl was its embeddable nature. When I started on Tcl I assumed that no-one besides myself would care about an embeddable command language, but I was wrong. Many people, including Don Libes and others, wanted to build applications that were scriptable and extensible but didn't have the time to build a language of their own. Because it was embeddable, Tcl provided a perfect solution for these people. As a result, Tcl ended up being used in applications ranging from stock trading to scientific visualization to manufacturing automation.

Out of this dichotomy in usage grew a discussion that reoccurred every year or two in the Tcl user community. Many Tcl users asserted that Tk was the only reason they used Tcl. Some of them didn't even like the Tcl language and wished that Tk could be mated with a different scripting language (in fact, this has happened several times, including TkPerl and Python). A different group argued that Tcl was all they cared about; they used it as an embedded scripting language, often without Tk, and liked the language's properties for this purpose. In a sense both groups were right.

The Tcl community

As Tcl usage grew in the early 1990s, a community formed around the language, contributing many powerful extensions and providing support for new users. Two of the earliest pioneers were Mark Diekhans and Karl Lehenbauer. They wrote TclX, which was one of the first freely available Tcl extensions. TclX provided file access, time and date manipulation, and many other useful facilities. Over time the features of TclX have proven so important that many of them have been incorporated into the base Tcl releases.

Many other Tcl pioneers wrote high-quality extensions and made them freely available. For example, Michael McLennan wrote [incr Tcl], an extension that provides object-oriented programming features in Tcl. George Howlett wrote BLT, which added several important widgets to Tk. Tom Poindexter wrote Sybtcl and Oratcl, which provided access to popular databases. Brian Smith and Larry Rowe wrote TclDP, which provided socket access and other distributed programming features. There are others as well; apologies to the many worthy contributors whom I have failed to recognize.

Other forms of community developed in this same period. I initially created a mailing list for communication among Tcl users, but within a few years it became so large as to be unwieldy. With help from the Tcl community the comp.lang.tcl newsgroup was created as a replacement for the mailing list; it provided a more effective forum for exchanging information and discussing new Tcl features.

In 1993 Larry Rowe recognized the need for people in the community to meet in person, so he organized the first Tcl Workshop at Berkeley that summer. About 60 people attended, and the workshop was so successful that it became an annual event. In 1995 the USENIX Association offered to help run the workshop, which it did for several years; the workshop quickly evolved into a full-fledged conference with several hundred attendees and a rich program of papers, tutorials, and other events. The Tcl Conference also became one of the world's premier events for upper body wear: it was not unusual to see 5-10 different T-shirt designs given away each year to recognize various extensions, events, etc.

During this period I made new releases of Tcl and Tk roughly every 6-12 months. Most of the new features were suggested by people in the Tcl community; in some cases users even supplied me with prototype code for the new features, which I then integrated into the core software. Typically each new release cycle would start off with a discussion period where I proposed a candidate list of new features and everyone in the community could vote for the ones they thought were most important. I tried to give priority to the features with the most votes.

Compared to other open-source projects, Tcl was more tightly controlled than most. I wanted to preserve cleanliness in both the architecture and implementation of Tcl, so I was fairly picky about which extensions I included in the core and I tended to do massive rewriting of contributed code before releasing it in the core. Unfortunately, this slowed down the accretion of functionality in Tcl: people could always distribute their extensions separately, but it was harder for other people to use such extensions and they didn't have an official stamp of approval. In retrospect, I should have been much more promiscuous about bundling things into the core releases, even though it would have violated my principles of cleanliness.

Ouster-votes

Over time the Tcl release cycle tended to align with the Tcl Conference. A new release would typically come out around the time of the conference, and I would use one session of the conference to discuss new features for the next release and get feedback. I would often call for an impromptu show of hands to see how the audience felt about various issues, and then attempt to make a rough count of how many hands were raised ("ten, twenty, thirty, ... looks like about 35 hands to me"). These became known as "Ouster-votes" and their accuracy has often been a topic of discussion. Some people have insinuated that my vote counting was less than totally objective....

The Sun years

In 1994 I decided to leave Berkeley for industry. I'd always wanted to spend part of my career in industry, and after 14 years in academia the time seemed right. After considering a variety of possibilities, I decided to accept an offer from Sun Microsystems. Eric Schmidt, Sun's Chief Technology Officer, and Bert Sutherland, Director of Sun Microsystems Laboratories, offered me the chance to build a team at Sun Labs and evolve Tcl into a universal scripting language for the Internet. I had always felt that Tcl would eventually need to have profitable businesses built around it in order to survive over the long term, and the Sun offer was a great opportunity to take Tcl to the next level.

I joined Sun in May of 1994 and began building a team of Tcl developers. Over the next three years the Sun Tcl team grew to a dozen members. Up until this point nearly every line of code in Tcl and Tk had been written by me personally, but over time more and more of the responsibility for Tcl code shifted to other members of the Sun Tcl team.

The additional resources provided by Sun allowed us to make major improvements to Tcl and Tk. Scott Stanton and Ray Johnson ported Tcl and Tk to Windows and the Macintosh, so that Tcl became an outstanding cross-platform development environment; Windows quickly came to be the most common platform. Jacob Levy and Scott Stanton overhauled the I/O system and added socket support, so that Tcl could easily be used for a variety of network applications. Brian Lewis built a bytecode compiler for Tcl scripts, which provided speedups of as much as a factor of 10x. Jacob Levy implemented Safe-Tcl, a powerful security model that allows untrusted scripts to be evaluated safely. Jacob Levy and Laurent Demailly built a Tcl plugin, so that Tcl scripts can be evaluated in a Web browser, and we created Jacl and TclBlend, which allow Tcl and Java to work closely together. We added many other smaller improvements, such as dynamic loading, namespaces, time and date support, binary I/O, additional file manipulation commands, and an improved font mechanism.

When I moved to Sun many people were worried that Tcl would become a proprietary language, and there were several "spirited discussions" on the comp.lang.tcl newsgroup. Fortunately, as part of my offer Sun had agreed that the core Tcl and Tk libraries would continue to be distributed freely in source form, and Sun honored this agreement faithfully. As new releases appeared with more and more improvements, it became clear that Sun's support for Tcl was a good thing that allowed the language to evolve much more rapidly than it could have otherwise. The concerns were gradually replaced with enthusiasm for all the new features.

During these years Tcl usage continued to explode. By the end of 1997 there were hundreds of thousands of Tcl developers; the rate of Tcl downloads from the Sun FTP site increased from 2000 per week at the end of 1995 to more than 10000 per week by early 1998. I continued to believe that there were interesting business opportunities around Tcl. It was also clear that the Tcl community had many needs that weren't being met by the open-source Tcl development. For example, there were almost no development tools for Tcl, and professional services such as training and support were only sporadically available. Many important extensions, such as ActiveX and CORBA support, still needed to be written.

At the same time, it became clear that Sun needed to focus its language evangelism around Java, which was released shortly after I arrived at Sun and had become hugely popular. Though Java and Tcl are very different languages, used for very different purposes, it would have been too hard for Sun to evangelize both of them simultaneously. I came to the conclusion that, overall, Java offered more benefits for Sun than Tcl did.

Scriptics

In late 1997 I decided that the best thing for Tcl was to spin out of Sun and form a startup that could focus entirely on Tcl. In January 1998 I founded Scriptics with Sarah Daniels, who became VP of Marketing and Sales. Within a month about half of the Sun Tcl team had joined us and begun development of TclPro, a set of development tools that would comprise our first product. TclPro 1.0 shipped in September of 1998.

Scriptics had also taken over from Sun the role of developing the Tcl core facilities, and it continued to release them freely just like Sun did. In 1998, Scriptics made several patch releases for Tcl 8.0 to fix bugs and add small new features, such as a better support for the [incr Tcl] extension. In April 1999, Scriptics made its first major open source release, Tcl/Tk 8.1. This release added Unicode support (for internationalization), thread safety (for multi-threaded server applications), and an all-new regular expression package by Henry Spencer that included many new features as well as Unicode support.

Tcl: award-winning software

In the spring of 1998 I learned the wonderful news that Tcl had won two major awards. The first is the ACM Software System Award, awarded each year for "a software system that has had a lasting influence". Past winners of the award include such seminal systems as the TCP/IP protocols, the first spreadsheets, the first relational databases, the World-Wide Web, Unix, PostScript, and Smalltalk. The second award was the USENIX Software Tools User Group (STUG) Award, given each year in recognition of an outstanding software tool.

Formation of the Tcl Core Team

The company strategy for Scriptics was a two-phase one. In the first phase the company would create development tools for Tcl. In the second phase, the company would develop interesting applications based on Tcl. In 1999, we decided that the application would be an XML-based business-to-business server (a hot area at the time). The first version of this product became available in early 2000. As the year unfolded it became clear that the dot-com bubble was about to end. Furthermore, our B2B server was younger and less mature than its competition and the B2B market had begun to consolidate with mergers. By mid-year we decided that it would be best to merge the company (which had been renamed Ajuba Solutions by then) into a larger company. Interwoven acquired the Ajuba Solutions in October of 2000.

Interwoven was not interested in Tcl (it purchased Ajuba Solutions for its server and XML technology and for its engineering team), so it did not continue supporting the open-source versions of Tcl. I had guessed that this was likely to happen, so during the period of July-October 2000 we transitioned the "ownership" of core development away from a single organization, and towards a broader group of open source developers. The Tcl Core Team was formed, and established its basic operating procedures.

In the years since then Tcl and Tk have continued to improve and evolve under the auspices of the Tcl Core Team. Jeff Hobbs, formerly the Tcl Ambassador at Scriptics/Ajuba, moved to ActiveState (already known for their Perl products and development) to head their Tcl technologies group. He has provided leadership for the Tcl Core Team, though many other people have also made important contributions.

Developments since 2000

After the formation of the Tcl Core Team I decided it was time for me to move on to new things, so I have not had an active role in Tcl's development since 2000. However, Tcl and Tck still seem to be widely used; in early 2008, Jeff Hobbs reported that downloads were running at 15000-30000 per month, which is comparable to Tcl's heyday in the late 1990's.

The rise of the Web has reduced the need for Tk because many GUI applications came to be delivered over the Web using HTML forms and, more recently, dynamic technologies such as AJAX. At the same time, Tcl and Tk still provide a higher level of interactivity than the Web can offer, and they also represent a truly cross-platform environment for creating GUI applications. Tcl continues to be heavily used as an embedded scripting language, a domain where it still seems to be the best alternative available. For example, it has become the industry standard scripting language for Electronic Design Automation tools.