Discover millions of ebooks, audiobooks, and so much more with a free trial

Only $11.99/month after trial. Cancel anytime.

No Code Required: Giving Users Tools to Transform the Web
No Code Required: Giving Users Tools to Transform the Web
No Code Required: Giving Users Tools to Transform the Web
Ebook940 pages11 hours

No Code Required: Giving Users Tools to Transform the Web

Rating: 0 out of 5 stars

()

Read preview

About this ebook

No Code Required presents the various design, system architectures, research methodologies, and evaluation strategies that are used by end users programming on the Web. It also presents the tools that will allow users to participate in the creation of their own Web.

Comprised of seven parts, the book provides basic information about the field of end-user programming. Part 1 points out that the Firefox browser is one of the differentiating factors considered for end-user programming on the Web. Part 2 discusses the automation and customization of the Web. Part 3 covers the different approaches to proposing a specialized platform for creating a new Web browser. Part 4 discusses three systems that focus on the customized tools that will be used by the end users in exploring large amounts of data on the Web. Part 5 explains the role of natural language in the end-user programming systems. Part 6 provides an overview of the assumptions on the accessibility of the Web site owners of the Web content. Lastly, Part 7 offers the idea of the Web-active end user, an individual who is seeking new technologies.

  • The first book since Web 2.0 that covers the latest research, development, and systems emerging from HCI research labs on end user programming tools
  • Featuring contributions from the creators of Adobe’s Zoetrope and Intel’s Mash Maker, discussing test results, implementation, feedback, and ways forward in this booming area
LanguageEnglish
Release dateMay 21, 2010
ISBN9780123815422
No Code Required: Giving Users Tools to Transform the Web
Author

Allen Cypher

Allen has provided, substantial contributions in the research, design, and implementation of innovative end-user applications. His primary interest is in creating simple interfaces for complex tasks, including intelligent interfaces, customizable software, user interface design, software design, and instruction. Edited the book "Watch What I Do: Programming by Demonstration", which was published by MIT Press in 1993 (~3000 sold). He has created several successful end-user programming systems: Eager, Stagecast Creator, and CoScripter. Ph.D. Computer Science, Yale University, A.B. A.B. Mathematics, Princeton University.

Related to No Code Required

Related ebooks

Internet & Web For You

View More

Related articles

Reviews for No Code Required

Rating: 0 out of 5 stars
0 ratings

0 ratings0 reviews

What did you think?

Tap to rate

Review must be at least 10 words

    Book preview

    No Code Required - Allen Cypher

    Pennsylvania

    PART

    1

    Introduction

    1. End user programming on the Web

    2. Why we customize the Web

    We begin this book with two chapters that set the context for the field of end user programming since its inception in the 1980s, and look to the Web as the new focus of this growing research area. Both chapters point to the Firefox browser and its extensibility model as one of the differentiating factors in end user programming on the Web as compared to the desktop. Firefox’s extensible platform combined with the simplicity and openness of HTML provide a consistent interface across many different tasks and applications, thereby enabling end user customization and automation in real settings. Cypher describes specific Web scenarios and the pain points that can be improved through end user programming, whereas Chilton et al. categorize existing browser extensions and the types of customizations that are available today.

    CHAPTER

    1

    End user programming on the Web

    Allen Cypher

    IBM Research – Almaden

    ABSTRACT

    This introduction explains who end users are and why they want to program. In the past 25 years, there have been two main approaches to enable end users to create their own programs: scripting languages and programming by demonstration. After outlining the challenges that confront these approaches, we shall see how the Web has removed some of the most significant barriers, opening the way for the recent progress that is detailed in this book.

    THE ORIGINS OF END USER PROGRAMMING

    For as long as there have been computers to program, there have been attempts to make programming easier, less technical, and available to a broader audience. The term end user programming proposes that although most computer users do not know how to program, they would appreciate having some of the power of programming, if only it could be obtained with little effort.

    Back in the 1960s, using a computer meant programming a computer. There was no need for the concept of end user programming because all end users were programmers. By the 1980s, this was beginning to change. I had a friend who – in 1980 – wrote her comparative literature thesis on punch cards. She was not a programmer, but the ability to add a sentence and not have to retype a chapter was revolutionary and compelling, and she was willing to spend some effort to get some of the capabilities that had previously been available only to programmers. Then the Macintosh came out in 1984 and changed everything. Soon, computers meant desktop computers, command languages were replaced by direct manipulation, and end users came into existence. These end users of computers had no desire to become programmers; they wanted to use computers to edit documents, create graphics, and, eventually, to communicate via email. Instead of programming, their goal was computer literacy, which meant knowing how to point-and-click in a word processor or spreadsheet.

    Nonetheless – as we shall see – it is inevitable that end users will encounter tasks that are painfully repetitive, or that require an inordinate number of actions to accomplish a simple goal. In these situations, end users could simplify their interaction with the computer, if only they could program. So, in the modern world of computer literacy, there is a need for end user programming.

    This chapter defines end user programming, summarizes the current state of the field, and shows how the Web has removed many of the obstacles to widespread adoption of end user programming.

    WHAT IS END USER PROGRAMMING?

    Who are end users?

    I consider the term end user to refer to the vast majority of personal computer users whose use of a computer consists of taking advantage of currently available software. The range of what they can do with a computer is determined by that software. The word end distinguishes this group from computer programmers, because programmers use computers to create tools and applications for others to use.

    End users have jobs in real estate, retail sales, car repair, and the performing arts. In their spare time, they are gardeners, cyclists, sports fans, and knitters. They use computers to communicate with their friends, to manage their photos, to find information, and to buy things.

    Why do end users want to program?

    Let’s begin with a variety of scenarios where end users could benefit from end user programming. These are situations in everyday computer use where it would be helpful if the computer could automate part of an activity. I will use real examples from my own experience, and I will briefly describe in more general terms the motivations for end user programming that these scenarios exemplify. Chapter 2, Why we customize the Web, offers a different perspective on why end users want to program.

    1) Forward phone to home

    I frequently work from home, so I like to forward my work phone to my home number. Fortunately, we have VOIP phones at work, so it is possible to use a Web site to forward the phone. The Web site offers many capabilities and many options. But I only use one of those capabilities, and I use it often. This activity requires nine actions, with a great deal of waiting because the Web site is slow. I would like to be able to click once and let my computer take care of forwarding the phone. See Figure 1.1.

    Motivation: Many capabilities and options. In general, Web site developers need to meet the needs of all of their users, so they need to include long lists of capabilities – such as Configure your Cisco Personal Address Book – that are useful for someone, but are just complicating clutter to others. And even a single activity, such as paying for a purchase, may require a variety of options to handle arbitrary user preferences, such as paying with Visa, MasterCard, or PayPal. Chapter 5, Collaborative scripting for the Web, describes a tool for automating activities with many actions.

    Motivation: Unique personal information. Even if there were a special Web page for forwarding one’s phone, Cisco is not going to know my home phone number. It takes a custom end user program to create Forward Allen Cypher’s work phone to his home number.

    2) Pay my monthly credit card bill

    Every month, I look over my credit card bill on the Web, and then schedule an automated payment for 3 days before the due date. This activity requires 15 actions. What I would really like is to click once to see the bill, and then when I’m ready, click one more time to complete the activity. See Figure 1.2.

    Motivation: Poorly designed applications and Web sites. Fifteen actions is an awful lot for paying a bill, and it’s likely that your credit card company has a better Web site. One reason end users create programs is to streamline and improve poorly designed applications and Web sites. It may happen that a software developer will eventually fix the problem, but it’s a problem for you now.

    3) Send text alerts

    I often forget to check my personal information manager for email, so I sometimes don’t see an important message until it’s too late. My iPhone makes an audible beep when a text alert arrives, so I would like to get a text alert whenever I receive an urgent email from a friend.

    FIGURE 1.1

    Forward phone to home.

    Motivation: Triggering automated responses. Chapter 7, Mixing the reactive with the personal, describes an end user programming system called Atomate that lets users automate a wide variety of tasks of the form Do A when situation B occurs. See Figure 1.6.

    4) Gather hotel information

    When I planned a trip to Florence and needed to find a hotel, I wanted to consult a variety of Web sites, read some reviews, select various hotels that seemed promising, and gather information about their location and price, a photo, and a sentence or two from a helpful review. A single button wouldn’t work in this case, but I would have appreciated assistance in the repetitive parts of this task, and I wish there had been a way to organize all of this information. See Figure 1.3.

    Motivation: Information gathering. Information gathering is such a prevalent and important use of the Web that it constitutes a new genre of activities that can be supported well by end user programming. Chapter 12, on Web summaries, addresses this genre directly.

    5) Add nutritional information

    Whenever I see a recipe on the Web, I wish the ingredients were accompanied by nutritional information. That information is available on other Web sites, but it requires of great deal of repetitive effort to compile the information. See Figure 1.4. Another end user might want to add price information from a local grocery store.

    FIGURE 1.2

    Pay my monthly credit card bill and the fifteen required actions.

    Motivation: Mashups. End users may have personal needs that are not addressed by a single application or Web site, so they may want to mash up multiple sites. Chapter 4, A goal-oriented Web browser, addresses this scenario directly, and mashups are considered in several other chapters as well.

    What is end user programming?

    As the examples above have illustrated, applications and Web sites are created to address specific needs and provide specific capabilities. But invariably, users find that in order to meet their own specific needs, they have to perform mundane, repetitive actions. Although it might be possible to meet these needs by writing a program, end users are not computer geeks, and one can’t rely on the fascination of programming to lure them into learning a traditional programming language.

    In short, end user programming is programming by people who are not professional programmers. The program is often something that only this one person wants. And because end users are engaged in performing real-world tasks, the solution often needs to be accomplished right away, while the user is performing the task.

    The challenge to researchers in this field is to find ways to make limited forms of programming sufficiently understandable and pleasant that end users will be willing and able to program. The goal is to bring programming to the rest of us, and advances in end user programming come when systems achieve an elegant balance between power and ease of use.

    The gray areas

    I would like to exclude solutions that are too limited in scope and power to be called programming. For instance, check boxes in a preference dialog can customize an application to better suit individual preferences, but they don’t qualify as end user programming.

    FIGURE 1.3

    Gather hotel information.

    FIGURE 1.4

    Add nutritional information.

    FIGURE 1.5

    Some typical spreadsheet formulas.

    I would also like to exclude approaches that are too hard and technical to be considered an advance in making programming accessible to nonprogrammers. This includes cases where the users who adopt the approach do not represent typical end users; for instance, they may be scientists or accountants. There is good work in this area: LabView (see http://www.ni.com/labview) is a successful tool that enables end users to wire together laboratory instrumentation.

    Another gray area is applications for end users that include programming as part of the application, to enable additional capabilities for advanced users. Adobe Photoshop includes a history mechanism that records each user action, and it is possible to group recorded actions into a macro for automation. Microsoft Word can be extended with the Visual Basic scripting language. But in neither of these cases is scripting essential to use of the application, nor is it a typical activity for the typical user. Perhaps one could call this amateur programming.

    The special case of spreadsheet programming

    Spreadsheet programmers are an important and unique subgroup of end user programmers; they are important because of their size – estimated at over 50 million people (Scaffidi, Shaw, & Myers, 2005) – and are unique because they are not averse to technical, mathematical solutions. Because spreadsheets are used to manipulate numeric data, spreadsheet programmers are comfortable with mathematical formulas and are not intimidated by abstraction. Figure 1.5 shows some typical spreadsheet formulas.

    Indeed, spreadsheet applications like Microsoft Excel can be considered to be programming environments, and the custom spreadsheets that users of Excel create are programs. So it is not that spreadsheet creators need occasional access to programming capabilities to enhance their interaction with the spreadsheet application; programming is the fundamental activity in spreadsheet creation.

    It is interesting to note that a whole continuum of spreadsheet users has evolved. Bonnie Nardi did a seminal study on the ecology of spreadsheet programming in her book, A Small Matter of Programming (Nardi, 1993), which was published in 1993. She describes the rich network of users that arises around spreadsheets, from expert programmers to individuals who only use spreadsheets created by others and do not do any formula writing themselves.

    Despite the phenomenal success of end user programming with spreadsheets, it is crucial not to assume that the solutions appropriate for this audience will also be appropriate for typical end users. I am continually dismayed by researchers who want to use spreadsheets as a paradigm for all classes of end user programmers.

    That said, there is significant research in the spreadsheet domain that is likely to be applicable to all types of end user programming. In particular, Margaret Burnett’s work on the Surprise-Explain-Reward strategy – as a technique for motivating end user programmers to test their spreadsheet programs – is likely to be effective for Web-based end user programming as well (Wilson et al., 2003).

    TWO APPROACHES TO END USER PROGRAMMING

    What is it about conventional programming languages that makes them daunting and unpalatable to ordinary end users? The answer is that the text you have to write to get something done is very remote from what is being done. To click a button, you have to write something like this:

       theEvent.initMouseEvent(mousedown, true, true, contentWindow, 1, (contentWindow.screenX + clickLocH), (contentWindow.screenY + clickLocV), clickLocH, clickLocV, ctrlP, false, false, false, 0, null)

    The syntax is obscure and unforgiving, and many of the details are abstract and only indirectly related to the simple action being performed. In short, traditional programming languages are obscure, abstract, and indirect. Don Norman (1986) has explained the analogous problem in user interfaces as a gulf of evaluation, and programming language researchers have referred to this issue as the closeness of mapping (Green & Petre, 1996).

    There have been two main approaches in end user programming to deal with the difficulties of programming languages: scripting and programming by demonstration. Of the 17 systems presented in this book, 9 use scripting and 11 use programming by demonstration. Each of these approaches offers novel means of overcoming the difficulties of traditional programming while retaining some of its power. There is a third approach – visual languages – but it has had limited success. One example is LabView, but it requires sophisticated users, and much of the actual programming for LabView is done by professional programmers outside of the visual tool. Another example is Stagecast Creator (Cypher & Smith, 1995), but it is intended specifically for children.

    Scripting languages

    Scripting languages approach end user programming by still using a programming language, but by making that language simpler and easier to use. To accomplish this, they may restrict their solutions to a limited domain – such as spreadsheets or Web pages – and offer only limited power within that domain.

    The syntax of scripting languages

    Although a programming language needs to be able to express arbitrary commands like the initMouseEvent command shown previously, a scripting language may only need to express a few actions, such as click and type; and a few objects, such as button and textbox. So instead of the myriad possibilities of commands using the obscure format of the initMouseEvent command, a scripting language can use simplified formats like click the button. The CoScripter system in Chapter 5 uses this simplified format, whereas the Chickenfoot system in Chapter 3 uses a hybrid of the simplified and complex formats.

    Scripting languages walk a fine line between power and ease of use. The formulas used in spreadsheets have opted for considerable power, at the expense of requiring significant effort to learn. For instance, a $ preceding a symbol in a formula means that the symbol will not be modified when the formula is moved to a different cell. Although it is true that it is easier to learn to write =SUM[A1:A12] than to create commands like the initMouseEvent shown previously, this is still a far cry from the simplicity required to turn most computer users into end user programmers. The Clip Connect Clone system in Chapter 8 uses spreadsheet formulas to connect and transform data from one Web page so that it can be used in another page, but this may limit its applicability, since many users of Web browsers are not spreadsheet programmers.

    Structure editors

    One approach that has helped make scripting languages easier is the structure editor, where the end user creates commands by selecting words from menus, and the editor guarantees that only legal combinations of words can be selected. Figure 1.6 shows the structure editor used in Atomate, which is described in Chapter 7.

    Programming by demonstration

    Instead of writing programs, the end user of a Programming by Demonstration (PbD) system demonstrates an activity, and the system writes a program to perform that activity. This simplifies program creation in three ways. First, it eliminates indirectness, because the user interacts directly with an application by clicking its buttons, typing into its boxes, and dragging its objects. There is no need to know that the button that was clicked is the dijit_form_ComboBox_0.downArrowNode. Second, it deals with abstractness by working with concrete instances of a program’s execution, rather than working directly with the abstractions. Third, it minimizes the problems of an obscure syntax by having the system write the commands for the user.

    The classic challenges that must be addressed in creating programs from user demonstrations are (1) inferring the user’s intent, and (2) presenting the created program to the user.

    Inferring intent

    Suppose I have a list of addresses that I want to add to my online address book (see Figure 1.7). After adding a few by hand, I wish I had a program that would finish this activity for me. In 1988, Witten and Mo’s (1993) TELS system used PbD to automate this kind of activity.

    Ideally, one would teach a PbD system just as one would teach another person: as you select the word John, you say "copy the first name and paste it into the First Name box in the Address Book. Actually, with a human assistant, you would probably just say copy this information into the address book." Both approaches rely on the fact that a human understands semantic concepts – people’s names, addresses, and phone numbers – and has enough experience with them to be able to identify those items in the text.

    FIGURE 1.6

    Atomate’s structure editor.

    FIGURE 1.7

    A list of unformatted addresses.

    Semantic knowledge

    The main reason this task is difficult for a PbD system is that the system doesn’t understand real-world semantic knowledge. This problem is actually not unique to PbD or even to end user programming; it is a challenge for all programmers. A professional programmer who wants to make a program that will take postal address information from a page of text and use it to fill in an address form on a Web page has to deal with exactly the same problem: how do you write a computer program that will figure out which part of the text is the first name, the last name, the street number, the street address, and so on?

    Witten and Mo’s system did what a programmer might do: it looked for patterns in the syntax, such as a series of digits, followed by a space, followed by a series of letters, and then a space followed by the letters Dr., or Rd., or Ave. that corresponded to the semantics.

    An end user might try to do the same thing with a scripting language; the huge improvement in ease of use that comes with a PbD system is that the system infers the syntactic pattern. Instead of writing a program, the end user just gives examples of what the system should do, and the PbD system writes the program.

    Semantic challenges come in a variety of forms. Consider a mashup that annotates a corporation’s list of approved hotels with the rating assigned by a travel Web site, such as TripAdvisor. (see Figure 1.8). A human can determine that Holiday Inn Philadelphia Historic Dst. is the same as Holiday Inn Philadelphia – Historic District, but this is problematic for a computer program. Once again, even professional programmers will find it challenging to handle this issue.

    Multiple interpretations

    When a user selects Bix in the example above, there are many syntactic inferences that the system could make about why that word was selected: because it is the second word, the first three-letter word, the first word that is followed by a comma and a space, or perhaps the second capitalized word. Examples such as Henry van der Zanden illustrate the complexity of the challenge.

    While generating syntactic rules for semantic concepts may account for many of the inferences that PbD systems need to make, there are plenty more that are not a matter of semantics. For example, consider the possible reasons why I might click on a link on a Web page. In order to do the right thing the next time I run my PbD-generated program on that Web site, it’s important to make the correct inference. When I visit my banking Web site, I always look at the charges for the second date in the list, since that is the most recently completed bill (see Figure 1.9a). Inferring from my demonstration that I always want to match the text February 3, 2009 would be wrong; I want to always select the second item. However, when I check the current traffic on http://cbs5.com, I always pick the information for South Bay. Sometimes this is the fourth item in the list, sometimes it’s the sixth, and sometimes it doesn’t appear at all (see Figure 1.9b). There is no way to know which inference is correct from observing a single demonstration.

    Presenting programs to the user

    As the examples in Figure 1.9 show, a PbD system can’t always make the correct inference from a single example. Sometimes, the best that PbD systems can do is to generate the reasonable alternatives and let the user pick the right one. A PbD system is nonetheless a great help to an end user, because recognizing the command for the correct inference is much easier than writing that command yourself. In order for users to choose an interpretation, PbD systems need to be able to present their inferences to the user. A representation of a program is important to users for other reasons as well. It may allow the user to edit the program, verify the program’s correctness, understand it, and trust that the program will do the right thing. Presenting programs to the user is therefore an important part of a PbD system. The problem of presenting abstractions to the user is still a major challenge for PbD systems and user interfaces in general, and it has not been simplified by the Web.

    FIGURE 1.8

    Determining that Holiday Inn Philadelphia Historic Dst. is the same as Holiday Inn Philadelphia – Historic District.

    HOW THE WEB HAS REMOVED BARRIERS TO END USER PROGRAMMING

    The most dramatic effect that the Web has had on end user programming is that it has removed several long-standing barriers that have kept end user programming from being useful in practice in the real world.

    Open access to applications

    The early work on end user programming was done at a time when desktop applications were distributed as proprietary source code that could not be modified, and user actions in the applications could not be recorded or even scripted. When Witten and Mo built their TELS system in 1988, they had to write their own text editor. This meant that no one was using TELS in their daily work. All testing had to be done in short, contrived experimental sessions, and regardless of the success and usefulness of their system, it would never be available for daily use.

    We are in a very different world today. Thanks to the open source movement, the Firefox browser – as a prime example – is freely and readily available to anyone with an Internet connection. What is most important for end user programming is that this browser has a complete system for extending its capabilities. This means that any researcher can now get full access to the internal workings of the browser and the Web pages it displays. There are millions of people around the world who use this browser for their activities on the Web, and more and more of their daily activities take place on the Web. Testing can now be ecologically valid – conducted on real tasks as their users are performing them – and if a system is successful, it can now be easily given to real users for real use. The potential for widespread adoption of end user programming technology is now a reality. As an indication of the importance of Firefox to the end user programming community, 10 of the 17 systems described in this book are implemented as Firefox extensions.

    FIGURE 1.9

    Match the ordinal or match the text? The correct inference for the selection in (a) is the second item, whereas the correct inference for the selection in (b) is the South Bay item.

    Cross-application scripting

    Until recently, all applications that were available to end users ran on top of a common desktop platform, such as the Microsoft Windows operating system or the Apple Macintosh operating system. These platforms are also proprietary, and they only provide limited access to their workings and to user actions. Researchers (Piernot & Yvon, 1993) have been concerned for a long time with finding ways to enable cross-application end user programming, because in practice, many of the tasks that users want to automate involve more than just a single application.

    The popularity of the Web means that many different kinds of applications, such as word processors, email, and chat programs, as well as online banking and retail shopping, are now implemented on a single platform: the Web browser. As a result, end user programming developers have direct access to all of the data in every Web page in the browser, so the barriers to cross-application scripting have vanished.

    The modern end user

    The Web is only partially responsible for another enabling change for end user programming: the advent of the modern end user. The end user of today differs dramatically from the end user of the 1980s. My nephew, who is 13 years old, cannot remember a time when he did not know how to use a computer. He learned how to use a mouse before he learned how to talk. Unlike many adults who are afraid that they will do something wrong, if he doesn’t know what a button does, he presses it. Users who are unafraid of technology and who are willing to try something new constitute a much more receptive audience for end user programming.

    Social networks

    Another trait of modern end users is that they create and share Web content, through sites like You-Tube and Wikipedia. And they readily join communities with common interests, from friends on Facebook to a forum for paragliding enthusiasts. The consequence of this is that end users are ready and willing to create and share scripts, if only the tools become available that make it sufficiently easy to do so. The Chickenfoot and CoScripter systems in Chapters 3 and 5 both have Web sites where users can contribute scripts. Furthermore, users may be further motivated to put in the effort to create scripts when they know that others in their community may benefit from the scripts. The rich variety of roles that Nardi observed in the spreadsheet world may arise in Web-based social networks as well.

    Semantic information

    As was noted previously, the inability to apply semantic information when inferring intent has been a major obstacle to progress in PbD systems. In the age of the Internet,

    1. large-scale semantic information is being collected in knowledge bases like ConceptNet (see http://conceptnet.media.mit.edu);

    2. programmers are writing data detectors (Nardi, Miller, & Wright, 1998) to recognize semi-structured information like addresses, and data mining techniques annotate Web pages with the detected semantic information;

    3. Web sites are formatting semantic information on their pages with microformats (see http://microformats.org); and, most importantly,

    4. this information and these programs are readily available, free of charge, and are being continually updated.

    As a result, it is now becoming possible for PbD systems to circumvent the entire problem of trying to approximate semantics with syntactic patterns. The Citrine system (Stylos, Myers, & Faulring, 2004), for instance, can take a line of text like that in Figure 1.7 and, in a single action, paste the appropriate parts into the various fields of a Web form (see Figure 1.10). There will still be plenty of idiosyncratic tasks for PbD systems to automate, but now those systems won’t be annoyingly stupid because they don’t have access to basic semantic concepts. This semantic information could potentially help scripting languages as well, if they have access to data detectors.

    Semantic information and HTML

    No discussion of the transformative power of the Web would be complete without a consideration of its underlying language: HTML. The simplicity of HTML is largely responsible for the widespread adoption of the Web. However, along with being simple, HTML is also impoverished. In the move from the desktop to the Web, developers lost the ability to drag and drop, to precisely arrange page layout, to draw anywhere on the page, and to update a small part of a page. Nuanced interactions were replaced with jumping from page to page.

    FIGURE 1.10

    Citrine’s address detector.

    This simplicity offers – at least temporarily – a great opportunity for end user programming. Any Web page written in HTML uses the same small set of interface elements, and because HTML is a declarative language, most of these elements have a semantically meaningful tag, which identifies them as buttons, textboxes, and pull-down menus (e.g., for a link; for a button). This immediately solves the problem for PbD systems of inferring the semantics of interface elements.

    However, HTML is now being augmented with techniques that bring back the richness of interaction of desktop applications, and these new techniques pose a challenge for the future success of end user programming. Adobe Flash, for instance, allows for rich user interactions, but no HTML appears on the part of a Web page that uses Flash. When users click and type in Flash, programming by demonstration systems get no indication at all that anything has happened. Similarly, the use of JavaScript, the AJAX programming style, and Web toolkits like YUI and Dojo are replacing the declarative format of HTML with procedures, or programs. For example, the buttons and textboxes in Dojo all use the semantically meaningless

    tag, and the only way to determine the semantics of a JavaScript procedure is to read and understand the program.

    As an example, consider the calendar widget in Figure 1.11. A click on the 26 in the calendar may get recorded as

    click at (119, 252) in the Date Selection window

    In terms of the semantics of the calendar widget, this click actually means

    set the Payment Date to August 26, 2009.

    In fact, because the date highlighted in blue (31) in this widget is the Payment Due Date, a highly specific interpretation of the user’s click would be

    set the Payment Date to 3 business days before the Payment Due Date

    Fortunately, the problem posed by toolkits may also afford its own solution. Toolkits enable Web site developers to use semantically rich user interface objects without having to build them by hand. This means that if just one person goes to the trouble of documenting the meaning of the items in a toolkit, then that information is available for every Web site that uses the toolkit. It is also fortunate that the need for Web site accessibility – for blind users in particular – is a strong motivation for adding just this sort of semantic annotation to a toolkit. The ARIA specification (see http://www.w3.org/WAI/intro/aria) is a standard for adding semantic annotations to toolkits, AJAX, and JavaScript. Chapter 18 describes a project that can make these semantic annotations available everywhere on the Web.

    HTML is still the dominant language on Web pages, and hopefully end user programming will gain success and widespread adoption soon. If a large number of end users recognize its importance, there will be a reason for developers to maintain the scriptability, recordability, and accessibility of their sites.

    FIGURE

    Enjoying the preview?
    Page 1 of 1