Chapter 11: Web programming: A crash course

¶ You are probably reading this in a web browser, so you are likely to be at least a little familiar with the World Wide Web. This chapter contains a quick, superficial introduction to the various elements that make the web work, and the way they relate to JavaScript. The three after this one are more practical, and show some of the ways JavaScript can be used to inspect and change a web-page.

¶ The Internet is, basically, just a computer network spanning most of the world. Computer networks make it possible for computers to send each other messages. The techniques that underlie networking are an interesting subject, but not the subject of this book. All you have to know is that, typically, one computer, which we will call the server, is waiting for other computers to start talking to it. Once another computer, the client, opens communications with this server, they will exchange whatever it is that needs to be exchanged using some specific language, a protocol.

¶ The Internet is used to carry messages for many different protocols. There are protocols for chatting, protocols for file sharing, protocols used by malicious software to control the computer of the poor schmuck who installed it, and so on. The protocol that is of interest to us is that used by the World Wide Web. It is called HTTP, which stands for Hyper Text Transfer Protocol, and is used to retrieve web-pages and the files associated with them.

¶ In HTTP communication, the server is the computer on which the web-page is stored. The client is the computer, such as yours, which asks the server for a page, so that it can display it. Asking for a page like this is called an 'HTTP request'.

¶ Web-pages and other files that are accessible through the Internet are identified by URLs, which is an abbreviation of Universal Resource Locators. A URL looks like this:

http://acc6.its.brooklyn.cuny.edu/~phalsall/texts/taote-v3.html

¶ It is composed of three parts. The start, http://, indicates that this URL uses the HTTP protocol. There are some other protocols, such as FTP (File Transfer Protocol), which also make use of URLs. The next part, acc6.its.brooklyn.cuny.edu, names the server on which this page can be found. The end of the URL, /~phalsal/texts/taote-v3.html, names a specific file on this server.

¶ Most of the time, the World Wide Web is accessed using a browser. After typing a URL or clicking a link, the browser makes the appropriate HTTP request to the appropriate server. If all goes well, the server responds by sending a file back to the browser, who shows it to the user in one way or another.

¶ When, as in the example, the retrieved file is an HTML document, it will be displayed as a web-page. We briefly discussed HTML in chapter 6, where we saw that it could refer to image files. In chapter 9, we found that HTML pages can also contain <script> tags to load files of JavaScript code. When showing an HTML document, a browser will fetch all these extra files from their servers, so it can add them to the document.

¶ Although a URL is supposed to point at a file, it is possible for a web-server to do something more complicated than just looking up a file and sending it to the client. ― It can process this file in some way first, or maybe there is no file at all, but only a program that, given a URL, has some way of generating the relevant document for it.

¶ Programs that transform or generate documents on a server are a popular way to make web-pages less static. When a file is just a file, it is always the same, but when there is a program that builds it every time it is requested, it could be made to look different for each person, based on things like whether this person has logged in or specified certain preferences. This can also make managing the content of web-pages much easier ― instead of adding a new HTML file whenever something new is put on a website, a new document is added to some central storage, and the program knows where to find it and how to show it to clients.

¶ This kind of web programming is called server-side programming. It affects the document before it is sent to the user. In some cases, it is also practical to have a program that runs after the page has been sent, when the user is looking at it. This is called client-side programming, because the program runs on the client computer. Client-side web programming is what JavaScript was invented for.

¶ Running programs client-side has an inherent problem. You can never really know in advance what kinds of programs the page you are visiting is going to run. If it can send information from your computer to others, damage something, or infiltrate your system, surfing the web would be a rather hazardous activity.

¶ To solve this dilemma, browsers severely limit the things a JavaScript program may do. It is not allowed to look at your files, or to modify anything not related to the web-page it came with. Isolating a programming environment like this is called sand-boxing. Allowing the programs enough room to be useful, and at the same time restricting them enough to prevent them from doing harm is not an easy thing to do. Every few months, some JavaScript programmer comes up with a new way to circumvent the limitations and do something harmful or privacy-invading. The people responsible for the browsers respond by modifying their programs to make this trick impossible, and all is well again ― until the next problem is discovered.

¶ One of the first JavaScript tricks that became widely used is the open method of the window object. It takes a URL as an argument, and will open a new window showing that URL.

var perry = window.open("http://www.pbfcomics.com");

¶ Unless you turned off pop-up blocking in chapter 6, there's a chance that this new window is blocked. There is a good reason pop-up blockers exist. Web-programmers, especially those trying to get people to pay attention to advertisements, have abused the poor window.open method so much that by now, most users hate it with a passion. It has its place though, and in this book we will be using it to show some example pages. As a general rule, your scripts should not open any new windows unless the user asked for them.

¶ Note that, because open (just like setTimeout and company) is a method on the window object, the window. part can be left off. When a function is called 'normally', it is called as a method on the top-level object, which is what window is. Personally, I think open sounds a bit generic, so I'll usually type window.open, which makes it clear that it is a window that is being opened.

¶ The value returned by window.open is a new window. This is the global object for the script running in that window, and contains all the standard things like the Object constructor and the Math object. But if you try to look at them, most browsers will (probably) not let you...

show(perry.Math);

¶ This is part of the sand-boxing that I mentioned earlier. Pages opened by your browser might show information that is meant only for you, for example on sites where you logged in, and thus it would be bad if any random script could go and read them. The exception to this rule is pages opened on the same domain: When a script running on a page from eloquentjavascript.net opens another page on that same domain, it can do everything it wants to this page.

¶ An opened window can be closed with its close method. If you didn't already close it yourself...

perry.close();

¶ Other kinds of sub-documents, such as frames (documents-within-a-document), are also windows from the perspective of a JavaScript program, and have their own JavaScript environment. In fact, the environment that you have access to in the console belongs to a small invisible frame hidden somewhere on this page ― this way, it is slightly harder for you to accidentally mess up the whole page.

¶ Every window object has a document property, which contains an object representing the document shown in that window. This object contains, for example, a property location, with information about the URL of the document.

show(document.location.href);

¶ Setting document.location.href to a new URL can be used to make the browser load another document. Another application of the document object is its write method. This method, when given a string argument, writes some HTML to the document. When it is used on a fully loaded document, it will replace the whole document by the given HTML, which is usually not what you want. The idea is to have a script call it while the document is being loaded, in which case the written HTML will be inserted into the document at the place of the script tag that triggered it. This is a simple way to add some dynamic elements to a page. For example, here is a trivially simple document showing the current time.

print(timeWriter);
var time = viewHTML(timeWriter);

time.close();

¶ Often, the techniques shown in chapter 12 provide a cleaner and more versatile way to modify the document, but occasionally, document.write is the nicest, simplest way to do something.

¶ Another popular application of JavaScript in web pages centers around forms. In case you are not quite sure what the role of 'forms' is, let me give a quick summary.

¶ A basic HTTP request is a simple request for a file. When this file is not really a passive file, but a server-side program, it can become useful to include information other than a filename in the request. For this purpose, HTTP requests are allowed to contain additional 'parameters'. Here is an example:

http://www.google.com/search?q=aztec%20empire

¶ After the filename (/search), the URL continues with a question mark, after which the parameters follow. This request has one parameter, called q (for 'query', presumably), whose value is aztec empire. The %20 part corresponds to a space. There are a number of characters that can not occur in these values, such as spaces, ampersands, or question marks. These are 'escaped' by replacing them with a % followed by their numerical value1, which serves the same purpose as the backslashes used in strings and regular expressions, but is even more unreadable.

¶ JavaScript provides functions encodeURIComponent and decodeURIComponent to add these codes to strings and remove them again.

var encoded = encodeURIComponent("aztec empire");
show(encoded);
show(decodeURIComponent(encoded));

¶ When a request contains more than one parameter, they are separated by ampersands, as in...

http://www.google.com/search?q=aztec%20empire&lang=nl

¶ A form, basically, is a way to make it easy for browser-users to create such parameterised URLs. It contains a number of fields, such as input boxes for text, checkboxes that can be 'checked' and 'unchecked', or thingies that allow you to choose from a given set of values. It also usually contains a 'submit' button and, invisible to the user, an 'action' URL to which it should be sent. When the submit button is clicked, or enter is pressed, the information that was entered in the fields is added to this action URL as parameters, and the browser will request this URL.

¶ Here is the HTML for a simple form:

<form name="userinfo" method="get" action="info.html">
  <p>Please give us your information, so that we can send
  you spam.</p>
  <p>Name: <input type="text" name="name"/></p>
  <p>E-Mail: <input type="text" name="email"/></p>
  <p>Sex: <select name="sex">
            <option>Male</option>
            <option>Female</option>
            <option>Other</option>
          </select></p>
  <p><input name="send" type="submit" value="Send!"/></p>
</form>

¶ The name of the form can be used to access it with JavaScript, as we shall see in a moment. The names of the fields determine the names of the HTTP parameters that are used to store their values. Sending this form might produce a URL like this:

http://planetspam.com/info.html?name=Ted&email=ted@zork.com&sex=Male

¶ There are quite a few other tags and properties that can be used in forms, but in this book we will stick with simple ones, so that we can concentrate on JavaScript.

¶ The method="get" property of the example form shown above indicates that this form should encode the values it is given as URL parameters, as shown before. There is an alternative method for sending parameters, which is called post. An HTTP request using the post method contains, in addition to a URL, a block of data. A form using the post method puts the values of its parameters in this data block instead of in the URL.

¶ When sending big chunks of data, the get method will result in URLs that are a mile wide, so post is usually more convenient. But the difference between the two methods is not just a question of convenience. Traditionally, get requests are used for requests that just ask the server for some document, while post requests are used to take an action that changes something on the server. For example, getting a list of recent messages on an Internet forum would be a get request, while adding a new message would be a post request. There is a good reason why most pages follow this distinction ― programs that automatically explore the web, such as those used by search engines, will generally only make get requests. If changes to a site can be made by get requests, these well-meaning 'crawlers' could do all kinds of damage.

¶ When the browser is displaying a page containing a form, JavaScript programs can inspect and modify the values that are entered in the form's fields. This opens up possibilities for all kinds of tricks, such as checking values before they are sent to the server, or automatically filling in certain fields.

¶ The form shown above can be found in the file example_getinfo.html. Open it.

var form = window.open("example_getinfo.html");

¶ When a URL does not contain a server name, it is called a relative URL. Relative URLs are interpreted by the browser to refer to files on the same server as the current document. Unless they start with a slash, the path (or directory) of the current document is also retained, and the given path is appended to it.

¶ We will be adding a validity check to the form, so that it only submits if the name field is not left empty and the e-mail field contains something that looks like a valid e-mail address. Because we no longer want the form to submit immediately when the 'Send!' button is pressed, its type property has been changed from "submit" to "button", which turns it into a regular button with no effect. ― Chapter 13 will show a much better way of doing this, but for now, we use the naive method.

¶ To be able to work with the newly opened window (if you closed it, re-open it first), we 'attach' the console to it, like this:

attach(form);

¶ After doing this, the code run from the console will be run in the given window. To verify that we are indeed working with the correct window, we can look at the document's location and title properties.

print(document.location.href);
print(document.title);

¶ Because we have entered a new environment, previously defined variables, such as form, are no longer present.

show(form);

¶ To get back to our starting environment, we can use the detach function (without arguments). But first, we have to add that validation system to the form.

¶ Every HTML tag shown in a document has a JavaScript object associated with it. These objects can be used to inspect and manipulate almost every aspect of the document. In this chapter, we will work with the objects for forms and form fields, chapter 12 talks about these objects in more detail.

¶ The document object has a property named forms, which contains links to all the forms in the document, by name. Our form has a property name="userinfo", so it can be found under the property userinfo.

var userForm = document.forms.userinfo;
print(userForm.method);
print(userForm.action);

¶ In this case, the properties method and action that were given to the HTML form tag are also present as properties of the JavaScript object. This is often the case, but not always: Some HTML properties are spelled differently in JavaScript, others are not present at all. Chapter 12 will show a way to get at all properties.

¶ The object for the form tag has a property elements, which refers to an object containing the fields of the form, by name.

var nameField = userForm.elements.name;
nameField.value = "Eugène";

¶ Text-input objects have a value property, which can be used to read and change their content. If you look at the form window after running the above code, you'll see that the name has been filled in.

¶ All we have to do now is determine what happens when people click the 'Send!' button. At the moment, it does not do anything at all. This can be remedied by setting its onclick property.

userForm.elements.send.onclick = function() {
  alert("Click.");
};

¶ Just like the actions given to setInterval and setTimeout (chapter 8), the value stored in an onclick (or similar) property can be either a function or a string of JavaScript code. In this case, we give it a function that opens an alert window. Try clicking it.

¶ Another trick related to form inputs, as well as other things that can be 'selected', such as buttons and links, is the focus method. When you know for sure that a user will want to start typing in a certain text field as soon as he enters the page, you can have your script start by placing the cursor in it, so he won't have to click it or select it in some other way.

userForm.elements.name.focus();

¶ Because the form sits in another window, it may not be obvious that something was selected, depending on the browser you are using. Some pages also automatically make the cursor jump to the next field when it looks like you finished filling in one field ― for example, when you type a zip code. This should not be overdone ― it makes the page behave in a way the user does not expect. If he is used to pressing tab to move the cursor manually, or mistyped the last character and wants to remove it, such magic cursor-jumping is very annoying.

detach();

¶ Test the validator. When you enter valid information and click the button, the form should submit. If the console was still attached to it, this will cause it to detach itself, because the page reloads and the JavaScript environment is replaced by a new one.

¶ If you haven't closed the form window yet, this will close it.

form.close();

¶ The above may look easy, but let me assure you, client-side web programming is no walk in the park. It can, at times, be a very painful ordeal. Why? Because programs that are supposed to run on the client computer generally have to work for all popular browsers. Each of these browsers tends to work slightly different. To make things worse, each of them contains a unique set of problems. Do not assume that a program is bug-free just because it was made by a multi-billion dollar company. So it is up to us, the web-programmer, to rigorously test our programs, figure out what goes wrong, and find ways to work around it.

¶ Some of you might think "I will just report any problems/bugs I find to the browser manufacturers, and they will certainly solve fix them immediately". These people are in for a major disappointment. The most recent version of Internet Explorer, the browser that is still used by some seventy percent of web-surfers (and that every web-developer likes to rag on) still contains bugs that have been known for over five years. Serious bugs, too.

¶ But do not let that discourage you. With the right kind of obsessive-compulsive mindset, such problems provide wonderful challenges. And for those of us who do not like wasting our time, being careful and avoiding the obscure corners of the browser's functionality will generally prevent you from running into too much trouble.

¶ Bugs aside, the by-design differences in interface between browsers still make for an interesting challenge. The current situation looks something like this: On the one hand, there are all the 'small' browsers: Firefox, Safari, and Opera are the most important ones, but there are more. These browsers all make a reasonable effort to adhere to a set of standards that have been developed, or are being developed, by the W3C, an organisation that tries to make the Web a less confusing place by defining standard interfaces for things like this. On the other hand, there is Internet Explorer, Microsoft's browser, which rose to dominance in a time when many of these standards did not really exist yet, and hasn't made much effort to adjust itself to what other people are doing.

¶ In some areas, such as the way the content of an HTML document can be approached from JavaScript (chapter 12), the standards are based on the method that Internet Explorer invented, and things work more or less the same on all browsers. In other areas, such as the way events (mouse-clicks, key-presses, and such) are handled (chapter 13), Internet Explorer works radically different from other browsers.

¶ For a long time, owing partially to the cluelessness of the average JavaScript developer, and partially to the fact that browser incompatibilities were much worse when browsers like Internet Explorer version 4 or 5 and old versions of Netscape were still common, the usual way to deal with such differences was to detect which browser the user was running, and litter code with alternate solutions for each browser ― if this is Internet Explorer, do this, if this is Netscape, do that, and if this is other browser that we didn't think of, just hope for the best. You can imagine how hideous, confusing, and long such programs were.

¶ Many sites would also just refuse to load when opened in a browser that was 'not supported'. This caused a few of the minor browsers to swallow their pride and pretend they were Internet Explorer, just so they would be allowed to load such pages. The properties of the navigator object contain information about the browser that a page was loaded in, but because of such lying this information is not particularly reliable. See what yours says2:

forEachIn(navigator, function(name, value) {
  print(name, " = ", value);
});

¶ A better approach is to try and 'isolate' our programs from differences in browsers. If you need, for example, to find out more about an event, such as the clicks we handled by setting the onclick property of our send button, you have to look at the top-level object called event on Internet Explorer, but you have to use the first argument passed to the event-handling function on other browsers. To handle this, and a number of other differences related to events, one can write a helper function for attaching events to things, which takes care of all the plumbing and allows the event-handling functions themselves to be the same for all browsers. In chapter 13 we will write such a function.

¶ (Note: The browser quirks mentioned in the following chapters refer to the state of affairs in early 2007, and might no longer be accurate on some points.)

¶ These chapters will only give a somewhat superficial introduction to the subject of browser interfaces. They are not the main subject of this book, and they are complex enough to fill a thick book on their own. When you understand the basics of these interfaces (and understand something about HTML), it is not too hard to look for specific information online. The interface documentation for the Firefox and Internet Explorer browsers are a good place to start.

¶ The information in the next chapters will not deal with the quirks of 'previous-generation' browsers. They deal with Internet Explorer 6, Firefox 1.5, Opera 9, Safari 3, or any more recent versions of the same browsers. Most of it will also probably be relevant to modern but obscure browsers such as Konqueror, but this has not been extensively checked. Fortunately, these previous-generation browsers have pretty much died out, and are hardly used anymore.

¶ There is, however, a group of web-users that will still use a browser without JavaScript. A large part of this group consists of people using a regular graphical browser, but with JavaScript disabled for security reasons. Then there are people using textual browsers, or browsers for blind people. When working on a 'serious' site, it is often a good idea to start with a plain HTML system that works, and then add non-essential tricks and conveniences with JavaScript.