9.3 Infrastructure Applications
There are some protocols that are essential to the smooth running of the Internet but that don't fit neatly into the strictly layered model. One of these is the Domain Name System (DNS)—not an application that users normally invoke directly, but rather a service that almost all other applications depend upon. This is because the name service is used to translate host names into host addresses; the existence of such an application allows the users of other applications to refer to remote hosts by name rather than by address. In other words, a name service is usually used by other applications, rather than by humans.
A second critical function is network management, which although not so familiar to the average user, is the operation performed most often by system administrators. Network management is widely considered one of the hard problems of networking and continues to be the focus of much research. We'll look at some of the issues and approaches to the problem below.
Name Service (DNS)
In most of this book, we have been using addresses to identify hosts.
While perfectly suited for processing by routers, addresses are not
exactly user friendly. It is for this reason that a unique name is
also typically assigned to each host in a network. Already in this
section we have seen application protocols like HTTP using names
www.princeton.edu. We now describe how a naming service can
be developed to map user-friendly names into router-friendly
addresses. Name services are sometimes called middleware because
they fill a gap between applications and the underlying network.
Host names differ from host addresses in two important ways. First, they are usually of variable length and mnemonic, thereby making them easier for humans to remember. (In contrast, fixed-length numeric addresses are easier for routers to process.) Second, names typically contain no information that helps the network locate (route packets toward) the host. Addresses, in contrast, sometimes have routing information embedded in them; flat addresses (those not divisible into component parts) are the exception.
Before getting into the details of how hosts are named in a network, we first introduce some basic terminology. First, a name space defines the set of possible names. A name space can be either flat (names are not divisible into components) or hierarchical (Unix file names are an obvious example). Second, the naming system maintains a collection of bindings of names to values. The value can be anything we want the naming system to return when presented with a name; in many cases, it is an address. Finally, a resolution mechanism is a procedure that, when invoked with a name, returns the corresponding value. A name server is a specific implementation of a resolution mechanism that is available on a network and that can be queried by sending it a message.
Because of its large size, the Internet has a particularly
well-developed naming system in place—the Domain Name System (DNS). We
therefore use DNS as a framework for discussing the problem of naming
hosts. Note that the Internet did not always use DNS. Early in its
history, when there were only a few hundred hosts on the Internet, a
central authority called the Network Information Center (NIC)
maintained a flat table of name-to-address bindings; this table was
HOSTS.TXT. Whenever a site wanted to add a new host to the
Internet, the site administrator sent email to the NIC giving the new host's
name/address pair. This information was manually entered into the table,
the modified table was mailed out to the various sites every few days,
and the system administrator at each site installed the table on every
host at the site. Name resolution was then simply implemented by a
procedure that looked up a host's name in the local copy of the table
and returned the corresponding address.
Believe it or not, there was also a paper book (like a phone book) published periodically that listed all the machines connected to the Internet and all people that had an Internet email account.
It should come as no surprise that the approach to naming did not work well as the number of hosts in the Internet started to grow. Therefore, in the mid-1980s, the Domain Naming System was put into place. DNS employs a hierarchical namespace rather than a flat name space, and the "table" of bindings that implements this name space is partitioned into disjoint pieces and distributed throughout the Internet. These subtables are made available in name servers that can be queried over the network.
What happens in the Internet is that a user presents a host name to an application program (possibly embedded in a compound name such as an email address or URL), and this program engages the naming system to translate this name into a host address. The application then opens a connection to this host by presenting some transport protocol (e.g., TCP) with the host's IP address. This situation is illustrated (in the case of sending email) in Figure 1. While this picture makes the name resolution task look simple enough, there is a bit more to it, as we shall see.
DNS implements a hierarchical name space for Internet objects. Unlike
Unix file names, which are processed from left to right with the naming
components separated with slashes, DNS names are processed from right to
left and use periods as the separator. (Although they are processed from
right to left, humans still read domain names from left to right.) An
example domain name for a host is
Notice that we said domain names are
used to name Internet "objects." What we mean by this is that DNS is not
strictly used to map host names into host addresses. It is more accurate
to say that DNS maps domain names into values. For the time being, we
assume that these values are IP addresses; we will come back to this
issue later in this section.
Like the Unix file hierarchy, the DNS hierarchy can be visualized as a tree, where each node in the tree corresponds to a domain, and the leaves in the tree correspond to the hosts being named. Figure 2 gives an example of a domain hierarchy. Note that we should not assign any semantics to the term domain other than that it is simply a context in which additional names can be defined.
Confusingly, the word domain is also used in Internet routing, where it means something different than it does in DNS, being roughly equivalent to the term autonomous system.
There was actually a substantial amount of discussion that took place
when the domain name hierarchy was first being developed as to what
conventions would govern the names that were to be handed out near the
top of the hierarchy. Without going into that discussion in any detail,
notice that the hierarchy is not very wide at the first level. There are
domains for each country, plus the "big six" domains:
.net. These six
domains were all originally based in the United States (where the
Internet and DNS were invented); for example, only U.S.-accredited
educational institutions can register an
.edu domain name. In recent
years, the number of top-level domains has been expanded, partly to
deal with the high demand for
.com domains names. The newer
top-level domains include
.info. There are now
over 1200 top-level domains.
The complete domain name hierarchy exists only in the abstract. We now turn our attention to the question of how this hierarchy is actually implemented. The first step is to partition the hierarchy into subtrees called zones. Figure 3 shows how the hierarchy given in Figure 2 might be divided into zones. Each zone can be thought of as corresponding to some administrative authority that is responsible for that portion of the hierarchy. For example, the top level of the hierarchy forms a zone that is managed by the Internet Corporation for Assigned Names and Numbers (ICANN). Below this is a zone that corresponds to Princeton University. Within this zone, some departments do not want the responsibility of managing the hierarchy (and so they remain in the university-level zone), while others, like the Department of Computer Science, manage their own department-level zone.
The relevance of a zone is that it corresponds to the fundamental unit of implementation in DNS—the name server. Specifically, the information contained in each zone is implemented in two or more name servers. Each name server, in turn, is a program that can be accessed over the Internet. Clients send queries to name servers, and name servers respond with the requested information. Sometimes the response contains the final answer that the client wants, and sometimes the response contains a pointer to another server that the client should query next. Thus, from an implementation perspective, it is more accurate to think of DNS as being represented by a hierarchy of name servers rather than by a hierarchy of domains, as illustrated in Figure 4.
Note that each zone is implemented in two or more name servers for the sake of redundancy; that is, the information is still available even if one name server fails. On the flip side, a given name server is free to implement more than one zone.
Each name server implements the zone information as a collection of resource records. In essence, a resource record is a name-to-value binding or, more specifically, a 5-tuple that contains the following fields:
(Name, Value, Type, Class, TTL)
Value fields are exactly what you would expect,
Type field specifies how the
Value should be
interpreted. For example, indicates that the
Value is an IP address.
A records implement the name-to-address mapping we have been
assuming. Other record types include:
Valuefield gives the domain name for a host that is running a name server that knows how to resolve names within the specified domain.
Valuefield gives the canonical name for a particular host; it is used to define aliases.
Valuefield gives the domain name for a host that is running a mail server that accepts messages for the specified domain.
Class field was included to allow entities other than the NIC to
define useful record types. To date, the only widely used
the one used by the Internet; it is denoted
IN. Finally, the
TTL) field shows how long this resource record is
valid. It is used by servers that cache resource records from other
servers; when the
TTL expires, the server must evict the record from
To better understand how resource records represent the information in
the domain hierarchy, consider the following examples drawn from the
domain hierarchy given in Figure 2. To simplify the
example, we ignore the
TTL field and we give the relevant
information for only one of the name servers that implement each zone.
First, a root name server contains an
NS record for each top-level
domain (TLD) name server. This identifies a server that can resolve
queries for this part of the DNS hierarchy (
example). It also has
A records that translates these names into the
corresponding IP addresses. Taken together, these two records
effectively implement a pointer from the root name server to one of the
(edu, a3.nstld.com, NS, IN) (a3.nstld.com, 184.108.40.206, A, IN) (com, a.gtld-servers.net, NS, IN) (a.gtld-servers.net, 220.127.116.11, A, IN) ...
Moving our way down the hierarchy by one level, the server has records for domains like this:
(princeton.edu, dns.princeton.edu, NS, IN) (dns.princeton.edu, 18.104.22.168, A, IN) ...
In this case, we get an
NS record and an
A record for the name
server that is responsible for the
princeton.edu part of the
hierarchy. That server might be able to directly resolve some queries
email.princeton.edu) while it would redirect others to a
server at yet another layer in the hierarchy (e.g., for a query about
(email.princeton.edu, 22.214.171.124, A, IN) (penguins.cs.princeton.edu, dns1.cs.princeton.edu, NS, IN) (dns1.cs.princeton.edu, 126.96.36.199, A, IN) ...
Finally, a third-level name server, such as the one managed by domain
A records for all of its hosts. It
might also define a set of aliases (
CNAME records) for each of those
hosts. Aliases are sometimes just convenient (e.g., shorter) names for
machines, but they can also be used to provide a level of
indirection. For example,
www.cs.princeton.edu is an alias for the
coreweb.cs.princeton.edu.This allows the site's web
server to move to another machine without affecting remote users; they
simply continue to use the alias without regard for what machine
currently runs the domain's web server. The mail exchange (
records serve the same purpose for the email application—they allow an
administrator to change which host receives mail on behalf of the
domain without having to change everyone's email address.
(penguins.cs.princeton.edu, 188.8.131.52, A, IN) (www.cs.princeton.edu, coreweb.cs.princeton.edu, CNAME, IN) coreweb.cs.princeton.edu, 184.108.40.206, A, IN) (cs.princeton.edu, mail.cs.princeton.edu, MX, IN) (mail.cs.princeton.edu, 220.127.116.11, A, IN) ...
Note that, although resource records can be defined for virtually any type of object, DNS is typically used to name hosts (including servers) and sites. It is not used to name individual people or other objects like files or directories; other naming systems are typically used to identify such objects. For example, X.500 is an ISO naming system designed to make it easier to identify people. It allows you to name a person by giving a set of attributes: name, title, phone number, postal address, and so on. X.500 proved too cumbersome—and, in some sense, was usurped by powerful search engines now available on the Web—but it did eventually evolve into the Lightweight Directory Access Protocol (LDAP). LDAP is a subset of X.500 originally designed as a PC front end to X.500. Today, widely used, mostly at the enterprise level, as a system for learning information about users.
Given a hierarchy of name servers, we now consider the issue of how a
client engages these servers to resolve a domain name. To illustrate the
basic idea, suppose the client wants to resolve the name
penguins.cs.princeton.edu relative to the
set of servers given in the previous subsection. The client could first
send a query containing this name to one of the root servers (as we'll
see below, this rarely happens in practice but will suffice to
illustrate the basic operation for now). The root server, unable to
match the entire name, returns the best match it has—the
edu which points to the TLD server
a3.nstld.com. The server
also returns all records that are related to this record, in this
A record for
a3.nstld.com. The client, having not
received the answer it was after, next sends the same query to the
name server at IP host
18.104.22.168. This server also cannot match the
whole name and so returns the
NS and corresponding
A records for
princeton.edu domain. Once again, the client sends the same
query as before to the server at IP host
22.214.171.124, and this
time gets back the
NS record and corresponding
A record for the
cs.princeton.edu domain. This time, the server that can fully
resolve the query has been reached. A final query to the server at
126.96.36.199 yields the
A record for
penguins.cs.princeton.edu, and the client learns that the
corresponding IP address is
This example still leaves a couple of questions about the resolution process unanswered. The first question is how did the client locate the root server in the first place, or, put another way, how do you resolve the name of the server that knows how to resolve names? This is a fundamental problem in any naming system, and the answer is that the system has to be bootstrapped in some way. In this case, the name-to-address mapping for one or more root servers is well known; that is, it is published through some means outside the naming system itself.
In practice, however, not all clients know about the root servers.
Instead, the client program running on each Internet host is initialized
with the address of a local name server. For example, all the hosts in
the Department of Computer Science at Princeton know about the server
dns1.cs.princeton.edu. This local name server, in turn, has
resource records for one or more of the root servers, for example:
('root', a.root-servers.net, NS, IN) (a.root-servers.net, 188.8.131.52, A, IN)
Thus, resolving a name actually involves a client querying the local
server, which in turn acts as a client that queries the remote servers
on the original client's behalf. This results in the client/server
interactions illustrated in Figure 5. One advantage of
this model is that all the hosts in the Internet do not have to be kept
up-to-date on where the current root servers are located; only the
servers have to know about the root. A second advantage is that the
local server gets to see the answers that come back from queries that
are posted by all the local clients. The local server caches these
responses and is sometimes able to resolve future queries without having
to go out over the network. The
TTL field in the resource records
returned by remote servers indicates how long each record can be safely
cached. This caching mechanism can be used further up the hierarchy as
well, reducing the load on the root and TLD servers.
The second question is how the system works when a user submits a
partial name (e.g.,
penguins) rather than a complete domain name
penguins.cs.princeton.edu). The answer is that the client
program is configured with the local domain in which the host resides
cs.princeton.edu), and it appends this string to any simple
names before sending out a query.
Just to make sure we are clear, we have now seen three different levels of identifiers—domain names, IP addresses, and physical network addresses—and the mapping of identifiers at one level into identifiers at another level happens at different points in the network architecture. First, users specify domain names when interacting with the application. Second, the application engages DNS to translate this name into an IP address; it is the IP address that is placed in each datagram, not the domain name. (As an aside, this translation process involves IP datagrams being sent over the Internet, but these datagrams are addressed to a host that runs a name server, not to the ultimate destination.) Third, IP does forwarding at each router, which often means that it maps one IP address into another; that is, it maps the ultimate destination's address into the address for the next hop router. Finally, IP engages the Address Resolution Protocol (ARP) to translate the next hop IP address into the physical address for that machine; the next hop might be the ultimate destination or it might be an intermediate router. Frames sent over the physical network have these physical addresses in their headers.
Network Management (SNMP)
A network is a complex system, both in terms of the number of nodes that are involved and in terms of the suite of protocols that can be running on any one node. Even if you restrict yourself to worrying about the nodes within a single administrative domain, such as a campus, there might be dozens of routers and hundreds—or even thousands—of hosts to keep track of. If you think about all the state that is maintained and manipulated on any one of those nodes—address translation tables, routing tables, TCP connection state, and so on—then it is easy to become depressed about the prospect of having to manage all of this information.
It is easy to imagine wanting to know about the state of various protocols on different nodes. For example, you might want to monitor the number of IP datagram reassemblies that have been aborted, so as to determine if the timeout that garbage collects partially assembled datagrams needs to be adjusted. As another example, you might want to keep track of the load on various nodes (i.e., the number of packets sent or received) so as to determine if new routers or links need to be added to the network. Of course, you also have to be on the watch for evidence of faulty hardware and misbehaving software.
What we have just described is the problem of network management, an issue that pervades the entire network architecture. Since the nodes we want to keep track of are distributed, our only real option is to use the network to manage the network. This means we need a protocol that allows us to read, and possibly write, various pieces of state information on different network nodes. A widely used protocol for this purpose is the Simple Network Management Protocol (SNMP).
While it is true that SNMP is widely used, and was historically "the" management protocol for switches and routers, system admins also have to administer servers, client desktop machines, and laptops. And to complicate matters even more, virtualization technology has led to the proliferation of virtual machines (VM), which now outnumber physical machines; admins have to administer these VMs too. New management tools are being invented every day to help address this challenge, but they typically leverage standard protocols like HTTP to retrieve monitoring data. This is a positive development in that it takes us out of the business of creating new protocols and puts us in the business of creating smarter and smarter management tools, many of which now starting to take advantage of Artificial Intelligence to determine if something is amiss. Today, SNMP is one of many sources of data being used by network automation and orchestration tools.
SNMP is essentially a specialized request/reply protocol that supports
two kinds of request messages:
SET. The former is used
to retrieve a piece of state from some node, and the latter is used to
store a new piece of state in some node. (SNMP also supports a third
operation, , which we explain below.) The following discussion focuses
GET operation, since it is the one most frequently used.
SNMP is used in the obvious way. A system administrator interacts with a client program that displays information about the network. This client program usually has a graphical interface. You can think of this interface as playing the same role as a web browser. Whenever the administrator selects a certain piece of information that he or she wants to see, the client program uses SNMP to request that information from the node in question. (SNMP runs on top of UDP.) An SNMP server running on that node receives the request, locates the appropriate piece of information, and returns it to the client program, which then displays it to the user.
There is only one complication to this otherwise simple scenario: Exactly how does the client indicate which piece of information it wants to retrieve, and, likewise, how does the server know which variable in memory to read to satisfy the request? The answer is that SNMP depends on a companion specification called the management information base (MIB). The MIB defines the specific pieces of information—the MIB variables—that you can retrieve from a network node.
The current version of MIB, called MIB-II, organizes variables into 10 different groups. You will recognize that most of the groups correspond to one of the protocols described in this book, and nearly all of the variables defined for each group should look familiar. For example:
System—General parameters of the system (node) as a whole, including where the node is located, how long it has been up, and the system's name
Interfaces—Information about all the network interfaces (adaptors) attached to this node, such as the physical address of each interface and how many packets have been sent and received on each interface
Address translation—Information about the Address Resolution Protocol, and in particular, the contents of its address translation table
IP—Variables related to IP, including its routing table, how many datagrams it has successfully forwarded, and statistics about datagram reassembly; includes counts of how many times IP drops a datagram for one reason or another
TCP—Information about TCP connections, such as the number of passive and active opens, the number of resets, the number of timeouts, default timeout settings, and so on; per-connection information persists only as long as the connection exists
UDP—Information about UDP traffic, including the total number of UDP datagrams that have been sent and received.
There are also groups for Internet Control Message Protocol (ICMP), Exterior Gateway Protocol (EGP), and SNMP itself. The tenth group is used by different media.
Returning to the issue of the client stating exactly what information it wants to retrieve from a node, having a list of MIB variables is only half the battle. Two problems remain. First, we need a precise syntax for the client to use to state which of the MIB variables it wants to fetch. Second, we need a precise representation for the values returned by the server. Both problems are addressed using Abstract Syntax Notation One (ASN.1).
Consider the second problem first. As we already saw in in a previous
chapter, ASN.1/Basic Encoding Rules (BER) defines a representation for
different data types, such as integers. The MIB defines the type of each
variable, and then it uses ASN.1/BER to encode the value contained in
this variable as it is transmitted over the network. As far as the first
problem is concerned, ASN.1 also defines an object identification
scheme. The MIB uses this identification system to assign a globally
unique identifier to each MIB variable. These identifiers are given in a
"dot" notation, not unlike domain names. For example, 184.108.40.206.220.127.116.11 is
the unique ASN.1 identifier for the IP-related MIB variable
ipInReceives; this variable counts the number of IP datagrams that
have been received by this node. In this example, the 18.104.22.168.2.1 prefix
identifies the MIB database (remember, ASN.1 object IDs are for all
possible objects in the world), the 4 corresponds to the IP group, and
the final 3 denotes the third variable in this group.
Thus, network management works as follows. The SNMP client puts the ASN.1 identifier for the MIB variable it wants to get into the request message, and it sends this message to the server. The server then maps this identifier into a local variable (i.e., into a memory location where the value for this variable is stored), retrieves the current value held in this variable, and uses ASN.1/BER to encode the value it sends back to the client.
There is one final detail. Many of the MIB variables are either tables or structures. Such compound variables explain the reason for the SNMP operation. This operation, when applied to a particular variable ID, returns the value of that variable plus the ID of the next variable, for example, the next item in the table or the next field in the structure. This aids the client in "walking through" the elements of a table or structure.