S4C6: The Internet

>>> Click here to access this episode of the Syllab Podcast on Spotify <<<

a) Concept and infrastructure

At a high level, the Internet is a meta-network of computer networks. Nothing truly innovative since at least one other inter-network system already existed in the domain of telephony in the form of the public switched telephone network (PSTN) that could connect any two users across large distances. The Internet, which is now so ubiquitous that I will use the lower-case version of “internet” going forward, was not born out of the need to communicate between individuals but, instead, to access and optimize the utilization of expensive computer resources: time sharing at a distance.

The reason why internet is so different in its technological and even societal impacts has to do with the versatility of what computers can do and the data they can store. The mention of data brings up the point that this usage and technological boom would not have been possible without the digital revolution, which goes hand in hand with the computer revolution, one feeding of the progress and market penetration of the other. Computing, digital data and now computer networks (including smartphones and other connected devices), this is the triad, the building blocks from which complexity and possibilities emerge.

We have already covered the transmission technologies in Chapter 4 (wired) and Chapter 5 (wireless), so this chapter is going to be much more oriented towards the infrastructure aspect and what makes communication possible, besides the transfer of data; this will be the topic of section b) on Internet protocols and in section c), d) and e) we will explore the broad types of applications made possible by internet.

So, a network of networks… let’s try to be a little more descriptive. There are two options for moving data between networks: #1 pay for the transit, #2 not pay because you have bartered the theoretical transit fee you would charge another network for the theoretical transit fees they would charge you; this is called free peering. This in turn means there are three network tiers: Tier 1 have settlement-free peering agreements with all other networks on the internet, not just other Tier 1 networks, Tier 2 have free peering with some networks and pay for transit or for peering with other networks (paid peering is like a paid subscription), and Tier 3 always pay for transit or peering. A lot of the Tier 1 Networks are telecom infrastructure companies and a lot of the Tier 2 are internet service providers (ISPs). Now, as is often the case, the money is not in the infrastructure building and maintenance but in the operations, in this case the moving of data as well as the content being provided. Consumers are all accessing the internet via Tier 3 networks but, as you will see shortly, it isn’t really a drawback because of the presence of other hardware within those networks.

As private users, we rely on ISPs mostly for providing us with internet access and transit, these are the companies we pay a monthly subscription to, which may also include mobile phone voice or data services, or both. These ISPs are then in charge of moving the data around, physically accessing other networks via nodes called internet exchange points (IXPs), the equivalent of the switches of the telephone network we described in S4 Section 4.d where hundreds or even sometimes more than a thousand networks can enter into mutual peering. Thus, ISPs are able to provide access to the entire network, unless for some reason they are disallowed, through payments of transit fees or reciprocal peering with other networks.

If IXPs are the interface between networks, then the interface between users (once the data is carried by wired or mobile telecommunication) and the local network is facilitated at a “point of presence” (PoP), which may or may not be at the same location as an IXP. Some of the hardware located there would include routers, switches and servers. Sometimes we don’t even go to the data, the data gets stored close to us, in caching servers. The idea is to store data within a local loop, in anticipation of it being used by customers in the area (to watch the latest Netflix series or see frequently accessed web pages) with three obvious benefits: the data can be moved outside of peak hours and avoid bandwidth bottlenecks (and probably more expensive transit prices), the data can be served much faster as it is already close by, and the same cache can be accessed by several users thus saving on total transit volume.

b) Internet protocols

The previous section was focused on the physical aspects of moving data from one user to another user or server, and vice-versa. But it is one thing to have the infrastructure to move a cargo from point A to B and another to know how to handle the cargo, which is called payload in the jargon. Let us use the analogy of data you want to transmit being in the form of a written letter and you can write instructions on a series of envelopes. How would you go about it when it comes to the internet?

What you would want is the relevant party (a computer program) during the journey to be able to read only the instructions written for it and then forward on another envelope contained within, and so on at each stage of the way. This means the instructions that need to be read first have to be on the outside envelop, the top-most layer of what is called the Internet Protocol suite known as TCP/IP – suite because there is a series of protocols, each corresponding to a different layer, 4 to be precise not counting the physical layer interfacing with the transmission medium itself.

The first layer is called the link layer and provides the relevant information for purpose of the local network segment, thus it enables the connection from the computer or smartphone which you may be using, called the “host”, to a network node.

The second envelope, encapsulated inside the first, enables the data to cross network boundaries. It makes the inter-connection, and therefore the very idea of internet, possible. For that reason, it is called the internet or network layer. The addressing system we know as IP address, now comprising 128 bits, is part of the Internet Protocol (IP), the main component of this layer. The IP address serves the same purpose as identification numbers on other types of network, whether local, personal or the telephone system, it contains information regarding identity and location so the right device can be found wherever it is connected from.

Layer number 3 is called transport, it assigns ports and channels between hosts receiving and sending the payload, thus establishing connectivity. These instructions are network-agnostic, which is the whole purpose of everybody using the same set of protocols and the best-known, and most widely-used, is called the Transfer Control Protocol (TCP). This is obviously the same TCP as in TCP/IP where IP is for Internet Protocol.

The fourth layer is read last and is also reserved for the relevant applications running on the host’s computer. It states which communication protocols and interface are being used for the purpose of this data stream, ensuring the data can be processed correctly. Examples include Simple Mail Transfer Protocol (SMTP) for emails and Hypertext Transfer Protocol (HTTP) for the World Wide Web.

c) The Web

The World Wide Web, or just “the Web”, is the facet of the internet we use to make available and access shared-content. Typically, this is done in the form of a website, which provides content on related topics, or owned by a same entity, whether it is an individual or an organization. To be sure, this content can actually be the interface for the performance of a service, for which we may or may not have to pay, that allows input to be provided and outputs to be displayed on a computer. In this case, we talk about web applications.

The defining feature of web, presented as pages with static or dynamic content, is the way the pages are linked to each other, allowing the user to easily navigate from one to the other by following hyperlinks. Hyperlinks are simply URLs embedded in a text, so when you click on this hypertext, often represented with a different colour font such as dark blue (or purple on this website), you are asking your browser to retrieve the content, hosted on a web server, which corresponds to a URL that is different from the one shown in the address bar of your browser.

Think of web pages as the pages of books, a website as a book and the web as a library. By specifying a URL, your web browser fetches the content of a page in a book somewhere in the library and shows it to you on a screen, or downloads it. URL means uniform resources locator and you can think about this as the standard way, shared by all websites and more, to write an address where the page can be found. The formatting for a website would start with “http” or “https” for the secure version, the same hypertext transfer protocol we saw in the previous section, followed by “://” and then a path. The path used to start with “www.” But this is now optional, it continues with the domain name of the website such as “labattu.com” and then the rest of the path which may be structured like branching pages separated by forward slashes (“/”).
The address bar is also often used to transfer inputs after the path name, right after a question mark (“?”) because this input is part of a query, so if you try to reserve a hotel room online you will see the inputs such as number of guests and the dates all being part of the now very long URL and what you are querying about is availability and price.

Technically, a website designates all the content made available over the web under a common domain name, and having control over the domain name provides control over the content of the website. Domain names consist of a top level and a second level parts, for Syllab it is “.com” and “labattu”, respectively, and the second level may be split into subdomains. The administration of these domain names is handled by domain name registrars accredited by the relevant authority looking after top-level domains (TLDs). For the generic TLDs such as ”.com” or “.org” it would be the Internet Assigned Numbers Authority, a part of the non-profit American association called ICANN (this goes back to the origins of the internet), and for the country code TLDs (ccTLD) such as “.be” for Belgium or “.io” for the British Indian Ocean Territory, these are managed by each country or territory.

Unless you are particularly computer savvy, you will be interacting with the web via a browser, a software that will do the fetching and displaying of the content. In some cases, as with Chrome OS, they can act as a full-fledged operating system – you may wish to refer to S4 Section 3.c if you are unfamiliar with the concept of OS. These applications allow us to bookmark websites (just storing the relevant URL in memory), to remember our inter-web-page navigation history so we can move back and forward between them, or reload the content, also called refreshing. Browsers should not be mistaken with search engines, although the interface is sometimes integrated by using the address bar of the browser as input which is fed to the engine that provides the results.

The job of a search engine, besides making money from advertising because it can very successfully influence your decisions to click or purchase goods and services, is three-fold although only the last one is really apparent to the user:

#1 crawling the web for new pages. This is done by crawlers nicknamed spiders that start with given URLs that may have been submitted by the website owners and follow hyperlinks from there.
#2 indexing of the pages. The search engine applies a series of rules to decide whether to index or not the URLs found and communicated by the crawlers. This can be one of several sore points in dealing with search engines for website owners. If you want to understand more about this process, I have included the relevant Wikipedia entry at the end of this chapter.
#3 displaying results by order of relevance. This is done by applying proprietary, undisclosed algorithms that are regularly tweaked, to the universe of indexed pages and quantify the expected value of each suggested link for the user. The assumed credibility or “authority” of a website, often a self-reinforcing artefact of popularity, accounts for a significant weight in the valuation algorithms. In other words, it is not a meritocracy out there in search engine results and “don’t be evil” has long been forgotten, if it ever was a thing. Yes, I do have first-hand experience with the ill-treatment of a website at the hands of the dominant search engine.

d) Internet-based communication

The Web and internet are often associated, rightly so from a functional standpoint but the Web is only part of the data that uses the internet to be moved between users or between users and servers. The inter-network is also used for sharing large files, including the delivery of video content via streaming websites – this we will look at in the next section – as well as for live telecommunications, be it voice only, video or text – this we will look at now with the mainstream technologies of emails and voice over IP (VoIP).

Emails (or e-mails) are the electronic version of the standard postal mail, and much more. It is possible to attach documents, forward messages and copy several persons on one single message and even if all of these features could in theory be replicated via the old non-dematerialized method, even ignoring the time and the price it would involve, this allows for the emergence of an entire new way of communicating, in particular within organizations, as a business tool. Clearly, time efficiency and near-zero cost should not be ignored because they make possible a boom in volume and style of communication. Not quite the more recent exchange of snippets we call messages but not full-fledged letters either because it is easy to write a second email or wait for an answer to orientate the conversation towards a particular topic. From monologue letters, we shifted to semi-conversational emails.

Emails can live on your computer or on a server, accessed and managed by a software (desktop or web application) called an email client or in the jargon, a mail user agent (MUA). This is the user interface where the content is introduced and formatted before being sent using the Simple Mail Transfer Protocol (SMTP) as the application layer. “Sent” here means submission to a mail server, the mail submission agent (MSA), and ultimately the delivery to the recipient by way of the mail delivery agent (MDA). However, the job of transferring across the network is fulfilled by two more agents called mail transfer agents, one at each end of the network path. The MSA passes on the information to a first MTA, this program finds the relevant server for delivery based on the recipient email address in the mail exchanger record (MX record) of the relevant domain name (the part of the address after the “@”) and connects to it to exchange the email with the second MTA located there. This MTA then passes on the data to the MDA and nowadays the email can then be accessed by the recipient with the Internet Messages Access Protocol (IMAP), also an application layer protocol, and most of the email storage is kept on remote servers rather than user desktops, as was typically the case in the early days when the main protocol was POP (Post Office Protocol).

Unlike the productivity gains made possible by emails compared to postal mail, the use of VoIP is driven by cost considerations. If you have ever been billed for roaming when making international calls, you and your wallet will very quickly appreciate the advantage of using your data subscription plan to route voice calls through the internet. This is quite an easy jump in many ways since mobile technology already makes use of digital data, even though the carrier signal is analog. The development in speed of connection using VoIP and the improvements in quality have been so significant that it has become the preferred option for mobile operators. Thus, operators of 4G and 5G telecom technologies route voice calls via data channels and infrastructure rather than the soon-to-be-legacy voice ones and for this they rely on Voice over LTE and Voice over NR standards respectively. In fact, the media for voice calls have come full circle and not only voice can be routed on the public switched telephone network but plain telephones can be used for VoIP thanks to adapters connected directly or indirectly to the internet and converting the analog signal into a digital signal, or the reverse for incoming calls.

As it happens, since video data can also be conveyed in a digital format, video calls can piggyback on VoIP technology and even though “voice” is in the name, it is now optimized for both video and voice. This begs the question of how – as in how is it optimized? Voice is already a fair number of bits and video is at least one order of magnitude more. The trick is of course to compress the data, through encoding, in a way that retains the maximum percentage of information with the least number of bits; this means it is a matter of trade off and there is not a single best algorithm, though clearly some do a much better job than others. To take an extremely simplified example of what compression can be like for video, imagine you are trying to describe the first 10 pixels of the top row of one video frame and the colours are RGBBBBGGRR with R representing red, G green and B blue and this information takes 8 bits for each colour; you could save a lot of bits by saying there is no change in colour after pixels 3, 4, 5, 7 and 9 and the symbol for indicating no change in colour would of course be much smaller than 8 bits. Information is actually not encoded quite this way but this is one of the central ideas of compression: communicating patterns such as a mathematical function rather than being purely descriptive.

This coding and decoding programs are suitably named codecs and one of the favoured algorithms for audio compression is linear predictive coding (LPC) while modified discrete cosine transform (MDCT) is popular for both audio and images.

LPC deconstructs speech into 3 layers: a signal with particular frequency and intensity coming from our vocal folds, a particular resonance coming from the vocal tract, and a filter in the form of stops (plosives) and hissing (sibilant) executed by our tongue, lips and throats. The values used to represent this are encoded into frames representing voice parameters up to 50 times per second.
MDCT generally encodes images in partially overlapping consecutive 8×8 pixel blocks and deconstructs their luminosity and chrominance (the colour aspect) into 64 coefficients, each corresponding to the individual contribution of 64 base cosine waves with different frequencies. These values are then shifted to be centred around 0 and divided by the intensity coefficient and all adjusted-coefficients will then be rounded to the nearest integer. As a result, many of the higher-frequency waves will have a 0 adjusted-coefficient ascribed to them and these values can be left out without any discernible loss of quality as far as the human eye is concerned on a standard-size screen. The overlapping aspect of modified DCT ensures better retention of information and smoothness at the boundaries of the pixel blocks.

e) Data transfer, streaming and the cloud

Internet is agnostic about data content and, to a certain extent, data formatting. All that is needed are the right protocols such as HTTP or FTP (File Transfer Protocol) for moving the data along network paths and across exchange points (the network boundaries). As mentioned earlier in this chapter, this data could be content displayed on the web, emails, or files made available on a directory for download. This is used widely in academia, and sometimes for illegitimate content as well.

The main architecture underpinning this is peer-to-peer (P2P) networking where resources are shared between computers forming the nodes of a decentralized network – essentially creating a virtual network riding on the internet where there is no client Vs server distinction and each participant fulfils both functions, all nodes being equal. This makes possible protocols such as BitTorrent with one initial uploader of File AB and each time a peer downloads AB it can be downloaded from this node by another peer, so each download becomes available for upload.

Part of the issue with the illegal sharing of downloaded files can be mitigated by enabling the consumption of media content as it reaches a computer, without storage in a hard disk drive. This is one of the commercial advantages of streaming from the perspective of the content owners and distributors and, oftentimes, the consumer is just happy listening to or watching content only once, as we would in the days of video rental. In addition, the improvement in streaming technologies makes live broadcasting over the internet in high resolution possible, as well as the ability for playback by using buffering.

In isolation, streaming doesn’t call for much new technology, yet the surge in the overall amount of data required to be moved creates bandwidth pressure on the networks that need to be addressed through a combination of hardware such as cache servers in local networks, software with more efficient compression algorithms, and resources optimization.

Finally, we cannot not conclude our discussion of the internet without talking about “the cloud”. The cloud is not a thing, it is a concept, and in fact it is the very concept which gave birth to the internet decades ago: the sharing of resources at a distance via a network. These resources can be storage capacity, data security, computing power, website deployment infrastructure, databases, APIs, etc. The cloud is a form of outsourcing, tapping on the expertise of service providers, and with operating systems now operating in the cloud, that is on remote servers, your personal computer can be shrunk in size, weight and power, and your data and applications need not be tied to a particular device.

f) Trivia – Firewalls

Being connected has obvious advantages in terms of the breadth of data and services one has access to, but it can cut both ways and if you have access to other personal computers or servers, then these also have a line into your computer and may not always be well-intentioned. Hence the need to protect oneself through antivirus, looking to block malicious programs, and just like the immune system, prevent physical access to some data in the first place. Conversely, some organizations may also want to avoid some types of content from leaving a specific network. This control of inbound and outbound traffic is the role of firewalls.

Actions that can be taken include the denying of entry/exit, the dropping of connections, the discarding of content or subjecting content and connection requests to further enquiry. All of this is driven by variables such as the source and destination of the traffic, including details such as the IP address and URLs, and the presence of keywords on the web pages being fetched.

Sometimes, an organization may define malicious content as content running contrary to its own interest. The best-known example is the Great Firewall in the PRC where the political party in power restricts access to websites and applications for which it cannot control the content, so the population within the country is not exposed to information it deems undesirable. It is mainly deployed at the exchange points where the national networks and other networks branching from there interface with international networks. If you are interested in learning a little more about some of the blocking and filtering techniques you can check out the relevant Wikipedia entry, I have included the hyperlink in the next section.