Web Fundamentals
This document is intended to be used as teaching material and roadmap for senior-junior pair on the topics of Internet, WWW and basic frontend web development. Updated Jan 2025 # Building simple websites !!! WARNING: Prerequisites Before starting this tutorial you should know - basic CLI usage, - unix file structure - python basics (functions) and have a web development environment set up (VSCode, Firefox developer). If you have no one to assist you with the first steps read the following: - [MDN - What tools do I actually need, right now?](https://developer.mozilla.org/en-US/docs/Learn/Getting_started_with_the_web/Installing_basic_software#what_tools_do_i_actually_need_right_now) ## Learn basic HTML and CSS Start with these three tutorials: - [MDN - Dealing with files](https://developer.mozilla.org/en-US/docs/Learn/Getting_started_with_the_web/Dealing_with_files) - [MDN - HTML basics](https://developer.mozilla.org/en-US/docs/Learn/Getting_started_with_the_web/HTML_basics) - [MDN - CSS basics](https://developer.mozilla.org/en-US/docs/Learn/Getting_started_with_the_web/CSS_basics) then go though following tutorials to revise learned and learn something extra: 1. [HTML Dog: beginner HTML](https://www.htmldog.com/guides/html/beginner/) 2. [HTML Dog: beginner CSS](https://www.htmldog.com/guides/css/beginner/) 3. [HTML Dog: intermediate HTML](https://www.htmldog.com/guides/html/intermediate/) 4. [HTML Dog: intermediate CSS](https://www.htmldog.com/guides/css/intermediate/) !!! Pay special attention to the `selectors`, they work similarly later in javascript to also target the elements. They re important because they re how we specify the element that we want to modify. Check these tutorials: - [HTML Dog: selectors](https://www.htmldog.com/guides/css/advanced/selectors/) - [HTML Dog: attributeselectors](https://www.htmldog.com/guides/css/advanced/attributeselectors/) Be sure that you understand that html document looks like a tree with subelements: **Project: google homepage** For your first frontend project you will make a clone of the google homepage. You are not allowed to use inspector or view source. Pro tip: imaging whole website as set of boxes with nested boxes. Searchbar is a wide box above main content thats a box containing sub boxes. Think of those boxes as div's with sub-div's and elements. **Project: CSS zen garden** Go to https://www.csszengarden.com/ and download the HTML and CSS files. Edit the CSS file locally to style the provided html. **Do not edit** the HTML file. Pro tips: - Lean how to use the [browser developer tools](https://developer.mozilla.org/en-US/docs/Learn/Common_questions/Tools_and_setup/What_are_browser_developer_tools). - You can use inspector on the CSS Zen Garden examples, but ideally go on your own. Don't copy 1:1. - You can find CSS inspirating in the [advanced css tutorial](https://www.htmldog.com/guides/css/advanced/). - You can use css to insert images. - Don't forget about the css box model: ## Learn basic Javascript Having learned a lot about HTML and CSS it is time for you to learn the final pillar of web development - the JavaScript. - Start with the basics in [MDN JS Basics](https://developer.mozilla.org/en-US/docs/Learn/Getting_started_with_the_web/JavaScript_basics) or go through [javascript beginner](https://www.htmldog.com/guides/javascript/beginner/) on HTML Dog - followed by [MDN JS First steps](https://developer.mozilla.org/en-US/docs/Learn/JavaScript/First_steps) - and the [MDN JS Building blocks](https://developer.mozilla.org/en-US/docs/Learn/JavaScript/Building_blocks). - You can see what available in the [MDN JS overview](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Language_Overview) **Project task - simple Javascript app** You are an owner of one-man shop web development agency. You have been contacted by potential client to build them a productivity app. They offered yout o build them either a simple calculator or a pomodoro timer. Aditionally, client wants a separate "landing page" that talks about your app and links to it This will later be used by the client to pitch his app and make sales on it. To collect the sweet imaginary client coins you have to: 1. Pair on design and app implementation progress with the "client" 2. Use plain HTML, CSS and JS to build the app. 3. Use the Bootstrap or Bulma CSS framework for the landing page. You need to keep CSS customizations minimum. ## Next steps - Read the style guides for [html/css](https://google.github.io/styleguide/htmlcssguide.html) and [js](https://google.github.io/styleguide/jsguide.html) - Learn more [HTML](http://diveintohtml5.info/) and [CSS](https://every-layout.dev/) - Learn more JS - [Objects](https://developer.mozilla.org/en-US/docs/Learn/JavaScript/Objects) - Learn about [BEM methodology](https://en.bem.info/methodology/) - Learn about accessibility ([url1](https://developer.mozilla.org/en-US/docs/Web/Accessibility), [url2](https://www.w3.org/WAI/tips/developing/)) and SEO ([url1](https://medium.com/welldone-software/seo-for-developers-a-quick-overview-5b5b7ce34679), [url2](https://learningseo.io/)) - Complete additional projects - build a CSS framework and demo page for it - pick and recreate complex website template (for e.g. admin dashboard template) # The World Wide Web How does web page get to your screen? It turns out there is an awful lot of computer science involved. For now we'll start out with a very high level overview and move down a little deeper later. ## Introduction Every web page you are seeing is the result of a web browser communicating with a web server. A number of important things happen during the conversation. - The browser and the server “talk” to each other using a relatively simple protocol called HTTP, which stands for the **Hyper Text Transfer Protocol**. - If we break down HTTP into its two components we get a little more detail: - **Hyper Text** - The idea of hyper text is that it links you from one page to another (we used to call the links "hyperlinks" but that has been shortened to just links). All transfered Hyper Text follows a standard called the Hyper Text Markup Language (HTML). - **Transfer Protocol** - when two programs want to talk to each other both programs have to “speak the same language.” A protocol is a very specific description of a series of messages that two programs use to communicate. - The web server sends the HTML to the browser over the internet and the browser renders the HTML into the nicely formatted page that you see on your screen. At the beginning of the World Wide Web there was only HTML, and web pages were a bit drab compared to what we are used to seeing today. Now modern web pages rely on three main technologies: - **HTML - Hyper Text Markup Language** - HTML describes the content of the page, giving the page some sctructure including common things like headings, and sections, and lists of things. - **CSS - Cascading Style Sheets** - CSS desribes the “look” of the page, the colors, the fonts, the margins, etc. - **Javascript** is used to change the behavior of the page. What happens when you click on a button, or scroll a scrollbar, or hover over something with your mouse. ## The Technology of the Web We now return to the question of how a page gets from the server to your browser. Let's use the idea of a URI as our starting point for this section. In fact let's start by addressing a very common point of confusion What is a URI and what is URL, you probably more commonly think of URL's. - URL - Uniform Resource Locator - URI - Uniform Resource Identifier All locators are also identifiers, so all URLs are URI's. Think about it this way, Bradley Miller is an identifier, it identifies me, but does not tell you how to locate me or how to contact me. 321 Olin Hall, Luther College, Decorah IA, is a locator and an identifier for a specific place in the world. On the web URL's have three main pieces, let's look at one and identify the parts: `https://interactivepython.org/ns/books/published/webfundamentals/index.html` This is a common URL with three parts: - `http` This part identifies the protocol that will be used for communication. It seems a bit redundant when we are always thinking about the web, but in fact there are several common protocols that are used in URL's including `https` for secure http communication, `mailto` to indicate that we are going to use a mail protocol, `ftp` the file transfer protocol. - interactivepython.org This is the name of a host on the web. Host names must be unique, there is only one `interactivepython.org` on the entire internet. If you dig more deeply you can find out that this name translates to a numeric address. If you have ever tried to configure your home router, or looked at the settings on your computer or phone you have undoubtedly come across these numeric addresses before. In the case of interactivepython.org the numeric address is `108.168.242.153`. This numerical address is also unique in all of the internet, and identifies a specific server running in a server room in Dallas TX. - `runestone/static/webfundamentals/index.html` identifies a particular file on the server. The server has a `runestone` folder, and inside that folder is another folder called `static`, which contains another folder called `webfundamentals` which finally contains a file called `index.html`. This is no different than how you probably organize your own files on your own computer. This URL contains a lot of information. When you type this into the address bar of your browser you have given your browser enough information to get the specific page you wanted. A diagram will help with the next part of this discussion. Let's suppose we continue with the URL above. You, the user, type in the URL to your browser, The browser knows that the host name is the part that comes after the first two slashes, but before the next slash. So it is able to extract that part of the URL. Using patterns like this occurs frequently in Computer Science. The name is then passed on to another server called the Domain Name Service (DNS) in the diagram above. The DNS is actually a very large and complex network of servers, but we'll just take it for granted that it works and gives us back the IP address correctly. With the address in hand, the browser establishes a connection to the server through a mechanism known as a `socket`. This socket allows the browser and the server to reliably communicate over the internet. ### An HTTP Request With the connection established the browser sends the following message: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ GET examples/webfundamentals/index.html HTTP/1.1 Host: interactivepython.org User-Agent: Mozilla/6.0 Accept-Language: en _ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The format of this message is defined by HTTP. Remember a protocol is a well defined set of rules for how one program communicates with another. Let us break down this message into its component parts. - `GET` – HTTP defines several commands, but the most common are GET and POST. As you might imagine, GET is a request to retrieve some resource from the server. - `examples/webfundamentals/index.html` This is the URI for the resource that we want to get. Recall that this will correspond to a file on the disk drive of the server. - `HTTP/1.1` tells the server that the browser is “speaking” the 1.1 version of the HTTP protocol. This is all of the information that can be contained on the first line of the message. If there is anything else on the line it may confuse the server, or the server may just ignore it. This line is called the request line. What follows the request line are a series of lines called the headers There may be just one header or there may be many. However, there can only be one header per line and they all follow the pattern of `name: value`. So the first line the name is `host` and the value is `interactivepython.org` this lets the server know the name of the machine that is contacting it. The `User-Agent` header tells the server that the user is running version 6.0 of the Mozilla browser. Finally the last line is blank. The server knows that there are no more headers when it gets a blank line. ### An HTTP Response The server extracts the URI from the request line and goes out to its disk to locate the file. Assuming the file is there it will make the following response through the socket back to the browser: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ HTTP/1.1 200 OK Date: Tues, 2 Sept 2014 15:39:21 Server: Apache/1.4.1 Content-Type: text/html <html> <head> </head> <body> <h1>Hello World</h1> </body> </html> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Just like the request, the format of the response is spelled out by the HTTP. The first line again has three parts `HTTP/1.1` tells the browser that the server is following the HTTP 1.1 protocol. The `200` is the response code that tells the browser that everything is fine and the OK is a descriptive message: Some other common response lines are: - HTTP/1.1 404 Not Found - you may have seen a 404 if you've ever mistyped a URL - HTTP/1.1 500 Internal Server Error - The server itself has crashed for some reason - HTTP/1.1 302 Resource Moved Following the response line are the headers from the server. Most of these are self-explanatory, but the Content-Type header is particularly important. This tells the browser that what follows is going to be some HTML. Another common Content-type is `image/jpg` this tells the browser that what comes next is going to be an image. Note that each image in a page will require its own request / response from the browser to the server. ## History The concept of the internet goes back more than a half-century: - 1962 - MIT researcher J.C.R. Licklider presents the idea of a group of computers connected to each other, enabling them to share data and programs together, no matter where each computer was physically located. - 1965 - Two other MIT researchers, Leonard Kleinrock and Lawrence G. Roberts, connect two computers to each other from across the country using a telephone line. - 1968 - The Defense Advanced Research Projects Agency (DARPA) expands on their predecessors' work and creates a system, ARPANET, used to send information between computers. - 1969 - A host computer connected to a separate computer at UCLA, and then researchers at Stanford connected their computers to the same network, which was part of the ARPANET project. And so the internet, based on ARPANET, was born. Tim Berners-Lee invented the World Wide Web in 1989, about 20 years after the first connection was established over what is today known as the Internet. At the time, Tim was a software engineer at CERN, the large particle physics laboratory near Geneva, Switzerland. Many scientists participated in experiments at CERN for extended periods of time, then returned to their laboratories around the world. These scientists were eager to exchange data and results, but had difficulties doing so. Tim understood this need, and understood the unrealized potential of millions of computers connected together through the Internet. Tim documented what was to become the World Wide Web with the submission of a proposal to his management at CERN, in late 1989. This proposal specified a set of technologies that would make the Internet truly accessible and useful to people. Believe it or not, Tim's initial proposal was not immediately accepted. However, Tim persevered. By October of 1990, he had specified the three fundamental technologies that remain the foundation of today's Web (and which you may have seen appear on parts of your Web browser): - HTML: HyperText Markup Language. The publishing format for the Web, including the ability to format documents and link to other documents and resources. - URI: Uniform Resource Identifier. A kind of “address” that is unique to each resource on the Web. - HTTP: Hypertext Transfer Protocol. Allows for the retrieval of linked resources from across the Web. Tim also wrote the first Web page editor/browser (“WorldWideWeb”) and the first Web server (“httpd“). By the end of 1990, the first Web page was served. By 1991, people outside of CERN joined the new Web community. Very important to the growth of the Web, CERN announced in April 1993 that the World Wide Web technology would be available for anyone to use on a royalty-free basis. Since that time, the Web has changed the world. It has arguably become the most powerful communication medium the world has ever known. Whereas only 25% of the people on the planet are currently using the Web (and the Web Foundation aims to accelerate this growth substantially), the Web has changed the way we teach and learn, buy and sell, inform and are informed, agree and disagree, share and collaborate, meet and love, and tackle problems ranging from putting food on our tables to curing cancer. Tim Berners-Lee and others realized that for the Web to reach its full potential, the underlying technologies must become global standards, implemented in the same way around the world. Therefore, in 1994, Tim founded the World *Wide Web Consortium (W3C)* as a place for stakeholders to reach consensus around the specification and guidelines to ensure that the Web works for everyone and that it evolves in a responsible manner. W3C standards have enabled a single World Wide Web of information and people, and an increasingly-rich set of capabilities: Web 2.0 (personal and dynamic), Web 3.0 (a semantic Web of linked data), Web services, voice access, mobile access, accessibility for people with disabilities and for people speaking many languages, richer graphics and video, etc. The Web Foundation supports the work of W3C to ensure that the Web and the technologies that underpin it remain free and open to all. ## Next steps - read more about [How the Web Works](https://developer.mozilla.org/en-US/docs/Learn/Getting_started_with_the_web/How_the_Web_works) - read overview of HTTP on [MDN](https://developer.mozilla.org/en-US/docs/Web/HTTP/Overview) - learn about a typical http session on [MDN](https://developer.mozilla.org/en-US/docs/Web/HTTP/Session) - extra: HTTP messages on [MDN](https://developer.mozilla.org/en-US/docs/Web/HTTP/Messages) - extra: HTTP authentication on [MDN](https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication) - extra: [How HTTPS works comic](https://howhttps.works/episodes/) # Internet & Web mechanics !!! Your learning goal in this section is to understand the underlaying concepts and ideas, you don't have to memorize all the details - we will touch on most important things again in the future. However, be sure to take notes and sketch diagrams! The internet’s become a crucial part of our daily lives. We use it to connect with other people, stream our favorite songs and TV shows, and even work from home. But for many of us, how it works remains a mystery. The good news is: It's not as complicated as you might think. You know a lot about Web mechanics already since you have been using the internet for years, to learn the details go though the following articles on the topics of Web mechanics: - [How does the Internet work?](https://developer.mozilla.org/en-US/docs/Learn/Common_questions/Web_mechanics/How_does_the_Internet_work) - [What is the difference between webpage, website, web server, and search engine?](https://developer.mozilla.org/en-US/docs/Learn/Common_questions/Web_mechanics/Pages_sites_servers_and_search_engines) - [What are hyperlinks?](https://developer.mozilla.org/en-US/docs/Learn/Common_questions/Web_mechanics/What_are_hyperlinks) - [What is a URL?](https://developer.mozilla.org/en-US/docs/Learn/Common_questions/Web_mechanics/What_is_a_URL) - [What is a Domain Name?](https://developer.mozilla.org/en-US/docs/Learn/Common_questions/Web_mechanics/What_is_a_domain_name) and [How DNS works comic](https://howdns.works/) - [What is a web server?](https://developer.mozilla.org/en-US/docs/Learn/Common_questions/Web_mechanics/What_is_a_web_server) ## OSI model Let's dive deeper. How a single bit travels from one computer to the next is a complex concept. In 1984, the **open systems interconnection** (OSI) model was published as a __framework__ for network communication. The model breaks down computer network communication into seven layers. All of the layers work together to create a digital message. The message is built as it moves down the protocol stack. However, it is not sent to another network until it reaches the physical layer. It was the first standard model for network communications, adopted by all major computer and telecommunication companies in the early 1980s. OSI Model The modern Internet is not based on OSI, but on the simpler TCP/IP model. However, the OSI 7-layer model is still widely used, as it helps visualize and communicate how networks operate, and helps isolate and troubleshoot networking problems. In other words, TCP/IP is a practical model that addresses specific communication challenges and relies on standardized protocols. In contrast, OSI serves as a comprehensive, protocol-independent framework designed to encompass various network communication methods. Let's take a detailed look at the OSI layers. #### Layer 1. Physical Layer The physical layer is where data moves across network interfaces as digital signals. Additionally, this is where the transmitting and receiving of network communication occurs. Starting with the Application Layer the message moves down the OSI model, and it eventually reaches the Physical Layer for transmission. When the message is received by the physical layer, the message will then move up the OSI layers until it reaches the final application layer. #### Layer 2. data-link layer Electrical signals received (or transmitted) to the physical layer are linked and translated to digital logic in the data-Link layer. Computer devices may be networked at the Data-Link layer, but only as a Local Area Network (LAN). Connecting a LAN to another LAN occurs at Layer 3. Within Layer 2, the Protocol Data Unit (PDU) known as a frame consists of a header, footer, and data. Understanding how a frame is structured is important for network traffic analysis. Additionally, within Layer 2, physical addresses are assigned and are also known as MAC addresses and/or hardware addresses in networking. MAC addresses are unique to each device on a local network. They are 48-bits in length and are assigned in hexadecimal characters. Some other things to note about Layer 2 is that there are a few protocols that reside in it that we should know about: - **Ethernet**: The most common type of LAN, Ethernet is the standard used to connect computing devices, routers, and switches in a wired network. - **IEEE 802.11**: “Wi-Fi” or “Wireless LAN.” - **Fiber Distributed Data Interface (FDDI)**: Network standard for fiber optic LAN connections. - **Link Layer Discovery Protocol (LLDP)**: A Link Layer protocol used for advertising neighbors, identity, and capabilities on a LAN. - **Address Resolution Protocol (ARP)**: Converts and links Internet Protocol (IP) addresses to MAC addresses on a LAN. - **Cisco Discovery Protocol (CDP)**: Similar to LLDP, but Cisco proprietary. The protocol collects neighbor information of directly connected LAN devices. Additionally, Layer 2 is split into two sublayers: - **Logical Link Control (LLC)**: Responsible for establishing the logical link between devices on a local network. - **Media Access Control (MAC)**: Responsible for the procedures used by devices across a network medium. #### Layer 3: network layer When we think of the internet, we are thinking of interconnected networks. Interconnecting networks refer to a Local Area Network (LAN) connection (such as home, office, or a building) to neighboring or remote networks. Layer 3 of the OSI model, the network layer, is where internetworking takes place and is where logical addresses are assigned to networked devices. A primary function of this layer is to route network packets from one LAN to another. Routing requires IP addresses and logical mapping of other networks across the internet to properly deliver messages. Another important function of Layer 3 is its ability to fragment and reassemble large communication. When Layer 3 passes a message down to Layer 2 for transmission, message length limits may be encountered in some cases. Additionally, Layer 3 is the layer where the protocols used to route communication between networks reside. Important protocol here is the **Internet Protocol (IP)**, specifically the most commonly used IPv4 version, but theres also IPv6 version. Within Layer 3, the Protocol Data Unit (PDU) is the packet. Packets encapsulate data intended for transmission with header and footer data. The IPv4 protocol encapsulates data with IPv4 header information necessary for delivery. For example, the 32-bit packet format contains the source address, the destination address, protocol, time-to-live (TTL), etc. in the IPv4 header data. #### Layer 4: transport layer The transport layer, Layer 4, is responsible for being the go-between the abstract layers of the OSI model (Layers 7-5) and the concrete communication layers (Layers 3-1). Depending on the type of application, the transportation of that application's communication will need to be handled in a specific way. For example, basic web browsing communication uses Hypertext Transfer Protocol (HTTP). HTTP communicates via a specific connection service type and port. The transport layer is responsible for delivering/receiving the HTTP communication and maintaining the connection throughout the HTTP communication. The Protocol Data Unit (PDU) at Layer 4 is known as a data segment. Segmentation is the process of dividing raw data into smaller pieces. Once the raw data is packaged from the higher application layers it is segmented at the transport layer before being passed to the Network Layer. The transport layer protocols are divided into two categories depending on their connection service type: ##### Connection-oriented services This connection type establishes a logical connection between two devices prior to beginning communication across a network. Connection-oriented protocols typically maintain service connection by following a set of rules that initiate, negotiate, manage, and terminate the communication. The Transport Layer protocols will also retransmit any data that is received without acknowledgment. The most common Connection-Oriented protocol is the **Transmission Control Protocol (TCP)** and its process to manage a connection between two devices is called the Three-Way Handshake. In TCP communication, the communicating devices typically share a client/server relationship where a client initiates communication with a service. The handshake involves the process of sending special TCP messages to synchronize a state of negotiated connection in communication. ##### Connectionless services In connectionless communication, the protocol does not establish a connection between client and server. Instead, once a request is made to the server, the server sends all data without initiation, negotiation, or management of connection. Connectionless protocols also do not attempt to correct any interruptions in data transmission. Once the server sends the data, the server is not concerned if the client receives it. When TCP or UDP are used to establish communication, the communication is assigned a port as the Layer 4 address. A port is a logical assignment given to processes and their respective application protocols on a computing system. A few important facts to memorize about ports are: - There are 65,535 valid port numbers available to assign to a communication process. - Ports 0 - 1023 are **Well-Known Ports**: Assigned to universal TCP/IP application protocols. These protocols are the most common such as HTTPS, SSH, FTP, DNS, and the list goes on. They are registered to these protocols by a global - Ports 1024 - 49,151 are **Registered Ports**: Reserved for application protocols that are not specified as universal TCP/IP application protocols. - Ports 49,152 - 65,535 are **Private/Dynamic Ports**: These ports may be used for any process without the need to register the port with the global assigning authority. - When TCP and IP are used together, a Layer 4 port and a Layer 3 IP address are assigned to the connection. This is called a socket. For example, 8.8.8.8:443 is a socket indicating that communication to IP address 8.8.8.8 is to connect to port 443 on the server. #### Layer 5: session layer The session layer starts, manages, and terminates sessions between end-user application processes. Sessions are considered the persistent connection between devices. A session is application-focused; sessions are not concerned with layers 1-4. Instead, the session layer controls dialog between two networked devices. It is considered to facilitate host-to-host communication. Sessions dialog may be controlled through synchronization checkpoints, and through management of communication modes. There are two modes of communication permitted at Layer 5: - Half-Duplex: Communication travels in both directions between sender and receiver, but only one device may transmit a message at a time. - Full-Duplex: Communication travels in both directions between sender and receiver, and messages may be sent simultaneously in either direction. The session layer resembles a phone conversation. For example, when a person picks up a phone and calls someone else a session is created. Once the communication on the call is completed, the session is terminated by hanging up the phone. In computing, software applications are making the phone call and establishing a session. Two common Layer 5 protocols still used today are: - NetBIOS - Remote Procedure Call (RPC) #### Layer 6: presentation layer The presentation layer is primarily responsible for presenting data so that the recipient will understand the data. Data formatting and encoding protocols apply at Layer 6 to ensure data is legible and presented properly in the application receiving it. Data compression is also a function of Layer 6. If necessary, data may be compressed to improve data throughput over network communication. Some common Layer 6 protocols are ASCII, JPEG, GIF, MPEG, and PNG. Another main function of the presentation layer is the encryption and decryption of data sent across a network. Most encryption communication protocols straddle multiple layers of the OSI model, but the actual encryption function is Layer 6. Two of the most common secure communication protocols are: - Transport Layer Security (TLS) - Secure Socket Layer (SSL) #### Layer 7: application layer The topmost layer of the OSI model is the application layer. On computer systems, applications display information to the user via the UI. !!! Note: Software applications running on a computer are NOT considered to reside in the application layer. Instead, they leverage application layer services and protocols that enable network communication. For example, the user can craft messages and access the network from the application layer. A web browser application allows a user to access a web page. The user may input information and receive information through the web browser. However, the application layer protocol HTTP performs the network communication function. The web browser and HTTP work closely together, and the distinction between the two may be subtle. Yet, HTTP is the web browsing protocol for all web browser applications. In contrast, no single web browser software exclusively utilizes HTTP. HTTP is one of many common application layer protocols. Below are a few additional protocols to know. It is also good practice to memorize the associated port assigned to the protocols: Protocol | Port Number(s) | Description ---------|----------------|------------- Domain Name System (DNS) | 53 | Translates internet names to their globally registered IP addresses. For example, “google.com” is registered in global DNS as IP address 8.8.8.8. Hypertext Transfer Protocol Secure (HTTPS) | 443 | Sends data to and from web browsers and web servers, but securely with the Secure Socket Layer (SSL) protocol. File Transfer Protocol FTP | 20, 21 | Transfers files from a client to a server and vice versa. Secure Shell (SSH) | 22 | Connects to computers remotely and in a secure, encrypted way. Simple Mail Transfer Protocol (SMTP) | 25 | Sends and receives email. Dynamic Host Configuration Protocol (DHCP) | 67 | Automatically assigns IP addresses to devices on a network. Internet Relay Chat (IRC) | 194 | Used in a client/server method. IRC clients communicate through an IRC server. Post Office Protocol 3 (POP3) | 110 (unsecured), 995 (secured) | Used for email where the client receives mail by downloading it locally to a computer from a server mailbox. ## Next steps - go through [what happens when](https://github.com/alex/what-happens-when#what-happens-when) - go through [how web works](https://github.com/vasanthk/how-web-works#how-web-works) # REST and APIs Pretty much all of modern web and applications make use of REST APIs. In this section we will learn what REST, API and RESTful API are. ## REpresentational State Transfer REST, or REpresentational State Transfer, is an architectural style for providing standards between computer systems on the web. We use this to make it easier for systems to communicate with each other. Pay close attention: you may be asked to define rest during an interview in tech careers. REST API Model REST-compliant systems, often called RESTful systems, are characterized by two things: 1. Separation of Client and Server 2. Statelessness **Separation of Client and Server** in the REST architectural style means that the implementation of the client (the code) and the implementation of the server (also code) can be done independently without each knowing about the other. This means that the code on the client side can be changed at any time without affecting the operation of the server, and the code on the server side can be changed without affecting the operation of the client. For example; OpenAI foundation created and maintain their own code, while individual developers and companies use provided ChatGPT REST API to create their own applications that make use of the AI. Users of REST intraface (API) are called clients. By using a REST interface, different clients hit the same REST endpoints, perform the same actions, and receive the same responses. **Statelessness** means that the server does not need to know anything about what state the client is in and vice versa. In this way, both the server and the client can understand any message received, even without seeing previous messages. ### Communication between Client and Server In the REST architecture, clients send requests to retrieve or modify resources, and servers send responses to these requests. Let's take a look at the standard ways to make requests and send responses. **Making Requests** REST requires that a client make a request to the server in order to retrieve or modify data on the server. A request generally consists of: - an HTTP verb, which defines what kind of operation to perform - a header, which allows the client to pass along information about the request - a path to a resource - an optional message body containing data **HTTP Verbs** There are 4 basic HTTP verbs we use in requests to interact with resources in a REST system: - GET — retrieve a specific resource (by id) or a collection of resources - POST — create a new resource - PUT — update a specific resource (by id) - DELETE — remove a specific resource by id ### (RESTful) API An application programming interface (API) is a way for two or more computer programs to communicate with each other. It is a type of software interface, offering a service to other pieces of software. A document or standard that describes how to build or use such a connection or interface is called an API specification. A computer system that meets this standard is said to implement or expose an API. The term API may refer either to the specification or to the implementation. Whereas a system's user interface dictates how its end-users interact with the system in question, its API dictates how to write code that takes advantage of that system's capabilities. In contrast to a user interface, which connects a computer to a person, an application programming interface connects computers or pieces of software to each other. It is not intended to be used directly by a person (the end user) other than a computer programmer who is incorporating it into the software. An API is often made up of different parts which act as tools or services that are available to the programmer. A program or a programmer that uses one of these parts is said to call that portion of the API. The calls that make up the API are also known as subroutines, methods, requests, or endpoints. An API specification defines these calls, meaning that it explains how to use or implement them. One purpose of APIs is to hide the internal details of how a system works, exposing only those parts that a programmer will find useful, and keeping them consistent even if the internal details change later. An API may be custom-built for a particular pair of systems, or it may be a shared standard allowing interoperability among many systems. There are APIs for programming languages, software libraries, computer operating systems, and computer hardware. A REST API is an API that conforms to the design principles of the REST, or representational state transfer architectural style. For this reason, REST APIs are sometimes referred to RESTful APIs. #### Practical example: Simple "todo list" web in python: ```python from flask import Flask, request app = Flask(__name__) todo_list = ['feed the cats', 'pay the utilities'] @app.route("/", methods=['GET', 'POST']) @app.route("/homepage") def homepage(): return '

Hello, world

' @app.route("/api/v1/ping", methods=['GET']) def ping(): return 'pong', 200 @app.route("/api/v1/todo", methods=['GET', 'POST']) def todo_api(): if request.method == "GET": return todo_list, 200 if request.method == "POST": todo_item = request.values.get("todo_entry") todo_list.append(todo_item) return 'OK', 200 app.run(port=5009) ``` #### Playing with the app Before we can continue set up a new python environment, install flask and run the app. **Web browser** - GET home page and check request and response - POST home page and check request and response - GET, POST, GET the todo api and check requests and responses **curl** `curl` is a computer software project providing a library and command-line tool for transferring data using various network protocols. The name stands for "Client for URL". We use it on the terminal. To check if you have it run `curl --version`. To send a GET request using `curl` we would type `curl URL`, and to send data using POST request we would type `curl --data-urlencode "KEY=VALUE" URL`. So, let's try it on our little web app: ``` $ curl localhost:5009

Hello, world

$ curl localhost:5009/homepage

Hello, world

``` As we can see both the root route (implied `/`) and the `/homepage` return same data. This would be our regular HTTP(S) website. In the server output we can see what was requested: ``` * Serving Flask app 'simple_todo' * Running on http://127.0.0.1:5009 Press CTRL+C to quit 127.0.0.1 - - [15/Nov/2023 00:09:48] "GET / HTTP/1.1" 200 - 127.0.0.1 - - [15/Nov/2023 00:11:39] "GET /homepage HTTP/1.1" 200 - ``` Line `127.0.0.1 - - [15/Nov/2023 00:11:39] "GET /homepage HTTP/1.1" 200 -` tells us client on address 127.0.0.1 sent a GET request for /homepage and our app returned 200. Now lets check the API part. We have a `/api/v1/ping` endpoint that can tell us if the server is alive by returning a simple string: ``` $ curl localhost:5009/api/v1/ping pong ``` The API part also has `/api/v1/todo` endpoint that supports both `GET` and `POST`, let's try it: ``` $ curl localhost:5009/api/v1/todo ["feed the cats","pay the utilities"] $ curl --data-urlencode "todo_entry=Learn web development" localhost:5009/api/v1/todo OK $ curl localhost:5009/api/v1/todo ["feed the cats","pay the utilities","Learn web development"] ``` ## Next steps **Reading: REST & CRUD** - [REST - Explained For Beginners](https://htmx.org/essays/rest-explained/) - [What is CRUD?](https://www.codecademy.com/article/what-is-crud) # Javascript recap This section is intended as recap for the knowledge you will need to build your next learning project. ## JSON JavaScript Object Notation - is a set of text formatting rules for storing and transferring data in a machine and human readable way. It looks a lot like the object literal syntax of JavaScript, and it is from there JSON originates. But JSON is not JavaScript. Officially it's a totally different language with its own specification but it plays such a big part in JavaScript and web development that it's important to cover. Example JSON: ```json { "name": "Yoda", age: 894, "lightsaber" : { "color": "green" } } ``` JSON is used to transfer information - between your browser to a server, or saved in text files for retrieval later - because it's simply text. That means you can't store complex data like a function, but you can store arrays, objects containing simple data, strings and numbers. To use JSON in javascript we use methods called stringify and parse. To convert javascript object into JSON string: ```js var jsonString = JSON.stringify({ make: "Volkswagen", model: "Golf", miles: 230000 }); ``` And to `parse` JSON string into javascript object: ```js var car = JSON.parse(jsonString); ``` now we can use the car javascript object normally: ```js car.model = "T-Roc"; ``` ## AJAX Until the late 1990s, most websites returned complete web pages when you visited a page on the site. To reload any data, you had to reload the complete page. AJAX solves this. Asynchronous JavaScript and XML (AJAX) is a programming technique that allows web developers to fetch server data without having to wait for the page to reload first. What makes AJAX possible is a web browser's built-in `XMLHttpRequest` (XHR) object. Example: ```js // create our XHR object const request = new XMLHttpRequest(); // create function and event listener that will get executed // once the request is successfully finished function requestListener() { console.log(this.responseText); } request.addEventListener("load", requestListener); // finally create and send the request request.open("GET", "/api/v1/todo"); request.send(); ``` Even though we call it Asynchronous JavaScript and **XML** we usually send and receive JSON, not the now outdated XML. AJAJ does't sound so cool tho :) Learn more about [AJAX](https://developer.mozilla.org/en-US/docs/Web/Guide/AJAX). Fetch API is the evolution of AJAX. Learn more about it [here](https://www.javascripttutorial.net/javascript-fetch-api/) and [here](https://web.dev/articles/introduction-to-fetch) before jquery. ## Events & callbacks In the browser most code is **event-driven** and writing interactive applications in JavaScript is often about waiting for and reacting to events, to alter the behavior of the browser in some way. Events occur when the page loads, when user interacts (clicks, hovers, changes) and myriad other times, and can be triggered manually too. To react to an event you listen for it and supply a function which will be called by the browser when the event occurs. This function is known as a callback. As we have seen in the AJAX example javascript had method available called [`addEventListener`](https://developer.mozilla.org/en-US/docs/Web/API/EventTarget/addEventListener). Here is an example of javascript code for button click event: ```js var handleClick = function (event) { // we can do anything here console.log("Button was clicked!") }; var button = document.querySelector('button'); // find the button element button.addEventListener('click', handleClick); // "listen" for the `click` event ``` `addEventListener` is a method found on all DOM elements. Here it's being called on an element saved in the button variable. The first argument is a string - the name of the event to listen for. Here it's `click` - that's browsers name for a click of the mouse or a tap of the finger on the touch screen. The second is the callback function - here we defined it as `handleClick`. Another handy pattern is to wait for the whole page to finish loading (rendering html, css, pulling all the resources) before we start running our scripts and logic. To do that we can simply place our code inside event listener that waits for 'DOMContentLoaded' event on the `document`. ```js document.addEventListener('DOMContentLoaded', function () { //our code goes here } ``` ## Checkpoint **Self-learning: Javascript** So far you should know basic javascript and the following concepts: - [scope](https://www.htmldog.com/guides/javascript/intermediate/scope/) - [objects](https://www.htmldog.com/guides/javascript/advanced/oo/) - [creating elements](https://www.htmldog.com/guides/javascript/advanced/creatingelements/) - [errors and exceptions](https://www.htmldog.com/guides/javascript/advanced/errors/) - [local storage](https://www.htmldog.com/guides/javascript/advanced/localstorage/) ### Project task: "To Do" web app A simple html/css/js "To Do" app is provided for you (download the HTML file [here](./demo/lecture3_todo.html)). The web server that serve the app on the `/` or `/homepage` route also provides `/ping` route to check if the server is up, and a simple CRUD API for "To Do" items (server file [here](./demo/lecture3_todo.py)). **API Reference**: ``` GET app_location/ - returns homepage GET app_location/ping - returns "pong", 200 GET app_location/api/v1/todo - returns [ {"task": "feed the cats", "status": "todo"}, {"task": "pay the utilities", "status": "todo"} ] POST app_location/api/v1/todo - accepts "todo_task" value - returns "OK", 200 PUT app_location/api/v1/todo - accepts "todo_task" value - accepts "todo_status" value ("done" or "todo") - returns "OK", 200 DELETE app_location/api/v1/todo - accepts "todo_task" value - returns "OK", 200 ``` **Your assigment**: Update the provided app using plain javascript to use the todo app's API to: 1. fetch and display list of todo tasks on the page load (GET) 2. add new tasks using the API (POST) 3. mark tasks as done (PUT) 4. delete tasks (DELETE) If the time allows: - implement server check and tell user server is offline if the server is unreachable - implement [local storage](https://developer.mozilla.org/en-US/docs/Learn/JavaScript/Client-side_web_APIs/Client-side_storage) to store the data until server is back online ## jQuery JavaScript is the most widely-used language for adding dynamic behavior to web pages. The JavaScript community contributes to a collection of libraries that extend and ease its use. Furthermore the built in set of tools used to allow modification and control of the DOM are a bit of mess because they differ across browsers, generally for historical reasons. To make your job easier, a number of javascript libraries have been created that hide these differences, providing a more uniform way of interacting with the DOM. Often they also provide AJAX functionality, taking the burden of that complexity away from you. jQuery is one of the most popular DOM libraries and it is used on a huge number of websites. !!! By the way, the way you interact with the DOM (or any service that you use via code) is called an Application Programming Interface or API. We use the browser provided DOM APIs to interface with the DOM. Let's take a look an example of how JavaScript is used to add dynamic behavior to a web page (don't worry about understanding the code). ```js const login = document.getElementById('login'); const loginMenu = document.getElementById('loginMenu'); login.addEventListener('click', () => { if(loginMenu.style.display === 'none'){ loginMenu.style.display = 'inline'; } else { loginMenu.style.display = 'none'; } }); ``` In this example, JavaScript is used to apply behavior to an HTML element with id login. The behavior allows a user to click a LOGIN button that toggles a login form. The code below accomplishes the same behavior with jQuery. ```js $('#login').click(() => { $('#loginMenu').toggle() }); ``` In this example, the same toggle functionality is accomplished using just three lines of code. So, the jQuery, it has a very distinctive syntax, all based around the dollar symbol: ```js $('.btn').click(function () { // do something }); ``` This example code attaches a click handler to all elements with a class of btn. This selector syntax is core to jQuery. (compare this to `addEventListener`, much shorter, isnt it!) Query syntax consists of selecting HTML elements and then performing some sort of action on them. ```js $(selector).action() ``` The following syntax is used: - **$()**: A leading function used to access jQuery. - **selector**: Uses CSS selector syntax for selecting HTML elements. - **action**: One of the many jQuery methods to be performed on the selected element(s). It is best practice to perform jQuery actions only after the document is finished loading, otherwise, some actions might fail. To insure this, it is best to enclose jQuery code inside a document-ready event: ```js $(document).ready(function(){ // jQuery code goes here }); ``` This can be shortened further to: ```js $(function () { //our code goes here }); ``` We saw example how to do that without jQuery (using plain javascript) in the previous lesson: ```js document.addEventListener('DOMContentLoaded', function () { //our code goes here } ``` To use jQuery in your page simply include it as a file using a script element, ```html ``` To see whats available in jQuey check the [jQuery API docs](https://api.jquery.com/), for another tutorial check [Sway Howes' jQuery](https://learn.shayhowe.com/advanced-html-css/jquery/). **AJAX** jQuery also has some AJAX helper methods that save time and are much easier to read, so the AJAX example from previous lesson would look like this in jQuery: ```js $.ajax({ url: '/api/v1/todo', method: 'GET', success: function (data) { console.log(data); } }); ``` ## MVC: Model, View, Controller MVC is short for Model, View, and Controller. MVC is a popular **way of organizing your code**. The big idea behind MVC is that each section of your code has a purpose, and those purposes are different. Some of your code holds the data of your app, some of your code makes your app look nice, and some of your code controls how your app functions. MVC is a way to organize your code's core functions into their own, neatly organized boxes. This makes thinking about your app, revisiting your app, and sharing your app with others much easier and cleaner. The parts of MVC: - **Model**: Model code typically reflects real-world things. This code can hold raw data, or it will define the essential components of your app. For instance, if you were building a To-do app, the model code would define what a “task” is and what a “list” is - since those are the main components of a todo app. - **View**: View code is made up of all the functions that directly interact with the user. This is the code that makes your app look nice, and otherwise defines how your user sees and interacts with it. - **Controller**: Controller code acts as a liaison between the Model and the View, receiving user input and deciding what to do with it. It's the brains of the application, and ties together the model and the view. **An analogy** MVC is a way to think about how an web application works. It's kind of like how you make Thanksgiving dinner. You have a fridge full of food, which is like the Model. The fridge (Model) contains the raw materials we will use to make dinner. You also probably have a recipe or two. A recipe (assuming you follow it exactly) is like the Controller of Thanksgiving dinner. Recipes dictate which stuff in the fridge you'll take out, how you'll put it together, and how long you need to cook it. Then, you have table-settings, silverware, etc., which are what your hungry friends and family use to eat dinner. Table-top items are like the View. They let your guests interact with your Model and Controller's creation. **MVC in the real-world** MVC is helpful when planning your app, because it gives you an outline of how your ideas should be organized into actual code. For instance, let's imagine you're creating a To-do list app. This app will let users create tasks and organize them into lists. The Model in a todo app might define what a “task” is and that a “list” is a collection of tasks. The View code will define what the todos and lists looks like, visually. The tasks could have large font, or be a certain color. Finally, the Controller could define how a user adds a task or marks another as complete. The Controller connects the View's add button to the Model, so that when you click “add task,” the Model adds a new task. ## Next steps Upgrade the "To Do" app to use jQuery. \pagebreak # Last steps !!! WARNING: Construction area - this part of the lesson is unfinished ## Deployment - explain hosting [url1](https://web.dev/articles/introduction-to-fetch), [url2](https://www.reddit.com/r/webdev/comments/of3nte/different_types_of_web_hosting_explained/) - then host https://developer.mozilla.org/en-US/docs/Learn/Getting_started_with_the_web/Publishing_your_website - host shared hosting? (ftp?) - install nginx? ## Final project **Design** - pair on design - simple jquery-based app (plan project together, revise on APIs) - use bootstrap or bulma for css, add custom styling on top. **Implementation** - use jquery for the js side. - the idea for this app is to have a copy of an github user page - profile image, - users name and nickname - the list of repositories - optionally: ability to specify which user to pull data for - protip: API endpoint for a user https://api.github.com/users/frainfreeze **Deployment** - finally, host the app on server ## Next steps !!! ERROR: Read this before venturing on - [Modern JavaScript Explained For Dinosaurs](https://peterxjang.com/blog/modern-javascript-explained-for-dinosaurs.html) - [Modern CSS Explained For Dinosaurs](https://medium.com/actualize-network/modern-css-explained-for-dinosaurs-5226febe3525) Keep in mind these are by now old articles and things changed further, but they serve as good checkpoint in time, overview and a roadmap for study. If you want to be a complete web developer you will also need to: - learn more javascript: [eloquentjs](https://eloquentjavascript.net/), [YDKJS](https://github.com/getify/You-Dont-Know-JS) - learn about package managers - learn about build tools (module bundlers, task runners, linters and formaters) - pick and learn a framework (vue.js/react/svetle) - learn about css preprocessors - learn about testing (unit, intergation and functional tests) - learn more about web security - learn about typescript - learn about GraphQL - learn about Server Side Rendering (SSR) - know what Static Site Generators are - know what Progressive Web Apps (PWA) are - know that its possible to create mobile apps using web tech (for eg. React native or Ionic) - know that its possible to create desktop apps using web tech (for eg. Electron) A good place to start would be the [FreeCodeCamp curriculum](https://www.freecodecamp.org/learn) and the [roadmap.sh](https://roadmap.sh/frontend) To prepare for the job interview check - [Front End Interview Handbook](https://www.frontendinterviewhandbook.com/), - [Front-end Developer Interview Questions](https://h5bp.org/Front-end-Developer-Interview-Questions/) Good luck!