Web Application Development

Jakub Klinkovský

:: Czech Technical University in Prague
:: Faculty of Nuclear Sciences and Physical Engineering
:: Department of Software Engineering

Academic Year 2024-2025

Network Communication Models

OSI Model

The OSI model was created in 1983 as an abstract foundation for standardizing communication protocols.

  • it is divided into 7 layers, each with well-defined functions
  • it does not specify a specific set of protocols

Internet Protocol Suite (TCP/IP)

The Internet Protocol Suite is a set of communication protocols used in the Internet network.

  • it has only 4 layers, it is a practical application of the OSI model
  • the name comes from the two most important protocols: TCP and IP

Comparison of OSI and TCP/IP Models

center contain

The lowest layer allows access to the physical transmission medium. It is specific to each network depending on its implementation.

Examples of networks: Ethernet, Wi-Fi, ADSL, PPP, ...

  • takes care of physical data transmission
  • basic unit of transmission: frame (in the OSI model: bits on the physical layer)

TCP/IP – Network Layer (Internet Layer)

This layer primarily provides network addressing, routing, and datagram forwarding.

  • independent of the specific transmission technology (uses the network access layer)
  • implemented in all network elements (routers and end devices)
  • main protocol: IP (IPv4, IPv6)
    • basic transmission unit: IP datagram
    • addressing mechanism: IP address (e.g. 192.168.0.10, fe80::862a:fdff:fe0f:10)
    • routing scheme for data transmission
  • other protocols: ARP, RARP, ICMP, IGMP, IGRP, IPSEC, ...

TCP/IP – Transport Layer

Provides transport services for pairs of application programs (endpoints), i.e. entities immediately above the layer.

  • implemented only in end devices (computers)
  • the level of reliability of data transmission depends on the chosen protocol (according to the needs of the application)
  • allows adapting the behavior of the network to the needs of the application

TCP/IP – Most Common Transport Layer Protocols

  • TCP (transmission control protocol)
    • reliable transport protocol (resends lost packets, ensures delivery in the correct order)
    • from the application's perspective, data transmission appears as a continuous stream
    • after establishing a connection, the transmission is fully duplex (data flow in both directions)
  • UDP (user datagram protocol)
    • unreliable protocol (does not provide integrity control)
    • provides efficient (fast) data transmission at the expense of reliability
    • unlike TCP, it does not have a connection establishment and termination phase
  • TCP and UDP use ports for addressing data to individual applications

TCP/IP – Ports in the TCP Protocol

A network port in the TCP and UDP protocols is a numerical designation of a communication point for a given connection.

  • the recipient of the data is the process that is currently connected to the given port
  • the relationship between the port and the application is dynamic (processes dynamically connect and disconnect from individual ports)
  • most common target port numbers: 22 (SSH), 53 (DNS), 80 (HTTP), 443 (HTTPS)

TCP/IP – Application Layer

The application layer includes protocols that serve to transmit specific data.

Examples:

  • DHCP – automatic configuration of network devices (IP address, network mask, default gateway, DNS server, and more)
  • DNS – hierarchical, decentralized system of domain names
    • mapping symbolic names (domains) to IP addresses
    • e.g. jlk.fjfi.cvut.cz: top-level domain cz, domain cvut, subdomain fjfi, etc.
  • HTTP – communication between web servers and clients (see below)
  • Telnet, SSH, FTP, POP3, IMAP, SMTP, ...

Data Flow in TCP/IP Model

contain center

HTTP Protocol

HTTP Communication Protocol

HTTP (Hypertext Transfer Protocol) is a protocol designed for communication with web servers. It is used for transmitting hypertext documents in HTML, XML, and generally any files (e.g. multimedia, text files).

  • usually uses TCP/80 port
  • does not allow encryption – secure protocol HTTPS (TLS connection over TCP/443)
  • client-server architecture: the client sends a request (request) and receives a response (response) from the server
  • it is stateless – it does not maintain information about previous connections and sent data (recall REST)

HTTP Protocol Operation

  1. establishing a connection
  2. sending a request from the client to the server
  3. sending a response from the server to the client
  4. terminating the connection

The most time-consuming part of loading a document is establishing a connection – especially for pages with many embedded objects. Therefore, various optimizations have been developed:

  • transmitting multiple requests and responses in one connection, maintaining persistent connections with the server (keep-alive) – from HTTP/1.1 version
  • header compression, prioritized request processing – HTTP/2 version
  • transport layer optimization using UDP – HTTP/3 version

HTTP Request Structure

method URL protocol
headers

message body

(Headers and request body are optional.)

Example:

GET /some_page/ HTTP/1.1
Host: localhost:8000
User-Agent: curl/7.88.1
Accept: */*

HTTP Response Structure

protocol status_code reason_phrase
headers

response body

(Headers and response body are optional.)

Example:

HTTP/1.1 200 OK
Date: Sun, 12 Mar 2023 21:35:04 GMT
Server: WSGIServer/0.2 CPython/3.10.9
Content-Type: text/html; charset=utf-8

<!DOCTYPE html><html lang="en">...................................</html>

HTTP Request Methods

The method determines the type of request that the client requests from the server. The client can use any defined method, but the given server may not support all of them.

  • GET – the most common method, a request for a document specified by URL
  • HEAD – identical to the GET method, but the server only sends the response headers
  • POST – sending data from the client to the server using the request body, modifying the server state (e.g. storing data in a database) – example
  • others: PUT, DELETE, CONNECT, OPTIONS, TRACE, PATCH

HTTP Status Codes

Status codes determine the general type of request processing result by the server. They are three-digit numeric codes divided into thematic groups according to the first digit:

  • 1xx – informational – the request has been received, processing continues
  • 2xx – success – the request has been received, understood, and accepted
  • 3xx – redirection – the client must perform further actions to complete the request
  • 4xx – client error – the request contains invalid data or cannot be processed
  • 5xx – server error – processing of a valid request has failed

HTTP Headers

Headers are used to supplement the request or response with metadata.

  • each header is a key: value pair
  • header encoding is either textual (HTTP/1.1) or binary (HTTP/2 and HTTP/3)
  • example request headers:
    User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:110.0) Gecko/20100101 Firefox/110.0
    Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
    Accept-Language: en-US,en;q=0.5
    Accept-Encoding: gzip, deflate, br
    
  • example response headers:
    Content-Length: 767
    Content-Type: text/html; charset=utf-8
    Date: Mon, 13 Mar 2023 09:29:38 GMT
    Server: WSGIServer/0.2 CPython/3.10.9
    
  • list of standard headers (custom keys can also be defined)

HTTP from the Perspective of Client Applications

Python – requests Package

Requests: HTTP for Humans™

  • general function requests.request(method, url, **kwargs)
  • "aliases" for individual HTTP methods:
    • requests.get(url, **kwargs)method == "GET"
    • requests.head(url, **kwargs)method == "HEAD"
    • requests.post(url, **kwargs)method == "POST"
    • ...
  • object representation of the HTTP protocol – exceptions, status codes, transparent encoding/decoding of data in the request/response body (e.g. JSON \leftrightarrow Python dict), headers, authentication, and many advanced features
  • allows only blocking/synchronous processing of requests

Python – httpx Package

HTTPX: A next-generation HTTP client for Python

  • API is mostly compatible with requests:
    • requests.get(...) \leftrightarrow httpx.get(...) etc.
  • provides blocking and asynchronous interfacesasync+await keywords

HTTP from the Perspective of the Server

Python Web Server for Static Web

Python includes the http.server module, which can be used directly for providing static content or for developing dynamic applications.

Direct use:

M:\>python -m http.server --help
usage: server.py [-h] [--cgi] [--bind ADDRESS] [--directory DIRECTORY] [port]

positional arguments:
  port                  specify alternate port (default: 8000)

options:
  -h, --help            show this help message and exit
  --cgi                 run as CGI server
  --bind ADDRESS, -b ADDRESS
                        specify alternate bind address (default: all interfaces)
  --directory DIRECTORY, -d DIRECTORY
                        specify alternate directory (default: current directory)

E.g. python -m http.server --directory C:\Users\klinkjak\Documents

Python Web Server for Static Web – Usage

  1. Open a terminal in the directory where you have the files for INTA
    (or use the cd command to change directories)
  2. Find the subdirectory that contains the application from the previous exercise,
    e.g. 01 (there should be files first-page.html etc.)
  3. Start the web server: python -m http.server --directory 01
  4. In the browser, open the web server address: http://0.0.0.0:8000/first-page.html (or equivalently http://localhost:8000/first-page.html)

Basic Dynamic Web Server

Creating a basic dynamic application using the http.server module is more complex than providing static content, but it still only requires a few lines of code. In the following example, we will demonstrate how it works.

Demo: see scratchpad


Note: Using the http.server module for developing a complete web application is too complicated and impractical. For this purpose, high-level frameworks have been developed, which significantly simplify the work.

![contain center w:800](images/TCP_port.jpg)

--- ## Asynchronous Programming in Python A high-level API for asynchronous operations was introduced in Python versions 3.4 to 3.7: - two new keywords: `async` and `await` (defining a _coroutine_) - standard module [`asyncio`](https://docs.python.org/3/library/asyncio.html) (API for running and managing _coroutines_) _Hello World_ example: ```python import asyncio async def main(): print('Hello ...') await asyncio.sleep(1) print('... World!') asyncio.run(main()) ``` --- ### Working with _coroutines_ 1. `asyncio.run()` – top-level interface for running an `async` function from a _synchronous context_ 2. _awaiting on coroutines_ – using the `await` keyword inside an `async` function - generally, there are 3 types of _awaitable_ objects: - __coroutines__ – Python functions defined as `async` - __Tasks__ – wrapper objects used to schedule _coroutines_ to run concurrently - __Futures__ – special low-level objects that represent the result of some operation that will complete in the future - `await` allows expressing which operations can be overlapped and processed concurrently --- #### Example with several _coroutines_ ```python import asyncio import time async def say_after(delay, what): await asyncio.sleep(delay) print(what) async def main(): print(f"started at {time.strftime('%X')}") await say_after(1, 'hello') await say_after(2, 'world') print(f"finished at {time.strftime('%X')}") asyncio.run(main()) ``` Possible output (note the duration): ```txt started at 17:13:52 hello world finished at 17:13:55 ``` --- ### Concurrent Execution of _coroutines_ `asyncio.create_task()` – schedules a _coroutine_ for concurrent execution __Example:__ ```python async def main(): task1 = asyncio.create_task( say_after(1, 'hello') ) task2 = asyncio.create_task( say_after(2, 'world') ) print(f"started at {time.strftime('%X')}") await task1 await task2 print(f"finished at {time.strftime('%X')}") ``` Note that after this modification, the code will run 1 second faster (processing tasks is overlapped when calling `asyncio.sleep(delay)` in the `say_after()` function). __Note:__ It is still not _parallel processing_ – Python uses only one thread. --- ### Concurrent Execution of _coroutines_ Using `TaskGroup` The [`asyncio.TaskGroup`](https://docs.python.org/3/library/asyncio-task.html#asyncio.TaskGroup) class – a modern alternative to the `asyncio.create_task()` function __Example:__ ```python async def main(): async with asyncio.TaskGroup() as tg: task1 = tg.create_task( say_after(1, 'hello') ) task2 = tg.create_task( say_after(2, 'world') ) print(f"started at {time.strftime('%X')}") # The await is implicit when the context manager exits. print(f"finished at {time.strftime('%X')}") ``` --- ### Synchronization Between Tasks The `asyncio` module provides basic tools for synchronization, similar to the `threading` module: - `asyncio.Lock` - `asyncio.Event` - `asyncio.Condition` - `asyncio.Semaphore` - `asyncio.BoundedSemaphore` - `asyncio.Barrier` --- ### Event Loop - `asyncio.run()` starts a low-level _event loop_ where all asynchronous tasks are scheduled - _coroutines_ themselves are useless until they are bound to an _event loop_ - the default _event loop_ in the CPython interpreter uses __only one thread__ - _event loops_ are _pluggable_ – the default _event loop_ can be replaced with another implementation - e.g. [uvloop](https://uvloop.readthedocs.io/) fast implementation in the Cython language (performance is comparable to Go and other statically compiled languages) - it is possible to develop a multi-threaded _event loop_ (but there is still the GIL limitation – global interpreter lock) --- ### Using Asynchronous Programming in Applications Asynchronous API is useful especially for hiding latency when waiting for data (IO). - HTTP requests – client and server packages (`httpx`, `starlette`) - databases – `sqlalchemy`, ... - web frameworks – Django, FastAPI, ... <br> > __Demo with the `httpx` package:__ see [GitLab](https://gitlab.fjfi.cvut.cz/18inta-2024/inta) and [notes](https://jlk.fjfi.cvut.cz/md/aqQ_xRsjR9ejUNymi7-iNQ) --- ## Asynchronous Programming in JavaScript JavaScript also has asynchronous functions, similar to Python: - `async` and `await` keywords with similar semantics - `async` functions can be called directly – JavaScript does not have an alternative to `asyncio.run()` - `async` functions always return a `Promise` object, which has 3 possible states: _pending_, _fulfilled_, or _rejected_ - `await` on a `Promise` object either returns the result value or raises an exception - same behavior as in Python in terms of code structuring - [interface for concurrent processing](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise#promise_concurrency): `Promise.all`, `Promise.allSettled`, `Promise.any`, `Promise.race` methods > __Demo:__ [Async Functions in JavaScript](https://thecodebarbarian.com/async-functions-in-javascript.html) > __Documentation:__ [async function](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function), [`Promise` object](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise) --- ## JavaScript Fetch API [Fetch API](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) allows asynchronous processing of HTTP requests in JavaScript. It is a modern alternative to the [XMLHttpRequest](https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest) interface. The basis is the `fetch` function: ```js async function f() { try { const url = "https://example.com"; await fetch(url, { mode: "cors" }); } catch(err) { alert(err); // Failed to fetch } } ``` __Note:__ Browsers block "unsafe" cross-origin requests (CORS) for security reasons. Details: [Fetch: Cross-Origin Requests](https://javascript.info/fetch-crossorigin).

TODO: advanced CORS - https://javascript.info/fetch-crossorigin - https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS