System Programming

(ELEC462)
Class Introduction

Dukyun Nam
HPC Lab@KNU
Contents
● Class Introduction
● Course Schedule
● Course Perspective
● Why Linux?
● Exascale Era
● Labs

2
Class Introduction
● Course name
○ System programming (ELEC0462)
● Prerequisites
○ Introduction to programming (COMP0204)
○ Data structure (COME0331)
○ Data structure applications (COMP0216)
● Lectures
○ Tuesday 9am-1pm (ELEC0462-001)
○ Office hour: Tuesday 2pm - 3pm
■ Please request an appointment via email
3
Class Introduction (cont.)
● Instructor
○ Dukyun Nam, Ph.D
○ Office: IT-4 #402
○ email : dynam at knu.ac.kr
● TA
○ 강현우 (cosmos041389 at gmail.com)
● Tutors
○ ELEC0462-001
• 김영훈 (xoals128 at naver.com)
• 홍지승 (wltmd3847 at naver.com)
4
Class Introduction (cont.)
● Textbook
○ [UULP] B. Molay, Understanding UNIX/LINUX Programming: A Guide to Theory and Practice,
Pearson, 2002
○ (Optional) [LPI] M. Kerrisk, The Linux Programming Interface, No Starch Press, 2010
■ https://learning.oreilly.com/library/view/the-linux-programming/9781593272203/
○ (Optional) [APUE] W.R. Stevens and S.A. Rago, Advanced Programming in the UNIX Environment,
Third Edition, Addison-Wesley Progressional, 2013
■ https://learning.oreilly.com/library/view/advanced-programming-in/9780321638014/

5
Class Introduction (cont.)
● Grading policy
○ Midterm exam 25%
○ Final exam 35%
○ Assignment 40%
■ 4~5 homeworks
■ 12~13 lab assignments

○ Lateness penalties: 50% per day

6
Course Schedule (Tentative)
● 1st half ● 2nd half
○ 1 - Class introduction ○ 9 - Processes and program
○ 2 - Users, files, and the manual ○ 10 - A programmable shell
○ 3 - Directories and file properties ○ 11 - I/O redirection and pipes
○ 4 - Focus on file systems ○ 12 - Connecting to processes
○ 5 - Connection control near and far

○ 6 - Programming for Humans ○ 13 - Socket programs

○ 7 - Event-driven programming ○ 14 - Threads

○ 15 - Final exam
○ 8 - Midterm exam
7
Simple Survey
● How many students are familiar with …
○ Operating system - Linux, Windows, …
○ Language - C, C++, Java, Python, Ruby, Scala, …
○ Editor - vim, sublime text, …
○ IDE - Visual studio code, IntelliJ, PyCharm, Eclipse, …
○ Version-control SW - git, svn, cvs, …
○ Container - Docker, Singularity, …
○ Cloud - AWS, Google cloud, MS Azure, IBM cloud, …

8
 Course Perspective
● Most Systems Courses are Builder-Centric
○ Computer Architecture
■ Design pipelined processor in Verilog
○ Operating Systems
■ Implement sample portions of operating system
○ Compilers
■ Write compiler for simple language
○ Networking
■ Implement and simulate network protocols

* These slides are modified from slides at http://csapp.cs.cmu.edu/3e/instructors.html 9

Course Perspective (cont.)
● This Course is Programmer-Centric
○ By knowing more about the underlying system, one can be more effective
as a programmer
○ Enable you to
■ Write programs that are more reliable and efficient
■ Incorporate features that require hooks into OS
● e.g., concurrency, signal handlers

10
Why Linux?
● Linux is good for education and research
○ Linux is open-source and well-specified
● Linux is good for programming
○ Linux is a variant of Unix
○ Unix has GNU, a rich open-source programming environment

11
Why Linux? (cont.)
● Linux is ubiquitously found on various types of hardware (wikipedia)

12
 High Performance Computing
• Supercomputer (from wikipedia)
• “a computer with a high level of performance as compared to a
general-purpose computer”
• Simplified view of HPC
• 3 axes of compute, I/O, and communication
performance

* Future Directions for NSF Advanced Computing Infrastructure to Support U.S. Science and Engineering in 2017-2020, 13
National Academies Press, 2016.
Exascale Era
● First exascale machine (May 30, 2022)
○ Frontier, ORNL in Tennessee, USA

14
Top 500
● The TOP500 project
○ “... ranks and details the 500 most powerful non-distributed computer
systems in the world. … publishes an updated list of the supercomputers
twice a year.” (wikipedia)
● Current list

15
Top 500 (cont.)

16
Top 500 (cont.)

17
Top 500 (cont.)
● Operating System Share (November 2023)
○ https://www.top500.org/statistics/list/

18
System Programming
(ELEC462)
Introduction

Dukyun Nam
HPC Lab @ KNU
Contents
● Background

● Introduction
● System Resources
● Unix from the User Perspective
● Unix and Linux
● Summary
● Appendix: vi editor

20
Background
● Computer
○ An electrical circuit-based machine that performs processes, operations
and calculations based on instructions defined as a software

Galaxy Nexus
(Samsung & Google, 2011)
IBM Q (IBM, 2019)

ENIAC
The first programmable, electronic,
general-purpose digital computer
(1945)

Summit (ORNL, 2018) Mac Studio (Apple, 2022) 21

Background (cont.)
● Abstraction layers in modern systems

22
The von Neumann Architecture

● 3(4) Basic Components of a Computer

○ 1. Memory: a long but finite sequence of cells (1D)
■ Each cell has a distinct address
■ Data in each cell: instruction, data or the address of another cell
○ 2. Control Unit: Fetches instructions from memory and decodes them
○ 3. Arithmetic Logic Unit: Does simple math operations on data
○ (Optional) 4. Input/Output: The connections with the outside world
23
 Introduction
● What is system programming, or systems programming?
○ The activity of programming computer system software
■ Computer programming is the process of performing a particular computation
■ System software is software designed to provide a platform for other software

* https://en.wikipedia.org/wiki/Systems_programming
24
* https://www.geeksforgeeks.org/software-concepts/
 Introduction (cont.)
● Application programming vs system programming
○ Application programming aims to produce software which provides
services to the user directly
■ e.g., Word processor
○ System programming aims to produce software and software platform
which provides service to other softwares

* https://en.wikipedia.org/wiki/Systems_programming 25
Introduction (cont.)
● The simple program model
○ Many programs are based on the model as shown below
■ A program: a piece of code that runs in a computer
● Data go into the program; the program does some tasks with the data, and the output
comes out of the program
● A human may type at a keyboard and monitor; the program may read or write to a disk; the
program may send data to a printer; many possibilities!

26
< An application program in a computer >
Introduction (cont.)
● The simple program model
○ The keyboard and screen are both connected to the program
■ Tip) GCC reports “errors” with the source file name and line number

< How application programs see user I/O >

27
Introduction (cont.)
● Face reality
○ There are several keyboards and displays, there may be several disks, one or
more printers, and there are certainly several programs running at once
○ Programs that get input from the keyboard and send output the display or
to the disk work fine

28
< Reality: many users, programs, and devices >
Introduction (cont.)
● The role of the operating system
○ To manage and protect all the resources and to connect various devices to
various programs
○ Operating system: called a kernel
○ Memory: space to store programs and data
■ User space
• Contains users programs
■ System space
• The operating system is stored

< An operating system is a program

>
29
Introduction (cont.)
● Providing services to programs
○ If a program wants to connect to or control any the devices, it needs to ask
the kernel
○ Access to the external objects are services the kernel provides to user
programs

< The kernel manages all connections > 30

System Resources
● Processors
○ A program: a set of (machine) instructions compiled from source code
○ A processor (CPU): the hardware executing instructions
■ The kernel assigns processors to programs
■ The kernel starts, pauses, resumes, and ends the execution of a program on a
processor

● Input/output
○ All data that flow in and out of a program go through the kernel
■ e.g., Data coming from/going out to users at terminals or disks cannot travel without
passing through the kernel
31
System Resources (cont.)
● Process management
○ Process: a program in execution
■ Consisting of i) memory, ii) open files, and iii) other system resources (e.g., socket,
device driver, etc.) needed
○ The kernel creates new processes and schedules them to work
cooperatively
● Memory
○ Computer’s memory is a resource.
○ The kernel:
■ Keeps track of which processes are using which sections of memory, and
■ Protects the memory of one process from being damaged by another process.
32
System Resources (cont.)
● Devices
○ One can attach all sorts of devices to a computer
■ e.g., Mice, scanners, printers, keyboards, USB drives, iPhones, ..
○ The kernel provides access to devices and takes care of complexities that
may derive from communications with the attached.
● Timers
○ Some programs depend on time.
■ e.g., Periodic tasks (by cron), sleep, elapsed time, …
○ The kernel makes timers available to processes.

33
System Resources (cont.)
● Interprocess communication (IPC)
○ In a real world, people need to communicate via phone, email, …
○ In a computer system running simultaneous programs, processes need such
communication as well
○ The kernel provides a variety of IPC forms
● Networking
○ A network allows processes on different computers, even on different
operating systems for data exchange
○ Network access is a kernel service
34
Our Goal
● Exploring the following questions:
○ What are the details of each type of service?
○ How does data get from a device to the program and back?
○ …
● We want to understand:
○ What the kernel is doing
○ How the kernel does it
○ How to write programs that use those services

35
Our Method
● Three simple steps to learn about system services by
○ 1. Looking at “real” programs
■ We will study standard Unix/Linux programs to see what they do and how they are
used in practice
■ We will see what system services these programs use
○ 2. Looking at the system calls
■ We will learn about the system calls we use to invoke these services
○ 3. Writing our own version
■ Once we understand the program, what system services it uses, and how those
services are used, we are ready to develop own system programs
■ Our own programs will be extensions of existing programs or ones that use similar
principles
36
Our Method (cont.)
● Three questions that will be asked throughout our course
○ 1. What does that (kernel service) do?
○ 2. How does that work?
○ 3. Can I try to do it?

37
Unix from the User Perspective
● Log in - Run programs - Log out
○ You login, run some programs, and then log out
○ The computer is the box at the left part of the picture
■ Inside the computer is memory that holds the kernel and user processes
■ The shell prints a prompt to tell the user it is ready to run a program
• e.g., date, fortune, etc.

< The kernel manages all connections >

38
Unix from the User Perspective (cont.)
● Working with directories
○ A tree of directories

< Part of the directory tree >

39
 Unix and Linux
● Unix is an operating system first developed by AT&T Bell Labs in the late 60s.
○ Primary contributors: Ken Thompson (B lang) and Dennis Ritchie (C Lang)
○ Written in C;
○ Commonly used in internet servers, workstations, etc
○ e.g., OS X, Solaris, BSD (Berkeley Software Distribution)
● Linux:
○ Developed by Linus Torvalds (responsible for kernel), combining GNU (GNU’s Not Unix)
tools from Richard Stallman
○ A “Unix-like” operating system: user-level interface very similar to Unix
○ Code base is different from original Unix code
○ Several distributions: e.g., Ubuntu, Fedora, SUSE, RedHat, …
* https://opensource.com/article/18/5/differences-between-linux-and-unix 40
 Unix and Linux (cont.)
Unix Linux
Cost Different cost structures according to vendors Freely distributed, but licensed versions

Development & Mostly AT&T Developed by

Distribution Open Source Community

Manufacturer Solaris (Oracle), AIX (IBM), Developed by the community, but Linus Torvalds
HP-UX (HP), OS X (Apple) oversees

Usage Internet Servers, Workstations, and Finance IT Cell phones, tablet PCs, video game consoles,
Infra with 24x7 availability mainframes/supercomputers.

Security 85 -120 viruses 60-100 viruses

See how amazing these are compared with that of Windows!

Inception Developed by AT&T employees at Bell Labs and Inspired by MINIX (a Unix-like system), later added many
Dennis Ritchie in 1969. Written in “C” language. features like GUI, and then Linus Torvalds developed it in
Designed to be a portable, multi-tasking and multi-user 1992.
system in a time-sharing configuration.

* https://www.diffen.com/difference/Linux_vs_Unix 41
Summary
● The purpose of the book is to explain how Unix works and to
demonstrate how to write programs that use Unix system services
● The chapter explains that the Unix kernel is a program that manages
multiple programs and system resources and connects programs to
resources
● The method used to explain these ideas is to study and write
common Unix programs

42