System Programming overview

• Application Vs System Programming

• C language and UNIX Introduction
• Running Program on a system and Program
translation
 Introduction and Essential Concepts of
System Programming
• This title is about system programming, which
is the practice of writing system software.
• The System Programming topic concentrates
on how programs run in user space and how
interact with the OS.

• What is System software?

 Cont. …
• System Software is a set of programs that control
and manage the operations of computer hardware. It
also helps application programs to execute correctly.
• System Software are designed to control the
operation and extend the processing functionalities
of a computer system.
• System software makes the operation of a computer
more fast, effective, and secure.
• System software lives at a low level, interfacing
directly with the kernel and core system libraries.
• List some examples of System programing
 Some system programing software
• Your shell and your text editor,
• your compiler and your debugger,
• your core utilities and system daemons are all system
software.
• But so are the network server, the web server, and the
database.
• These components are entirely system software, primarily if not
exclusively interfacing with the kernel and the C library.
• Other software (such as high-level GUI applications) lives at a
higher level, delving into the low level only on occasion.
• Traditionally, all Unix programming was system-level
programming. Unix systems historically did not include many
higher-level abstractions.
 Application Software
• Application Software acts as a mediator
between the end-user and System Software.
• It is also known as an application package.
• This type of software is written using a high-
level language like C, Java, VB. Net, etc.
• It is a user-specific and is designed to meet the
requirements of the user.
 Application Vs. System Programming
• We can compare and contrast system programming with
application programming, which differ in some important
aspects but are quite similar in others.
• System programming’s hallmark is that the system
programmer must have an acute awareness of the
hardware and the operating system on which they work.
• Where system programs interface primarily with the kernel
and system libraries,
• Application programs also interface with high-level
libraries. These libraries abstract away the details of the
hardware and operating system
 KEY DIFFERENCES:
• System software are designed to manage the resources of the system, like
memory and process management, security, etc. whereas Application
software are designed to fulfil the requirements of the user for
performing specific tasks.
• The System Software is a general-purpose software while the Application
Software is specific purpose software.
• System Software is written in a low-level language like a machine or
assembly language but Application software is a high-level language is
used to write Application Software.
• System Software is capable of running independently while Application
software can't run independently.
• The System Software starts running when the system is powered on and
runs until the system is powered off while the Application Software starts
when the user begins, and it ends when the user stops it.
• System software are independent of the application software while an
Application software needs system software to run.
 Operating system
• Separates applications from the hardware they access
– Software layer
– Manages software and hardware to produce desired results
• Operating systems primarily are resource managers
– Hardware
• Processors
• Memory
• Input/output devices
• Communication devices
– Software applications
 Key. …
• All operating systems provide services for
programs they run.
• Typical services include executing a new
program,
• opening a file,
• reading a file,
• allocating a region of memory,
• getting the current time of day, and so on.
 Core Operating System Components
• User interaction with operating system
– Often, through special application called a shell
– Kernel
• Software that contains core components of operating system
• Typical operating system components include:
– Process scheduler
– Memory manager
– I/O manager
– Inter process communication (IPC) manager
– File system manager
 The Function of OS
• Two main functions:
• Manage physical resources:
– It drives various devices
• Eg: CPU, memory, disks, networks, displays, cameras,
etc
– Efficiently, reliably, tolerating and masking failures, etc
• Provide an execution environment to the applications
running on the computer (programs like Word, Emacs)
– Provide virtual resources and interfaces
• Eg: files, directories, users, threads, processes, etc
– Simplify programming through high-level abstractions
– Provide users with a stable environment, mask failures
 Operating System Goals
• Users expect certain properties of operating
systems
– Efficiency
– Robustness
– Scalability
– Extensibility
– Portability
– Security
– Protection
– Interactivity
– Usability
 Linux Systems
linux torvalds is authore of the Linux kernel
They are manly used in main frame computer and super
computer. But only around 1% in desk top computers.
• services provided by various versions of the UNIX operating
system?
-init
-logins from terminal
Syslog (recording erors)
Graphical user interface
Networking
Network file system,mail,print,file system lay out
…
 Cornerstones of System Programming
• There are three cornerstones of system programming in Linux: system calls, the C library, and
the C compiler. Each deserves an introduction.
System Calls
• System programming starts and ends with system calls.
• System calls (often shortened to syscalls) are function invocations
made from user space—your text editor, favourite game, and so on—
into the kernel (the core internals of the system) in order to request
some service or resource from the operating system.
• System calls range from the familiar, such as read() and write(), to the
exotic, such as get_thread_area() and set_tid_address().
• Linux implements far fewer system calls than most other operating
system kernels. For example, a count of the x86-64 architecture’s
system calls comes in at around 300, compared with the suspected
thousands of system calls on Microsoft Windows
 Invoking system calls
• It is not possible to directly link user-space applications with
kernel space. For reasons of security and reliability, user-space
applications must not be allowed to directly execute kernel
code or manipulate kernel data.

User Mode
Mode bit = 1 Resume process
User process System Call

Trap Kernel Mode Return

Mode bit = 0 Mode bit = 0 Mode bit = 1

Save Caller’s state Execute system call Restore state

 The C Library
• The C library (libc) is at the heart of Unix applications.
• Even when you’re programming in another language, the C
library is most likely in play, wrapped by the higher-level
libraries, providing core services, and facilitating system call
invocation.
• On modern Linux systems, the C library is provided by GNU libc,
abbreviated glibc, and pronounced gee-lib-see or, less
commonly, glib-see.
• The GNU C library provides more than its name suggests.
• In addition to implementing the standard C library, glibc
provides wrappers for system calls, threading support, and
basic application facilities.
 The C Compiler
• In Linux, the standard C compiler is provided by the GNU
Compiler Collection (gcc).
• Originally, gcc was GNU’s version of cc, the C Compiler.
• Thus, gcc stood for GNU C Compiler. Over time, support was
added for more and more languages.
• Consequently, nowadays gcc is used as the generic name for
the family of GNU compilers.
• However, gcc is also the binary used to invoke the C compiler.
• The compiler used in a Unix system—Linux included—is
highly relevant to system programming, as the compiler
helps implement the C standard
 C language and UNIX review

• The UNIX operating system’s development started in 1969, and

its code was rewritten in C in 1972.
• The C language was actually created to move the UNIX kernel
code from assembly to a higher level language, which would do
the same tasks with fewer lines of code.
• Oracle database development started in 1977, and its code was
rewritten from assembly to C in 1983. It became one of the most
popular databases in the world.
• Linux kernel development started in 1991, and it is also written
in C. The next year, it was released under the GNU license and
was used as part of the GNU Operating System.
• The GNU operating system itself was started using C and Lisp
programming languages, so many of its components are written
in C.
 Program translation
• On the very earliest computers programs were written and entered in
binary form.
• Some computers required the program to be entered one binary word
at a time, using switches on the front panel of the computer, one
switch for each bit in the word.(video earlyprogramming 7 th min) (
https://youtu.be/nwDq4adJwzM )
• Because of that
– the size and the complexity of the programs were severe limited
– the debugging was very difficult task
– in general, development of the programs was very difficult
and error prone.
Idea: use the computer itself to ease the programmer’s work
by translation from a more human-readable form of the
program into executable binary code.
 Assembly and High-level languages.
• First translator programs, known as the assemblers, were available at
the beginning of 1950’s.
• Translation from assembly languages to the binary form was fairly
straightforward (on command-by-command basis)
• Later, the high-level procedural languages have appeared.
– FORTRAN (1957), Algol (1958), COBOL (1960)
– today there are hundreds of third-generation high-level
programming languages (C, Pascal, BASIC, etc.)
• Based on function-oriented programming concept, these
languages attempted to free the programmer from the
details of the machine’s instructions.
• Translator for these languages became known as compilers
• different type of high-level language translator, known as an
interpreter, appeared around 1962
 Compilers vs. Interpreters.
• Both compilers and interpreters are used to
convert a program written in a high-level language
into machine code understood by computers.
• However, there are differences between how an
interpreter and a compiler works.
• A compiler operates on the entire program,
translate it and generating a permanent binary
module representing the program. This module
can then be executed.
• An interpreter translate source code and executes
it, one source code program line at a time.
 Compilers vs. Interpreters.
• Execution of interpreted code is much more slowly
than compiled code.
• However, when a program is to be executed only once
or twice, or when a program is frequently modified, it
is easier and faster not to have to compile in order to
run the program.
• The translation process for interpreters is simplified,
since the code does not need to take into account
different possible conditions; only actual conditions
do matter.
• Some techniques are more easy to implement with
interpreter than with a compiler, e.g self-modifying code.
 Programming Tools Overview
• Compilers
• Editors
• Linkers
• Assemblers
• Loaders
• Debuggers
• Interpreters

Integrated Development Environments (IDEs) combine several of

the above programming tools
 The Program Translation Process

Source

Translator Object

Linker

Loader

Executable
Programming Language Categories
• Machine Language
– Binary coded instructions
• Assembly Language
– Symbolic coded instructions
• Procedural Languages
– procedural statements or arithmetic notation
• Object-oriented Languages
– Combination of objects and procedures
 Assembly Language
• When to use
– When speed or size of program is critical
– Hybrid approach
– Hardware Drivers
– Can use specialized instructions
• Disadvantages
– Inherently machine specific
– Architectures may become obsolete
– Lack of programming structure
 Assembly …
• Assemblers like a compiler,
– a translator from source code to target code;
– it converts symbolic machine code to binary machine
code, and
– from symbolic data to binary data.
• Linker
– combines one or more program object files and probably some library
objects files into one executable file.
• Loader
– An integrated part of the operating system (which makes
it essentially invisible) which loads the content of an executable code
file into main memory.
 Compilers
• Translates high-level language into low-level
instructions
• High-level language: Source code
• Machine-level: Object code
• Changes, including bug fixes, require
recompiling
 Compilation Steps
• Preprocessing (#):- modify the source code before it is
compiled.
• translation (compiling):- The modified source code is
compiled into binary object code. This code is not yet
executable
• assembling mnemonics: A mnemonic is a term, symbol or
name used to define or specify a computing function.
• Linking:-the object code is combined with required
supporting code to make an executable program. This step
typically involves adding in any libraries that are required.
 Language Components
• The lexical component
– All legal words in the language
– elementary expressions
– Meaning and type
• The syntactical component (Syntax)
– Syntax define the form and structures of legal expressions.
– grammar rules
– separate statements with a semi-colon
– enclose the conditional expression of an IF statement inside
parentheses
– data types and variables must be declared before the first
executable statement
 Language components …
• Semantics
– deals with the meaning of the expressions.
– For the programming languages semantic component
defines a course of action to be performed, while
executing a particular (fragment of a) program.
 Why semantic and syntax rule?
• The precise description of a programming language is needed
for the translator:
– to analyze of the (source) program, and
– to generate correct and unambiguous code
• Key part of the precise description of any Programming
language is the syntax rules that make up its grammar
• These rules are used in the parsing process.
• The parsing process consists of determining which rule applies
in a particular instance as one traces through the sentence.
 Pre-processing.
• Pre-processing is a kind of translation from a
high-level code to some other high-level code.
• Sometime it is used for the translation of object-
oriented (fourth-generation) programming
languages:
– pre-processing translates a source code (say, in C+
+) into the code in some lower-level programming
language (e.g. in C)
– then the compilation of the result is performed
The Compilation Process
 Lexical analysis (scanning)
• text of the program is broken down into the smallest
possible meaningful language components (tokens).
• These will consist e.g. number constants, identifiers
(labels, variable names), etc.
–symbol table will be build of the various tokens
used in the program, including the information on
the type of every token
(e.g. comparative token, variable token, etc.)
– Thus, the program text is reduced to the sequence
of tokens.
 Lexical analysis … example
#include <stdio.h>
Lexeme Token
int maximum(int x, int y) { Int Keyword
// This will compare 2 maximum Identifier

numbers ( Operator
int Keyword
if (x > y) x Identifier
return x; , Operator
int Keyword
else
Y Identifier
{ ) Operator
return y; { Operator
If Keyword
}
}
 Symbol Table
• Symbol table: is a data structure which holds
different information, whether that
information(variable is defined or not).
• Compiler create and maintains symbol table to
keep detail about the following:
• Variable names
• Function name
• Objects
• Classes
• Interfaces
 Symbol table … function
• For analysis and synthesis, symbol table is used by
compiler
• It helps to determine the scope resolution
• It helps to determine whether a variable is defined
already or not.
• To add a new name to the table
• To access information associated with a given name
• To add new information with given name
• To delete a name or group of names from table
• Type checking for semantic correctness determination
 Symbol table … example
Int var1;
Int prcA()
Var 1 var Int
{ procA Proc Int
Int var2,var3; procB proc int
-----
{int var 4, var 5;
---} Symbol table(procA Symbol table(procB
Int var 6; Var 2 var Int Var 9 var Int
---
Var 3 var Int Var 10 var Int
{--
Int var 7, var 8; Var 6 var int Var 13 var int
--}}
Int procB()
{int var9,var10;
----
Var4 var Int Var 11 var Int
{ Var 5 var Int Var 12 var Int
Int var 11, var 12;
---- Inner scope -1 Inner scope -3
}
Int var 13; Var7 var Int
} Var 8 var Int
Inner scope -2
 Syntax analysis.
• Once lexical analysis is complete, the compiler moves to the
syntax analysis stage to parse the result of lexical analysis into
individual productions.
• Most difficult part of compilation process, especially
because of detecting possible syntactical mistakes in
programs.
• Semantic analysis is usually active at the same time.
– It is responsible in particularly for the detecting data types
inconsistencies.
 Code generation.
• Memory locations and registers will be
assigned to the various data objects: variables,
arrays, etc.
• The code will be generated for each of the
productions identified during the syntax and
semantic steps
 Optimization
• Compiler analyzes code in order to
– Reduce amount of code
– Eliminate repeated operations
– Reorganize parts of the program to execute faster and more efficiently
– Use computer resources more effectively
• Different compilers can produce different results!
 Process of Parsing
• Lexical analysis
– Also known as scanning
– Divides the string of input characters into single
elements, tokens, based on strict computer
punctuation
• Syntactic analysis
– Checks for errors in grammar rules
• Semantic parsing
– Determines the meaning of the string
 Linking

Object file

Object Executable
file or object Linker
file
module

C library
 Linkers… function
• Searches program libraries to find library routines
used by the program
– Library: collection of pre-written functions and
subroutines made available to perform commonly required
activities
• Determines the memory locations that code from
each module will occupy and relocates instructions
by adjusting absolute references
• Resolves references among files
 Why Link?
• Construct single executable program from multiple
object code files compiled at different times
• Program can be subdivided into components and
parceled out to different developers
• Example
– Main program and multiple subroutines written and
compiled by different programmers at different times
 Loader
• Loads binary files that have been linked into main
memory
• Program is ready for execution
• Determines size of the text and data segments
• Creates a new address space for the program
• Copies program from the executable into the new
address space
• Copies arguments to the stack
• Initializes the machine registers
• Jumps to a startup routine and calls the main
 Conclusion
• In this lecture we tried to discuss:
• System programing essential concepts, what is
System programing, what is the d/c b/n Syst
soft and app soft, why we have to learn
system programing? Corner stone of system
programing
• Little introduction to OS