OverviewOnFiles Up
OverviewOnFiles Up
Email : [email protected] 1
STORING MEDIA
There are different levels of storing media:
1- PROCESSOR Very fast (a few ns), volatile, size of a few machine words.
REGISTERS
2-CACHE MEMORY BETWEEN Faster than MM (a few tens of ns), volatile, very small size.
PROCESSOR AND MAIN MEMORY
(MM)
5-DIFFERENT TYPES OF SECONDARY Mainly magnetic disks (or hard drives HDD) and flash-disks
STORAGE (SS) (SSD...), slower than MC, non-volatile, large size and less
expensive than MC.
Registe
r
CPU Primary
storage &
Caches low level
Archiving memories
MEMOIRES
Low level memories
Flip-Flops: Responsible for storing and maintaining a single binary bit of information
until a new input signal updates their state.
Register: Refers to a small high-speed storage unit within the central processing unit
(CPU) of a computer, used to temporarily hold data that is actively being processed or
manipulated during computational tasks.
Caches memories
Level 1 Cache: A very small, extremely fast memory unit located
directly within the central processing unit (CPU) that stores
frequently accessed instructions and data to speed up
processing.
CPU
Level 2 Cache: A larger memory unit located between the
central processing unit (CPU) and the main memory, serving as Cache Cache Cache Main
an intermediate storage space for frequently accessed data, L1 L2 L3 Memory
thereby improving overall system performance.
Unlike dynamic memory, it does not need to periodically refresh its content.
SRAM (Static RAM) Like dynamic memory, it is volatile (In the primary role, SRAM serves as
cache memory, acting as an interface between DRAM and the CPU) →
(faster than DRAM).
SDR SDRAM (Single Data
1993 A high-speed computer memory that transfers data at every clock cycle
Rate Synchronous Dynamic
RAM)
Double Data Rate is a type of computer memory technology that
DDR SDRAM (Double Data 2000 transfers data on both the rising and falling edges of the clock
Rate Synchronous signal, effectively doubling the data transfer rate compared to
DDR1, DDR2, SDR (Single Data Rate) memory
Dynamic RAM)
DDR*
Uses lasers to read and write data on optical discs, such as CDs, DVDs,
Optical storage and Blu-ray discs, for the purpose of storing and retrieving digital
information
8
THE HISTORY OF STORAGE TECHNOLOGIES
Cloud
• 2000 CD & DVD
• Storage solution Solid State USB & microSD • 1980-2000
accessible via the Drive(SSD) • 2000 • Optical storage for
internet
• 1970 ~ 2005 • Removable storage for software, music,
• Based on NAND flash saving, transferring, and with larger capacity
memory, faster and more transporting data
durable than HDDs
9
TYPES OF STORAGE MEDIA Hard drive (HDD)
Platter
❑ Disks: A disk is a magnetic disk covered with a thin layer of magnetic Motor
particles (platters). Data is stored in the form of bits on the disk. Read/Write
❑ Platter: A circular disk made of a magnetic material where data is stored. Actuator
10
TYPES OF STORAGE MEDIA Hard drive (HDD)
11
TYPES OF STORAGE MEDIA
12
TYPES OF STORAGE MEDIA
Solid State Driver (SSD)
❑ The memory is divided into blocks (e.g., 256 KB) and each
block is composed of a certain number of pages (e.g., 4 KB).
….
BLOC
There are 3 possible operations : Page Page Page Page Page 64
K
1 2 3 4
❑ Read a page → very fast (20 microseconds)
13
TYPES OF STORAGE MEDIA
….
BLOCK
Page Page Page Page Page 64
1 2 3 4
❑ Blocks support a fixed number of erasures, beyond which they become unusable. It is therefore
necessary to distribute erasures uniformly across all blocks of the disk ⇒ This is the
WearLeveling technique.
❑ When updating the content of a page, it is preferable to write the new content to a new page
(already erased).
❑ Physical page numbers must therefore be hidden from users. ⇒ This is the role of the FTL (Flash
Translation Layer)
EXTERNAL MEMORY ACCESS INTERFACE
Regardless of the type of disk (HDD or SSD), the interface remains the same: The unit of transfer = a
physical block
Prog_4 Prog_5
Physical Bloc _4 Bloc _5 Bloc _6 Bloc _7
I/O
Buffer _1 Buffer _2 Bloc _... Bloc _... Bloc _... Bloc _...
ReadBloc(i)
WriteBloc(i,X)
Buffer
Buffer _3 Bloc
_m Bloc _... Bloc _... Bloc _...
_N-1
Cache
Main Memory Secondary Storage
17
Power
Memory Connector
Processor Slots
FS
B
Clock BIOS
ARCHITECTURE Generator
Daughter
S-ATA Storage
Cards
Data from the hard drive is loaded Once the data is in RAM, the
into RAM. CPU can access it.
19
DATA STRUCTURE
❑ A data structure is an organized arrangement of different data that takes into account not only the
recorded entities themselves, but also their interconnections, based on their usage context, such as
how a student is linked to grades, and patients to treatments.
Student Patient
Name String Name String
Modules Table or List of String Medical tests & report Liste (String and float )
Prescriptions and doses Liste (String and float)
Notes Table or List of float
20
DATA STRUCTURE
Storage
Output processing
Storage
Data Structure
Interne Memory
Externes
Raw Data
Human Machine -
Input processing
File Structure
Interface
Users Retrieve 22
FILES
A file is the concept through which a program or application stores data in external memory.
Files representation
Regardless of the file type, it is used at different levels of abstraction with different semantics.
Logical level
(Applicatif)
Physical level
(Interne)
Users Developers, application users (data entry agent, doctor,...) Operating systems, file managers
23
NOTIONS OF FILES AT THE LOGICAL LEVEL
File: A file at the logical level (application, "high-level programming language") is considered as:
❑ A large list of logically linked information, addressed in the form of records composed of different fields (typed).
String Name;
int Age; // Integer (size of 32 bits)
❑ Or a stream of raw bytes without a well-defined structure. These streams generalize the notion of I/O to be independent of the
type of device used (Non-typed).
24
NOTIONS OF FILES AT THE LOGICAL LEVEL
Data Structure Structured in records with defined fields Raw byte stream without defined structure
Data Types Specific data types for each field No explicit specification of data types
Easier to interpret due to the defined structure, even with basic software More difficult to interpret due to the absence of structure:
Ease of Interpretation
(Notepad) requires dedicated software (dev++ to run the executable)
Flexibility Less flexible in terms of data types More flexible for storing various data types
Example Relational database, CSV, Excel files Binary files (images, audio, executables), text files
25
NOTIONS OF FILES AT THE LOGICAL LEVEL
In the case of typed files, records are formed by a set of fields (or attributes). Among these fields, one or more can
play the role of search key .
Student
ID_etudiant String
Search
Name String
Key
Age Integer
Modules Table or liste of String
Marks Table or list of float
Example : Search for students over 20 years old and group them in the same section.
26
NOTIONS OF FILES AT THE
PHYSICAL LEVEL
Rec Rec
Rec 1 Rec 4 Rec 7
10 n-2
Rec Rec
Rec 2 Rec 5 Rec 8
11 n-1
Rec
Rec 3 Rec 6 Rec 9 Rec n
12
Access to the contents of the blocks is done through input/output (I/O) operations. 27
NOTIONS OF FILES AT THE PHYSICAL LEVEL:
FILES IN AN FMS
Concept Description
- Files are divided into data blocks of fixed or variable
size.
- Each block is identified by a block number or a
physical address.
Data Blocks
- Files are spread across multiple blocks due to their
size.
- The data of a file is stored sequentially in the order of
addition.
- Each FMS has its own structure for organizing files
and data blocks.
File Management - Stores information such as the location of blocks,
System (FMS) authorizations, and metadata.
- The bytes in the data blocks are not interpreted by the
file system.
- The FMS does not know if the data is text, images,
etc.
Uninterpreted Bytes
- The interpretation of the data is the responsibility of the
applications that read the files. Note: The NTFS file system of Windows uses
- FMSs manage the allocation and deallocation of block MFT (Master File Table) to store metadata about
Space Management spaces. files (characteristics and information).
- They maintain a table to track the blocks used by files. 28
OVERVIEW ON FILE CHARACTERISTICS
❑ Files characteristics
Characteristic Description
File Name Unique identifier of the file.
File Extension Indicates the type/format of the file content.
Size Quantity of data contained in the file.
Content Type Nature of the data (text, image, audio, etc.).
Location The physical location where the file is stored.
Creation/Modification Date Timestamp of creation and last modification.
Access Permissions Controls access to the file (read, write).
File Formats Specific format defining the structure.
Program Associations Associated program for opening the file.
29
FILE CHARACTERISTIC MANAGEMENT
❑ Information Storage : This information is stored in reserved locations on the disk. For example, a characteristic
table can be stored at a fixed address on the disk.
❑ Information Retrieval : When an application wants to open a file, the system retrieves its information from the
characteristic table.
❑ Usage in Sequential Secondary memory: In sequential storage media like magnetic tapes, this characteristic
information was generally placed at the beginning of each file, called a header block.
❑ Specific Characteristics : Certain applications also need to manage specific characteristics for manipulating their
data files. These characteristics can be stored at the beginning of the file, before the data.
30
FILE OPERATIONS AND ACTIVITY
❑ File Operations : Represents the flow of input/output of file data between the processor, RAM, and secondary storage :
❑ Creating a file.
❑ Inserting records or fields into a file.
❑ Modifying data within the file.
❑ Reading data from a file.
❑ Deleting a record or a file.
❑ Merging files or splitting a file
❑ File Activity : A set of indicators for all the operations and actions that can be performed on a file.
Description
Consultation Rate Frequency of file reading by users or applications.
Renewal Rate Frequency of file updates or modifications.
Popularity Relevance and usefulness of the file for users or applications.
Version History Number of different versions of the file created over time.
Consultation Time Duration during which the file is open or read.
Parallel Access Number of users or applications simultaneously accessing the file.
Backups and Restaurations Frequency of file backups to prevent data loss. 31
BUFFER CACHE
All operations at the application level are translated into low-level operations to access the physical blocks of the storage
memory(SM). slow !!!!!!!!!!!!
Prog 1
Use A A D
The system maintains a buffer cache in main memory to temporarily A A A A
store copies of physical blocks, using strategies to choose the most Prog 2 B C
relevant ones Use B A
A C B B B
Prog 1
Use B B B
❑ During physical reading, the system copies the block into the buffer Main Memory Secondary Memory
area. If there is no space, it can overwrite an existing block by first
saving it to disk, depending on the replacement strategy. I/O physical
❑ Updates to records by applications are first made in the buffer area in main memory. Physical writing to disk is deferred until later
(for example, during synchronization).
Note : To prevent data loss in case of a failure, the system synchronizes its cache with secondary storage by
periodically writing all modified blocks in main memory.
32
SUMMARY OF THE INTERACTION BETWEEN LOGICAL AND PHYSICAL
LEVELS
Physical level
MM
Processor
7 6 F F 1. Application Instruction
1 1
Caches 2. Operating System (OS) and
33
FILE STRUCTURES: DESCRIPTION
34
FILE STRUCTURES: ACCESS AND FUNCTIONS
Bloc 1 Rec 1
Rec 2 Ahmed
Rec 3 Benarab
19 y
F3 F3 F3 F3 F1 F1 F1
Bloc 4 Leila
F2 F3 F3 Soualmi
Rec 13 26 y
F2 F2 F2 F1
Rec 14
F3
When dealing with files and optimizing performance, two main criteria are generally taken into account: the
number of physical Input/Output operations (I/O) and the occupation of memory space.
Number of Input/Output Number of read/write operations to/from physical storage Minimize the number of I/O operations to
(I/O) (hard drive, SSD, etc.). improve performance.
36
ABSTRACT MACHINE FOR THE
CONCEPTUALIZATION OF FILE MANIPULATION IN
MEMORY
User HMI hardware Hardware
- OS
- Driver
- FMS
Absract
Machine -Programs
Open
Allocate -Data Bases
Write Management
Characteristics
.
.
Close
How ?
❑ Is a conceptual (virtual) model that describes how file management
operations are performed within a computer information system.
❑ F1 A file of a vector of BLOCKS in the BUFFER T1. Type TBloc = Struct // Content of the bloc
tab : Table [MaxE] of Tenreg // Table of record
NE : integer // Number of records 0<NE<MaxE
❑ F2 A file of list of BLOCKS in the BUFFER T2
End.
❑ F3 A file of a vector of BLOCKS in the BUFFERS T3,
F3 : FILE of TBloc BUFFER buff HEADER(type1,
T4
type2, ...typem);
❑ For each file, there are header-blocks to ensure
proper manipulation of files and these data blocks.
38
ABSTRACT MACHINE FOR FILE MANIPULATION IN
MEMORY: MODELS AND ALGORITHMS
❑ To write algorithms on file structures, we will use the abstract machine defined by
the following model:
❑ In this model, we manipulate block numbers relative to the beginning of each file
(these are therefore virtual numbers (VCN)).
❑ The use of physical addresses is not particularly useful at this level of presentation.
❑ In this model, a file is therefore a set of virtually numbered blocks (1, 2, 3, ...n).
39
ABSTRACT MACHINE FOR FILE MANIPULATION IN
MEMORY: MODELS AND ALGORITHMS
Mode 'A': Opens an existing file for both reading and writing. The term 'A' likely stands for 'Ancien'.
Open( F , Filename , mode )
Mode 'N': Creates a new file for reading and writing. The term 'N' stands for ‘New’.
The characteristics are allocated in the MM during the creation of the file.
Closes the file.
Close( F )
File characteristics are saved in secondary storage ‘SS'
ReadDIR( F , i , buf ) Reads the contents of the i-th block of file 'F' and stores it in the variable 'buf'
WriteDIR( F , i , buf ) Writes the contents of the variable 'buf' into the i-th block of file 'F'.
Returns the value of the i-th characteristic associated with file 'F'. This could be information like file size,
Header( F , i )
creation date, permissions, etc.
Set_ENTETE( F , i , v ) Sets the value of the i-th characteristic of file 'F' to 'v'. This allows you to modify file attributes.
Allocates a new block to file 'F' and returns the number of the newly allocated block. This is used for extending
ALLOC_BLOC( F )
a file.
40
ABSTRACT MACHINE FOR FILE MANIPULATION IN
MEMORY: ALGORITHMS (EXAMPLE)
CONST MaxE=10; // Maximum number of records
Patients : FILE of TBloc BUFFER T1, T2,.. HEADER (NE1 : entier , NE2: entier );
41
ABSTRACT MACHINE FOR FILE MANIPULATION IN
MEMORY: ALGORITHMS (EXAMPLE)
//Print the content of file
BEGIN
43
FILES IN THE C LANGUAGE:
DEFINITION
❑ When implemented by the execution support, a file is seen as a linear
sequence of bytes without a particular structure.
❑ The elements of a file are therefore not typed, it is up to the programmer to
ensure their management.
❑ Files in C are manipulated by the <stdio.h> library.
Text files contain human-readable ASCII or Unicode characters, Binary files can contain any sequence of bytes, including data that is
typically organized in lines of text. not human-readable.
Larger than binary files for storing the same amount of More compact because they store raw data without any character
information. overhead.
Text files can be easily read and modified by humans using text
Binary files are not human-readable and often require specific software
editors.
to be interpreted.. 44
FILES IN THE C LANGUAGE: OPENING AND
CLOSING (TEXT / BINARY)
Opening :
read
write/create
append (add to the end of file) Opening mode
Action
read Text Binary
Read "r" "rb"
write Write "w" "wb"
❑ FILE * If everything goes well (Access to data) Read/Write at the fin "a+" "ab+"
❑ NULL If error (ex: inexistent file, Absence of access rights . . . )
FILES IN THE C LANGUAGE: OPENING AND
CLOSING (TEXT / BINARY)
Closing :
//fclose(FILE *file)
❑ Disconnect the file pointer from the physical file.
fclose(FILE *file)
47
FILES IN THE C LANGUAGE:
OPENING AND CLOSING (TEXT
FILES)
// Exemple: Lire un fichier et le recopier dans un autres fichiers
1. #include <stdio.h>
By line: fgets / fputs 2. #include <stdlib.h>
fgets: 3. int main() {
char *fgets(char * str, int size, FILE *file) 4. FILE *source = fopen("source.txt", "r");
5. if (source == NULL) {
❑ Used to read a complete line of text from a 6. printf("Error while loading the file");
file. 7. return 1;
8. }
❑ It takes three main arguments :
9. // Open the file in writing mode
▪ str : A pointer to a character array where
10. FILE *destination = fopen("destination.txt", "w");
the read line will be stored.
11. if (destination == NULL) {
▪ size : The maximum length of the line to
12. printf("Error while loading the file");
read.
13. fclose(source);
▪ file : A pointer to the file.
14. return 1; }
15. char line[2048]; // a table to store the retrieved line from the file
16. while (fgets(line, sizeof(ligne), source) != NULL) {
fputs: 17. // if the line not empty, write it in the file
18. if (line[0] != '\n') {
int fputs(const char * str, FILE *file);
19. fputs(ligne, destination);
20. }
❑ Used to write a whole line of text to a file 21. }
22.
23. fclose(source);
24. fclose(destination); return 0;
25. }
48
OPENING AND CLOSING (TEXT
FILES) #include <stdio.h>
struct Patient {
char name[100];
int age;
Frormatted R/W Formater : fscanf / char diagnostic[100]; };
struct Patient patients[3];
fprintf
fscanf: int main() {
fscanf(FILE *file, const char *format, ...); FILE *file = fopen("patients.txt", "w");
if (fichier == NULL) {
❑ Used to read formatted data from a text file printf ("Erreur lors de l'ouverture du fichier"); return 1; }
for (int i = 0; i < 3; i++) {// fprintf to write the patient data into the file
fprintf(file, "Name: %s\n", patients[i].name);
fprintf(file, "Age: %d\n", patients[i].age);
❑ It takes two main arguments : fprintf(file, "Diagnostic: %s\n", patients[i].diagnostic);
▪ file : A pointer to a character array the file fprintf(file, "\n"); } // an empty line to separate the patients’ records
text. fclose(fichier);
▪ format : A format string specifying how the file = fopen("patients.txt", "r");
data should be read (%d, %s, %f, ....) if (fichier == NULL) {
printf("Error while loading the file"); return 1; }
for (int i = 0; i < 3; i++) {// fscanf to read and print the data from the file to the screen
fscanf(file, "Name: %s\n", patients[i].name);
fprintf: fscanf(file, "Age: %d\n", patients[i].age);
fprintf(FILE *fichier, const char *format, ...); fscanf(file, "Diagnostic: %s\n", patients[i].diagnostic);
fgetc(fichier); // read the empty line that exists at the end of each record
// print the data
❑ Used to read formatted data to a text file printf("Nam: %s\n", patients[i].name);
printf("Age: %d\n", patients[i].age);
printf("Diagnostic: %s\n", patients[i].diagnostic);
}
fclose(fichier);
return 0;}
49
FILES IN THE C LANGUAGE: READING / WRITING (BINARY FILE)
❑ fread and fwrite are the two standard C library functions needed to read and write binary data from/to
binary files
❑ These functions are particularly useful for manipulating raw data, such as structures or arrays, and
storing or extracting them from binary files.
fread : fwrite :
fread(void *ptr, size_t size, size_t count, FILE fwrite(const void *ptr, size_t size, size_t count, FILE
*file); *file);
❑ ptr : A pointer to the memory area where the read data ❑ ptr : A pointer to the memory area where the data will be stored.
are stored.
❑ size : The size in bytes of each element to be read. ❑ size : The size in bytes of each element to be written.
❑ count : The total number of elements to be read. ❑ count : The total number of elements to be written .
❑ file : A pointer to the binary file from which to read the ❑ file : A pointer to the binary file in which to write the data.
data.
50
FILES IN THE C LANGUAGE: READING / WRITING (BINARY
FILE)
1. #include <stdio.h>
2. // Record to represent a patient
3. struct Patient {
4. char name[100]; 1. // Open binary file for reading
2. file = fopen("patients.dat", "rb");
5. int age;
3. if (file == NULL) {
6. char diagnostic[100]; }; 4. printf("Error while loading the file"); return 1; }
7. struct Patient patients[3]; // create a table of patients 5. printf("Patients data retireved from the files :\n");
8. int main() { 6. // read the data from the binary file
9. //Fill the table with the patients’data 7. size_t elements = fread(patients, sizeof(struct Patient), 3, file);
10. struct Patient patient1 = {"Ahmed Benarab", 35, « Flue"}; 8. if (elements != 3) {
11. struct Patient patient2 = {"Leila slimani", 28, « Covid19"}; 9. printf (" Error while loading data from the file s");
12. struct Patient patient3 = {" Laamri Moustfaoui", 82, " Parkinson"}; 10. } else
patients[0] = patient1; patients[1] = patient2; patients[2] = patient3; 11. { for (int i = 0; i < 3; i++) {
13. // Open binary file for writting 12. printf("Patient %d:\n", i + 1);
14. FILE *file = fopen("patients.dat", "wb"); 13. printf("Name: %s\n", patients[i].name);
15. if (file == NULL) { 14. printf("Age: %d\n", patients[i].age);
15. printf("Diagnostic: %s\n", patients[i].diagnostic);
16. printf("Error while loading the file"); return 1; }
16. printf("\n"); }
17. // Write data in the file 17. }
18. size_t element = fwrite(patients, sizeof(struct Patient), 3, file); 18. fclose(file);
19. if (elements != 3) 19. return 0;
20. { printf(« Error while uploading data in the file "); 20. }
21. }
22. else {
23. printf(« Success.\n"); }
24. fclose(file);
51
FILES IN THE C LANGUAGE: CHANGING POSITION (BINARY FILE)
fseek :
1. #include <stdio.h>
int fseek(FILE *file, long offset, int origin);
2. int main() {
3. FILE *file = fopen("data.bin", "rb");
4. if (fichier == NULL) {
❑ The function fseek() in the C language is used to move the 5. printf("Error while loading the file"); return 1; }
6. // Move the pointe with 2 Bytes from the actual position
file position pointer within a binary file. It allows you to specify 7. int val= fseek(file, 2, SEEK_CUR);
the new position for reading or writing within the file 8. If(val!=0) {
❑ Parameters: 9. printf("Error while moving the pointer to the indicated
▪ File : A pointer to the binary file. position");
10. fclose(file); return 1; }
▪ Offset : The amount of movement, in bytes, relative to the
11. // At this point, the file position pointer has been moved 2
specified origin. bytes from its initial position.
▪ Origine : The origin from which to perform the movement. This 12. if (feof(file)) { // Test if we reached the end of the file
can be one of the following constants : 13. printf("\nEnd of file .\n"); }
▪ SEEK_SET : Move from the beginning of the file.. 14. else {
▪ SEEK_CUR : Move from the current position of the pointer. 15. printf("\nThe pointer is not at the end of file.\n");
▪ SEEK_END : Move from the end of the file. 16. }
17. fclose(file);
18. return 0;
19. }
Note: feof(File * file) allows us to test if we have reached the
end of the file 52
FILES IN C LANGUAGE: RECAPITULATIVE
fgets Reads a line of text (up to a newline character) from the file. char *fgets(char *string, int size, FILE *file);
fputs Writes a string of characters to the file. int fputs(const char *string, FILE *file);
fscanf Reads formatted values from the file. int fscanf(FILE *file, const char *format, ...);
fprintf Writes formatted values to the file. int fprintf(FILE *file, const char *format, ...);
fread Reads a block of binary data from the file. size_t fread(void *ptr, size_t size, size_t count, FILE *file);
fwrite Writes a block of binary data to the file. size_t fwrite(const void *ptr, size_t size, size_t count, FILE *file);
fseek Moves the file position pointer within the file. int fseek(FILE *file, long offset, int origin);
53