IIT Bombay

Computer Programming
Dr. Deepak B Phatak
(with Dr. Supratik Chakraborty)
Department of Computer Science and Engineering
IIT Bombay

Session: Handling data in external files

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 1

Objectives
IIT Bombay

• Introduction to files and standard input/output

• Read and write data from/to text files
• Variable length ‘records’, delimited by ‘newline’ (/n)
• Create binary files
• May contain records, images, audio, video, etc.
• Introduction to direct access and update (binary files)

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 2

 IIT Bombay

Introduction

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 3

Memory Hierarchy
IIT Bombay

• Main Memory (Electronic, very fast)

• Cache Memory
• Solid state Disks
• Magnetic (Mechanical disks)

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 4

Memory Hierarchy
IIT Bombay

• Main Memory (Electronic, very fast, volatile)

• Few hundred MB to few GB capacity
• Cache Memory(Electronic, fast, volatile)
• Solid state Disks (EEPROM, fast, non-volatile)
• Typically 256 GB to 512 GB, up to 1 TB
• Pen drives are typically 8 GB to 64 GB
• Magnetic disks (Mechanical, slow, non-volatile)
• Typically 1 TB, 8 TB drives now available

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 5, Up to 16

Introduction
IIT Bombay

• Each file stored on an external device has

• Name, extension, path (location), size, access permissions
• All files are handled by the Operating System
• A C++ program defines and uses FILE objects (pointers)
• Name of the object is the internal filename for C++
• It has to be “associated” with an external file
• A ‘file’ is essentially a sequence of Bytes

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 6

Example of a digital image
IIT Bombay

• A file contains a photograph of a scene

• N rows by M columns
• Each byte stores intensity value (0-255) for a ‘pixel’
• The image has a poor contrast
• All intensity values are in a small range (between 120 – 200)

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 7

A grayscale picture
IIT Bombay

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 8

Histogram equalization of a digital image
IIT Bombay

• A simple technique to improve contrast

• We can ‘stretch’ the histogram to cover all possible
values
– This is called histogram ‘equalization’
– We will get a better contrast
• We need to ‘map’ existing pixel values to new values
– A value ‘v’ should be mapped by a function to h (v)
[120 mapped to 0, 200 mapped to 255]
• Use cumulative distribution function (cdf) of histogram
Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 9
Picture with enhanced contrast
IIT Bombay

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 10

Original picture, equalization, modified picture
IIT Bombay

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 11

Overview of text files
IIT Bombay

• Text data may be independently entered in a text file using

any text editor
• Data in several other file-types can be saved as plain text
• Such as data from a spread sheet
• C++ can read data from, and write data to text files

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 12

Standard input/output
IIT Bombay

• The operators ‘cin’ and ‘cout’

• Read/write a sequence of ASCII characters
• stdin (standard input file)
• stdout (standard output file)
• These files are automatically connected to keyboard and
terminal
• Can be redirected to actual files on disk

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 13

Redirection
IIT Bombay

$ myprog < infile.txt > outfile.txt

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 14

Formatted input/output
IIT Bombay

• Standard input and output

• cin and cout
• Special functions to perform formatted input and output
• Functions: cout.width(…)
int n = -77;
cout.width(6);
cout << left << n << endl;
- 77
cout.width(6);
cout << right << n << endl;
- 77
Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 15
Another method
IIT Bombay

• setw(…)
• Header file: iomanip
• Same example where n is -77

cout << setw(6);

cout << n << endl; - 77

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 16

 IIT Bombay

Read and write data (text files)

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 17

Classes to read text file IIT Bombay

• Stream classes for handling files in C++

ofstream: for write in files
ifstream: for reading from files
fstream: for reading and writing
• Header file: fstream

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 18

Program 1: Reading and printing text file IIT Bombay

• Open the file

• Check whether it was opened successfully
• Read line by line till End of File
• Print each line
• Close the file

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 19

Program 1: Reading and printing text file IIT Bombay

#include<iostream> ifstream myfile("inputdata.txt", ios::in);

#include<fstream> if (myfile.is_open()) {
#include<string> while(getline(myfile,line)) {
using namespace std; cout << line << endl;
}
// read_text_file.cpp myfile.close();
int main() { }
string line; else cout << "unable to read file" << endl;
return 0;
} // End of main
Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 20
Example of a spread-sheet
IIT Bombay

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 21

CSV data (Comma Separated Values)
IIT Bombay

10101,Anil Shah,112,12.5
10102,Amit Jadhav,111,15
10103,Shefali Pandya,112,17
10104,Rajesh Mashruwala,111,19
10105,Nandan Meshram,111,16
10106,Avinash Adsule,112,14
10107,Srikant Rao,112,14.5
10108,Nilmani Raut,111,11.5
10110,Rajesh Singh,112,10
10115,Ketan Maheshwari,111,12
Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 22
A program to process data from a CSV file
IIT Bombay

• Read one line from the input file in a string

• string linestr;
For example, the first line is: 10101,Anil Shah,112,12.5
• Separate the four parts in four different strings
• Now put these four values, separated by blank spaces, in the
output file
• Repeat this procedure to process all lines from the input file

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 23

Program logic
IIT Bombay

While (read a line till EOF){

Process input string, separate parts in four strings
Prepare an output text string with these four values
write this to output file
count number of records read!
}

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 24

Separating parts in four strings
IIT Bombay

|1|0|1|0|1|,|A|n|I|l| |S|h|a|h|,|1|1|2|,|1|2|.|5|

sroll
sname
sbatch
smarks

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 25

Program 2: Reading and writing
IIT Bombay

#include<iostream>
#include<fstream>
#include<string>
#include<sstream>
#include<iomanip>
using namespace std;

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 26

Program …
IIT Bombay

int main() {
string linestr, sroll, sname, sbatch, smarks;
int count = 0;
ifstream fin("CSV_data.txt", ios::in);
if(!fin.is_open()){
cout << "Could not open file" << endl;
return -1;
}
Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 27
Program …
IIT Bombay

ofstream fout("marks_data.txt", ios::out);

if(!fout.is_open()){
cout << "Could not open file" << endl;
return -1;
}

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 28

Program …
IIT Bombay

while(getline(fin,linestr,'\n')){
stringstream X(linestr);
getline(X,sroll,','); getline(X,sname,',');
getline(X,sbatch,','); getline(X,smarks,',');
fout << sroll << " " << setw(30) << sname << " " << sbatch
<< " " << smarks << endl;
cout << sroll << " " << setw(30) << sname << " " << sbatch
<< " " << smarks << endl;
count++; //No. of records
} // End of while
Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 29
Program …
IIT Bombay

cout << "\ninput file has been read and printed\n";

cout << "output file marks_data.txt created\n";
cout << count << " records written to output file\n";

fin.close();
fout.close();

return 0;
}
Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 30
Execution results
IIT Bombay

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 31

 IIT Bombay

Create binary files

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 32

Overview
IIT Bombay

• We will study a program to create a binary file, in which

fixed length records are written
• Later, we will see how these records can be directly
accessed, read, and updated

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 33

Files, records, and fields
IIT Bombay

• Recall the format of the text file created (marksdata.txt)

10101 Anil Shah 112 12.50
10102 Amit Jadhav 111 15.00
10103 Shefali Pandya 112 17.00
• Each line contains a record of one student’s information
• A record has 4 fields or attributes
• Roll number, name, batch number, marks
• Each field, and thus each record has a fixed length (in bytes)
Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 34
Fixed length records
IIT Bombay

• We know the record size (number of bytes in a record), say S

• If data for 10,000 students is written to a disk file, then the
file will contain 10,000 x S bytes
• If we know which is the relative position r, of the record of a
student in the file, then we can directly read the data for
that student
• r* (S-1) will be the starting byte position of the record
• Next S bytes will contain the record itself

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 35

A record structure
IIT Bombay

• One good way is to define a structure for our record

struct studentinfo{
int roll;
char fname[40];
char lname[40]
int batch;
float marks;
};

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 36

Size and elements of a structure
IIT Bombay

• Individual elements of s can now be accessed by

s.roll, s.fname, s.lname, s.batch , and s.marks
• The size (in bytes) of a structure can be found by
int rec_size; rec_size = sizeof(struct studentinfo)
• Most compilers will count the size of our record as 92
bytes
• Elements need to be allocated at word boundary
(divisible by 4)
Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 37
Program logic for creating a database file
IIT Bombay

• Open input text file and output binary file

• While (read a line till EOF)
• Process input string, separate parts in five strings
• Prepare an output record with these values
• write this to output file
• Verify
• Go to the first position in the file (binary file opened as input)
• Read the binary file till EOF record by record and print output
• Close both the files

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 38

Program
IIT Bombay

#include<iostream> struct studentinfo{

#include<fstream> int roll;
#include<string> char fname[40];
#include<sstream> char sname[40];
int batch;
#include<iomanip>
float marks;
#include<cstdlib> };
using namespace std;

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 39

Program
IIT Bombay

int main() {
struct studentinfo s;
string linestr;
ifstream fin("CSV_data.txt", ios::in);
if(!fin.is_open()){
cout << "Could not open text file" << endl;
return -1;
}

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 40

Program
IIT Bombay

fstream fout("marks_data.bin", ios::binary | ios::in

| ios::out | ios::trunc);
if(!fout.is_open()){
cout << "Could not open binary file" << endl;
return -1;
}
int count = 0;
string temp;

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 41

Program
IIT Bombay

fstream fout("marks_data.bin", ios::binary | ios::in

| ios::out | ios::trunc);
if(!fout.is_open()){
cout << "Could not open binary file" << endl;
return -1;
}
int count = 0;
string temp;

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 42

Program
IIT Bombay

while(getline(fin,linestr,'\n')){
/* extract differentt values in member variables of s
e.g. s.roll, etc */

fout.write((char*)&s, sizeof(s));
count++;
}

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 43

Program
IIT Bombay

//Go to the first location in the binary file

fout.seekg(0, ios::beg);

//Read till EOF

while(fout.read((char*)&s, sizeof(s))){
cout << s.roll << " " << s.fname << " “ << s.lname << s.batch
<< " " << s.marks << endl;
}

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 44

Program
IIT Bombay

//End of while loop, Entire input file has been processed

cout << "\ninput file has been read and printed\n";
cout << "output file created\n";
cout << count << " records written to output file\n";

fin.close();
fout.close();
return 0;
}
Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 45
Results of execution
IIT Bombay

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 46

Organizing records in a direct access file
IIT Bombay

• We can directly access data in a binary file giving position

fp POS
• Current position can be found using member function tellg
long POS; POS = fin.tellg();
• Reading/writing happens at this position
• You may set the position to a value POS by
fin.seekg(POS, ios::beg)

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 47

Summary
IIT Bombay

• Records in binary files can be directly accessed and processed

• We typically search on a key attribute, such as roll_number
• Need to know the record number of the desired record
• We need a mapping from rollnumber to record position
• Refer to C++ tutorials and reference section on the web at:
http://www.cplusplus.com/reference/cstdio
• Handouts posted will provide a summary of file processing,
along with the program listings

Dr. Deepak B. Phatak & Dr. Supratik Chakraborty, IIT Bombay 48