Introduction to dataframe

What’s pandas
Pandas is an open source package that provides numerous tools for data analysis. It
offers fast, flexible and expressive data structures that can be used for many
different data manipulation tasks.

In order to use Pandas in your Python IDE you need to import the Pandas library first :

import pandas as pd

Pandas data structures

The two primary data structures of pandas are:

1. Series: is one-dimensional array. It can store data of any type. Its values are mutable
but the size cannot be changed.
2. DataFrame: is two-dimensional data with mutable size, it allows to store and
manipulate tabular data in rows of observations and columns of variables.

How to create series

A series may be created from:
1. A numpy array:

import pandas as pd

import numpy as np
array = np.array(["blue", "yellow", "pink", "purple"]) # get the array

series1 = pd.Series(array) #create the series from the array

print(series1)

2. A list:

list = [19, 175, 41, 22]

series2 = pd.Series(list) #create the series from the list

print(series2)```

https://www.tutorialspoint.com/python_pandas/python_pandas_series.htm

https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.add.html

https://subscription.packtpub.com/book/big_data_and_business_intelligence/9781787123137/3/
ch03lvl1sec31/re-indexing-a-series

Series Changing Index

A big advantage we gain compared to NumPy arrays is that we can create a Series
using our own indexes.
For example:

import pandas as pd

color=["pink", "white", "black", "blue"]

occurence = [20, 15, 6, 43]

S=pd.Series(occurence, index=color)

print(S)

Series addition
If we add two series with the same index, we get a new series with the same index
and the corresponding values will be added :

import pandas as pd

color=["pink", "white", "black", "blue"]

S1=pd.Series([20, 15, 6, 43], index=color)

S2=pd.Series([3, 22, 9, 10], index=color)

print(S1+S2)

Dataframe Introduction
DataFrame is a 2-dimensional labeled data structure with columns of potentially
different types.

 Dataframe columns are made up of pandas Series.

You can think of it like a spreadsheet or SQL table,or a dictionary of Series.

How to create DataFrame

1. From a list:

import pandas as pd

list = [['Jack', 34, 'Paris'], ['Thomas', 30, 'Roma'],

['Alexandre', 16, 'New York']]

df = pd.DataFrame(list, columns =['name', 'age', 'city'])

2. From dictionary:

dictionary = { 'name' : ['Jack', 'Thomas', 'Alexandre'],

'age' : [34, 30, 16],

'city' : ['Paris', 'Roma', 'New York']}

df = pd.DataFrame(dictionary)
print(df)

3. From numpy array:

import numpy as np

import pandas as pd

my_numpy_array=np.random.randn(3,4)

df=pd.DataFrame(my_numpy_array, columns=list("abcd"))

print(df)

4. From csv file:

Let's create a dataframe from this csv file

import pandas as pd

df=pd.read_csv("csv file example", sep=";")

https://databricks.com/glossary/pandas-dataframe

Dataframe exportation to csv

1. We created a dataframe using dictionary.
2. We uploaded it into a csv file.
3. We created a new dataframe from our csv file.
4. We used the head command to show the first 5 rows.
https://datatofish.com/export-dataframe-to-csv/

Getting information about dataframe

To show general information of different columns such as the type, we write:

df.info()

 Viewing our data

df.head() #to show the first 5 rows of our data

df.tail() # to show the last 5 rows of our data

 Describing our data

Now, let’s use the describe command for calculating some statistical data for one
specific column.
df.describe() # we will get a detailed description of numerical variables of our data
such as mean, min, std, max...etc

https://note.nkmk.me/en/python-pandas-head-tail/

https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.describe.html
Dataframe Bracket Selection
In this course, we will often have to select specific rows or columns from our
DataFrame.
One of the easiest ways to do that is to use brackets:

https://datatofish.com/select-rows-pandas-dataframe/

https://pandas.pydata.org/pandas-docs/stable/reference/api/
pandas.DataFrame.loc.html

https://pandas.pydata.org/pandas-docs/stable/reference/api/
pandas.DataFrame.iloc.html

Dataframe loc/iloc
 Dataframe loc
The loc() method allows us to extract rows and columns by labeled index.

df.index=["Jack", "Thomas", "Alexandre", "Anne"] #get the index labeled

df.loc[["Jack", "Thomas"]] #select the first and second row

 Dataframe iloc
The iloc() follows the same rules as loc(). It extracts rows and columns by selecting
indexes.

df.iloc[:, 1:3] #select the second and third columns with keeping all rows

print(df)
Setting index in dataframe
We can use the set_index() function if we want to replace the index using one or
more existing column.

 Old Index

 New Index

https://pandas.pydata.org/pandas-docs/stable/reference/api/
pandas.DataFrame.set_index.html
Dataframe Concatenate

https://pandas.pydata.org/pandas-docs/stable/reference/api/
pandas.DataFrame.drop.html

https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.concat.html

Dataframe drop
To drop specified labels from rows or columns, we simply use drop() method.
For example, we want to delete the country column we added previously:

df.drop("country", axis=1)

drop() method has inplace=False as default, you can see that the country column is
not gone. Take a break & make some research.

https://pandas.pydata.org/pandas-docs/stable/reference/api/
pandas.DataFrame.drop.html

Dataframe with Pandas Recap

 Pandas is an open source library which offers several tools for data analysis. It
provides fast, flexible and expressive data struc tures that can be used for numerous
data manipulation tasks.
 We can create DataFrame from list, dictionary, numpy array or csv file.
 To convert data to a csv file : data.to_csv(“file.csv”)
 To see information about columns : dataframe.info()
 To see a brief description about columns and their values : dataframe.describe()
 To view the first 5 rows : dataframe.head()
 To view the last 5 rows : dataframe.tail()
 There are multiple ways to select rows and columns from Pandas DataFrames. Iloc
and loc are the main operations for retrieving data: The iloc method is used to select
indexes for Pandas Dataframe. Whereas loc method ensure the extraction by selecting
labels .
 We can also select specific rows or columns from using the brackets.