EE2211 hack

Lecture 7
Supervised learning: given feature(s) x, we want to predict
target. Algorithm for it:

Above is an objective function.

• Most supervised learning algorithms can be formulated as the

following optimization problem
• Data-Loss(w) quantifies fitting error to training set given
parameters w: smaller error => better fit to training data
• Regularization(w) penalizes more complex models

Overfitting, Underfitting, Just Nice

Regularisation
Bias VS Variance = Test error (Mean squared error) = Bias Squared + Variance +
Irreducible Noise (Aim for lower variance and bias)
Bias-Variance Decomposition theorem

Lecture 8 - Optimization and Gradient descent

Loss function & Learning modele

Lecture 10
Useful Methods for EE2211:

scipy.integrate.quad(function, lower_limit, upper_limit) 2

• This method is for basic integration 2
scipy.stats.norm 3
• Gives you methods for normal continuous distribution functions 3
collections.defaultdict 3
numpy.matlib.repmat(a,m,n): 4

How do i write the fact that I have good communication skills ( i know 3 languages, english
tamil and chinese), good leadership skills as i have been teaching tuition for 2 years and
am a good team member for a job
scipy.integrate.quad(function, lower_limit,
upper_limit)

• This method is for basic integration

 • The function parameter takes in the function that you’d want to integrate
• The other two parameters are the lower and upper limits respectively

scipy.stats.norm

• Gives you methods for normal continuous distribution

functions
• Some methods are pdf and cdf
• pdf(x, loc, scale)
• x - is the x-value
• Loc - Is the mean
• Scale - is the standard deviation
• cdf is similar, just call cdf.
• cdf doesn’t need integration but pdf needs

collections.defaultdict
• Allows you to group a sequence of key-value pairs into a dictionary of lists
• This means that if a key has been addressed multiple times by the values, these values are put
into and array and the newly created array is made the value of the key
• You can choose what you want the key to be
• You first have to implement as a function:
• d = defaultdict(default_factory)
• default_factory can be sets, tuples or list depending on the type that you’d want to
arrange
• ‘d’ is still a dictionary and the dict methods apply to it

matplotlib.pyplot
• This method allows you to plot a graph
• .plot(x_axis_values, y_axis_values, marker_characters)
• .plot.xlabel lets you label the x-axis
• .plot.ylabel lets you label the y-axis

numpy.polyfit(x_values, y_values, degree_of_polynomial)

● this solves for the regression line for given data
● polyfit adds bias for you
● returns an array arranged with the highest power first
numpy.hstack(tuple)
● takes in a tuple of arrays in the order to be stacked
● returns stacked array

numpy.append(array, old_array/values, axis)

● takes in 2 arrays and appends them
● axis specifies along which direction to append, 1 is the first column, if none is given the array is
flattened

numpy.ones(shape)
● takes in the shape of new array, eg: 3 x 1
● returns array of given array, filled with ones

numpy.sign(array)
● takes in array and returns signum output of elements in array

numpy.argmax(array, axis)
● takes in array to find the index of the maximum elements
● by default, array is flattened
● if axis=0, then in each column, the maximum element is found and it’s row index is returned
● if axis=1, then in each row, the maximum element is found and it’s column index is returned

numpy.argsort([array])
● it returns the indexes into the original list so that if you use them in that order applied to the
original list, the list is sorted

numpy.matlib.repmat(a,m,n):

● repeat a 0-D to 2-D array or matrix MxN times

● a: array like to be repeated
● m,n the number of times a is repeated in the first and second axes

numpy.array(list)
• Takes in a list to display it in a matrix form
• You can access matrix elements with array indexing
• E.g: nparray[:,0] - returns elements in the first column
• nparray[[0,1,2] : [0,1,0]] - returns the elements in the [0,0], [1,1] and [2,0] positions

numpy.matmul(x1, x2, out=None)

● matrix product of 2 arrays
● x1,x2 input arrays
● out the array in which the result is to be stored

numpy.where(condition[,x,y])
● return elements choses form x or y depending on condition
 ● condition: when True, yield x, otherwise yield y
● returns an array with elements from x where conditions is True and elements from y elsewhere
● np.where(a < 5, a, 10*a)
array([ 0, 1, 2, 3, 4, 50, 60, 70, 80, 90])

pandas.read_csv(file_path, delimiter)
• Allows you to read in comma separated values
• delimiter/separator is usually Automatically detected, or else it can be specified

Csv dataframes
• When you read in csv data, it gets put into a data frame
• It’s basically a miniature excel type table
• Its fields can be accessed by Dataframe[‘field_name’]
• To access the rows and columns by label, we use dataframe.loc[‘what_you_want_to_find’]
• .index returns the row_labels, usually the left most columns
• Dataframes are made up of index and columns

Dataframe.describe()
• Gives all the descriptive statistics of the dataframe

Dataframe.replace(original, new)
• Replace the old values with new ones

sklearn.preprocessing.scale(dataframe/ndarray)
• Sometimes we need to scale our values into a common range
• If not the machine learning will be bias towards the larger values
• This one function does it all for us
• And the scaled data has unit variance and zero mean

sklearn.preprocessing.MinMaxScaler(feature-range = tuple(min, max))

• Is used to declare a min-max function
• This will set the range of the scaled data
• Default is [0,1]
• .fit_transform(dataframe) will return scaled value

poly = sklearn.preprocessing.PolynomialFeatures(degree, interaction_only, include_bias)

● this generates a polynomial with features
● degree = degree of polynomial
● interaction_only => if true, only includes mixed parameter products, default is false
● include_bias => if true, adds offset term, default is true
poly.fit_transform(array[, target_array])
● this takes in the array which we need to transform with more features.
sklearn.preprocessing.OneHotEncoder(sparse=False).fit_transform (X)
● X is the array to be encoded
● returns encoded array, since sparse is false
● .transform(y_array) transforms the array

sklearn.metrics.accuracy_score(y_true, y_pred, *, normalize=True, sample_weight=None)

● normalize: if false, returns the number of correctly classified samples, else the fraction of
correctly classified samples.

sklearn.metrics.mean_squared_error(y_true, y_pred)
● returns the mean square error as an array or float

sklearn.neural_network.MLPClassifier(hidden_layer_sizes=(100, ), activation='relu', *,
solver='adam', alpha=0.0001, batch_size='auto', learning_rate='constant', learning_rate_init=0.001,
power_t=0.5, max_iter=200, shuffle=True, random_state=None, tol=0.0001, verbose=False,
warm_start=False, momentum=0.9, nesterovs_momentum=True, early_stopping=False,
validation_fraction=0.1, beta_1=0.9, beta_2=0.999, epsilon=1e-08, n_iter_no_change=10,
max_fun=15000)
● a multilayer perceptron classifier
● the number of neurons in the hidden layer, total - 2
● activation: to specify the activation function
● solver: the solver for weight optimization
● alpha: L2 penalty parameter
● batch_size: not for lbfgs
● learning_rate: for sgd
● max_iter: the number of iterations
● .loss_: the current loss computed with the loss function
● .classes_: class labels for each output
● .fit(X,y): fit the model data matrix X and target y
● predict(X): predict the classes using mlp

sklearn.neural_network.MLPRegressor()
● same as above
bias_variance_decomp(estimator, X_train, y_train, X_test, y_test, loss='0-1_loss',
num_rounds=200, random_seed=None)

estimator : object A classifier or regressor object or class implementing a fit predict method similar
to the scikit-learn API.

● X_train : array-like, shape=(num_examples, num_features)

A training dataset for drawing the bootstrap samples to carry out the bias-variance
decomposition.
● y_train : array-like, shape=(num_examples)
Targets (class labels, continuous values in case of regression) associated with the X_train
examples.
● X_test : array-like, shape=(num_examples, num_features)
The test dataset for computing the average loss, bias, and variance.
● y_test : array-like, shape=(num_examples)
Targets (class label
● s, continuous values in case of regression) associated with the X_test examples.
● loss : str (default='0-1_loss')
Loss function for performing the bias-variance decomposition. Currently allowed values are
'0-1_loss' and 'mse'.
● num_rounds : int (default=200)
Number of bootstrap rounds for performing the bias-variance decomposition.
● random_seed : int (default=None)
Random seed for the bootstrap sampling used for the bias-variance decomposition.

sklearn.model_selection.KFold(n_splits=5, *, shuffle=False, random_state=None)

● n_splits: number of folds, must be at least 2, default 5
● .split(X[,y,groups]): gives you the indices of the split arrays which can be destructed into
train and test indices
● get_n_splits(): returns the number of splitting iterations in the cross-validator

sklearn.ensemble.RandomForestRegressor(n_estimators=100, *, criterion='mse',
max_depth=None, min_samples_split=2, min_samples_leaf=1, min_weight_fraction_leaf=0.0,
max_features='auto', max_leaf_nodes=None, min_impurity_decrease=0.0,
min_impurity_split=None, bootstrap=True, oob_score=False, n_jobs=None, random_state=None,
verbose=0, warm_start=False, ccp_alpha=0.0, max_samples=None)
● n_estimators: the number of trees in the forest
● max_depth: the maximum depth of the trees, default None
● criterion: the function to measure the quality of the split. “mse” or “mae”
● max_features: the number of features to consider when looking for the best fit: “auto, sqrt,
log2”
● impurity can be specified
● .fit(X,y): build a forest from the training set X,y, # reshape X , it is necessary because tree
expects 2D array
● .predict(X): predicts the regression target values for X
● .score(X,y): returns the coefficient of determination R^2 of the prediction

sklearn.tree.DecisionTreeClassifier(*, criterion='gini', splitter='best', max_depth=None,

min_samples_split=2, min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features=None,
random_state=None, max_leaf_nodes=None, min_impurity_decrease=0.0,
min_impurity_split=None, class_weight=None, presort='deprecated', ccp_alpha=0.0)
● a decision tree classifier
● criterion: function to measure the quality of split. (“gin”,”entropy”)
● max_depth: the maximum depth of the tree
● .classes_: gives an ndarray of class labels
● .n_features_: the number of features when fit is performed
● .predict(X): predicts the class value for elements in X
● .fit(X,y): build a decision tree classifier front the training set (X,y)
● .score(X,y): returns the mean accuracy on the given test data and labels

sklearn.tree.plot_tree(decision_tree, *, max_depth=None, feature_names=None,

class_names=None, label='all', filled=False, impurity=True, node_ids=False, proportion=False,
rotate='deprecated', rounded=False, precision=3, ax=None, fontsize=None)
● plot a decision tree
● decision_tree: the tree to be plotted, must be in tree format
● max_depth: the max depth to be displayed
sklearn.cluster.KMeans(n_clusters=8, *, init='k-means++', n_init=10, max_iter=300, tol=0.0001,
precompute_distances='deprecated', verbose=0, random_state=None, copy_x=True,
n_jobs='deprecated', algorithm='auto')
● does the full k-means clustering
● n_clusters: the number of clusters to form
● max_iter: the number of iterations for a single run
● .cluster_centers: returns the coordinates of cluster centers
● labels_: labels/classes of each point
 ● inertia_: sum of squared distances of samples to their cluster center
● .fit(X, y): compute k-means clustering
● .fit_predict(X,y): Compute cluster centers and predict cluster index for each sample.
● .fit_tranform(X,y): Compute clustering and transform X to cluster-distance space.
● .predict(X): predicts the closest cluster for each sample in X belongs to.

sklearn.datasets.make_blobs(n_samples=100, n_features=2, *, centers=None, cluster_std=1.0,

center_box=(-10.0, 10.0), shuffle=True, random_state=None, return_centers=False)
● number of samples, number of features for each sample, the number of centers to generate
or fix them

numpy.matlib.repmat(a,m,n)
● repeats the matrix a MxN times
● m,n: the number of times a is repeated in the row and columns

numpy.random.uniform(low, high, size)

● returns a scalar or array of samples from a parameterized uniform distribution
● size can be tuple to indicate dimensional shape

If your array has weights (ie. your distribution is not uniform) you can use
Numpy.average(array, weights) to calculate the expected value of such distribution.

numpy.dot(a, b)
● for taking matrix multiplication of matrix a and matrix b
● returns a computed matrix.
● input must be of numpy.array type
numpy.linalg.inv(a)
● returns the inverse of matrix a
● input must be of numpy.
numpy.linalg.pinv(a)
● takes in a matrix and returns its pseudo inverse
● pseudo inverse rhs before multiplying with y for overdetermined system
numpy.unique(array, axis)
● this method returns the unique elements of the input array
● the result is sorted
● axis is the axis to work on, i.e. if axis=NONE, the input array is flattened, if 0, it looks for unique
rows, if 1, looks for unique columns.
numpyArray.reshape(a, newshape)
● returns the array in the new rows and columns
● a is the array
● if newshape is an integer then returns 1-D array of that length, or can be tuple
Mean Squared error: the average squared difference between the estimated values and what is
estimated.

numpy.random.randn(d0,d1….)
● returns a sample from the “standard normal” distribution
● d0,d1 the dimensions of the returned array

For random samples from , use:

sigma * np.random.randn(...) + mu

numpy.c_[]
● translates slice objects to concatenation along the second axis

numpy.column_stack(tup)
● stacks 1-D arrays as columns into 2-D array
● tup: tuple of arrays to stack, must have the same first dimension
● returns stacked 2_D array
our goal is to find a line that minimizes the squared distances to these different points.

MSE = numpy.square(numpy.subtract(original values, prediction_values)).mean()

sklearn.metrics.mean_squared_error(y_true, y_pred, *, sample_weight=None,

multioutput='uniform_average', squared=True)
Required Probability=P[ (A and notB) or(B and not A)]

Confusion Matrix

Basic slicing and indexing of lists

Leaf Node
Leaf nodes are the nodes of the tree that have no additional nodes coming off them. They
don't split the data any further

Decision trees are examples of models with low bias and high variance. The tree makes
almost no assumptions about target function but it is highly susceptible to variance in data.

There are ensemble algorithms, such as bootstrapping aggregation and random forest, which
aim to reduce variance at the small cost of bias in decision trees.
Relations:

u got it? wtv answer u get just put might not have time to take the best ss 1 or 2 options might have
been left off

1.D
2. 1-0.125 , 2-0.649.
3.false
4.b
5.
6.F
7.False
8.
9.
10.a)wo = 0, w1 = 0.240 and w2 = -0.420.
b)girl
c)1.862 m
11.C
12.
13.False
14.
15.
16.
17.D
18.a)wo ≈ 0.915 and w1 ≈ 1.139.
b)wo is 2.935 and w1 is 1.471
c)ynew = 3.638
19.F
20.F