模式识别 课程实验报告
学 院 专 业 班 级 姓 名 学 号 指导教师 提交日期
1 Data Preprocessing
The provide dataset includes a training set with 3605 positive samples and 10055 negative samples, and a test set with 2043 positive samples and 4832 negative samples. A 2330-dimensional Haar-like feature was extracted for each sample. For high dimensional data, we keep the computation manageable by sampling , feature selection, and dimension reduction.
1.1 Sampling
In order to make the samples evenly distributed, we calculate the ratio of negative samples and positive samples int test set. And then randomly select the same ratio of negative samples in training set. After that We use different ratios of negative and positive samples to train the classifier for better training speed.
1.2 Feature Selection
Univariate feature selection can test each feature, measure the relationship between the feature and the response variable so as to remove unimportant feature variables. In the experiment, we use the method sklearn.feature_selection.SelectKBest to implement feature selection. We use chi2 which is chi-squared stats of non-negative features for classification tasks to rank features, and finally we choose 100 features ranked in the top 100.
2 Logistic regression
In the experiment, we choose the logistic regression model, which is widely used in real-world scenarios. The main consideration in our work is based on the binary classification logic regression model.
2.1 Introduction
A logistic regression is a discriminant-based approach that assumes that instances of a class are linearly separable. It can obtain the final prediction model by directly estimating the parameters of the discriminant. The logistic regression model does not model class conditional density, but rather models the class condition ratio.
1
2.2 process
The next step is how to obtain the best evaluation parameters, making the training of the LR model can get the best classification effect. This process can also be seen as a search process, that is, in an LR model of the solution space, how to find a design with our LR model most match the solution. In order to achieve the best available LR model, we need to design a search strategy.
The intuitive idea is to evaluate the predictive model by judging the degree of matching between the results of the model and the true value. In the field of machine learning, the use of loss function or cost function to calculate the forecast. For the classification, logistic regression uses the Sigmoid curve to greatly reduce the weight of the points that are far from the classification plane through the nonlinear mapping, which relatively increases the weight of the data points which is most relevant to the classification.
2.3 Sigmoid function
We should find a function which can separate two in the two classes binary classification problem. The ideal function is called step function. In this we use the sigmoid function.
??z??11?e?z
When we increase the value of x, the sigmoid will approach 1, and when we decrease the value of x, the sigmoid will gradually approaches 0. The sigmoid looks like a step function On a large enough scale.
2.4 gradient descent
?
t?1n?L(?)t?????????(yi??(?Txi))xi
??i?1tThe parameter α called learning rate, in simple terms is how far each step. this parameter is very critical. parameter?is sigmoid function that we introduce in 2.3.
2