Experience Seamless Full Ebook Downloads for Every Genre at textbookfull.

com

Feature Engineering for Machine Learning and Data

Analytics First Edition Dong

https://textbookfull.com/product/feature-engineering-for-
machine-learning-and-data-analytics-first-edition-dong/

OR CLICK BUTTON

DOWNLOAD NOW

Explore and download more ebook at https://textbookfull.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Feature engineering for machine learning principles and

techniques for data scientists First Edition Casari

https://textbookfull.com/product/feature-engineering-for-machine-
learning-principles-and-techniques-for-data-scientists-first-edition-
casari/
textboxfull.com

The Art of Feature Engineering: Essentials for Machine

Learning 1st Edition Pablo Duboue

https://textbookfull.com/product/the-art-of-feature-engineering-
essentials-for-machine-learning-1st-edition-pablo-duboue/

textboxfull.com

AI and Machine Learning Paradigms for Health Monitoring

System: Intelligent Data Analytics Hasmat Malik

https://textbookfull.com/product/ai-and-machine-learning-paradigms-
for-health-monitoring-system-intelligent-data-analytics-hasmat-malik/

textboxfull.com

Advanced Data Analytics Using Python: With Machine

Learning, Deep Learning and NLP Examples Mukhopadhyay

https://textbookfull.com/product/advanced-data-analytics-using-python-
with-machine-learning-deep-learning-and-nlp-examples-mukhopadhyay/

textboxfull.com
Recent Developments in Machine Learning and Data Analytics
IC3 2018 Jugal Kalita

https://textbookfull.com/product/recent-developments-in-machine-
learning-and-data-analytics-ic3-2018-jugal-kalita/

textboxfull.com

Machine Learning and Big Data Analytics Paradigms:

Analysis, Applications and Challenges Aboul Ella Hassanien

https://textbookfull.com/product/machine-learning-and-big-data-
analytics-paradigms-analysis-applications-and-challenges-aboul-ella-
hassanien/
textboxfull.com

Data Processing with Optimus: Supercharge big data

preparation tasks for analytics and machine learning with
Optimus using Dask and PySpark Leon
https://textbookfull.com/product/data-processing-with-optimus-
supercharge-big-data-preparation-tasks-for-analytics-and-machine-
learning-with-optimus-using-dask-and-pyspark-leon/
textboxfull.com

Intelligent Feature Selection for Machine Learning Using

the Dynamic Wavelet Fingerprint Mark K. Hinders

https://textbookfull.com/product/intelligent-feature-selection-for-
machine-learning-using-the-dynamic-wavelet-fingerprint-mark-k-hinders/

textboxfull.com

Scala and Spark for Big Data Analytics Explore the

concepts of functional programming data streaming and
machine learning 1st Edition Md. Rezaul Karim
https://textbookfull.com/product/scala-and-spark-for-big-data-
analytics-explore-the-concepts-of-functional-programming-data-
streaming-and-machine-learning-1st-edition-md-rezaul-karim/
textboxfull.com
FEATURE ENGINEERING FOR
MACHINE LEARNING AND
DATA ANALYTICS
Chapman & Hall/CRC
Data Mining and Knowledge Series
Series Editor: Vipin Kumar

RapidMiner
Data Mining Use Cases and Business Analytics Applications
Markus Hofmann and Ralf Klinkenberg
Computational Business Analytics
Subrata Das
Data Classification
Algorithms and Applications
Charu C. Aggarwal
Healthcare Data Analytics
Chandan K. Reddy and Charu C. Aggarwal
Accelerating Discovery
Mining Unstructured Information for Hypothesis Generation
Scott Spangler
Event Mining
Algorithms and Applications
Tao Li
Text Mining and Visualization
Case Studies Using Open-Source Tools
Markus Hofmann and Andrew Chisholm
Graph-Based Social Media Analysis
Ioannis Pitas
Data Mining
A Tutorial-Based Primer, Second Edition
Richard J. Roiger
Data Mining with R
Learning with Case Studies, Second Edition
Luís Torgo
Social Networks with Rich Edge Semantics
Quan Zheng and David Skillicorn
Large-Scale Machine Learning in the Earth Sciences
Ashok N. Srivastava, Ramakrishna Nemani, and Karsten Steinhaeuser
Data Science and Analytics with Python
Jesus Rogel-Salazar
Feature Engineering for Machine Learning and Data Analytics
Guozhu Dong and Huan Liu

For more information about this series please visit:

https://www.crcpress.com/Chapman--HallCRC-Data-Mining-and-Knowledge-Discovery-Series/book-series/CHDAMINODIS
FEATURE ENGINEERING FOR
MACHINE LEARNING AND
DATA ANALYTICS

Edited by
Guozhu Dong and Huan Liu
MATLAB• is a trademark of The MathWorks, Inc. and is used with permission. The MathWorks
does not warrant the accuracy of the text or exercises in this book. This book’s use or discussion
of MATLAB• software or related products does not constitute endorsement or sponsorship by The
MathWorks of a particular pedagogical approach or particular use of the MATLAB• software.

CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742

© 2018 by Taylor & Francis Group, LLC

CRC Press is an imprint of Taylor & Francis Group, an Informa business

No claim to original U.S. Government works

Printed on acid-free paper

Version Date: 20180301

International Standard Book Number-13: 978-1-1387-4438-7 (Hardback)

This book contains information obtained from authentic and highly regarded sources. Reasonable
efforts have been made to publish reliable data and information, but the author and publisher cannot
assume responsibility for the validity of all materials or the consequences of their use. The authors and
publishers have attempted to trace the copyright holders of all material reproduced in this publication
and apologize to copyright holders if permission to publish in this form has not been obtained. If any
copyright material has not been acknowledged please write and let us know so we may rectify in any
future reprint.

Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced,
transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or
hereafter invented, including photocopying, microfilming, and recording, or in any information
storage or retrieval system, without written permission from the publishers.

For permission to photocopy or use material electronically from this work, please access
www.copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc.
(CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization
that provides licenses and registration for a variety of users. For organizations that have been granted
a photocopy license by the CCC, a separate system of payment has been arranged.

Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and
are used only for identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
To my family, especially baby Hazel [G. D.]

To my family [H. L.]

To all contributing authors [G. D. & H. L.]

Contents

Preface xv

Contributors xvii

1 Preliminaries and Overview 1

Guozhu Dong and Huan Liu
1.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.1 Features . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.2 Feature Engineering . . . . . . . . . . . . . . . . . . . 3
1.1.3 Machine Learning and Data Analytic Tasks . . . . . . 3
1.2 Overview of the Chapters . . . . . . . . . . . . . . . . . . . . 4
1.3 Beyond this Book . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3.1 Feature Engineering for Specific Data Types . . . . . . 8
1.3.2 Feature Engineering on Non-Data-Specific Topics . . . 9

I Feature Engineering for Various Data Types 13

2 Feature Engineering for Text Data 15
Chase Geigle, Qiaozhu Mei, and ChengXiang Zhai
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.2 Overview of Text Representation . . . . . . . . . . . . . . . . 17
2.3 Text as Strings . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.4 Sequence of Words Representation . . . . . . . . . . . . . . . 19
2.5 Bag of Words Representation . . . . . . . . . . . . . . . . . . 21
2.5.1 Term Weighting . . . . . . . . . . . . . . . . . . . . . 22
2.5.2 Beyond Single Words . . . . . . . . . . . . . . . . . . . 27
2.6 Structural Representation of Text . . . . . . . . . . . . . . . 28
2.6.1 Semantic Structure Features . . . . . . . . . . . . . . . 30
2.7 Latent Semantic Representation . . . . . . . . . . . . . . . . 31
2.7.1 Latent Semantic Analysis . . . . . . . . . . . . . . . . 31
2.7.2 Probabilistic Latent Semantic Analysis . . . . . . . . . 33
2.7.3 Latent Dirichlet Allocation . . . . . . . . . . . . . . . 35
2.8 Explicit Semantic Representation . . . . . . . . . . . . . . . 37
2.9 Embeddings for Text Representation . . . . . . . . . . . . . 37
2.9.1 Matrix Factorization for Word Embeddings . . . . . . 38
2.9.2 Neural Networks for Word Embeddings . . . . . . . . 40

vii
viii Contents

2.9.3 Document Representations from Word Embeddings . . 41

2.10 Context-Sensitive Text Representation . . . . . . . . . . . . 42
2.11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

3 Feature Extraction and Learning for Visual Data 55

Parag S. Chandakkar, Ragav Venkatesan, and Baoxin Li
3.1 Classical Visual Feature Representations . . . . . . . . . . . 57
3.1.1 Color Features . . . . . . . . . . . . . . . . . . . . . . 57
3.1.2 Texture Features . . . . . . . . . . . . . . . . . . . . . 61
3.1.3 Shape Features . . . . . . . . . . . . . . . . . . . . . . 63
3.2 Latent Feature Extraction . . . . . . . . . . . . . . . . . . . 66
3.2.1 Principal Component Analysis . . . . . . . . . . . . . 67
3.2.2 Kernel Principal Component Analysis . . . . . . . . . 68
3.2.3 Multidimensional Scaling . . . . . . . . . . . . . . . . 69
3.2.4 Isomap . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3.2.5 Laplacian Eigenmaps . . . . . . . . . . . . . . . . . . . 70
3.3 Deep Image Features . . . . . . . . . . . . . . . . . . . . . . 71
3.3.1 Convolutional Neural Networks . . . . . . . . . . . . . 72
3.3.1.1 The Dot-Product Layer . . . . . . . . . . . . 72
3.3.1.2 The Convolution Layer . . . . . . . . . . . . 73
3.3.2 CNN Architecture Design . . . . . . . . . . . . . . . . 75
3.3.3 Fine-Tuning Off-the-Shelf Neural Networks . . . . . . 76
3.3.4 Summary and Conclusions . . . . . . . . . . . . . . . . 79

4 Feature-Based Time-Series Analysis 87

Ben D. Fulcher
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
4.1.1 The Time Series Data Type . . . . . . . . . . . . . . . 87
4.1.2 Time-Series Characterization . . . . . . . . . . . . . . 89
4.1.3 Applications of Time-Series Analysis . . . . . . . . . . 90
4.2 Feature-Based Representations of Time Series . . . . . . . . 92
4.3 Global Features . . . . . . . . . . . . . . . . . . . . . . . . . 95
4.3.1 Examples of Global Features . . . . . . . . . . . . . . 95
4.3.2 Massive Feature Vectors and Highly Comparative Time-
Series Analysis . . . . . . . . . . . . . . . . . . . . . . 98
4.4 Subsequence Features . . . . . . . . . . . . . . . . . . . . . . 102
4.4.1 Interval Features . . . . . . . . . . . . . . . . . . . . . 102
4.4.2 Shapelets . . . . . . . . . . . . . . . . . . . . . . . . . 103
4.4.3 Pattern Dictionaries . . . . . . . . . . . . . . . . . . . 105
4.5 Combining Time-Series Representations . . . . . . . . . . . . 106
4.6 Feature-Based Forecasting . . . . . . . . . . . . . . . . . . . 108
4.7 Summary and Outlook . . . . . . . . . . . . . . . . . . . . . 109
 Contents ix

5 Feature Engineering for Data Streams 117

Yao Ma, Jiliang Tang, and Charu Aggarwal
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
5.2 Streaming Settings . . . . . . . . . . . . . . . . . . . . . . . . 119
5.3 Linear Methods for Streaming Feature Construction . . . . . 121
5.3.1 Principal Component Analysis for Data Streams . . . 121
5.3.2 Linear Discriminant Analysis for Data Streams . . . . 123
5.4 Non-Linear Methods for Streaming Feature Construction . . 125
5.4.1 Locally Linear Embedding for Data Streams . . . . . 125
5.4.2 Kernel Learning for Data Streams . . . . . . . . . . . 126
5.4.3 Neural Networks for Data Streams . . . . . . . . . . . 128
5.4.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 132
5.5 Feature Selection for Data Streams with Streaming Features 132
5.5.1 The Grafting Algorithm . . . . . . . . . . . . . . . . . 133
5.5.2 The Alpha-Investing Algorithm . . . . . . . . . . . . . 133
5.5.3 The Online Streaming Feature Selection Algorithm . . 134
5.5.4 Unsupervised Streaming Feature Selection in
Social Media . . . . . . . . . . . . . . . . . . . . . . . 135
5.6 Feature Selection for Data Streams with Streaming Instances 135
5.6.1 Online Feature Selection . . . . . . . . . . . . . . . . . 136
5.6.2 Unsupervised Feature Selection on Data Streams . . . 136
5.7 Discussions and Challenges . . . . . . . . . . . . . . . . . . . 136
5.7.1 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . 137
5.7.2 Number of Features . . . . . . . . . . . . . . . . . . . 137
5.7.3 Heterogeneous Streaming Data . . . . . . . . . . . . . 137

6 Feature Generation and Feature Engineering for Sequences 145

Guozhu Dong, Lei Duan, Jyrki Nummenmaa, and Peng Zhang
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
6.2 Basics on Sequence Data and Sequence Patterns . . . . . . . 148
6.3 Approaches to Using Patterns in Sequence Features . . . . . 149
6.4 Traditional Pattern-Based Sequence Features . . . . . . . . . 150
6.5 Mined Sequence Patterns for Use in Sequence Features . . . 151
6.5.1 Frequent Sequence Patterns . . . . . . . . . . . . . . . 152
6.5.2 Closed Sequential Patterns . . . . . . . . . . . . . . . 154
6.5.3 Gap Constraints for Sequence Patterns . . . . . . . . . 155
6.5.4 Partial Order Patterns . . . . . . . . . . . . . . . . . . 156
6.5.5 Periodic Sequence Patterns . . . . . . . . . . . . . . . 158
6.5.6 Distinguishing Sequence Patterns . . . . . . . . . . . . 158
6.5.7 Pattern Matching for Sequences . . . . . . . . . . . . . 160
6.6 Factors for Selecting Sequence Patterns as Features . . . . . 161
6.7 Sequence Features Not Defined by Patterns . . . . . . . . . . 161
6.8 Sequence Databases . . . . . . . . . . . . . . . . . . . . . . . 162
6.9 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . 163
x Contents

7 Feature Generation for Graphs and Networks 167

Yuan Yao, Hanghang Tong, Feng Xu, and Jian Lu
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
7.2 Feature Types . . . . . . . . . . . . . . . . . . . . . . . . . . 168
7.3 Feature Generation . . . . . . . . . . . . . . . . . . . . . . . 169
7.3.1 Basic Models . . . . . . . . . . . . . . . . . . . . . . . 170
7.3.2 Extensions . . . . . . . . . . . . . . . . . . . . . . . . 175
7.3.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 179
7.4 Feature Usages . . . . . . . . . . . . . . . . . . . . . . . . . . 181
7.4.1 Multi-Label Classification . . . . . . . . . . . . . . . . 181
7.4.2 Link Prediction . . . . . . . . . . . . . . . . . . . . . . 181
7.4.3 Anomaly Detection . . . . . . . . . . . . . . . . . . . . 182
7.4.4 Visualization . . . . . . . . . . . . . . . . . . . . . . . 182
7.5 Conclusions and Future Directions . . . . . . . . . . . . . . . 183
7.6 Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

II General Feature Engineering Techniques 189

8 Feature Selection and Evaluation 191
Yun Li and Tao Li
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
8.2 Feature Selection Frameworks . . . . . . . . . . . . . . . . . 192
8.2.1 Search-Based Feature Selection Framework . . . . . . 193
8.2.2 Correlation-Based Feature Selection Framework . . . . 194
8.3 Advanced Topics for Feature Selection . . . . . . . . . . . . . 196
8.3.1 Stable Feature Selection . . . . . . . . . . . . . . . . . 196
8.3.2 Sparsity-Based Feature Selection . . . . . . . . . . . . 199
8.3.3 Multi-Source Feature Selection . . . . . . . . . . . . . 200
8.3.4 Distributed Feature Selection . . . . . . . . . . . . . . 203
8.3.5 Multi-View Feature Selection . . . . . . . . . . . . . . 204
8.3.6 Multi-Label Feature Selection . . . . . . . . . . . . . . 205
8.3.7 Online Feature Selection . . . . . . . . . . . . . . . . . 206
8.3.8 Privacy-Preserving Feature Selection . . . . . . . . . . 208
8.3.9 Adversarial Feature Selection . . . . . . . . . . . . . . 210
8.4 Future Work and Conclusion . . . . . . . . . . . . . . . . . . 211

9 Automating Feature Engineering in Supervised Learning 221

Udayan Khurana
9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
9.1.1 Challenges in Performing Feature Engineering . . . . 224
9.2 Terminology and Problem Definition . . . . . . . . . . . . . . 225
9.3 A Few Simple Approaches . . . . . . . . . . . . . . . . . . . 226
9.4 Hierarchical Exploration of Feature Transformations . . . . . 227
9.4.1 Transformation Graph . . . . . . . . . . . . . . . . . . 228
9.4.2 Transformation Graph Exploration . . . . . . . . . . . 229
 Contents xi

9.5 Learning Optimal Traversal Policy . . . . . . . . . . . . . . . 231

9.5.1 Feature Exploration through Reinforcement Learning 233
9.6 Finding Effective Features without Model Training . . . . . . 235
9.6.1 Learning to Predict Useful Transformations . . . . . . 237
9.7 Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . . 239
9.7.1 Other Related Work . . . . . . . . . . . . . . . . . . . 239
9.7.2 Research Opportunities . . . . . . . . . . . . . . . . . 240
9.7.3 Resources . . . . . . . . . . . . . . . . . . . . . . . . . 240

10 Pattern-Based Feature Generation 245

Yunzhe Jia, James Bailey, Ramamohanarao Kotagiri, and Christopher
Leckie
10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
10.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
10.2.1 Data and Patterns . . . . . . . . . . . . . . . . . . . . 247
10.2.2 Patterns for Non-Transactional Data . . . . . . . . . . 248
10.3 Framework of Pattern-Based Feature Generation . . . . . . . 251
10.3.1 Pattern Mining . . . . . . . . . . . . . . . . . . . . . . 251
10.3.2 Pattern Selection . . . . . . . . . . . . . . . . . . . . . 252
10.3.3 Feature Generation . . . . . . . . . . . . . . . . . . . . 253
10.4 Pattern Mining Algorithms . . . . . . . . . . . . . . . . . . . 254
10.4.1 Frequent Pattern Mining . . . . . . . . . . . . . . . . 254
10.4.2 Contrast Pattern Mining . . . . . . . . . . . . . . . . . 256
10.5 Pattern Selection Approaches . . . . . . . . . . . . . . . . . . 258
10.5.1 Post-Processing Pruning . . . . . . . . . . . . . . . . . 258
10.5.2 In-processing Pruning . . . . . . . . . . . . . . . . . . 260
10.6 Pattern-Based Feature Generation . . . . . . . . . . . . . . . 262
10.6.1 Unsupervised Mapping Functions . . . . . . . . . . . . 262
10.6.2 Supervised Mapping Functions . . . . . . . . . . . . . 263
10.6.3 Feature Generation for Sequence Data and Graph Data 265
10.6.4 Comparison with Similar Techniques . . . . . . . . . . 265
10.7 Pattern-Based Feature Generation for Classification . . . . . 266
10.7.1 Problem Statement . . . . . . . . . . . . . . . . . . . . 266
10.7.2 Direct Classification in the Pattern Space . . . . . . . 267
10.7.3 Indirect Classification in the Pattern Space . . . . . . 268
10.7.4 Connection with Stacking Technique . . . . . . . . . . 269
10.8 Pattern-Based Feature Generation for Clustering . . . . . . . 269
10.8.1 Clustering in the Pattern Space . . . . . . . . . . . . . 269
10.8.2 Subspace Clustering . . . . . . . . . . . . . . . . . . . 270
10.9 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271

11 Deep Learning for Feature Representation 279

Suhang Wang and Huan Liu
11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
11.2 Restricted Boltzmann Machine . . . . . . . . . . . . . . . . . 280
xii Contents

11.2.1 Deep Belief Networks and Deep Boltzmann Machine . 281

11.2.2 RBM for Real-Valued Data . . . . . . . . . . . . . . . 283
11.3 AutoEncoder . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
11.3.1 Sparse Autoencoder . . . . . . . . . . . . . . . . . . . 286
11.3.2 Denoising Autoencoder . . . . . . . . . . . . . . . . . 287
11.3.3 Stacked Autoencoder . . . . . . . . . . . . . . . . . . . 287
11.4 Convolutional Neural Networks . . . . . . . . . . . . . . . . . 288
11.4.1 Transfer Feature Learning of CNN . . . . . . . . . . . 290
11.5 Word Embedding and Recurrent Neural Networks . . . . . . 291
11.5.1 Word Embedding . . . . . . . . . . . . . . . . . . . . . 291
11.5.2 Recurrent Neural Networks . . . . . . . . . . . . . . . 294
11.5.3 Gated Recurrent Unit . . . . . . . . . . . . . . . . . . 295
11.5.4 Long Short-Term Memory . . . . . . . . . . . . . . . . 296
11.6 Generative Adversarial Networks and Variational Autoencoder 296
11.6.1 Generative Adversarial Networks . . . . . . . . . . . . 297
11.6.2 Variational Autoencoder . . . . . . . . . . . . . . . . . 298
11.7 Discussion and Further Readings . . . . . . . . . . . . . . . . 299

III Feature Engineering in Special Applications 309

12 Feature Engineering for Social Bot Detection 311
Onur Varol, Clayton A. Davis, Filippo Menczer, and Alessandro Flammini
12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 312
12.2 Social Bot Detection . . . . . . . . . . . . . . . . . . . . . . . 312
12.2.1 Holistic Approach . . . . . . . . . . . . . . . . . . . . 313
12.2.2 Pairwise Account Comparison . . . . . . . . . . . . . . 313
12.2.3 Egocentric Analysis . . . . . . . . . . . . . . . . . . . 314
12.3 Online Bot Detection Framework . . . . . . . . . . . . . . . 314
12.3.1 Feature Extraction . . . . . . . . . . . . . . . . . . . . 315
12.3.1.1 User-Based Features . . . . . . . . . . . . . . 316
12.3.1.2 Friend Features . . . . . . . . . . . . . . . . 316
12.3.1.3 Network Features . . . . . . . . . . . . . . . 318
12.3.1.4 Content and Language Features . . . . . . . 318
12.3.1.5 Sentiment Features . . . . . . . . . . . . . . 319
12.3.1.6 Temporal Features . . . . . . . . . . . . . . . 320
12.3.2 Possible Directions for Feature Engineering . . . . . . 320
12.3.3 Feature Analysis . . . . . . . . . . . . . . . . . . . . . 320
12.3.4 Feature Selection . . . . . . . . . . . . . . . . . . . . . 323
12.3.4.1 Feature Classes . . . . . . . . . . . . . . . . . 323
12.3.4.2 Top Individual Features . . . . . . . . . . . . 324
12.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
12.5 Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334
 Contents xiii

13 Feature Generation and Engineering for Software Analytics 335

Xin Xia and David Lo
13.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
13.2 Features for Defect Prediction . . . . . . . . . . . . . . . . . 337
13.2.1 File-level Defect Prediction . . . . . . . . . . . . . . . 337
13.2.1.1 Code Features . . . . . . . . . . . . . . . . . 338
13.2.1.2 Process Features . . . . . . . . . . . . . . . . 340
13.2.2 Just-in-time Defect Prediction . . . . . . . . . . . . . 341
13.2.3 Prediction Models and Results . . . . . . . . . . . . . 343
13.3 Features for Crash Release Prediction for Apps . . . . . . . . 343
13.3.1 Complexity Dimension . . . . . . . . . . . . . . . . . . 344
13.3.2 Time Dimension . . . . . . . . . . . . . . . . . . . . . 345
13.3.3 Code Dimension . . . . . . . . . . . . . . . . . . . . . 346
13.3.4 Diffusion Dimension . . . . . . . . . . . . . . . . . . . 346
13.3.5 Commit Dimension . . . . . . . . . . . . . . . . . . . . 347
13.3.6 Text Dimension . . . . . . . . . . . . . . . . . . . . . . 347
13.3.7 Prediction Models and Results . . . . . . . . . . . . . 348
13.4 Features from Mining Monthly Reports to Predict Developer
Turnover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348
13.4.1 Working Hours . . . . . . . . . . . . . . . . . . . . . . 349
13.4.2 Task Report . . . . . . . . . . . . . . . . . . . . . . . . 349
13.4.3 Project . . . . . . . . . . . . . . . . . . . . . . . . . . 350
13.4.4 Prediction Models and Results . . . . . . . . . . . . . 351
13.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351

14 Feature Engineering for Twitter-Based Applications 359

Sanjaya Wijeratne, Amit Sheth, Shreyansh Bhatt, Lakshika Balasuriya,
Hussein S. Al-Olimat, Manas Gaur, Amir Hossein Yazdavar,
Krishnaprasad Thirunarayan
14.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 359
14.2 Data Present in a Tweet . . . . . . . . . . . . . . . . . . . . 361
14.2.1 Tweet Text-Related Data . . . . . . . . . . . . . . . . 362
14.2.2 Twitter User-Related Data . . . . . . . . . . . . . . . 363
14.2.3 Other Metadata . . . . . . . . . . . . . . . . . . . . . 364
14.3 Common Types of Features Used in Twitter-Based Applications 364
14.3.1 Textual Features . . . . . . . . . . . . . . . . . . . . . 365
14.3.2 Image and Video Features . . . . . . . . . . . . . . . . 368
14.3.3 Twitter Metadata-Related Features . . . . . . . . . . . 369
14.3.4 Network Features . . . . . . . . . . . . . . . . . . . . . 370
14.4 Twitter Feature Engineering in Selected Twitter-Based Studies 370
14.4.1 Twitter User Profile Classification . . . . . . . . . . . 371
14.4.2 Assisting Coordination during Crisis Events . . . . . . 372
14.4.3 Location Extraction from Tweets . . . . . . . . . . . . 375
14.4.4 Studying the Mental Health Conditions of Depressed
Twitter Users . . . . . . . . . . . . . . . . . . . . . . . 377
xiv Contents

14.4.5 Sentiment and Emotion Analysis on Twitter . . . . . . 379

14.5 Twitris: A Real-Time Social Media Analysis Platform . . . . 381
14.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383
14.7 Acknowledgment . . . . . . . . . . . . . . . . . . . . . . . . . 384

Index 395
Preface

Feature engineering plays a vital role in big data analytics. Machine learning
and data mining algorithms cannot work without data. Little can be achieved
if there are few features to represent the underlying data objects, and the
quality of results of those algorithms largely depends on the quality of the
available features. Data can exist in various forms such as image, text, graph,
sequence, and time series. A common way to represent data for data analytics
is to use feature vectors. Feature engineering meets the needs in the generation
and selection of useful features, as well as several other issues.
This book is devoted to feature engineering. It covers various aspects
of feature engineering, including feature generation, feature extraction, fea-
ture transformation, feature selection, and feature analysis and evaluation. It
presents concepts, methods, examples, as well as applications.
Feature engineering is often data type specific and application dependent.
This calls for multiple chapters on different data types that require specialized
feature engineering techniques to meet various data analytic needs. Hence, this
book contains chapters on feature engineering for major data types such as
texts, images, sequences, time series, graphs, streaming data, software engi-
neering data, Twitter data, and social media data. It also contains generic
feature generation approaches, as well as methods for generating tried-and-
tested, hand-crafted, domain-specific features.
This book contains many useful feature engineering concepts and tech-
niques, which are an important part of machine learning and data analytics.
They can help readers to meet their needs in multiple scenarios: (a) gener-
ate features to represent the data when there are no features, (b) generate
effective features when (one may be concerned that) existing features are
not good/competitive enough, (c) select features when there are too many
features, (d) generate and select effective features for specific types of appli-
cations, and (e) understand the challenges associated with, and the needed
approaches to handle, various data types. This list is certainly not exhaustive.
The first chapter is an introduction, which defines the concepts of fea-
tures and feature engineering, offers an overview of the book, and provides
pointers to topics not covered in this book. The next six chapters are devoted
to feature engineering, including feature generation, for specific data types,
namely texts, images, sequences, time series, graphs, and streaming data. The
subsequent four chapters cover generic approaches for feature engineering,
namely feature selection, feature transformation-based feature engineering,

xv
xvi Preface

deep learning–based feature engineering, and pattern-based feature genera-

tion and engineering. The last three chapters discuss feature engineering for
social bot detection, software management, and Twitter-based applications
respectively.
Getting familiar with the concepts and techniques covered in this book will
boost one’s understanding and expertise in machine learning and big data
analytics. This book can be used as a reference for data analysts, big data
scientists, data preprocessing workers, project managers, project developers,
prediction modelers, professors, researchers, graduate students, and upper-
level undergraduate students. This book can be used as the primary text for
courses on feature engineering, and as supplementary materials for courses on
machine learning, data mining, and big data analytics.
We wish to express our profound gratitude to the contributing authors of
the chapters of the book; without their expertise and dedicated efforts, this
book would not be possible. We are grateful to Randi Cohen and Veronica
Rodriguez who provided guidance and assistance on the publishing side of this
effort. We are indebted to Jiawei Han, Jian Pei, Nicholas Skapura, Xintao Wu,
and Junjie Zhang who kindly suggested domain experts as potential authors
and so on, and also to Vineeth Rakesh Mohan who provided useful feedback
on parts of this book.

Guozhu Dong, Dayton, Ohio

Huan Liu, Phoenix, Arizona

Contributors

Charu Aggarwal Bloomington, Indiana, USA

IBM Research
Yorktown Heights, New York, USA Ben D. Fulcher
Monash University
Hussein S. Al-Olimat Melbourne, Victoria, Australia
Wright State University
Dayton, Ohio, USA Manas Gaur
James Bailey Wright State University
The University of Melbourne Dayton, Ohio, USA
Parkville, Victoria, Australia
Chase Geigle
Lakshika Balasuriya University of Illinois at
Wright State University Urbana-Champaign
Dayton, Ohio, USA Urbana-Champaign, Illinois, USA
Shreyansh Bhatt Yunzhe Jia
Wright State University
The University of Melbourne
Dayton, Ohio, USA
Parkville, Victoria, Australia
Parag S. Chandakkar
Arizona State University Ramamohanarao Kotagiri
Phoenix, Arizona, USA The University of Melbourne
Parkville, Victoria, Australia
Clayton A. Davis
Indiana University Christopher Leckie
Bloomington, Indiana, USA The University of Melbourne
Parkville, Victoria, Australia
Guozhu Dong
Wright State University Baoxin Li
Dayton, Ohio, USA
Arizona State University
Lei Duan Phoenix, Arizona, USA
Sichuan University
Chengdu, Sichuan, China Tao Li
Nanjing University of Posts and
Alessandro Flammini Telecommunications
Indiana University Nanjing, Jiangsu, China

xvii
xviii Contributors

Yun Li Hanghang Tong

Nanjing University of Posts and Arizona State University
Telecommunications Phoenix, Arizona, USA
Nanjing, Jiangsu, China
Onur Varol
Huan Liu Indiana University
Arizona State University Bloomington, Indiana, USA
Phoenix, Arizona, USA
Ragav Venkatesan
David Lo Arizona State University
Singapore Management University Phoenix, Arizona, USA
Singapore
Suhang Wang
Jian Lu Arizona State University
Nanjing University Phoenix, Arizona, USA
Nanjing, Jiansu, China
Sanjaya Wijeratne
Yao Ma
Wright State University
Michigan State University Dayton, Ohio, USA
East Lansing, Michigan, USA
Xin Xia
Qiaozhu Mei
Monash University
University of Illinois at
Melbourne, Victoria, Australia
Urbana-Champaign
Urbana-Champaign, Illinois, USA Feng Xu
Nanjing University
Filippo Menczer
Nanjing, Jiansu, China
Indiana University
Bloomington, Indiana, USA
Yuan Yao
Jyrki Nummenmaa Nanjing University
University of Tampere Nanjing, Jiansu, China
Tampere, Finland
Amir Hossein Yazdavar
Amit Sheth Wright State University
Wright State University Dayton, Ohio, USA
Dayton, Ohio, USA
ChengXiang Zhai
Jiliang Tang University of Illinois at
Michigan State University Urbana-Champaign
East Lansing, Michigan, USA Urbana-Champaign, Illinois, USA

Krishnaprasad Thirunarayan Peng Zhang

Wright State University Sichuan University
Dayton, Ohio, USA Chengdu, Sichuan, China
Chapter 1
Preliminaries and Overview

Guozhu Dong
Wright State University, Dayton, Ohio, USA

Huan Liu
Arizona State University, Phoenix, Arizona, USA

1.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.1 Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.2 Feature Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1.3 Machine Learning and Data Analytic Tasks . . . . . . . . . . . . 3
1.2 Overview of the Chapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3 Beyond this Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3.1 Feature Engineering for Specific Data Types . . . . . . . . . . . 8
1.3.2 Feature Engineering on Non-Data-Specific Topics . . . . . . 9
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

At the end of the day, some machine learning projects succeed

and some fail. What makes the difference? Easily the most
important factor is the features used.
Feature engineering is the key.

Pedro Domingos, 2012 [2].

This chapter first provides preliminaries on the basic concepts related to

features, feature engineering, and data analytic tasks. It then gives an overview
of this book. Finally, it offers pointers to other feature engineering topics not
covered in other chapters of this book.

1.1 Preliminaries
1.1.1 Features
In machine learning, data mining, and data analytics, a feature is an
attribute or variable used to describe some aspect of individual data objects.

1
2 Feature Engineering for Machine Learning and Data Analytics

Example features include age and eye color for persons, and major and grade
point average for students.
Informative features are the basis for data analytics. They are useful for
describing the underlying objects, and for distinguishing and characterizing
different (explicit or latent) groups of objects. They are also vital for producing
accurate and easy-to-explain predictive models, and yielding good results in
various data analytic tasks. “Feature,” “variable,” and “attribute” are often
used as synonyms.
For a given application and a fixed point in time, often a fixed set of
features is implicitly chosen to describe all underlying data objects; each object
takes a particular value for each of those features. This results in a feature-
vector-based representation of the data objects.
Features are divided into several feature types, including categorical, ordi-
nal, and numerical. Different feature types require different kinds of analysis,
due to structural differences in their domains.
• The domain of a categorical feature is a set of discrete val-
ues. For example, color is a categorical feature whose domain is
{black, blue, brown, green, red, white, yellow}.

• A special type of categorical feature is binary, whose domain has exactly

two values.
• The domain of an ordinal feature is a set of ordered values. The degree
feature is an ordinal feature whose domain is {Bachelor, M aster, P hD}
and the three values are ordered as follows: Bachelor < M aster < P hD.

• The domain of a numerical feature is a set of numerical values. A nu-

merical feature is also called quantitative or continuous. For example,
the age feature is a numerical one whose domain is the set of integers
between 0 and 150.
There is a less-than (denoted by <) relationship between each pair of
numerical values, e.g., 3 < 4. This relationship and the implied nearby-
ness among numerical values are often exploited in machine learning and
data analytics.
• A numerical feature is a ratio-scaled feature if the following is true: a
value u is twice as much as a value v with respect to some meaning (e.g.,
warmness or length) associated with the feature, whenever it is the case
that u = 2 ∗ v.
In many applications categorical features are represented as numerical val-
ues. Care is needed in such cases as the order implied by the numerical values
is typically meaningless. In R packages categorical variables are treated as fac-
tors. A so-called “one-hot encoding” is often used to transform such features.
The usefulness of a feature is measured ultimately in terms of the im-
provement the feature adds to the data analytic task at hand. For example,
 Preliminaries and Overview 3

in classification this is mostly in terms of the improvement in classification

accuracy. The understandability and interpretability of the feature are also of
significant interest.

1.1.2 Feature Engineering

This book uses a very general definition for feature engineering. It includes
the topics of feature transformation, feature generation, feature extraction,
feature selection, feature analysis and evaluation, general automatic feature
engineering methodology, and feature engineering applications. We briefly ex-
plain these concepts below.

(1) Feature transformation is about constructing new features from existing

features; this is often achieved using mathematical mappings.

(2) Feature generation is about generating new features that are often not
the result of feature transformations. For example, assuming that one
does not view a pixel in an image as a feature, one generates new features
for images. Moreover, it makes sense to say that features defined from
patterns are generated features. Many domain-specific ways for defining
features also belong in the feature generation category. Sometimes the
term feature extraction is used for feature generation.
(3) Feature selection is about selecting a small set of features from a very
large pool of features. The reduced feature set size makes it computa-
tionally feasible to use certain algorithms. Feature selection may also
lead to improved quality on the result of those algorithms.

(4) Feature analysis and evaluation is about concepts, methods, and mea-
sures for evaluating the usefulness of features and feature sets. This is
often included as part of feature selection.
(5) General automatic feature engineering methodology is about generic ap-
proaches for automatically generating a large number of features and
selecting an effective subset of the generated features.
(6) Feature engineering applications involve feature engineering but the fo-
cus is to solve some other data analytic tasks in specific contexts. Ex-
amples include analyzing Twitter data to improve the quality of disaster
response and relief efforts.

1.1.3 Machine Learning and Data Analytic Tasks

Generally speaking, all machine learning, data mining, and data analytic
tasks rely on and can benefit from effective feature engineering. Specific tasks
include classification, regression, clustering, outlier detection, pattern/rule
4 Feature Engineering for Machine Learning and Data Analytics

mining, predictive modeling, contrasting and characterizing data classes, con-

cept discovery, distance learning, probability estimation, ontology/taxonomy
construction, information retrieval, business intelligence, and so on. Below we
provide basic concepts for some of these tasks. More information can be found
in textbooks on machine learning and data mining such as [6, 7, 26, 30].
Classification is the task of learning a function f that maps each data
object x to one member of some given set of predefined class labels y. The
resulting function f is often referred to as a classification model. For a given
application, the learning of a classification model is achieved by applying some
method to a training dataset, namely a set of (x, y) pairs where x is a data
object and y is a class label.
Regression is similar to classification except that y is a numerical attribute
instead of a class label.
Clustering is the task of partitioning a given dataset D into a number
of subsets (the clusters) so that the objects within a cluster are highly sim-
ilar to each other and objects in different clusters are highly different from
each other. Similarity is often measured using a distance function, but other
approaches also exist. Clustering is also referred to as segmentation and as
concept discovery.
Outlier detection (also called anomaly detection) is the task of identifying
objects that do not conform to an expected pattern exhibited by the majority
of objects in a given dataset.
Pattern mining is the task of identifying patterns that are interesting in
some sense. A pattern can be viewed as a condition of individual objects that
can be evaluated as true or false. The pattern mining process can be applied
to one dataset, in which case one wants to mine interesting frequent patterns;
it can also be applied to multiple datasets (e.g., classes), in which case one
wants to mine interesting patterns that distinguish one of the datasets from
the other datasets.

1.2 Overview of the Chapters

This book has three parts, together with this introductory chapter. Part
1 consists of six chapters (2 to 7) on feature engineering for various data
types. Part 2 contains four chapters (8 to 11) on general feature engineering
techniques which are not specific to data types. Part 3 includes three chapters
(12 to 14) on feature engineering in special applications.
Chapter 2 provides a systematic review of the main techniques for feature
representations for text data. Text data can be regarded as data reported by
human sensors, which are supplementary to data collected by physical sen-
sors. Text data are useful in many applications especially for supporting de-
cision making and analyzing people’s opinions and preferences. This chapter
 Preliminaries and Overview 5

discusses (a) the dominant bag of words–based text representation, (b) ap-
proaches that use multiple words as features, and (c) structural features that
require natural language processing techniques or statistical pattern analysis
methods. It further describes how to learn latent semantic representations us-
ing methods such as probabilistic topic models and neural networks, and how
text data can be analyzed together with non-textual context data to extract
contextualized text representations.
A majority of visual computing tasks involve prediction, regression or deci-
sion making using features extracted from the original, raw visual data (images
or videos). Chapter 3 presents a hierarchy of feature representations for im-
age data, starting with classic, hand-crafted features. The classic features are
designed by human experts and they are based on task-specific prior knowl-
edge. They are easily interpretable and characterize fundamental aspects of
images such as color, texture and shape. The features at the next level are
latent feature representations. Such features represent task-specific structures
in the feature space such as sparsity, decorrelation of reduced dimension, low
rank, etc.
Time series is an important type of data that are frequently encountered
in data analytics. Chapter 4 provides an overview of a vast literature of rep-
resentations and analysis methods for time series. It first presents discussion
on global distances between time-series values including Euclidean and elastic
distance measures like DTW. It then discusses three kinds of features, namely
subsequences that provide more localized shape-based information, global fea-
tures that capture higher order structure, and interval features that capture
discriminative properties in time-series subsequences. It also discusses factors
that influence the selection of the most useful method for a given task.
Chapter 5 provides an overview of feature engineering for streaming data,
with a focus on streaming feature construction and selection. It first summa-
rizes the typical streaming settings and their corresponding formal defini-
tions. Then it reviews automated feature construction algorithms including
linear and non-linear methods. Next it gives an overview of feature selection
algorithms with different streaming settings. Finally it discusses some open
questions and possible research directions about feature engineering for data
streams.
Sequence data occur in many applications including bioinformatics, mu-
sic, literature, health care, and security. Chapter 6 first discusses the basic
concepts for sequence data. It then discusses three major classes of sequence
features, namely traditional pattern-based sequence features, general pattern-
based features, and sequence features that do not involve the use of patterns. It
presents several approaches for using sequence patterns as sequence features,
and it provides an overview of sequence pattern types as well as methods to
mine such patterns. It also considers factors that are important for selecting
patterns as features.
Graph and network data are essential for various graph analysis tasks
such as social network analysis, protein–protein interaction analysis, and
6 Feature Engineering for Machine Learning and Data Analytics

chemical molecule toxicity analysis. Chapter 7 focuses on feature genera-

tion for graphs and networks. It first discusses the feature types for graphs,
including neighborhood-level features and global-level features. Next, it de-
scribes existing feature generation methods, divided into feature extraction
approaches and feature learning approaches. Finally, it presents several ap-
plications to illustrate graph feature usages, including the applications of
multi-label classification on nodes, link prediction, anomaly detection, and
visualization.
Feature selection is one of the key problems for machine learning and data
mining. Chapter 8 reviews recent developments on this topic. A brief histor-
ical background of the field is given, followed by a selection of topics which
are of particular current interests, such as stable feature selection, multi-view
feature selection, distributed feature selection, multi-label feature selection,
online feature selection and adversarial feature selection. The chapter then
reviews recent research advances of these topics.
The process of predictive modeling requires extensive feature engineering.
It often involves the transformation of a given feature space, typically using
mathematical functions, with the objective of reducing the modeling error for
a given target. However, there is no well-defined basis for performing effec-
tive feature engineering. It involves domain knowledge, intuition, and most
of all, a lengthy process of trial and error. The human attention involved in
overseeing this process significantly influences the cost of model generation.
Moreover, when the data presented is not well described and labeled, effective
manual feature engineering becomes an even more prohibitive task. Chapter
9 discusses ways to algorithmically tackle the problem of feature engineering
using transformation functions in the context of supervised learning.
Frequent patterns are combinations of conditions on features that have
a high frequency of co-occurrence, which can represent interesting interac-
tion relationships among features in a given dataset. Features generated using
patterns can be more discriminative than individual features. Chapter 10
provides a systematic overview on pattern-based feature generation. Specifi-
cally, it presents approaches for generating patterns, techniques for pruning
large pattern sets, strategies for constructing new features using patterns, and
applications of pattern-based feature generation for classification and cluster-
ing.
Deep learning methods have become increasingly popular in recent years
because of their tremendous success in image classification, speech recognition
and natural language processing tasks. The great success of deep learning
mainly comes from specially designed structures of deep nets, which are able
to learn discriminative non-linear features that can facilitate the task at hand.
In essence, the majority of existing deep learning algorithms can be used as
powerful feature learning/extraction tools, i.e., the latent features extracted
by deep learning algorithms are the learned new representations. Chapter 11
reviews various classical and popular deep learning algorithms and explains
how they can be used for feature representation learning. It also discusses
 Preliminaries and Overview 7

how they are used for hierarchical and disentangle representation learning,
and how they can be applied for various domains.
Increasing evidence suggests that social platforms like Twitter accommo-
date an increasing number of autonomous entities known as social bots, which
are controlled by software that generates content and establishes interactions
with other accounts. Chapter 12 considers feature engineering for social bot
detection in the context of social media. It describes the setting of such de-
tection, and it presents various kinds of features, some of which are unique
for social media, including their definition, selection, and usefulness for social
bot detection. It also describes a system called Botometer that analyzes pub-
lic information about a Twitter account, extracting over a thousand features
describing the account and its neighbors, and discusses experiments where the
extracted features were used to build classifiers for bot detection.
Chapter 13 considers feature generation and engineering for software
analytics. It shows how domain-specific features can be designed and used
to automate three software engineering tasks: (1) detecting defective software
modules, (2) identifying a crashing mobile app release, and (3) predicting who
will leave a software team. For each task, different sets of features are extracted
from a diverse set of software artifacts, and used to build predictive models.
The chapter also discusses recent advances as well as their potential.
Chapter 14 presents studies concerning feature engineering for Twitter-
based applications. It first discusses how Twitter data can be downloaded
from the Twitter Application Programming Interface (API) and the kinds of
data available in the downloaded tweets. Then, it discusses various textual
features, image and video features, Twitter metadata-related features, and
network features that can be extracted. Next, it discusses the uses of different
feature types along with an analysis of why certain features perform well
in the context of informal short text messages typically found in tweets. It
then presents five real-world Twitter applications that utilize different feature
types. For each application, it also highlights the features that perform well
in the corresponding application setting. Finally, it concludes the chapter by
discussing Twitris, a real-time semantic social web analytics platform that has
already been commercialized, and its use of Twitter features.

1.3 Beyond this Book

No single book can give due attention to the rich variety of topics of fea-
ture engineering. This section provides some pointers to topics not covered in
other parts of the book. It includes topics on speech features, music features,
malware detection data features, log data features, transfer learning–based
feature engineering, numerical feature discretization, feature engineering in-
side machine learning and data analytic algorithms, and early papers and
8 Feature Engineering for Machine Learning and Data Analytics

books on feature engineering. Due to the broadness of the feature engineering

field, this chapter is certainly not complete.

1.3.1 Feature Engineering for Specific Data Types

There have been numerous studies on speech feature generation (e.g.,
[11,22]), often conducted as part of speech recognition (see [20]). Reference [28]
gives an account of acoustic features used for emotional speech recognition;
these features include the pitch, the formants, the vocal-tract cross-section ar-
eas, the mel-frequency cepstral coefficients, the Teager energy operator-based
features, the intensity of the speech signal, and the speech rate.
Music data mining and analysis have attracted a lot of attention [13, 29],
whose tasks include genre classification, emotion and mood detection [25],
instrument detection, and music characteristic identification. Within a piece
of music one may be interested in analyzing emotion or mood differences
between different parts [12], and finding repeating patterns [8]. Music data
is quite complex, as it includes an acoustic part (pitch, intensity, etc.), a
music score part, and a text part (the lyrics). Music data can be treated as
large sequences, although the alphabet of the sequences is quite large and
different elements are related to each other in significant ways, and one needs
to consider issues such as the handling of several concurrent sequences for
each of the acoustic, music score and lyrics parts. Music data and speech data
are also related. Chapter 4 on time series analysis and Chapter 6 on sequence
feature engineering are related to music data analysis.
Analyzing executable codes to detect malware is an important problem.
Several types of static features were used in [23], namely DLL-related features,
system call features, and string/sequence features. The DLL-related features
include the DLLs contained in the binary code, the DLL function calls, and
the number of different system calls used within each DLL. Function length
features were used for malware classification in [27], and network behavior
features were used in [18]. Reference [5] gives a survey of malware classification.
Another kind of data with distinct characteristics is the execution and net-
work navigation log data. An example is network intrusion data such as the
1999 KDD Cup dataset [10]. This KDD Cup dataset contains logs (raw TCP
dump data) collected from a typical LAN. Reference [9] discusses generat-
ing and selecting features for this dataset; it considered four types of features,
namely basic features (e.g., those derived from packet headers without inspect-
ing the payload), content features, time-based traffic features, and host-based
traffic features.
Analyzing game-related data can be useful in several ways, including un-
derstanding human behavior, designing winning game playing strategies, and
improving game designs. However, game-play data lack structure and has big
volumes. Reference [1] discusses issues related to feature generation and se-
lection for game-play data.
 Preliminaries and Overview 9

1.3.2 Feature Engineering on Non-Data-Specific Topics

Transfer learning can be used to find effective features for a new dataset
from another dataset, although algorithms in this field are often originally
designed for other data analytic tasks such as classification and clustering.
Transfer learning can also help avoid much expensive data labeling efforts.
Transfer learning can also mine similar structures such as shared decision
trees [3]. For a survey, see [19].
Feature discretization (also called binning) of numerical data is useful
for feature transformation and feature generation, and sometimes feature se-
lection. Feature discretization is about constructing informative categorical
representations of numerical features so that the categorical values retain as
much information in the original numerical values as possible. Representa-
tive research includes [4, 15]. Reference [21] is a recent a survey on feature
discretization. This is still an active field of research [17, 24].
Implicit feature generation/transformation is often a part of machine learn-
ing and data mining algorithms aimed at solving specific problems. Often new
features are automatically constructed and selected for use in machine learn-
ing systems such as deep learning (see Chapter 11). Also, meta classification
uses predicted values of other classifiers as features [14].
Reference [16] is an early book that was devoted to several aspects of fea-
ture engineering. It contains chapters on feature selection, feature extraction,
and feature construction. For example, it includes chapters on feature extrac-
tion using adaptive wavelets and using neural networks, feature transformation
by function decomposition, and automatic fractal feature extraction for image
recognition. This current book is more complete and more up to date; it covers
a wider range of topics and techniques for feature engineering, and it includes
many hand-crafted domain-specific feature generation techniques.

Bibliography
[1] Alessandro Canossa. Meaning in gameplay: Filtering variables, defining
metrics, extracting features and creating models for gameplay analysis.
In Game Analytics, pages 255–283. Springer, 2013.
[2] Pedro Domingos. A few useful things to know about machine learning.
Communications of the ACM, 55(10):78–87, 2012.
[3] Guozhu Dong and Qian Han. Mining accurate shared decision trees from
microarray gene expression data for different cancers. In Proceedings of
the International Conference on Bioinformatics & Computational Biology
(BIOCOMP), 2013.
10 Feature Engineering for Machine Learning and Data Analytics

[4] Usama Fayyad and Keki Irani. Multi-interval discretization of

continuous-valued attributes for classification learning. In Proceedings
of the 13th International Joint Conference on Artificial Intelligence (IJ-
CAI), pages 1022–1029, 1993.
[5] Ekta Gandotra, Divya Bansal, and Sanjeev Sofat. Malware analysis and
classification: A survey. Journal of Information Security, 5(02):56, 2014.
[6] Jiawei Han, Jian Pei, and Micheline Kamber. Data Mining: Concepts and
Techniques. Elsevier, 2011.
[7] David J Hand, Heikki Mannila, and Padhraic Smyth. Principles of Data
Mining. MIT Press, 2001.
[8] Jia-Lien Hsu, Arbee LP Chen, and C-C Liu. Efficient repeating pattern
finding in music databases. In Proceedings of the Seventh International
Conference on Information and Knowledge Management, pages 281–288.
ACM, 1998.
[9] H Günes Kayacik, A Nur Zincir-Heywood, and Malcolm I Heywood. Se-
lecting features for intrusion detection: A feature relevance analysis on
KDD 99 intrusion detection datasets. In Proceedings of the Third Annual
Conference on Privacy, Security and Trust, 2005.
[10] KDD Cup. 1999. http://kdd.ics.uci.edu/databases/kddcup99
[11] Yelin Kim, Honglak Lee, and Emily Mower Provost. Deep learning
for robust feature generation in audiovisual emotion recognition. In
IEEE International Conference on Acoustics, Speech and Signal Process-
ing (ICASSP), pages 3687–3691. IEEE, 2013.
[12] Hua-Fu Li. Memsa: Mining emerging melody structures from music query
data. Multimedia Systems, 17(3):237–245, 2011.
[13] Tao Li, Mitsunori Ogihara, and George Tzanetakis. Music Data Mining.
CRC Press, 2011.
[14] Wei-Hao Lin and Alexander Hauptmann. News video classification using
SVM-based multimodal classifiers and combination strategies. In Proceed-
ings of the Tenth ACM International Conference on Multimedia, pages
323–326. ACM, 2002.
[15] Huan Liu, Farhad Hussain, Chew Lim Tan, and Manoranjan Dash. Dis-
cretization: An enabling technique. Data Mining and Knowledge Discov-
ery, 6(4):393–423, 2002.
[16] Huan Liu and Hiroshi Motoda. Feature Extraction, Construction and
Selection: A Data Mining Perspective, volume 453. Springer Science &
Business Media, 1998.
Exploring the Variety of Random
Documents with Different Content
 PATTIES À LA PONTIFE. (ENTRÉE.)

(A fast day, or Maigre dish.)

Mince, but not very small, the yolks of six fresh hard-boiled eggs;
mince also and mix with them a couple of fine truffles,[120] a large
saltspoonful of salt, half the quantity of mace and nutmeg, and a
fourth as much of cayenne. Moisten these ingredients with a
spoonful of thick cream, or béchamel maigre (see page 109), or with
a dessertspoonful of clarified butter; line the patty-moulds, fill them
with the mixture, cover, and bake them from twelve to fifteen minutes
in a moderate oven. They are excellent made with the cream-crust of
page 347.
120. The bottled ones will answer well for these.
Yolks hard-boiled eggs, 6; truffles, 2 large; seasoning of salt,
mace, nutmeg, and cayenne; cream, or béchamel maigre, 1
tablespoonful, or clarified butter, 1 dessertspoonful: baked moderate
oven, 12 to 15 minutes.
Obs.—A spoonful or two of jellied stock or gravy, or of good white
sauce, converts these into admirable patties: the same ingredients
make also very superior rolls or cannelons. For Patties à la
Cardinale, small mushroom-buttons stewed as for partridges,
Chapter XIII., before they are minced, must be substituted for
truffles; and the butter in which they are simmered should be added
with them to the eggs.
 EXCELLENT MEAT ROLLS.

Pound, as for potting (see page 305), and with the same
proportion of butter and of seasonings, some half-roasted veal,
chicken, or turkey. Make some forcemeat by the receipt No. 1,
Chapter VI., and form it into small rolls, not larger than a finger; wrap
twice or thrice as much of the pounded meat equally round each of
these, first moistening it with a teaspoonful of water; fold them in
good puff-paste, and bake them from fifteen to twenty minutes, or
until the crust is perfectly done. A small quantity of the lean of a
boiled ham may be finely minced and pounded with the veal, and
very small mushrooms, prepared as for a partridge (page 329), may
be substituted for the forcemeat.
 SMALL VOLS-AU-VENTS, OR PATTY-CASES.

These are quickly and easily made with two round paste-cutters,
of which one should be little more than half the size of the other: to
give the pastry a better appearance, they should be fluted. Roll out
some of the lightest puff-paste to a half-inch of thickness, and with
the larger of the tins cut the number of patties required; then dip the
edge of the small shape into hot water, and press it about half
through them. Bake them in a moderately quick oven from ten to
twelve minutes, and when they are done, with the point of a sharp
knife, take out the small rounds of crust from the tops, and scoop all
the crumb from the inside of the patties, which may then be filled
with shrimps, oysters, lobster, chicken, pheasant, or any other of the
ordinary varieties of patty meat, prepared with white sauce. Fried
crumbs may be laid over them instead of the covers, or these last
can be replaced.
For sweet dishes, glaze the pastry, and fill it with rich whipped
cream, preserve, or boiled custard; if with the last of these put it back
into a very gentle oven until the custards are set.
 ANOTHER RECEIPT FOR TARTLETS.

For a dozen tartlets, cut twenty-four rounds of paste of the usual

size, and form twelve of them into rings by pressing the small cutter
quite through them; moisten these with cold water, or white of egg,
and lay them on the remainder of the rounds of paste, so as to form
the rims of the tartlets. Bake them from ten to twelve minutes, fill
them with preserve while they are still warm, and place over it a
small ornament of paste cut from the remnants, and baked gently of
a light colour. Serve the tartlets cold, or if wanted hot for table put
them back into the oven for one minute after they are filled.
 A SEFTON, OR VEAL CUSTARD.

Pour boiling, a pint of rich, clear, pale veal gravy on six fresh eggs,
which have been well beaten and strained: sprinkle in directly the
grated rind of a fine lemon, a little cayenne, some salt if needed, and
a quarter-teaspoonful of mace. Put a paste border round a dish, pour
in, first two ounces of clarified butter, and then the other ingredients;
bake the Sefton in a very slow oven from twenty-five to thirty
minutes, or until it is quite firm in the middle, and send it to table with
a little good gravy. Very highly flavoured game stock, in which a few
mushrooms have been stewed, may be used for this dish with great
advantage in lieu of veal gravy; and a sauce made of the smallest
mushroom buttons, may be served with it in either case. The mixture
can be baked in a whole paste, if preferred so, or in well buttered
cups; then turned out and covered with the sauce before it is sent to
table.
Rich veal or game stock, 1 pint; fresh eggs, 6; rind, 1 lemon; little
salt and cayenne; pounded mace, 1/4 teaspoonful; butter, 2 oz.:
baked, 25 to 30 minutes, slow oven.
 APPLE CAKE, OR GERMAN TART.

Work together with the fingers, ten ounces of butter and a pound
of flour, until they resemble fine crumbs of bread; throw in a small
pinch of salt, and make them into a firm smooth paste with the yolks
of two eggs and a spoonful or two of water. Butter thickly, a plain tin
cake, or pie mould (those which open at the sides, see plate, page
344, are best adapted for the purpose); roll out the paste thin, place
the mould upon it, trim a bit to its exact size, cover the bottom of the
mould with this, then cut a band the height of the sides, and press it
smoothly round them, joining the edge, which must be moistened
with egg or water, to the bottom crust; and fasten upon them, to
prevent their separation, a narrow and thin band of paste, also
moistened. Next, fill the mould nearly from the brim with the following
marmalade, which must be quite cold when it is put in. Boil together,
over a gentle fire at first, but more quickly afterwards, three pounds
of good apples with fourteen ounces of pounded sugar, or of the
finest Lisbon, the strained juice of a large lemon, three ounces of
fresh butter, and a teaspoonful of pounded cinnamon, or the lightly
grated rind of a couple of lemons: when the whole is perfectly
smooth and dry, turn it into a pan to cool, and let it be quite cold
before it is put into the paste. In early autumn, a larger proportion of
sugar may be required, but this can be regulated by the taste. When
the mould is filled, roll out the cover, lay it carefully over the
marmalade that it may not touch it; and when the cake is securely
closed, trim off the superfluous paste, add a little pounded sugar to
the parings, spread them out very thin, and cut them into leaves to
ornament the top of the cake, round which they may be placed as a
sort of wreath.[121] Bake it for an hour in a moderately brisk oven;
take it from the mould, and should the sides not be sufficiently
coloured put it back for a few minutes into the oven upon a baking
tin. Lay a paper over the top, when it is of a fine light brown, to
prevent its being too deeply coloured. This cake should be served
hot.
121. Or, instead of these, fasten on it with a little white of egg, after it is taken from
the oven, some ready-baked leaves of almond-paste (see page 355), either
plain or coloured.
Paste: flour, 1 lb.; butter, 10 oz.; yolks of eggs, 2; little water.
Marmalade: apples, 3 lbs.; sugar, 14 oz. (more if needed); juice of
lemon, 1; rinds of lemons, 2; butter, 3 oz.: baked, 1 hour.
 TOURTE MERINGUÉE, OR TART WITH ROYAL ICING.[122]
122. The limits to which we are obliged to confine this volume, compel us to omit
many receipts which we would gladly insert; we have, therefore, rejected
those which may be found in almost every English cookery book, for such as
are, we apprehend, less known to the reader: this will account for the small
number of receipts for pies and fruit tarts to be found in the present chapter.

Lay a band of fine paste round the rim of a tart-dish, fill it with any
kind of fruit mixed with a moderate proportion of sugar, roll out the
cover very evenly, moisten the edges of the paste, press them
together carefully, and trim them off close to the dish; spread equally
over the top, to within rather more than an inch of the edge all round,
the whites of three fresh eggs beaten to a quite solid froth and mixed
quickly at the moment of using them with three tablespoonsful of dry
sifted sugar. Put the tart into a moderately brisk oven, and when the
crust has risen well and the icing is set, either lay a sheet of writing-
paper lightly over it, or draw it to a part of the oven where it will not
take too much colour. This is now a fashionable mode of icing tarts,
and greatly improves their appearance.
Bake half an hour.
 A GOOD APPLE TART.

A pound and a quarter of apples weighed after they are pared and
cored, will be sufficient for a small tart, and four ounces more for one
of moderate size. Lay a border of English puff-paste, or of cream-
crust round the dish, just dip the apples into water, arrange them
very compactly in it, higher in the centre than at the sides, and strew
amongst them from three to four ounces of pounded sugar, or more
should they be very acid: the grated rind and the strained juice of
half a lemon will much improve their flavour. Lay on the cover rolled
thin, and ice it or not at pleasure. Send the tart to a moderate oven
for about half an hour. This may be converted into the old-fashioned
creamed apple tart, by cutting out the cover while it is still quite hot,
leaving only about an inch-wide border of paste round the edge, and
pouring over the apples when they have become cold, from half to
three-quarters of a pint of rich boiled custard. The cover divided into
triangular sippets, was formerly stuck round the inside of the tart, but
ornamental leaves of pale puff-paste have a better effect. Well-
drained whipped cream may be substituted for the custard, and be
piled high, and lightly over the fruit.
 TART OF VERY YOUNG GREEN APPLES. (GOOD.)

Take very young apples from the tree before the cores are formed,
clear off the buds and stalks, wash them well, and fill a tart-dish with
them after having rolled them in plenty of sugar, or strew layers of
sugar between them; add a very small quantity of water and bake
the tart rather slowly, that the fruit may be tender quite through. It will
resemble a green apricot-tart if carefully made. We give this receipt
from recollection, having had the dish served often formerly, and
having found it very good.
 BARBERRY TART.

Barberries, with half their weight of fine brown sugar, when they
are thoroughly ripe, and with two ounces more when they are not
quite so, make an admirable tart. For one of moderate size, put into
a dish bordered with paste three quarters of a pound of barberries
stripped from their stalks, and six ounces of sugar in alternate layers;
pour over them three tablespoonsful of water, put on the cover, and
bake the tart for half an hour. Another way of making it is, to line a
shallow tin pan with very thin crust, to mix the fruit and sugar well
together with a spoon before they are laid in, and to put bars of paste
across instead of a cover; or it may be baked without either.[123]
123. The French make their fruit-tarts generally thus, in large shallow pans.
Plums, split and stoned (or if of small kinds, left entire), cherries and currants
freed from the stalks, and various other fruits, all rolled in plenty of sugar, are
baked in the uncovered crust; or this is baked by itself, and then filled
afterwards with fruit previously stewed tender.
 THE LADY’S TOURTE, AND CHRISTMAS TOURTE À LA
CHÂTELAINE.

To make this Tourte, which, when filled, is

of pretty appearance, two paste-cutters are
requisite, one the size, or nearly so, of the
inside of the dish in which the entremets is
Lady’s Tourte. to be served, the other not more than an
inch in diameter, and both of them fluted, as
will be seen by the engraving. To make the
paste for it, throw a small half saltspoonful of salt into half a pound of
the finest flour, and break lightly into it four ounces of fresh butter,
which should be firm. Make these up smoothly with cold milk or
water, of which nearly a quarter of a pint will be sufficient, unless the
butter should be very hard, when a spoonful or two more must be
added. Roll the paste out as lightly as possible twice or thrice if
needful, to blend the butter thoroughly with it, and each time either
fold it in three by wrapping the ends over each other, or fold it over
and over like a roll pudding. An additional ounce, or even two, of
butter can be used for it when very rich pastry is liked, but the tourte
will not then retain its form so well. Roll it out evenly to something
more than three-quarters of an inch in thickness, and press the large
cutter firmly through it; draw away the superfluous paste, and lay the
tourte on a lightly floured baking-tin. Roll the remainder of the paste
until it is less than a quarter of an inch thick, and stamp out with the
smaller cutter—of which the edge should be dipped into hot water, or
slightly encrusted with flour—as many rounds as will form the border
of the tourte. In placing them upon it, lay the edge of one over the
other just sufficiently to give a shell-like appearance to the whole;
and with the finger press lightly on the opposite part of the round to
make it adhere to the under paste. Next, with a sharp-pointed knife,
make an incision very evenly round the inside of the tourte nearly
close to the border, but be extremely careful not to cut too deeply
into the paste. Bake it in a gentle oven, from twenty to thirty minutes.
When it is done, detach the crust from the centre, where it has been
marked with the knife, take out part of the crumb, fill the space high
with apricot-jam, or with any other choice preserve, set it again for an
instant into the oven, and serve it hot or cold. Spikes of blanched
almonds, filberts, or pistachio-nuts, may be strewed over the
preserve, when they are considered an improvement; and the border
of the pastry may be glazed or ornamented to the fancy; but if well
made, it will generally please in its quite simple form. It may be
converted into a delicious entrée, by filling it either with oysters, or
sliced sweetbreads, stewed, and served in thick, rich, white sauce,
or béchamel. Lobster also prepared and moulded as for the new
lobster patties of page 359, will form a superior dish even to these.
Obs.—Six ounces of flour, and three of butter, will make sufficient
paste for this tourte, when it is required only of the usual moderate
size. If richer paste be used for it, it must have two or three additional
turns or rollings to prevent its losing its form in the oven.
Christmas Tourte à la Châtelaine.—Make the case for this tourte
as for the preceding one, and put sufficient mincemeat to fill it
handsomely into a jar, cover it very securely with paste, or with two
or three folds of thick paper, and bake it gently for half an hour or
longer, should the currants, raisins, &c., not be fully tender. Take out
the inside of the tourte, heap the hot mincemeat in it, pour a little
fresh brandy over; just touch it with a strip of lighted writing-paper at
the door of the dining-room, and serve it in a blaze; or if better liked
so, serve it very hot without the brandy, and with Devonshire cream
as an accompaniment.[124]
124. Sufficient of cream for this purpose can easily be prepared from good milk.
 GENOISES À LA REINE, OR HER MAJESTY’S PASTRY.

Make some nouilles (see page 5), with the yolks of four fresh
eggs, and when they are all cut as directed there, drop them lightly
into a pint and a half of boiling cream (new milk will answer quite as
well, or a portion of each may be used), in which six ounces of fresh
butter have been dissolved. When these have boiled quickly for a
minute or two, during which time they must be stirred to prevent their
gathering into lumps, add a small pinch of salt, and six ounces of
sugar on which the rinds of two lemons have been rasped; place the
saucepan over a clear and very gentle fire, and when the mixture
has simmered from thirty to forty minutes take it off, stir briskly in the
yolks of six eggs, and pour it out upon a delicately clean baking-tin
which has been slightly rubbed in every part with butter; level the
nouilles with a knife to something less than a quarter of an inch of
thickness, and let them be very evenly spread; put them into a
moderate oven, and bake them of a fine equal brown: should any air-
bladders appear, pierce them with the point of a knife. On taking the
paste from the oven, divide it into two equal parts; turn one of these,
the underside uppermost, on to a clean tin or a large dish, and
spread quickly over it a jar of fine apricot-jam, place the other half
upon it, the brown side outwards, and leave the paste to become
cold; then stamp it out with a round or diamond-shaped cutter, and
arrange the genoises tastefully in a dish. This pastry will be found
delicious the day it is baked, but its excellence is destroyed by
keeping. Peach, green-gage, or magnum bonum jam, will serve for it
quite as well as apricot. We strongly recommend to our readers this
preparation, baked in pattypans, and served hot; or the whole
quantity made into a pudding. From the smaller ones a little may be
taken out with a teaspoon, and replaced with some preserve just
before they are sent to table; or they may thus be eaten cold.
Nouilles of 4 eggs; cream or milk, 1-1/2 pint; butter, 6 oz.; sugar 6
oz.; rasped rinds of lemons, 2; grain of salt: 30 to 40 minutes. Yolks
of eggs, 6: baked from 15 to 25 minutes.
 ALMOND PASTE.

For a single dish of pastry, blanch seven ounces of fine Jordan

almonds and one of bitter;[125] throw them into cold water as they
are done, and let them remain in it for an hour or two; then wipe, and
pound them to the finest paste, moistening them occasionally with a
few drops of cold water, to prevent their oiling; next, add to, and mix
thoroughly with them, seven ounces of highly-refined, dried, and
sifted sugar; put them into a small preserving-pan, or enamelled
stewpan, and stir them over a clear and very gentle fire until they are
so dry as not to adhere to the finger when touched; turn the paste
immediately into an earthen pan or jar, and when cold it will be ready
for use.
125. When these are objected to, use half a pound of the sweet almonds.
Jordan almonds, 7 oz.; bitter almonds, 1 oz.; cold water, 1
tablespoonful; sugar, 7 oz.
Obs.—The pan in which the paste is dried, should by no means be
placed upon the fire, but high above it on a bar or trevet: should it be
allowed by accident to harden too much, it must be sprinkled
plentifully with water, broken up quite small, and worked, as it
warms, with a strong wooden spoon to a smooth paste again. We
have found this method perfectly successful; but, if time will permit, it
should be moistened some hours before it is again set over the fire.
 TARTLETS OF ALMOND PASTE.

Butter slightly the smallest-sized pattypans, and line them with the
almond-paste rolled as thin as possible; cut it with a sharp knife
close to their edges, and bake or rather dry the tartlets slowly at the
mouth of a very cool oven. If at all coloured, they should be only of
the palest brown; but they will become perfectly crisp without losing
their whiteness if left for some hours in a very gently-heated stove or
oven. They should be taken from the pans when two-thirds done,
and laid, reversed, upon a sheet of paper placed on a dish or board,
before they are put back into the oven. At the instant of serving, fill
them with bright-coloured whipped cream, or with peach or apricot
jam; if the preserve be used, lay over it a small star or other
ornament cut from the same paste, and dried with the tartlets. Sifted
sugar, instead of flour, must be dredged upon the board and roller in
using almond paste. Leaves and flowers formed of it, and dried
gradually until perfectly crisp, will keep for a long time in a tin box or
canister, and they form elegant decorations for pastry. When a fluted
cutter the size of the pattypans is at hand, it will be an improvement
to cut out the paste with it, and then to press it lightly into them, as it
is rather apt to break when pared off with a knife. To colour it,
prepared cochineal, or spinach-green, must be added to it in the
mortar.
 FAIRY FANCIES.

(Fantaisies de Fées.)
A small, but very
inexpensive set of
tin cutters must be
had for this pretty
form of pastry,
which is, however,
quite worthy of so slight a cost. The short crust, of page 349,
answers for it better than puff paste. Roll it thin and very even, and
with the larger tin, shaped thus, cut out a dozen or more of small
sheets; then, with a couple of round cutters, of which one should be
about an inch in diameter, and the other only half the size, form four
times the number of rings, and lay them on the sheets in the manner
shown in the engraving. The easier mode of placing them regularly,
is to raise each ring without removing the small cutter from it, to
moisten it with a camel’s hair brush dipped in white of egg, and to lay
it on the paste as it is gently loosened from the tin When all the
pastry is prepared, set it into a very gentle oven, that it may become
crisp and yet remain quite pale. Before it is sent to table, fill the four
divisions of each fantaisie with preserve of a different colour. For
example: one ring with apple or strawberry jelly, another with apricot
jam, a third with peach or green-gage, and a fourth with raspberry
jelly. The cases may be iced, and ornamented in various ways
before they are baked. They are prettiest when formed of white
almond-paste, with pink or pale green rings: they may then be filled,
at the instant of serving, with well-drained whipped cream.
 MINCEMEAT.

(Author’s Receipt.)
To one pound of an unsalted ox-tongue, boiled tender and cut free
from the rind, add two pounds of fine stoned raisins, two of beef
kidney-suet, two pounds and a half of currants well cleaned and
dried, two of good apples, two and a half of fine Lisbon sugar, from
half to a whole pound of candied peel according to the taste, the
grated rinds of two large lemons, and two more boiled quite tender,
and chopped up entirely, with the exception of the pips, two small
nutmegs, half an ounce of salt, a large teaspoonful of pounded
mace, rather more of ginger in powder, half a pint of brandy, and as
much good sherry or Madeira. Mince these ingredients separately,
and mix the others all well before the brandy and the wine are
added; press the whole into a jar or jars, and keep it closely covered.
It should be stored for a few days before it is used, and will remain
good for many weeks. Some persons like a slight flavouring of
cloves in addition to the other spices; others add the juice of two or
three lemons, and a larger quantity of brandy. The inside of a tender
and well-roasted sirloin of beef will answer quite as well as the
tongue.
Of a fresh-boiled ox-tongue, or inside of roasted sirloin, 1 lb.;
stoned raisins and minced apples, each 2 lbs.; currants and fine
Lisbon sugar, each 2-1/2 lbs.; candied orange, lemon or citron rind, 8
to 16 oz.; boiled lemons, 2 large; rinds of two others, grated; salt, 1/2
oz.; nutmegs, 2 small; pounded mace, 1 large teaspoonful, and
rather more of ginger; good sherry or Madeira, 1/2 pint; brandy, 1/2
pint.
Obs.—The lemons will be sufficiently boiled in from one hour to
one and a quarter.
 SUPERLATIVE MINCEMEAT.

Take four large lemons, with their weight of golden pippins pared
and cored, of jar-raisins, currants, candied citron and orange-rind,
and the finest suet, and a fourth part more of pounded sugar. Boil the
lemons tender, chop them small, but be careful first to extract all the
pips; add them to the other ingredients, after all have been prepared
with great nicety, and mix the whole well with from three to four
glasses of good brandy. Apportion salt and spice by the preceding
receipt. We think that the weight of one lemon, in meat, improves
this mixture; or, in lieu of it, a small quantity of crushed macaroons
added just before it is baked.
 MINCE PIES. (ENTREMETS.)

Butter some tin pattypans well, and line them evenly with fine puff
paste rolled thin; fill them with mincemeat, moisten the edges of the
covers, which should be nearly a quarter of an inch thick, close the
pies carefully, trim off the superfluous paste, make a small aperture
in the centre of the crust with a fork or the point of a knife, ice the
pies or not, at pleasure, and bake them half an hour in a well-heated
but not fierce oven: lay a paper over them when they are partially
done, should they appear likely to take too much colour.
1/2 hour.
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

textbookfull.com