Download the Full Version of textbook for Fast Typing at textbookfull.

com

Java Deep Learning Essentials 1st Edition Sugomori

Yusuke

https://textbookfull.com/product/java-deep-learning-
essentials-1st-edition-sugomori-yusuke/

OR CLICK BUTTON

DOWNLOAD NOW

Download More textbook Instantly Today - Get Yours Now at textbookfull.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

R Deep Learning Essentials 1st Edition Wiley

https://textbookfull.com/product/r-deep-learning-essentials-1st-
edition-wiley/

textboxfull.com

Programming PyTorch for Deep Learning Creating and

Deploying Deep Learning Applications 1st Edition Ian
Pointer
https://textbookfull.com/product/programming-pytorch-for-deep-
learning-creating-and-deploying-deep-learning-applications-1st-
edition-ian-pointer/
textboxfull.com

Deep Learning Pipeline: Building a Deep Learning Model

with TensorFlow 1st Edition Hisham El-Amir

https://textbookfull.com/product/deep-learning-pipeline-building-a-
deep-learning-model-with-tensorflow-1st-edition-hisham-el-amir/

textboxfull.com

Deep Learning on Windows Building Deep Learning Computer

Vision Systems on Microsoft Windows 1st Edition Thimira
Amaratunga
https://textbookfull.com/product/deep-learning-on-windows-building-
deep-learning-computer-vision-systems-on-microsoft-windows-1st-
edition-thimira-amaratunga/
textboxfull.com
Deep Learning with Python 1st Edition François Chollet

https://textbookfull.com/product/deep-learning-with-python-1st-
edition-francois-chollet/

textboxfull.com

Deep Learning in Biometrics 1st Edition Mayank Vatsa

https://textbookfull.com/product/deep-learning-in-biometrics-1st-
edition-mayank-vatsa/

textboxfull.com

Deep Learning with R 1st Edition François Chollet

https://textbookfull.com/product/deep-learning-with-r-1st-edition-
francois-chollet/

textboxfull.com

Deep Learning With Pytorch 1st Edition Eli Stevens

https://textbookfull.com/product/deep-learning-with-pytorch-1st-
edition-eli-stevens/

textboxfull.com

Learning Java Through Games 1st Edition Stanchev Lubomir

https://textbookfull.com/product/learning-java-through-games-1st-
edition-stanchev-lubomir/

textboxfull.com
[1]
Java Deep Learning Essentials

Dive into the future of data science and learn how to

build the sophisticated algorithms that are fundamental
to deep learning and AI with Java

Yusuke Sugomori

BIRMINGHAM - MUMBAI
Java Deep Learning Essentials

Copyright © 2016 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval
system, or transmitted in any form or by any means, without the prior written
permission of the publisher, except in the case of brief quotations embedded in
critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy
of the information presented. However, the information contained in this book is
sold without warranty, either express or implied. Neither the author, nor Packt
Publishing, and its dealers and distributors will be held liable for any damages
caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the
companies and products mentioned in this book by the appropriate use of capitals.
However, Packt Publishing cannot guarantee the accuracy of this information.

First published: May 2016

Production reference: 1250516

Published by Packt Publishing Ltd.

Livery Place
35 Livery Street
Birmingham B3 2PB, UK.

ISBN 978-1-78528-219-5

www.packtpub.com
 Credits

Author Project Coordinator

Yusuke Sugomori Izzat Contractor

Reviewers Proofreader
Wei Di Safis Editing
Vikram Kalabi
Indexer
Commissioning Editor Mariammal Chettiyar
Kartikey Pandey
Graphics
Acquisition Editor Abhinash Sahu
Manish Nainani
Production Coordinator
Content Development Editor Arvindkumar Gupta
Rohit Singh
Cover Work
Technical Editor Arvindkumar Gupta
Vivek Arora

Copy Editor
Ameesha Smith Green
 About the Author

Yusuke Sugomori is a creative technologist with a background in information

engineering. When he was a graduate school student, he cofounded Gunosy with his
colleagues, which uses machine learning and web-based data mining to determine
individual users' respective interests and provides an optimized selection of daily
news items based on those interests. This algorithm-based app has gained a lot of
attention since its release and now has more than 10 million users. The company has
been listed on the Tokyo Stock Exchange since April 28, 2015.

In 2013, Sugomori joined Dentsu, the largest advertising company in Japan based on
nonconsolidated gross profit in 2014, where he carried out a wide variety of digital
advertising, smartphone app development, and big data analysis. He was also
featured as one of eight "new generation" creators by the Japanese magazine Web
Designing.

In April 2016, he joined a medical start-up as cofounder and CTO.

 About the Reviewers

Wei Di is a data scientist. She is passionate about creating smart and scalable
analytics and data mining solutions that can impact millions of individuals and
empower successful businesses.

Her interests also cover wide areas including artificial intelligence, machine learning,
and computer vision. She was previously associated with the eBay Human Language
Technology team and eBay Research Labs, with a focus on image understanding for
large scale applications and joint learning from both visual and text information.
Prior to that, she was with Ancestry.com working on large-scale data mining and
machine learning models in the areas of record linkage, search relevance, and
ranking. She received her PhD from Purdue University in 2011 with focuses on data
mining and image classification.

Vikram Kalabi is a data scientist. He is working on a Cognitive System that

can enable smart plant breeding. His work is primarily in predictive analytics and
mathematical optimization. He has also worked on large scale data-driven decision
making systems with a focus on recommender systems. He is excited about data
science that can help improve farmer's life and help reduce food scarcity in the
world. He is a certified data scientist from John Hopkins University.
 www.PacktPub.com

eBooks, discount offers, and more

Did you know that Packt offers eBook versions of every book published, with PDF
and ePub files available? You can upgrade to the eBook version at www.PacktPub.com
and as a print book customer, you are entitled to a discount on the eBook copy. Get in
touch with us at [email protected] for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign
up for a range of free newsletters and receive exclusive discounts and offers on Packt
books and eBooks.
TM

https://www2.packtpub.com/books/subscription/packtlib

Do you need instant solutions to your IT questions? PacktLib is Packt's online digital
book library. Here, you can search, access, and read Packt's entire library of books.

Why subscribe?
• Fully searchable across every book published by Packt
• Copy and paste, print, and bookmark content
• On demand and accessible via a web browser
 Table of Contents
Preface v
Chapter 1: Deep Learning Overview 1
Transition of AI 2
Definition of AI 2
AI booms in the past 3
Machine learning evolves 8
What even machine learning cannot do 11
Things dividing a machine and human 13
AI and deep learning 14
Summary 22
Chapter 2: Algorithms for Machine Learning – Preparing for
Deep Learning 23
Getting started 23
The need for training in machine learning 24
Supervised and unsupervised learning 27
Support Vector Machine (SVM) 28
Hidden Markov Model (HMM) 31
Neural networks 32
Logistic regression 33
Reinforcement learning 33
Machine learning application flow 34
Theories and algorithms of neural networks 40
Perceptrons (single-layer neural networks) 40
Logistic regression 48
Multi-class logistic regression 51
Multi-layer perceptrons (multi-layer neural networks) 57
Summary 66

[i]
Table of Contents

Chapter 3: Deep Belief Nets and Stacked

Denoising Autoencoders 67
Neural networks fall 67
Neural networks' revenge 68
Deep learning's evolution – what was the breakthrough? 69
Deep learning with pre-training 70
Deep learning algorithms 76
Restricted Boltzmann machines 76
Deep Belief Nets (DBNs) 90
Denoising Autoencoders 96
Stacked Denoising Autoencoders (SDA) 103
Summary 105
Chapter 4: Dropout and Convolutional Neural Networks 107
Deep learning algorithms without pre-training 107
Dropout 108
Convolutional neural networks 120
Convolution 122
Pooling 125
Equations and implementations 126
Summary 142
Chapter 5: Exploring Java Deep Learning Libraries – DL4J,
ND4J, and More 143
Implementing from scratch versus a library/framework 144
Introducing DL4J and ND4J 146
Implementations with ND4J 148
Implementations with DL4J 154
Setup 154
Build 157
DBNIrisExample.java 157
CSVExample.java 163
CNNMnistExample.java/LenetMnistExample.java 166
Learning rate optimization 172
Summary 175
Chapter 6: Approaches to Practical Applications – Recurrent
Neural Networks and More 177
Fields where deep learning is active 178
Image recognition 178
Natural language processing 180
Feed-forward neural networks for NLP 180
Deep learning for NLP 186

[ ii ]
 Table of Contents

The difficulties of deep learning 196

The approaches to maximizing deep learning
possibilities and abilities 198
Field-oriented approach 199
Medicine 199
Automobiles 200
Advert technologies 201
Profession or practice 201
Sports 202
Breakdown-oriented approach 202
Output-oriented approach 205
Summary 206
Chapter 7: Other Important Deep Learning Libraries 207
Theano 207
TensorFlow 212
Caffe 217
Summary 220
Chapter 8: What's Next? 221
Breaking news about deep learning 221
Expected next actions 224
Useful news sources for deep learning 229
Summary 232
Index 233

[ iii ]
 Preface
With an increasing interest in AI around the world, deep learning has attracted a
great deal of public attention. Every day, deep learning algorithms are used across
different industries. Deep learning has provided a revolutionary step to actualize
AI. While it is a revolutionary technique, deep learning is often thought to be
complicated, and so it is often kept from much being known of its contents. Theories
and concepts based on deep learning are not complex or difficult. In this book,
we'll take a step-by-step approach to learn theories and equations for the correct
understanding of deep learning. You will find implementations from scratch, with
detailed explanations of the cautionary notes for practical use cases.

What this book covers

Chapter 1, Deep Learning Overview, explores how deep learning has evolved.

Chapter 2, Algorithms for Machine Learning - Preparing for Deep Learning, implements
machine learning algorithms related to deep learning.

Chapter 3, Deep Belief Nets and Stacked Denoising Autoencoders, dives into Deep Belief
Nets and Stacked Denoising Autoencoders algorithms.

Chapter 4, Dropout and Convolutional Neural Networks, discovers more deep learning
algorithms with Dropout and Convolutional Neural Networks.

Chapter 5, Exploring Java Deep Learning Libraries – DL4J, ND4J, and More, gains an
insight into the deep learning library, DL4J, and its practical uses.

Chapter 6, Approaches to Practical Applications – Recurrent Neural Networks and More, lets
you devise strategies to use deep learning algorithms and libraries in the real world.

[v]
Preface

Chapter 7, Other Important Deep Learning Libraries, explores deep learning further with
Theano, TensorFlow, and Caffe.

Chapter 8, What's Next?, explores recent deep learning movements and events, and
looks into useful deep learning resources.

What you need for this book

We'll implement deep learning algorithms using Lambda Expressions, hence Java 8
or above is required. Also, we'll use the Java library DeepLearning4J 0.4 or above.

Who this book is for

This book is for Java developers who want to know about deep learning algorithms
and wish to implement them in applications.

Since this book covers the core concepts of and approaches to both machine learning
and deep learning, no previous experience in machine learning is required.

Also, we will implement deep learning algorithms with very simple codes, so
elementary Java developers will also find this book useful for developing both
their Java skills and deep learning skills.

Conventions
In this book, you will find a number of text styles that distinguish between different
kinds of information. Here are some examples of these styles and an explanation of
their meaning.

Code words in text, database table names, folder names, filenames, file extensions,
pathnames, dummy URLs, user input, and Twitter handles are shown as follows:
"Let's take a look at CNNMnistExample.java in the package of convolution."

A block of code is set as follows:

<dependency>
<groupId>org.nd4j</groupId>
<artifactId>nd4j-jcublas-7.0</artifactId>
<version>${nd4j.version}</version>
</dependency>

[ vi ]
 Preface

Any command-line input or output is written as follows:

[[7.00,7.00]
[7.00,7.00]
[7.00,7.00]]

New terms and important words are shown in bold. Words that you see on the
screen, for example, in menus or dialog boxes, appear in the text like this: "If you're
using IntelliJ, you can import the project from File | New | Project from existing
sources."

Warnings or important notes appear in a box like this.

Tips and tricks appear like this.

Reader feedback
Feedback from our readers is always welcome. Let us know what you think about
this book—what you liked or disliked. Reader feedback is important for us as it helps
us develop titles that you will really get the most out of.

To send us general feedback, simply e-mail [email protected], and mention

the book's title in the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing
or contributing to a book, see our author guide at www.packtpub.com/authors.

Customer support
Now that you are the proud owner of a Packt book, we have a number of things to
help you to get the most from your purchase.

[ vii ]
Preface

Downloading the example code

You can download the example code files for this book from your account at
http://www.packtpub.com. If you purchased this book elsewhere, you can visit
http://www.packtpub.com/support and register to have the files e-mailed directly
to you.

You can download the code files by following these steps:

1. Log in or register to our website using your e-mail address and password.
2. Hover the mouse pointer on the SUPPORT tab at the top.
3. Click on Code Downloads & Errata.
4. Enter the name of the book in the Search box.
5. Select the book for which you're looking to download the code files.
6. Choose from the drop-down menu where you purchased this book from.
7. Click on Code Download.

You can also download the code files by clicking on the Code Files button on the
book's webpage at the Packt Publishing website. This page can be accessed by
entering the book's name in the Search box. Please note that you need to be logged in
to your Packt account.

Once the file is downloaded, please make sure that you unzip or extract the folder
using the latest version of:

• WinRAR / 7-Zip for Windows

• Zipeg / iZip / UnRarX for Mac
• 7-Zip / PeaZip for Linux

The code bundle for the book is also hosted on GitHub at https://github.com/
PacktPublishing/Java-Deep-Learning-Essentials. We also have other code
bundles from our rich catalog of books and videos available at https://github.
com/PacktPublishing/. Check them out!

[ viii ]
 Preface

Errata
Although we have taken every care to ensure the accuracy of our content, mistakes
do happen. If you find a mistake in one of our books—maybe a mistake in the text or
the code—we would be grateful if you could report this to us. By doing so, you can
save other readers from frustration and help us improve subsequent versions of this
book. If you find any errata, please report them by visiting http://www.packtpub.
com/submit-errata, selecting your book, clicking on the Errata Submission Form
link, and entering the details of your errata. Once your errata are verified, your
submission will be accepted and the errata will be uploaded to our website or added
to any list of existing errata under the Errata section of that title.

To view the previously submitted errata, go to https://www.packtpub.com/books/

content/support and enter the name of the book in the search field. The required
information will appear under the Errata section.

Piracy
Piracy of copyrighted material on the Internet is an ongoing problem across all
media. At Packt, we take the protection of our copyright and licenses very seriously.
If you come across any illegal copies of our works in any form on the Internet, please
provide us with the location address or website name immediately so that we can
pursue a remedy.

Please contact us at [email protected] with a link to the suspected

pirated material.

We appreciate your help in protecting our authors and our ability to bring you
valuable content.

Questions
If you have a problem with any aspect of this book, you can contact us at
[email protected], and we will do our best to address the problem.

[ ix ]
 Deep Learning Overview
Artificial Intelligence (AI) is a word that you might start to see more often these
days. AI has become a hot topic not only in academic society, but also in the field of
business. Large tech companies such as Google and Facebook have actively bought
AI-related start-ups. Mergers and acquisitions in these AI areas have been especially
active, with big money flowing into AI. The Japanese IT/mobile carrier company
Softbank released a robot called Pepper in June 2014, which understands human
feelings, and a year later they have started to sell Pepper to general consumers. This
is a good movement for the field of AI, without a doubt.

The idea of AI has been with us for decades. So, why has AI suddenly became a
hot field? One of the factors that has driven recent AI-related movements, and is
almost always used with the word AI, is deep learning. After deep learning made
a vivid debut and its technological capabilities began to grow exponentially, people
started to think that finally AI would become a reality. It sounds like deep learning is
definitely something we need to know. So, what exactly is it?

To answer the previous questions, in this chapter we'll look at why and how AI
has become popular by following its history and fields of studies. The topics covered
will be:

• The former approaches and techniques of AI

• An introduction to machine learning and a look at how it has evolved into
deep learning
• An introduction to deep learning and some recent use cases

If you already know what deep learning is or if you would like to find out about the
specific algorithm of the deep learning/implementation technique, you can skip this
chapter and jump directly to Chapter 2, Algorithms for Machine Learning – Preparing for
Deep Learning.

[1]
Deep Learning Overview

Although deep learning is an innovative technique, it is not actually that

complicated. It is rather surprisingly simple. Reading through this book, you
will see how brilliant it is. I sincerely hope that this book will contribute to your
understanding of deep learning and thus to your research and business.

Transition of AI
So, why is it now that deep learning is in the spotlight? You might raise this question,
especially if you are familiar with machine learning, because deep learning is not that
different to any other machine learning algorithm (don't worry if you don't know
this, as we'll go through it later in the book). In fact, we can say that deep learning is
the adaptation of neural networks, one of the algorithms of machine learning, which
mimics the structure of a human brain. However, what deep learning can achieve
is much more significant and different to any other machine learning algorithm,
including neural networks. If you see what processes and research deep learning has
gone through, you will have a better understanding of deep learning itself. So, let's go
through the transition of AI. You can just skim through this while sipping your coffee.

Definition of AI
All of a sudden, AI has become a hot topic in the world; however, as it turns out,
actual AI doesn't exist yet. Of course, research is making progress in creating actual
AI, but it will take more time to achieve it. Pleased or not, the human brain—which
could be called "intelligence"—is structured in an extremely complicated way and
you can't easily replicate it.

But wait a moment - we see many advertisements for products with the phrase by
AI or using AI all over them. Are they fraudulent? Actually, they are! Surprised?
You might see words like recommendation system by AI or products driven by AI, but
the word AI used here doesn't indicate the actual meaning of AI. Strictly speaking,
the word AI is used with a much broader meaning. The research into AI and the AI
techniques accumulated in the past have achieved only some parts of AI, but now
people are using the word AI for those parts too.

Let's look at a few examples. Roughly divided, there are three different categories
recognized as AI in general:

• Simple repetitive machine movements that a human programmed

beforehand. For example, high speed processing industrial robots
that only process the same set of work.
• Searching or guessing answers to a given assignment following rules set by
a human. For example, the iRobot Roomba can clean up along the shape of a
room as it can assume the shape of a room by bumping into obstacles.
[2]
 Chapter 1

• Providing an answer to unknown data by finding measurable regularity

from the existing data. For example, a product recommendation system
based on a user's purchase history or distributing banner ads among ad
networks falls under this category.

People use the word AI for these categories and, needless to say, new technology
that utilizes deep learning is also called AI. Yet, these technologies are different
both in structure and in what they can do. So, which should we specifically call AI?
Unfortunately, people have different opinions about that question and the answer
cannot be objectively explained. Academically, a term has been set as either strong
AI or weak AI depending on the level that a machine can achieve. However, in this
book, to avoid confusion, AI is used to mean (Not yet achieved) human-like intelligence
that is hard to distinguish from the actual human brain. The field of AI is being drastically
developed, and the possibility of AI becoming reality is exponentially higher when
driven by deep learning. This field is booming now more than ever in history. How
long this boom will continue depends on future research.

AI booms in the past

AI suddenly became a hot topic recently: however, this is not the first AI boom.
When you look back to the past, research into AI has been conducted for decades
and there has been a cycle of being active and inactive. The recent boom is the third
boom. Therefore, some people actually think that, at this time, it's just an evanescent
boom again.

However, the latest boom has a significant difference from the past booms. Yes,
that is deep learning. Deep learning has achieved what the past techniques could
not achieve. What is that? Simply put, a machine itself is able to find out the feature
quantity from the given data, and learn. With this achievement, we can see the
great possibility of AI becoming a reality, because until now a machine couldn't
understand a new concept by itself and a human needed to input a certain feature
quantity in advance using past techniques created in the AI field.

It doesn't look like a huge difference if you just read this fact, but there's a world
of difference. There has been a long path taken before reaching the stage where a
machine can measure feature quantity by itself. People were finally able to take a
big step forward when a machine could obtain intelligence driven by deep learning.
So, what's the big difference between the past techniques and deep learning? Let's
briefly look back into the past AI field to get a better sense of the difference.

[3]
Deep Learning Overview

The first AI boom came in the late 1950s. Back then, the mainstream research and
development of a search program was based on fixed rules—needless to say, they
were human-defined. The search was, simply put, dividing cases. In this search, if we
wanted a machine to perform any process, we had to write out every possible pattern
we might need for the process. A machine can calculate much faster than a human can.
It doesn't matter how enormous the patterns are, a machine can easily handle them. A
machine will keep searching a million times and eventually will find the best answer.
However, even if a machine can calculate at high speed, if it is just searching for an
answer randomly and blindly it will take a massive amount of time. Yes, don't forget
that constraint condition, "time." Therefore, further studies were conducted on how to
make the search more efficient. The most popular search methods among the studies
were depth-first search (DFS) and breadth-first search (BFS).

Out of every possible pattern you can think of, search for the most efficient path
and make the best possible choice among them within a realistic time frame. By
doing this, you should get the best answer each time. Based on this hypothesis,
two searching or traversing algorithms for a tree of graph data structures were
developed: DFS and BFS. Both start at the root of a graph or tree, and DFS explores
as far as possible along each branch before backtracking, whereas BFS explores
the neighbor nodes first before moving to the next level neighbors. Here are some
example diagrams that show the difference between DFS and BFS:

These search algorithms could achieve certain results in a specific field, especially
fields like Chess and Shogi. This board game field is one of the areas that a machine
excels in. If it is given an input of massive amounts of win/lose patterns, past game
data, and all the permitted moves of a piece in advance, a machine can evaluate the
board position and decide the best possible next move from a very large range
of patterns.

[4]
 Chapter 1

For those of you who are interested in this field, let's look into how a machine plays
chess in more detail. Let's say a machine makes the first move as "white," and there
are 20 possible moves for both "white" and "black" for the next move. Remember the
tree-like model in the preceding diagram. From the top of the tree at the start of the
game, there are 20 branches underneath as white's next possible move. Under one
of these 20 branches, there's another 20 branches underneath as black's next possible
movement, and so on. In this case, the tree has 20 x 20 = 400 branches for black,
depending on how white moves, 400 x 20 = 8,000 branches for white, 8,000 x 20 =
160,000 branches again for black, and... feel free to calculate this if you like.

A machine generates this tree and evaluates every possible board position from
these branches, deciding the best arrangement in a second. How deep it goes (how
many levels of the tree it generates and evaluates) is controlled by the speed of the
machine. Of course, each different piece's movement should also be considered
and embedded in a program, so the chess program is not as simple as previously
thought, but we won't go into detail about this in this book. As you can see, it's not
surprising that a machine can beat a human at Chess. A machine can evaluate and
calculate massive amounts of patterns at the same time, in a much shorter time than
a human could. It's not a new story that a machine has beaten a Chess champion; a
machine has won a game over a human. Because of stories like this, people expected
that AI would become a true story.

Unfortunately, reality is not that easy. We then found out that there was a big wall
in front of us preventing us from applying the search algorithm to reality. Reality
is, as you know, complicated. A machine is good at processing things at high speed
based on a given set of rules, but it cannot find out how to act and what rules to
apply by itself when only a task is given. Humans unconsciously evaluate, discard
many things/options that are not related to them, and make a choice from millions
of things (patterns) in the real world whenever they act. A machine cannot make
these unconscious decisions like humans can. If we create a machine that can
appropriately consider a phenomenon that happens in the real world, we can assume
two possibilities:

• A machine tries to accomplish its task or purpose without taking into account
secondarily occurring incidents and possibilities
• A machine tries to accomplish its task or purpose without taking into account
irrelevant incidents and possibilities

Both of these machines would still freeze and be lost in processing before they
accomplished their purpose when humans give them a task; in particular, the latter
machine would immediately freeze before even taking its first action. This is because
these elements are almost infinite and a machine can't sort them out within a realistic
time if it tries to think/search these infinite patterns. This issue is recognized as one
of the important challenges in the AI field, and it's called the frame problem.
[5]
Deep Learning Overview

A machine can achieve great success in the field of Chess or Shogi because the
searching space, the space a machine should be processing within, is limited (set in
a certain frame) in advance. You can't write out an enormous amount of patterns,
so you can't define what the best solution is. Even if you are forced to limit the
number of patterns or to define an optimal solution, you can't get the result within an
economical time frame for use due to the enormous amounts of calculation needed.
After all, the research at that time would only make a machine follow detailed rules
set by a human. As such, although this search method could succeed in a specific
area, it is far from achieving actual AI. Therefore, the first AI boom cooled down
rapidly with disappointment.

The first AI boom was swept away; however, on the side, the research into AI
continued. The second AI boom came in the 1980s. This time, the movement of
so-called Knowledge Representation (KR) was booming. KR intended to describe
knowledge that a machine could easily understand. If all the knowledge in the world
was integrated into a machine and a machine could understand this knowledge, it
should be able to provide the right answer even if it is given a complex task. Based
on this assumption, various methods were developed for designing knowledge for
a machine to understand better. For example, the structured forms on a web page—
the semantic web—is one example of an approach that tried to design in order for a
machine to understand information easier. An example of how the semantic web is
described with KR is shown here:

[6]
 Chapter 1

Making a machine gain knowledge is not like a human ordering a machine what to
do one-sidedly, but more like a machine being able to respond to what humans ask
and then answer. One of the simple examples of how this is applied to the actual
world is positive-negative analysis, one of the topics of sentiment analysis. If you
input data that defines a tone of positive or negative for every word in a sentence
(called "a dictionary") into a machine beforehand, a machine can compare the
sentence and the dictionary to find out whether the sentence is positive or negative.

This technique is used for the positive-negative analysis of posts or comments on a

social network or blog. If you ask a machine "Is the reaction to this blog post positive
or negative?" it analyzes the comments based on its knowledge (dictionary) and
replies to you. From the first AI boom, where a machine only followed rules that
humans set, the second AI boom showed some progress.

By integrating knowledge into a machine, a machine becomes the almighty. This idea
itself is not bad for achieving AI; however, there were two high walls ahead of us
in achieving it. First, as you may have noticed, inputting all real-world knowledge
requires an almost infinite amount of work now that the Internet is more commonly
used and we can obtain enormous amounts of open data from the Web. Back then,
it wasn't realistic to collect millions of pieces of data and then analyze and input that
knowledge into a machine. Actually, this work of databasing all the world's data has
continued and is known as Cyc (http://www.cyc.com/). Cyc's ultimate purpose
is to build an inference engine based on the database of this knowledge, called
knowledge base. Here is an example of KR using the Cyc project:

[7]
Deep Learning Overview

Second, it's not that a machine understands the actual meaning of the knowledge.
Even if the knowledge is structured and systemized, a machine understands it as
a mark and never understands the concept. After all, the knowledge is input by
a human and what a machine does is just compare the data and assume meaning
based on the dictionary. For example, if you know the concept of "apple" and "green"
and are taught "green apple = apple + green", then you can understand that "a green
apple is a green colored apple" at first sight, whereas a machine can't. This is called
the symbol grounding problem and is considered one of the biggest problems in the
AI field, as well as the frame problem.

The idea was not bad—it did improve AI—however, this approach won't achieve
AI in reality as it's not able to create AI. Thus, the second AI boom cooled down
imperceptibly, and with a loss of expectation from AI, the number of people who
talked about AI decreased. When it came to the question of "Are we really able to
achieve AI?" the number of people who answered "no" increased gradually.

Machine learning evolves

While people had a hard time trying to establish a method to achieve AI, a
completely different approach had steadily built a generic technology . That
approach is called machine learning. You should have heard the name if you have
touched on data mining even a little. Machine learning is a strong tool compared
to past AI approaches, which simply searched or assumed based on the knowledge
given by a human, as mentioned earlier in the chapter, so machine learning is very
advanced. Until machine learning, a machine could only search for an answer from
the data that had already been inputted. The focus was on how fast a machine
could pull out knowledge related to a question from its saved knowledge. Hence,
a machine can quickly reply to a question it already knows, but gets stuck when it
faces questions it doesn't know.

On the other hand, in machine learning, a machine is literally learning. A machine

can cope with unknown questions based on the knowledge it has learned. So, how
was a machine able to learn, you ask? What exactly is learning here? Simply put,
learning is when a machine can divide a problem into "yes" or "no." We'll go through
more detail on this in the next chapter, but for now we can say that machine learning
is a method of pattern recognition.

[8]
 Chapter 1

We could say that, ultimately, every question in the world can be replaced with
a question that can be answered with yes or no. For example, the question "What
color do you like?" can be considered almost the same as asking "Do you like red?
Do you like green? Do you like blue? Do you like yellow?..." In machine learning,
using the ability to calculate and the capacity to process at high speed as a weapon,
a machine utilizes a substantial amount of training data, replaces complex questions
with yes/no questions, and finds out the regularity with which data is yes, and
which data is no (in other words, it learns). Then, with that learning, a machine
assumes whether the newly-given data is yes or no and provides an answer. To sum
up, machine learning can give an answer by recognizing and sorting out patterns
from the data provided and then classifying that data into the possible appropriate
pattern (predicting) when it faces unknown data as a question.

In fact, this approach is not doing something especially difficult. Humans also
unconsciously classify data into patterns. For example, if you meet a man/woman
who's perfectly your type at a party, you might be desperate to know whether
the man/woman in front of you has similar feelings towards you. In your head,
you would compare his/her way of talking, looks, expressions, or gestures to past
experience (that is, data) and assume whether you will go on a date! This is the same
as a presumption based on pattern recognition.

Machine learning is a method that can process this pattern recognition not by
humans but by a machine in a mechanical manner. So, how can a machine recognize
patterns and classify them? The standard of classification by machine learning is a
presumption based on a numerical formula called the probabilistic statistical model.
This approach has been studied based on various mathematical models.

Learning, in other words, is tuning the parameters of a model and, once the
learning is done, building a model with one adjusted parameter. The machine then
categorizes unknown data into the most possible pattern (that is, the pattern that fits
best). Categorizing data mathematically has great merit. While it is almost impossible
for a human to process multi-dimensional data or multiple-patterned data, machine
learning can process the categorization with almost the same numerical formulas.
A machine just needs to add a vector or the number of dimensions of a matrix.
(Internally, when it classifies multi-dimensions, it's not done by a classified line or a
classified curve but by a hyperplane.)

[9]
Deep Learning Overview

Until this approach was developed, machines were helpless in terms of

responding to unknown data without a human's help, but with machine learning
machines became capable of responding to data that humans can't process.
Researchers were excited about the possibilities of machine learning and jumped on
the opportunity to start working on improving the method. The concept of machine
learning itself has a long history, but researchers couldn't do much research and
prove the usefulness of machine learning due to a lack of available data. Recently,
however, many open-source data have become available online and researchers
can easily experiment with their algorithms using the data. Then, the third AI
boom came about like this. The environment surrounding machine learning also
gave its progress a boost. Machine learning needs a massive amount of data before
it can correctly recognize patterns. In addition, it needs to have the capability to
process data. The more data and types of patterns it handles, the more the amount
of data and the number of calculations increases. Hence, obviously, past technology
wouldn't have been able to deal with machine learning.

However, time is progressing, not to mention that the processing capability of

machines has improved. In addition, the web has developed and the Internet is
spreading all over the world, so open data has increased. With this development,
everyone can handle data mining only if they pull data from the web. The
environment is set for everyone to casually study machine learning. The web is a
treasure box of text-data. By making good use of this text-data in the field of machine
learning, we are seeing great development, especially with statistical natural
language processing. Machine learning has also made outstanding achievements
in the field of image recognition and voice recognition, and researchers have been
working on finding the method with the best precision.

Machine learning is utilized in various parts of the business world as well. In the
field of natural language processing, the prediction conversion in the input method
editor (IME) could soon be on your mind. The fields of image recognition, voice
recognition, image search, and voice search in the search engine are good examples.
Of course, it's not limited to these fields. It is also applied to a wide range of fields
from marketing targeting, such as the sales prediction of specific products or the
optimization of advertisements, or designing store shelf or space planning based on
predicting human behavior, to predicting the movements of the financial market. It
can be said that the most used method of data mining in the business world is now
machine learning. Yes, machine learning is that powerful. At present, if you hear
the word "AI," it's usually the case that the word simply indicates a process done by
machine learning.

[ 10 ]
 Chapter 1

What even machine learning cannot do

A machine learns by gathering data and predicting an answer. Indeed, machine
learning is very useful. Thanks to machine learning, questions that are difficult for
a human to solve within a realistic time frame (such as using a 100-dimensional
hyperplane for categorization!) are easy for a machine. Recently, "big data" has been
used as a buzzword and, by the way, analyzing this big data is mainly done using
machine learning too.

Unfortunately, however, even machine learning cannot make AI. From the
perspective of "can it actually achieve AI?" machine learning has a big weak point.
There is one big difference in the process of learning between machine learning
and human learning. You might have noticed the difference, but let's see. Machine
learning is the technique of pattern classification and prediction based on input data.
If so, what exactly is that input data? Can it use any data? Of course… it can't. It's
obvious that it can't correctly predict based on irrelevant data. For a machine to learn
correctly, it needs to have appropriate data, but then a problem occurs. A machine is
not able to sort out what is appropriate data and what is not. Only if it has the right
data can machine learning find a pattern. No matter how easy or difficult a question
is, it's humans that need to find the right data.

Let's think about this question: "Is the object in front of you a human or a cat?" For
a human, the answer is all too obvious. It's not difficult at all to distinguish them.
Now, let's do the same thing with machine learning. First, we need to prepare the
format that a machine can read, in other words, we need to prepare the image data of
a human and a cat respectively. This isn't anything special. The problem is the next
step. You probably just want to use the image data for inputting, but this doesn't
work. As mentioned earlier, a machine can't find out what to learn from data by
itself. Things a machine should learn need to be processed from the original image
data and created by a human. Let's say, in this example, we might need to use data
that can define the differences such as face colors, facial part position, the facial
outlines of a human and a cat, and so on, as input data. These values, given as inputs
that humans need to find out, are called the features.

[ 11 ]
Deep Learning Overview

Machine learning can't do feature engineering. This is the weakest point of machine
learning. Features are, namely, variables in the model of machine learning. As this
value shows the feature of the object quantitatively, a machine can appropriately
handle pattern recognition. In other words, how you set the value of identities will
make a huge difference in terms of the precision of prediction. Potentially, there are
two types of limitations with machine learning:

• An algorithm can only work well on data with the assumption of the training
data - with data that has different distribution. In many cases, the learned
model does not generalize well.
• Even the well-trained model lacks the ability to make a smart meta-decision.
Therefore, in most cases, machine learning can be very successful in a very
narrow direction.

Let's look at a simple example so that you can easily imagine how identities have a
big influence on the prediction precision of a model. Imagine there is a corporation
that wants to promote a package of asset management based on the amount of assets.
The corporation would like to recommend an appropriate product, but as it can't ask
a personal question, it needs to predict how many assets a customer might have and
prepare in advance. In this case, what type of potential customers shall we consider
as an identity? We can assume many factors such as their height, weight, age,
address, and so on as an identity, but clearly age or residence seem more relevant
than height or weight. You probably won't get a good result if you try machine
learning based on height or weight, as it predicts based on irrelevant data, meaning
it's just a random prediction.

As such, machine learning can provide an appropriate answer against the question
only after the machine reads an appropriate identity. But, unfortunately, the machine
can't judge what the appropriate identity is, and the precision of machine learning
depends on this feature engineering!

Machine learning has various methods, but the problem of being unable to do
feature engineering is seen across all of these. Various methods have been developed
and people compete against their precision rates, but after we have achieved
precision to a certain extent, people decide whether a method of machine learning is
good or bad based on how great a feature they can find. This is no longer a difference
in algorithms, but more like a human's intuition or taste, or the fine-tuning of
parameters, and this can't be said to be innovative at all. Various methods have been
developed, but after all, the hardest thing is to think of the best identity and a human
has to do that part anyway.

[ 12 ]
Random documents with unrelated
content Scribd suggests to you:
 In 442 the three commissioners for conducting a colony to Ardea
were prosecuted by the tribunes on the ground that, by enrolling
Ardeates in place of Romans in the list of colonists, they had
circumvented the law which called their commission into being. The
action would probably have been finable; but the accused avoided
trial by remaining in the colony.[1740] In 423 M. Postumius and T.
Quinctius, retired tribunes with consular power, were tried for
mismanagement of the war with Veii. The former was fined 10,000
asses; the latter was exculpated by all the tribes.[1741] In 401 two
other retired tribunes with consular power were prosecuted by the
tribunes of the plebs and fined each 10,000 asses.[1742] The
imposition of a fine on Camillus, 391, has already been considered.
[1743] In 389 a tribune of the plebs brought an accusation against Q.
Fabius on the ground that the latter while ambassador to the Gauls
had fought against them in violation of the law of nations. The
accused suddenly died, possibly by suicide, before the day of trial.
[1744] In 362 the dictator of the preceding year, L. Manlius, was
prosecuted by a tribune because, though appointed for the sole
purpose of driving the nail, he had nevertheless made a levy of
troops and that with extreme cruelty. But the prosecutor dropped the
accusation, intimidated by the son of the accused.[1745] This is the
view of Livy, whereas Cicero[1746] states the ground of the charge to
have been the addition of a few days to his dictatorship. If historical,
the prosecution may possibly have been for perduellio, and in that
case it would have come before the centuries.
The following cases are historically more certain. Lucius
Postumius, prosecuted by the tribunes of the plebs in 293 for the
misuse of his consulship of the preceding year, escaped trial by
becoming the legatus of the consul Carvilius. The charge was that in
his campaign he had not limited himself to the province assigned him
by the senate.[1747] Evidently the intention of the prosecutor was not
serious.[1748] The consul Q. Fabius Gurges of the year 292,
defeated in battle, was recalled, and his conduct was impugned
before the people. The past services and the promises of his father
saved him, and he continued his consulship with greater success.
The accusation probably did not take the form of a trial, but was
presented in a resolution to remove him from office[1749] or at least
from the command of the army. L. Postumius, third time consul in
291, employed his army to work on his own estate; and on the
expiration of his office he was brought to trial therefor by the tribunes
and condemned.[1750]
In the period under discussion, 449-287, a single effort to hold the
plebeian tribunes responsible for their official conduct is reported. In
293 two retired tribunes were condemned to a fine of 10,000 asses
each on a charge of having favored the patres by interceding against
the proposals of colleagues.[1751] This instance, if historical, is the
only one of the kind before the revolution. The tribunes doubtless felt
that the prosecution of their predecessors rendered their own future
unsafe.
Several attempts were also made by legislation to reach results
equivalent to judicial sentences. In spite of the prohibition of
privilegia by a law of the Twelve Tables, Sp. Maelius, a tribune of the
plebs in 436, tried to carry a resolution for the confiscation of the
property of Servilius Ahala; but the people rejected it.[1752] Another
privilegium was the resolution of the plebs of 368 which threatened
M. Furius Camillus with a fine of 500,000 asses, should he use his
dictatorship to obstruct the Licinian-Sextian bills then under
discussion.[1753] It was certainly not supported by a senatus
consultum, and probably the proposers had no serious intention of
carrying it into effect.
In reviewing the finable actions alleged to have been brought by
the plebeian tribunes during the two centuries which intervened
between the institution of their office and the Hortensian legislation,
as in the case of the capital actions,[1754] we are struck by the
relatively small number belonging to the latter part of the period; in
fact to the time following 362 two cases only are assigned, one of
which is insignificant. The conclusion we must draw from this fact is
similar to that expressed with relation to the capital cases—that the
finable actions attributed to the earlier period are in all probability
largely unhistorical, and that before the enactment of the Hortensian
law the jurisdiction of the tribunes in finable cases was limited and
rare.

b. AEDILICIAN

For some time after their institution the tribunes of the plebs,
having no viatores or at least none that were recognized as public
officials,[1755] depended upon the two plebeian aediles as bailiffs for
making arrests and for executing sentences.[1756] The latter
functionaries seem to have stood in some such relation to the
tribunes as the quaestors toward the consuls. It was accordingly as
deputies of the tribunes that they acquired jurisdiction.[1757] The
earliest mentioned case, 454, is the trial and condemnation of a
retired consul in a finable action for official misconduct.[1758] It
should be placed in the same mythical category with the numerous
tribunician prosecutions of the period.[1759] After the institution of
curule aediles, 367, the aediles of the plebs continued indeed to
serve occasionally as bailiffs of the tribunes,[1760] but acquired in
addition, along with those of curule rank, an independent jurisdiction.
In 357 C. Licinius Stolo was prosecuted by M. Popillius Laenas on
the charge of having circumvented his own law by emancipating his
son in order that he and his son might each possess five hundred
iugera of the public land. He was fined 10,000 asses.[1761] From the
cases to be mentioned below the inference may be drawn that the
accuser was an aedile. In 298 several persons were prosecuted by
the aediles, whether curule or plebeian is not stated, for violation of
the same law, and hardly one was acquitted.[1762] In 295 the
plebeian aediles made considerable money by fining those who had
trespassed against the article of the Licinian-Sextian statute which
related to pasturage;[1763] and two years afterward violators of the
same provision were again fined, on this occasion by the curule
aediles.[1764] Actions against usurers were brought by aediles in
344,[1765] 304,[1766] and 295.[1767]
 Shortly before 328, M. Flavius was prosecuted before the people
by the aediles for the crimen stupratae matris familiae, and
acquitted.[1768] In 295 Q. Fabius Gurges, a curule aedile,[1769]
accused several matrons before the people, also of stuprum, and
fined them.
In the period between the Licinian-Sextian and the Hortensian
legislation, accordingly, the jurisdiction of the aediles, so far as is
known, was limited to usury, stuprum, and the violation of laws
regarding the occupation and pasturage of the public land. They had
nothing to do with perduellio or related offences, or with the
accountability of magistrates, or with any capital actions whatsoever.
All their trials were finable, and in case the fine exceeded thirty cattle
and two sheep, or the equivalent, 3020 heavy asses,[1770] an appeal
could be made to the tribes. The plebeian aediles equally with the
tribunes[1771] lacked the power to summon patricians, whereas the
curule aediles as patrician magistrates[1772] possessed the right; but
no distinction in the composition of these tribal assemblies,
corresponding to the form of presidency, is suggested by the
sources.[1773]

III. Legislative

The legislative function of the tribal assembly under tribunician

presidency after the decemvirate (451-450)[1774] is represented as
bringing forthwith into being the Icilian and Duillian plebiscites of 449.
That of Icilius granted amnesty to those who had seceded from the
decemvirs.[1775] The first plebiscite of Duillius provided for the
election of consuls cum provocatione.[1776] Both acts are alleged to
have been passed, however, before the resolutions of the plebs had
acquired the force of law. The second Duillian plebiscite, which
followed the enactment of the Valerian-Horatian statute, and which
was therefore valid for all the citizens, threatened with scourging and
death any one who left the plebs without tribunes or who caused the
election of a magistrate without appeal.[1777] Its first provision was
merely the expression of a principle on which the plebeians, had
from the beginning insisted as essential to the continuance of the
office from year to year;[1778] the second clause precluded the
recurrence of an elective magistracy like the decemvirate just past.
[1779] According to Diodorus[1780] an agreement was made in this
year between the patricians and plebeians by which one consul at
least should be a plebeian. Although Diodorus generally drew from
sources more ancient than those of Livy, he is wrong in assigning
this provision to so early a date.[1781]
For the same year is recorded another Icilian plebiscite, which
granted the privilege of a triumph to the consuls after the senate had
refused it.[1782] The alleged act is suspicious, in the first place,
because the two consuls must have had the support of a majority in
the senate, as the acceptance of their great constitutional statute
proves. Then, too, a resolution of the people for granting the triumph
could not avail in this period without the consent of the senate. The
last observation applies as well to the alleged refusal of the senate to
ratify an act of the people in 356 for granting a triumph to the first
plebeian dictator.[1783] Such a resolution merely assured the
triumphator that the people would be present at the festival. Without
the consent of the senate, they could not appropriate the necessary
funds for the occasion;[1784] but the general always had a right to
triumph, in earlier time within the city and later on the Alban Mount,
at his own expense.[1785] If the senate decreed the triumph, as
remained the rule,[1786] ratification by the people was unnecessary,
though it sometimes occurred.[1787]
The Trebonian plebiscite, 448 or more probably 401, has already
been discussed.[1788] The interest of the plebs in enhancing the
dignity and importance of their own order manifested itself not only in
this act but also in the Canuleian plebiscite of 445, which
reëstablished conubium between the patricians and plebeians after it
had been forbidden by a law of the Twelve Tables.[1789] Closely
related is the centuriate law of the same year for the institution of
tribunes of the soldiers with consular power to be elected
indiscriminately from the two social classes.[1790]
 Slightly earlier, if we may trust our sources, the people were given
an unwonted opportunity to share in the decision of questions
relating to foreign affairs; and the favor fell to the comitia tributa
under patrician presidency, which had convened in this form for the
first time in 447 for the election of quaestors.[1791] The question
before this assembly in 446 was the arbitration of a dispute between
Ardea and Aricia concerning a piece of territory. The contestants
brought the case before the Roman senate, which usually decided
such matters on its own responsibility, but which in this instance
requested the consuls to refer the business to the tribes. The aim of
the senate must have been to throw the odium of the decision upon
the people, who, disregarding the claims of the two contestants, lost
little time in adjudging the disputed property to Rome.[1792] This act
did not serve as a precedent for further interference of the assembly
in foreign affairs; and when in 427 the people acquired the right to
declare an offensive war,[1793] the function fell to the centuries rather
than to the tribes. Apart from this gain the comitia made little
progress[1794] in the period between the Canuleian and the Licinian-
Sextian legislation, 445-367. Few legislative acts of the tribes are
recorded: the plebiscite which provided in a time of famine for the
election of a prefect of the market, 440;[1795] the historically
questionable plebiscite which forbade candidates for office to whiten
their garments, 432;[1796] the plebiscite of 414 for the creation of a
special court to try a case of murder;[1797] the act, probably a
plebiscite, which forbade a patrician to dwell on the Capitoline, 384.
[1798]

Doubtless in this period there was much agrarian agitation on the

part of the tribunes, although we cannot be sure that any of the bills
mentioned by Livy[1799] are historical. In like manner the leaders of
the plebs, as candidates for the consular tribunate, are represented
as agitating for the institution of pay for military service, the money to
be derived from rents of public lands.[1800] When the reform came,
however, it was by a voluntary concession of the senate extremely
annoying to the tribunes, who found themselves thus deprived of a
useful ground for complaint, 406.[1801] Epoch-making were the
Licinian-Sextian laws, the first, 368, increasing the duoviri sacris
faciundis to decemviri and providing that five should be plebeian,
[1802] the second, 367, containing in Livy’s opinion four articles: (1)

that one consul must be plebeian,[1803] (2) that the interest already
paid on debts should be deducted from the principal and the balance
rendered in three equal annual instalments, (3) that no one should
occupy more than five hundred iugera of the public land,[1804] (4)
that the right to pasture cattle and sheep on the public land should
also be limited.[1805]
Thereafter we find the tribal assembly more active in legislation. To
the year 358 is assigned the first well-authenticated lex de ambitu,
the Poetelian plebiscite, which forbade candidates for office to visit
markets and meeting-places outside the city for electioneering
purposes.[1806] The motive, however, which Livy attributes to the
author—to prevent the further enlargement of the patricio-plebeian
nobility through the admission of new men—was hardly possible at
this early date.
In 357 tribal comitia under patrician chairmanship passed a law for
placing a tax of five per cent on manumissions of slaves. The
circumstances attending this meeting were peculiar; the consul Cn.
Manlius summoned to it the soldiers of his army in the camp at
Sutrium.[1807] It must have been composed, therefore, of a small
minority of the citizens, lacking not only those who were too old for
service, but doubtless a majority of the men of military age.
Difficulties regarding the auspices, too, and other formalities might
have arisen; and yet in spite of the fact that the enactment of the law
was an intrusion within the administrative domain of the senate, the
patres gave their sanction;[1808] and the legality of the measure was
never called in question.[1809] In contrast with the general
prevalence of free labor in early Rome, the number of slaves since
the conquest of Veii had become considerable; and wealthy
individuals were evidently beginning the practice of building up a
political following through the clientage of their freedmen, to the
disadvantage of the older plebs. The majority of the patricians must
have been in sympathy with the effort of their consul to check this
new development, although they could not approve the peculiar
means by which the law was passed. Nor could the tribunes of the
plebs allow legislation to be thus removed beyond the sphere of their
control. The repetition of the procedure was immediately forbidden
accordingly by a plebiscite which threatened with the death penalty
any magistrate who held comitia away from the city.[1810] In the
same year the people took a further step in the administration of
finance by enacting the Duillian-Menenian plebiscite for establishing
the rate of interest at ten per cent[1811]—thereby confirming a law of
the Twelve Tables[1812]—and five years later the consular law of P.
Valerius Publicola and C. Marcius Rutilus for the institution of a bank
under the direction of five commissioners to assist debtors in
meeting their obligations (352).[1813] The latter was followed in 347
by a plebiscite which reduced the maximal rate of interest to five per
cent and provided for the payment of the principal in four equal
annual instalments.[1814]
This activity of the people in financial legislation is to be explained
by the economic distress which lasted many years, and which the
measures thus far mentioned failed to remedy. There can be no
doubt that the general indebtedness and the resultant discontent of
the masses, assigned by the annalists to the earliest years of the
republic, belong in reality to the period now under consideration. The
murmurings of the debtors culminated in 342 in a military mutiny,
with which the masses of citizens seem to have been in full
sympathy. The demands of the soldiers and civilians were met (1) by
a law of the dictator Valerius, which, remedying other grievances of
the soldiers, is said to have proclaimed an abolition of debts,[1815]
(2) by the plebiscite of L. Genucius, tribune of the same year. The
provisions of the latter were as follows: (1) it forbade the lending of
money on interest; (2) it ordered that no one should fill the same
office within a period of ten years, or two offices at the same time;(3)
it allowed both consuls to be plebeian.[1816] Although Livy, failing to
find the Genucian law in all his sources, hesitates to accept it as
historical, there seems to be no cogent ground for disbelieving that
such a statute was actually passed.[1817] The legal rate of interest
had recently been lowered one-half; and the plebeians, not satisfied
with the temporary relief afforded by the cancellation of debts, hoped
for all time to free themselves from an intolerable affliction by one
sweeping legislative act. This article of the plebiscite, however,
probably remained from the beginning a dead letter. The second
continued unenforced for many years,[1818] whereas the provision
regarding two consuls had to wait more than a century for its first
practical application.[1819] The patricians had often violated the
Licinian-Sextian statute by placing two of their number together in
the consulship. Perhaps the third article of the Genucian law was
intended to make them respect the earlier statute by a threat to
exclude them entirely from this office. If this was the object of
Genucius, his means certainly proved effective.[1820]
Three years later the dictator Publilius Philo passed through the
centuriate assembly the statute (1) that plebi scita should be binding
on all the quirites; (2) that before the voting began the patres should
give their auctoritas to proposals brought before the comitia
centuriata; (3) that one censor at least should be plebeian (339).
[1821] All three articles were alike aimed against the political
dominance of the patricians. The second freed centuriate legislation
from their control;[1822] the third[1823] assured to the plebeians a just
share in the function of determining the composition of the tribes,
hence of the civil and political status of every Roman. It was not long
afterward that the censors were to be given in addition the function
of revising the list of senators.[1824]
The first article has substantially the same form as the
corresponding provision of the Valerian-Horatian statute, 449, and of
the Hortensian, 287.[1825] All manner of conjectures as to the
relation of these three laws to one another has been offered, the
readiest theory being that the Valerian-Horatian statute had become
obsolete, and required reënactment.[1826] The explanation is proved
impossible by the circumstance that important plebi scita were
passed under the Valerian-Horatian provision, the last being the
Genucian. The Valerian-Horatian law could not have become
obsolete in three years. The true explanation is to be found in the
fact, now well known to historians, that the political ideas and
political struggles assigned by our sources to the fifth century b.c.
belong mostly to the fourth. The setting of the law of Publilius Volero,
471, was inaccurately transferred to it from the law of Publilius Philo,
339. The very existence of the latter statute is proof that the
patricians were at that time declaring plebi scita invalid on the
ground that they were passed by only a part of the people—a
complaint recorded against neither the Canuleian nor the Licinian
plebiscite. Hence, as the sources indicate, the patricians were in the
assembly which passed these two measures. We may legitimately
apply to the period from 449 to 339 the story of the long but finally
successful struggle on the part of the tribunes to expel the patricians
from the comitia tributa under plebeian chairmanship—a story which
the sources assign to the period ending in 367. The struggle must be
accepted as historical, for there was in later time no motive for
creating it; and as it must have been a matter of tradition rather than
of record, it could not well be placed earlier than the fourth century
b.c. We may suppose that the patricians yielded the more readily
because they at last recognized their inability simply by their votes to
control the tribunician assembly, and because from the beginning
they disliked to submit to the authority of a plebeian president.
Hence their withdrawal from that form of comitia was in the first
instance voluntary. The assembly, therefore, which adopted the
Genucian plebiscite was de facto, though not de jure, exclusively
plebeian. When accordingly the patricians objected to its validity on
the ground that it was passed by but a part of the people, Publilius
Philo, the most eminent plebeian statesman of his age, carried
through the centuriate assembly the law above mentioned, that the
resolutions of the tribunician assembly as then constituted, of plebs
only, should be valid for all the people. This interpretation throws
light on the otherwise inexplicable circumstance that the Genucian
plebiscite was so indifferently enforced. The exclusion of the
patricians was in line, too, with the general policy followed by the
plebeians against them in the fourth century: the plebeians shut the
patricians out (1) from the plebeian tribunate, probably 401. (2) from
five places in the college of decemviri sacris faciundis, 368, and from
one of the consular places, 367. (3) by agreement from the two
curule aedileships on alternate years, (4) from one of the censorial
places, 339. (5) from a fixed number of places in the college of
augurs and of pontiffs, 300. It was in accord with this tendency to
convert the earlier privileges of the patricians into disabilities that a
vote of the people excluded them from those comitia tributa which
were presided over by tribunes. This state of affairs was formulated
in the antiquarian and juristic definitions of populus and plebs, lex
and plebi scitum. The condition, however, seems to have been only
transient. The dwindling of the patriciate in numbers and strength,
with the corresponding growth of a plebeian nobility, which converted
the tribunate and assembly of plebs into most potent organs of the
senatorial government, obliterated distinctions between patricians
and plebeians within the political assemblies, to such a degree that
for the period after the Hortensian legislation no reference to an
exclusively plebeian assembly is made by any ancient author.
Although this article of the Publilian statute was never formally
repealed, we may feel certain that the principle involved was no
longer remembered in the age of Cicero.[1827]
The Publilian statute of 339 is not known to have provided for an
extension of the field of competence of the tribal assembly; yet we
find the comitia tributa soon afterward attending to business
heretofore managed by the senate or in one or two instances by the
centuries. Although about a hundred years earlier the centuriate
comitia had acquired the right to ratify or reject declarations of
offensive war,[1828] we find no record of a ratification of a treaty of
peace by the people before the year 321, in which occurred the
disaster at Caudium; and in this case it was not only the common
opinion in Livy’s time, but also the understanding of Claudius, the
historian, that the treaty made by the consuls, without the sanction of
the senate or the people, was regular and valid[1829]—a “foedus
summae religionis,” as Cicero declares.[1830] Even Livy, who aims to
prove the procedure defective, admits that the tribunes of the
plebs[1831] and Postumius,[1832] one of the consuls who made it,
looked upon it as legitimate. But according to Livy[1833] the senate
itself declared the treaty invalid on the ground that it lacked popular
confirmation;[1834] and in that body the principle was then
enunciated that nothing which was to bind the people could be
sanctioned without their order[1835]—the first recorded expression of
the doctrine of popular sovereignty among the Romans. In this
period, however, the people were never called upon to ratify the
acceptance of a submission or of an alliance on unequal terms. Such
agreements granting Rome the superior right were negotiated, as in
earlier time, by the magistrate or senate or by both in conjunction.
[1836] The details, too, of every treaty were still left to the magistrates
and senate, so that to the end of the republic the senatus consultum
continued to be indispensable.[1837] But from the time of the Caudine
misfortune, and in consequence of it, the principle was established
that a treaty involving a concession of even equal rights on the part
of Rome required the sanction of a popular vote. Recorded instances
of such ratification for this period (321-287) are rare.[1838] The
function fell to the comitia tributa under patrician or plebeian
presidency, which in its exercise showed more independence[1839]
than did the comitia centuriata in the declaration of wars. In this way
the tribal assembly took its place by the side of the centuriate in
international affairs.[1840]
The absolute power to bestow the citizenship exercised by the
kings[1841] would naturally pass undiminished to the consuls, and
thence to the censors on the institution of the latter. It is in fact the
opinion of Lange[1842] that these magistrates respectively exercised
full rights in the matter, and that they consulted the senate in
important cases only. At all events the question is simply as to the
relative participation of the magistrates and the senate in the
function. The final settlement of Latium after the war, involving the
bestowal of citizenship, 338, the senate seems to have attended to
alone through a consultum, no mention being made of the people.
[1843] In the whole course of Roman history to 332 there is no record

of a grant of citizenship by popular vote.[1844] As the Acerrani were

left out of account by the senatus consultum above mentioned, L.
Papirius in 332 through the first recorded pretorian law granted them
the civitas sine suffragio.[1845] In opinion of Lange,[1846] based upon
a statement of Velleius,[1847] the censors of the year, Q. Publilius
Philo and Sp. Postumius, while enrolling the new citizens, probably
obtained a senatus consultum requesting the praetor to bring this
subject before the tribes. That a senatorial decree was essential is
proved by the case of the Privernates mentioned below. We may
well believe that the great plebeian statesman Publilius gladly
embraced the opportunity to make the tribal assembly a partner in
the important function of imparting the rights of the city. Three years
afterward an order of the people, doubtless of the tribes, ex
auctoritate patrum, granted the citizenship to the Privernates, 329.
[1848] By what authority the Hernicans received the civitas sine

suffragio in 306 is not stated.[1849] Long after the Hortensian

legislation the principle was established that the people alone
without the authorization of the senate had a right to bestow the ius
suffragii on whomsoever they pleased.[1850] Logically the function
should have fallen to the comitia centuriata as the source of
censorial power; but the tribal assembly assumed it because of its
connection with the making of treaties.[1851]
It was the province of the centuriate assembly to introduce
permanent regulations of existing magistracies and to institute new
ones;[1852] but the function was now transferred, silently so far as we
know, to the tribes. Far-reaching in its effect was the creation of the
promagistracy in 327. No prolongation of an official power is known
to have occurred before this date. The extension of the territory of
Rome and of her military operations was now calling for greater
elasticity in the duration of commands; but in the face of a strong
movement toward popular rights the senate dared not assume the
responsibility of so sweeping an innovation. It placed in the hands of
the tribunes, accordingly, the business of bringing before the people
a rogation for prolonging the imperium of the consul Q. Publilius
Philo to the end of the war with Naples, instituting by this precedent
the promagistracy.[1853] Again in 295 the imperium of the consul
Volumnius was prolonged for a year by a decree of the senate
ratified by a plebiscite.[1854] After the custom had been established,
however, the senate ordinarily attended to the prolonging of the
imperium, as in 308,[1855] in 296,[1856] and in 294,[1857] consulting
the people, as it seems, only in cases of tribunician opposition.[1858]
No instance of popular interference in the assignment of provinces is
mentioned before 295, when a resolution of the comitia tributa, under
what form of presidency is not stated, granted Etruria to the patrician
Fabius in preference to the plebeian Decius.[1859] This act was an
inroad upon the right of the magistrates to divide the business of
their office among themselves by agreement or lot. In 292 another
resolution of the people recalled from the field the consul Q. Fabius
Gurges because of ill-success in war with the Samnites. The senate
was the prime mover in the matter, but the form of assembly is
unknown. The question concerned either the abrogation of his
magistracy or more probably his transfer to some other activity.[1860]
Even in the latter case the act of the people was a remarkable
deviation from their usual modest policy of dealing with officials.
In 318 a law, doubtless tribal, was passed for sending praefecti
iure dicundo to Capua;[1861] and similar laws were from time to time
enacted for assigning the same kind of officials to other communities
of Italy.[1862] These prefects continued to be appointed by the urban
praetor till after 124.[1863] Whether the law of 318 was pretorian or
tribunician cannot be determined.[1864] Similar in character was the
Atilian-Marcian plebiscite for the election of sixteen military tribunes
instead of six, 311.[1865] The substitution of election for appointment
was in effect the institution of a magistracy—in this case merely an
increase in number within a magisterial college which had existed
since 362. In the act of 311 the tribes usurped a function which had
hitherto belonged to the centuries.[1866] Although the elective military
tribunes remained subordinate to the consuls, the change increased
their dignity and in some degree their independence, while it tended
to impair the efficiency of the service. Naturally the office became a
stepping-stone to political honors. The Decian plebiscite of the same
year instituted the duumviri navales charged with the function of
repairing, equipping, and commanding the fleet.[1867] The two
plebiscites of this year have the appearance of a compromise
between continental and commercial interests under the influence of
Appius Claudius Caecus the censor. Closely related is the article of
the Ogulnian plebiscite, 300, which provided for an increase in the
number of augurs and pontiffs.[1868] Here, too, belongs the plebiscite
of 296 for the appointment of commissioners for conducting colonies.
[1869] Henceforth it was the custom of the senate to refer to the
people the creation of all extraordinary offices, and their election to
the comitia tributa usually under pretorian presidency.[1870]
The people made a further advance when they undertook to
regulate by law the composition of the senate itself. To the period
between the Publilian legislation of 339 and the censorship of Appius
Claudius Caecus, 312, belongs the famous Ovinian plebiscite
concerning the revision of the senate list.[1871] It transferred the
function from the consuls to the censors, and required the latter
under oath (iurati; MS. curiati) to enroll all who were worthy among
the retired magistrates of every rank, from the curule functionaries
down through those of plebeian standing to the quaestors.[1872]
The Valerian-Horatian and Publilian statutes are evidence of the
right of the people to legislate regarding the composition and powers
of their assemblies. No longer content, however, with the making and
repeal of laws,—a right guaranteed by the Twelve Tables,[1873]—
they began the practice of occasionally suspending laws to the
advantage or disadvantage of individuals or of classes—in other
words, the voting of privilegia.[1874] There were repeated violations
of that article of the Genucian plebiscite which forbade reëlection to
an office within a period of ten years,[1875] and no mention is made
of the necessity of a dispensation before the year 298, when Q.
Fabius Maximus is alleged to have objected to further reëlection on
the ground that such conduct was forbidden by law. Thereupon the
tribunes of the plebs declared that to remove the obstacle they would
propose to the people that he should be absolved from the legal
requirement.[1876] But in fact, as Lange[1877] has noticed, Fabius
had not been consul for ten years and was therefore legally eligible.
Lange suggests that this story of the dispensation may belong to his
next election in 295.[1878] At all events the custom of granting
dispensations began about this time,[1879] although we need not
suppose that the patricians attached much importance to the
Genucian statute, which was adopted by an exclusively plebeian
assembly. This function assumed by the people of freeing from the
power of the law, often exercised in historical time by the senate as
well, marks a great advance toward popular sovereignty. The idea
that the law was sovereign, which had arisen in the early republic,
was now yielding to the idea that it was subject to the caprice of
every popular gathering.[1880] The aristocracy was giving way to a
democracy, which under the conditions destined to prevail at Rome
could only mean mob-rule.
The right of the people in their tribal assemblies to legislate
concerning religion had already been established by the precedent
of the Licinian-Sextian plebiscite on the decemviri sacris
faciundis[1881] and of other less important acts.[1882] Immediately
after the Publilian legislation the comitia of tribes became more
active in this field. To the period of the great Latin war according to
Cicero,[1883] hence necessarily to 338,[1884] belongs the consular lex
Maenia, which added to the Ludi Romani the day called
instauraticius,[1885] although less trustworthy accounts assign the
establishment of this day to 491.[1886] The law initiated by the senate
in 304 forbidding the dedication of a temple or altar except by
permission of the senate or of a majority of the college of
tribunes[1887] was probably passed by the comitia tributa plebis. In
the opinion of Lange[1888] it was either identical with, or afterward
supplemented by, the lex Papiria tribunicia, which forbade the
consecration of a temple, precinct, or altar without an order of the
plebs.[1889] The latter is the more probable; it seems reasonable
that, as Lange suggests, the right of the people in this matter
developed from the necessity of referring to them cases in which the
senate and the tribunes could not agree. Technically religious,
though of vast political consequence, was the Ogulnian plebiscite of
300, which increased the number of augurs and pontiffs to nine
each, and provided that four augurs and five pontiffs should be
plebeian.[1890] It was the last step in the opening of offices to the
plebs.
In their effort to gain control of the more important judicial business
the people made slower progress. In all probability it was not till after
the Publilian legislation that the centuriate and tribal assemblies
began regularly to exercise the function of appellate courts—a right
established long before by legislation[1891] and confirmed for the
centuries by the Valerian law of appeal in 300.[1892] The creation of
special judicial commissions—quaestiones extraordinariae—
belonged originally to the senate; and the establishment of such a
court de caede through a plebiscite in 414, if historical, was merely
the execution of a senatus consultum.[1893] The task of trying and
condemning the matrons for poisoning in 331 must have fallen to
such a quaestio extraordinaria not expressly mentioned. Whether it
was instituted by the senate or the tribes cannot be known.[1894] The
special quaestio, too, concerning conspiracy, at first under dictatorial
and afterward under consular presidency, seems to have been
instituted solely by a senatus consultum.[1895] The Flavian rogation
of 323 for punishing the Tusculans for having given aid and
encouragement to the enemies of Rome[1896] may have aimed to
create a special court for the purpose, or it may have been an
attempt to dispense justice by means of legislation.[1897] However
that may be, it was rejected by all the tribes but one. The Satricans,
who revolted to the Samnites after the Caudine disaster and were
conquered in 319, were punished by the senate acting as a special
court on the authority of the Antistian plebiscite.[1898]
The right of the people both in the centuries and in the tribes to
legislate on finance had before 339 been well established by
precedent. Economic as well as social in character was the lex
Poetelia, which prohibited loans on the security of the person,[1899]
and which was proposed to the tribes, or possibly to the centuries,
by C. Poetelius Libo as consul in 326 or as dictator in 313.[1900] It
abolished contractual but not judicial servitude, though it probably
mitigated the latter.[1901] Politically more significant than this
individual act was the long-continued popular effort to gain control of
the disposal of the public land. It was to the detriment of the
senatorial prerogative that the tribunes of the plebs took up the
agrarian question from the time of Sp. Cassius,[1902] and continued
almost unceasingly to agitate for the limitation of the use of public
land by the rich and the division of the surplus among the poor, till
they succeeded in embodying their ideas in the Licinian-Sextian law
on these subjects. Equally to the province of the senate belonged
the planting of colonies[1903] both from the military and from the
financial point of view. Here, too, the tribunes in the economic
interest of their constituents began early to agitate for a share in the
administration.[1904] It was not till 296 that they met with any success
in this direction, and then at the will of the senate, which charged the
tribunes with the business of introducing a plebiscite for ordering the
praetor to appoint triumviri for conducting colonies to certain
specified places.[1905] This was the modest outcome of centuries of
agrarian and colonial agitation on the part of the tribunes.
The fact is that after the enactment of the Genucian and Publilian
laws the plebeians continued for about a generation relatively
content with their economic condition. Frequent victories brought
booty,[1906] and conquests made extensive assignments of land
possible.[1907] But the people must have found the third Samnite war
oppressive. Although of far shorter duration than the second, it
required larger armies and longer and more distant campaigns.
Under the burden of military service the plebs again fell into debt, in
spite of the unusual distributions of booty among the soldiers when
victorious.[1908] Their burden was rendered the heavier by the
circumstance that many of the wealthy were violating the Licinian-
Sextian restrictions on the use of public land and pasture, and were
doubtless failing to pay their dues[1909]—a course of conduct which
rendered necessary not only the assignment of the spoil of 293 to
the aerarium but also the imposition of a tributum especially
vexatious to the plebs.[1910] The distress was augmented by a
pestilence which began in 295 and continued for several years.[1911]
Whereas all on actual service were by law exempt from prosecution
for debt, many citizens who remained at home were the victims of
the usurers, who were occasionally fined for their illegal exactions.
[1912] Again all the commons incurred hopeless debts, which at the
close of the war (290) the creditors must have proceeded to exact
with their usual ruthlessness. The institution of the tresviri capitales
in the following year[1913] is proof of the intention of the government
to enforce the criminal law with the utmost rigor. A new movement
for the relief of debtors had already set in, and the creditors were
organizing resistance to the popular demands. As long as the nobility
could rely upon the tribunate of the plebs,[1914] they felt secure.
Even if a bill for the benefit of the poor should be presented, they
believed their interests to be well fortified by tribunician intercession
and by the senate, which, composed chiefly of creditors, would
certainly refuse its sanction to such a measure. The grave economic
distress, however, at length filled the tribunate with men who were at
one in demanding a radical measure of relief, and who accordingly
presented a bill for the abolition of debts. Many times they offered it
to the tribes in vain; the senate refused its assent; for the creditors,
among whom must be counted a majority of the senators, hoped to
recover both principal and interest. Willing to compromise, the
tribunes then offered the senate, if it should yield, a choice of two
alternatives, neither of which can be deduced with certainty from the
mutilated fragment of Dio Cassius, our authority for this event. One
of them, however, is conjectured to be that the principal alone should
be recovered,[1915] in what way cannot be made out; and the other
that the interest already paid should be deducted from the principal,
and the balance rendered in three equal annual instalments—a
repetition of the Licinian-Sextian provision regarding debts. At first
the debtors were willing to grant this concession through fear of
failing to obtain any degree of relief; but the creditors, now hoping to
recover everything, refused to be conciliated. After a time both
parties shifted their attitude; the creditors expressed themselves as
satisfied to recover the principal merely, while the debtors would no
longer accept either alternative of the compromise. The sedition, for
such the conflict became, continued interminably; and although the
creditors yielded, little by little, far more than they had intended in the
beginning, the debtors made each concession the basis of a new
demand. They brought the long, serious struggle to a climax by
seceding to the Janiculum, at the very time when the Tarentines
were completing the organization of a coalition of Etruscans, Gauls,
Samnites, and several other peoples against Rome.[1916] Q.
Hortensius, appointed dictator to meet this crisis, carried through the
comitia centuriata a group of provisions for satisfying the demands of
the seceders:

(1) Doubtless a clause for the relief of debtors, of which no

mention is made in our scant sources.
(2) A provision that without the consent either of the senate
or of the patrician portion of it a resolution of the plebs should
be valid for all the citizens.[1917]
At the time when the Valerian-Horatian statute provided that
with the consent of the senate resolutions of the tribunician
comitia tributa should have the force of law, the senate was
still composed exclusively of patricians; and the phrase
senatus consultum in this law was therefore considered a full
equivalent of the patrum auctoritas, the only difference being
that the consultum was given in advance of a popular vote
and the auctoritas subsequently to it. But when with the
appearance of plebeians in the senate the two acts began to
drift more widely apart, the patricians successfully claimed an
exclusive right to the auctoritas, which, as we have seen,
[1918] was reduced to a formality, so far as centuriate
legislation was concerned, by an article of the Publilian law.
So long as the patricians voted in the tribunician comitia
tributa, however, and constituted a majority in the senate, they
were willing to abide by the specific declaration of the
Valerian-Horatian statute which conditioned the validity of the
plebiscite on the senatus consultum. But from 339 they were
legally excluded from the tribunician comitia tributa, and they
foresaw, moreover, the end of their majority in the senate. In
the period between 339 and 287, accordingly, they set up a
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

textbookfull.com