See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/24274563

CowLog: Open-source software for coding behaviors from digital video

Article in Behavior Research Methods · June 2009

DOI: 10.3758/BRM.41.2.472 · Source: PubMed

CITATIONS READS

208 800

2 authors:

Laura Hänninen Matti Pastell

University of Helsinki Natural Resources Institute Finland (Luke)
118 PUBLICATIONS 2,756 CITATIONS 198 PUBLICATIONS 3,357 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Laura Hänninen on 19 May 2014.

The user has requested enhancement of the downloaded file.

 This document is downloaded from
HELDA - The Digital Repository of
University of Helsinki.

Title CowLog; open-source software for coding behaviors from

digital video
Author(s) Hänninen, Laura; Pastell, Matti
Citation Hänninen , L & Pastell , M 2009 , ' CowLog : open-source
software for coding behaviors from digital video ' , Behavior
Research Methods , vol 41 , no. 2 , pp. 472-476 .
Date 2009
URL http://hdl.handle.net/10138/27390
Version Peer reviewed

HELDA - The Digital Repository of University of Helsinki - Terms and User Rights

By using HELDA - The Digital Repository of University of Helsinki you are bound by the following Terms & Conditions.
Please read them carefully.

I have read and I understand the following statement:

All material supplied via HELDA is protected by copyright and other intellectual property rights, and duplication or sale of all
or part of any of the repository collections is not permitted, except that material may be duplicated by you for your research
use or educational purposes in electronic or print form. You must obtain permission for any other use. Electronic or print
copies may not be offered, whether for sale or otherwise to anyone who is not an authorised user.
Behavior Research Methods
2009, 41 (2), 472-476
doi:10.3758/BRM.41.2.472

CowLog: Open-source software for coding

behaviors from digital video
Laura Hänninen and Matti Pastell
University of Helsinki, Helsinki, Finland

We have developed CowLog, which is open-source software for recording behaviors from digital video and is
easy to use and modify. CowLog tracks the time code from digital video files. The program is suitable for coding
any digital video, but the authors have used it in animal research. The program has two main windows: a coding
window, which is a graphical user interface used for choosing video files and defining output files that also has but-
tons for scoring behaviors, and a video window, which displays the video used for coding. The windows can be used
in separate displays. The user types the key codes for the predefined behavioral categories, and CowLog transcribes
their timing from the video time code to a data file. CowLog comes with an additional feature, an R package called
Animal, for elementary analyses of the data files. With the analysis package, the user can calculate the frequencies,
bout durations, and total durations of the coded behaviors and produce summary plots from the data.

Most behavior observations rely on video recordings CowLog is the only solution that can be used without
because direct observation has a high risk of disturbing any software costs, because it works on free operating
the observed subject. The observer scores the video data, systems. This makes it especially appealing to students.
usually with the help of behavior coding software. For The use of the GPL license means that the source code
complex video readings, the observer needs to be able is available to all users, and users are free to use, modify,
to move forward and backward in the video, as well as and redistribute the source code. Therefore, future devel-
to use different playback speeds. To read such video data opments of open-source software, unlike developments
successfully and fluently, the coding program needs to be of freeware, are not dependent on a single developer.
able to track the video time code accurately. Digital video recorders use a variety of different video
For behavioral recording from the videos, one can use formats to encode video, and thus it is also important
different sampling methods. When recording behaviors that the coding software has the ability to be used with as
continuously, one records each time the behavior occurs many formats as possible without the need to transcode
(frequency), the length of the behavioral sequence (bout the video.
duration), and the total time the subject exhibits each Several behavioral data collection programs are avail-
of the behaviors during the time period. During instan- able, some of which are also freeware. However, these are
taneous behavioral recording, an observed behavior is not open-source programs and operate only in Microsoft
scored in predefined time intervals. During one–zero Windows—mfor example, Etholog (Ottoni, 2000), which
sampling, the observer records only frequencies—for ex- does not read the video time code, and ODCS (MacLin
ample, whether the behavior occurs (frequency) during & MacLin, 2005), which supports only mpeg and AVI
the observed time periods (Martin & Bateson, 1995). video formats and does not provide any analysis tools.
We have developed the open-source software CowLog More powerful programs, such as The Observer (Nol-
for recording behaviors from digital video. CowLog is dus, Wageningen, The Netherlands), also read the video
easy to use and modify, and tracks the time code from time code and have multiple analysis tools; however, The
video files and records the time with the coded behavior Observer is expensive and its license is restricted to a
to a data file. The program is suitable for coding any predefined number of computers. In many studies, the
digital video, but the authors have used it only in animal user needs only the simpler features of programs, such
research. as coding the video and calculating total duration, bout
The new program is different from existing, similar duration, and frequencies from the data.
software in three significant ways: (1) CowLog is the The aim of this article is to introduce CowLog as a
only software targeting Linux and BSD operating sys- new open-source tool for behavioral research. We pre-
tems, (2) it is open-source software released under the sent information on the basic features, use, and struc-
GNU GPL license, and (3) it supports all popular video ture of the software and also present the basic behavioral
codecs, because it uses MPlayer as a video player. analysis of the data using a package called Animal for

M. Pastell, [email protected]

© 2009 The Psychonomic Society, Inc. 472

 CowLog     473

on one screen, and the coding window can be visible on

another).
Users type the key codes for their predefined behavioral
categories, and CowLog transcribes their timing from the
video time code to a data file. The data file can be used to
calculate frequencies, bout durations, and total durations
for each of the behaviors.
CowLog comes with an additional analysis component,
a package called Animal, for elementary analysis of data
files. Animal is written with the open-source statistical
software R. Packages provide a convenient way to distrib-
ute users’ own functions in order to extend the functional-
ity of R. With the analysis program, the user can calculate
the frequencies, bout durations, and total durations of the
coded behaviors.
The most recent version of CowLog can be freely
downloaded from its homepage: www.mm.helsinki
.fi/~mpastell/CowLog.

Use
Before starting the behavioral observations, the user
must assign behavioral codes. The behavioral class is a set
of behaviors, in which each occurrence is exclusive (i.e.,
the subject can either stand or walk, but not both). The
behaviors belonging to parallel classes may occur simul-
taneously (e.g., standing and eating).
At the moment, the program consists of three behav-
ioral classes, each with eight behaviors. The names for the
key codes are always the same (numbers from 1 to 24).
The user decides which key code represents each of the
coded behaviors. No initial setup is needed, since the key
code definitions are always the same; however, the user
needs to keep track of codes elsewhere. The user should
also note that in this program the behavioral states (i.e.,
behaviors for which the user wants to record both the oc-
currence and duration) and events (behaviors for which
only the number of occurrences is recorded) are defined
Figure 1. The coding window of CowLog. The name of the video in the analysis stage. Coding is performed continuously
file being used is shown in the upper file dialog, and the user-
defined name for the data file under it. B1, B2, and B3 are the
behavioral classes available for the scoring, and squares below
show the status of the users’ previous recordings in each of the
behavioral classes. The left column of keys are the operator keys
for the video window.

elementary analysis of data files, written with the open-

source statistical software R, supplied with CowLog.

CowLog

The program reads all common digital video formats

and also works on older computers. The recommended
operating system for CowLog is Ubuntu Linux 8.04, but
it has also been tested on FreeBSD 7.0.
The graphical user interface for the CowLog program
has two windows: a coding window (Figure 1), for choos-
ing the video files, defining the output file, and scoring
behaviors, and a video window (Figure 2), showing the
video used for coding. The windows can be used in sepa- Figure 2. The video window of CowLog with a test picture from
rate displays (i.e., the video can be viewed in full screen a calf study.
474     Hänninen and Pastell

Table 1 point, which makes the removal of the wrong codes pos-
Example Output From the Software sible while using the descriptive analyses function.
Time (sec) Code
1.3 3 Software Description and System Requirements
49.3 4 The recommended operating system for CowLog is
53.7 3 Ubuntu Linux 8.04 (Canonical Ltd.), but it has also been
71.6 4
97.5 3 tested with FreeBSD 7.0 (The FreeBSD Project). The pro-
103.7 4 gram should run on most Linux and BSD distributions,
140.5 3 as well as Mac OS X (Apple Inc.), provided the software
159.3 4 dependencies are met. CowLog supports all video formats
171.0 3
playable with MPlayer (which means most common video
formats). The codecs for the required video formats need
to be installed on the computer.
by logging the time each behavior starts. In the analysis The program consists of a graphical front end made
stage, the type of the behavior is given as an input for the with Kommander 1.3 (Britton, King, Laffoon, & Man-
analysis function, and the start time of the next state is tja, 2008), and it uses the MPlayer media player running
used as the end time of the previous state. Accordingly, in slave mode to show the video files and read the time
the start time of events is used to mark the occurrence code. A set of underlying shell and Gawk (Robbins, 2007)
of a behavior. An example output from the software is scripts are used to pass commands between the graphical
shown in Table 1. user interface and MPlayer.
After the program has been started, the user defines Because of its modular structure, the program needs
the video observation file to be used for coding and the other software packages to run. All of the dependencies
prefix and location of the data files. After the files have are open source and can be easily installed in Ubuntu
been defined, the user simply presses the start button to using the graphical package manager. The following soft-
start coding. The user can enter the scores by pressing the ware packages are needed by CowLog.
buttons in the coding window (Figure 1) or by pressing Kommander 1.3. Kommander is a tool for creating
keyboard shortcuts. Each of the behavioral classes has a graphical applications with a simple user interface. Kom-
representative display from which the user can follow the mander produces noncompiled dialogs and very small
five last-entered codes. If the user wants to code several applications that run fast and can be distributed without
behaviors with the same time code, he or she pauses the binary compatibility issues. It was used to create the cod-
video, enters the codes, and resumes playing. The cod- ing window of the CowLog, and the commands passed on
ing is ended simply by closing the program, and can be to MPlayer are called from Kommander scripts.
directly resumed to the same data files by moving to the MPlayer. MPlayer is a movie player that runs on many
correct position in the video. The program does not auto- systems. It plays almost all existing video formats, such
matically give the user the resuming point; the user needs as MPEG/VOB, AVI, Ogg/OGM, ASF/WMA/WMV, and
to check the time stamp in the last row of the data file. DivX 3/4/5. CowLog runs MPlayer in slave mode to play
The user can run the video at different speeds—up the videos and read the time codes.
to five times the normal speed. To ease the search for a Gawk. Gawk is a GNU implementation of the popular
wanted point in a video, CowLog can automatically jump Awk programming language (Robbins, 2007). The Awk
30 sec forward or backward. utility interprets a special-purpose programming language
CowLog saves the outputs for each behavioral class in that makes it possible to handle simple data-reformatting
separate data files. The output files have the extensions jobs with just a few lines of code. CowLog calls Gawk to
.bh1, .bh2, and .bh3, where the number corresponds to the format the output from MPlayer to a cleaner form before
behavioral class. The files contain the complete data of the writing to the data file.
observed sessions. The program saves the data after each
scoring, so that no data are lost if the computer crashes or Example Output From the Software
is disconnected. CowLog writes the output files in plain text format. The
Coding errors are easy to remove manually from the data output data are organized in columns: The first column shows
files after the recording has finished. Users can also define the time in seconds from the beginning of video recording,
one key code in each of the classes as an error key. Pressing and the second column represents the behavioral key code
this key after the coding error has occurred marks the data (Table 1). Each row represents each of the observations.

Table 2
An Example Output From the Analysis Function
Behavioral Feature Standing Still Locomotion Lying Down Hidden
Frequency 270 269 30 9
Bout length (min) 0.8 0.37 37.33 4.03
Total duration of behaviors (min) 214.81 99.35 1,120.03 36.28
 CowLog     475

Input parameters from function

call

Order
observations
by time

Calculate Event State Remove

frequency of
duplicated codes
events Observation
type

Merge Calculate total

results duration, bout
duration, and
frequency of states

Show
table of
results

Plot
results

Figure 3. A flowchart showing the operation of the main analysis function in the Animal
R package.

Descriptive analyses appear on-screen and can be saved to a variable and to a

of recorded data file. An example function is shown below:

The data files created by the program can be processed cowAnalyze(file5'filename.bh1',states5c(1,2,3),

with almost any statistical software. We have written an events5c(4,5),state.names5c('state1','state2',
analysis package, Animal, with the open-source statistical 'state3'), event.names5c('event4','event5'))
software R 2.72 (R Development Core Team, 2008). Pack- The function also removes double-state (duplicated)
ages provide a convenient way to extend the functionality errors for state codes. The operation of the main analy-
of R. The package has a main analysis function to process sis function in the package is shown as a flowchart in
CowLog data files. Other functions are called internally Figure 3.
from the main function to perform specific tasks. We have
also written a basic R manual for the package, and all its An example project
code is accessible for users to modify. The Animal pack-
age is available for download with CowLog. The sample files are obtained from the project study-
The results from the analysis can be exported to other ing calves’ activity, in which calves’ overall activity was
programs using the clipboard or R’s functions for saving scored continuously for 24 h. We show here an example of
data. The main function (cowAnalyze) takes the data file the behavior of 1 calf scored from a 24-h .avi video file.
name, labels of the codes, and type (event or state) of the The data file was analyzed with the provided R function.
codes as inputs, and gives a summary table (Table 2) and a The output obtained from the analysis is shown in Table 2
plot of the results as output. The descriptive statistics will and the function produced is shown in Figure 4.
476     Hänninen and Pastell

Bout Duration of Different Behaviors

120

Bout Duration (min) 100

80

60

40

20

Standing Still Locomotion Lying Hidden

Behavior
Figure 4. An example output from the analysis function: A boxplot of the bout
duration, in minutes, for different scored behaviors.

Discussion and further needed. Because CowLog is an open-source program, and

development the main window is created with a graphical tool, users can
also modify the interface and data file formats for their own
At the moment, CowLog is a basic program designed purposes. The use of an open-source license contributes to
for coding the behavior of 1 subject from one video at the reproducible research, because everyone can see ex-
a time. The development version of CowLog also works actly how the coding and analysis are executed.
with two parallel videos. This can be beneficial in study The program is freely available from its Web page,
designs in which 1 subject is filmed with several videos where users may share their comments and applications
simultaneously, using synchronized time codes. In this or ask for help from other users when troubleshooting.
case, only one data file is saved. Using several videos,
however, requires the messages from the coding window Author Note
to be passed to several instances of MPlayer, and this is We thank DVM Ann-Helena Hokkanen for her valuable comments
not yet implemented in the distributed version. The maxi- on the user interface and for testing of the software. Correspondence
mum number of video files the program can read simul- concerning this article should be addressed to M. Pastell, Department
taneously depends on how many instances of MPlayer the of Agrotechnology, P.O. Box 28, 00014 University of Helsinki, Hel-
sinki, Finland (e-mail: [email protected]).
computer can run.
The R package Animal is provided with the software References
for easy calculation of elementary statistics. The package
can be freely modified by users, just like CowLog. The Britton, M., King, T., Laffoon, E., & Mantja, A. (2008).
The Kommander handbook. Available from http://docs.kde
analyzed data can also be exported in ASCII format or .org/kde3/en/kdewebdev/kommander/index.html.
via the clipboard to other statistical software for the user’s MacLin, O. H., & MacLin, M. K. (2005). Coding observational data: A
own calculations. The use of the analysis package requires software solution. Behavior Research Methods, 37, 224-231.
a basic knowledge of how to use R, but the user can get de- Martin, P., & Bateson, P. (1995). Measuring behavior: An introduc-
tory guide (2nd ed.). Cambridge: Cambridge University Press.
scriptive statistics of a data file with a single function call. Ottoni, E. B. (2000). EthoLog 2.2: A tool for the transcription and tim-
No advanced programming is needed. We are currently ing of behavior observation sessions. Behavior Research Methods,
working on a graphical front end so that the user does not Instruments, & Computers, 32, 446-449.
need to use R to get the descriptive statistics. R Development Core Team (2008). R: A language and environ-
The program can be used as is, because it does not need ment for statistical computing. Vienna: R Foundation for Statistical
Computing.
any modifications for the codes to be used. The behavioral Robbins, A. D. (2007). GAWK: Effective AWK programming. A user’s
codes are only saved as numbers; thus, the labels for dif- guide for GNU Awk. Boston: Free Software Foundation.
ferent behaviors need to be supplied in the analysis stage.
Thus, users can change states to events and vice versa, (Manuscript received December 18, 2008;
and return to already transcribed data for new analyses, if revision accepted for publication February 25, 2009.)

View publication stats