41703
41703
https://textbookfull.com/product/reachability-problems-12th-
international-conference-rp-2018-marseille-france-
september-24-26-2018-proceedings-igor-potapov/
https://textbookfull.com/product/reversible-computation-10th-
international-conference-rc-2018-leicester-uk-
september-12-14-2018-proceedings-jarkko-kari/
https://textbookfull.com/product/exploring-service-science-9th-
international-conference-iess-2018-karlsruhe-germany-
september-19-21-2018-proceedings-gerhard-satzger/
https://textbookfull.com/product/belief-functions-theory-and-
applications-5th-international-conference-belief-2018-compiegne-
france-september-17-21-2018-proceedings-sebastien-destercke/
Text Speech and Dialogue 21st International Conference
TSD 2018 Brno Czech Republic September 11 14 2018
Proceedings Petr Sojka
https://textbookfull.com/product/text-speech-and-dialogue-21st-
international-conference-tsd-2018-brno-czech-republic-
september-11-14-2018-proceedings-petr-sojka/
https://textbookfull.com/product/web-information-systems-and-
applications-15th-international-conference-wisa-2018-taiyuan-
china-september-14-15-2018-proceedings-xiaofeng-meng/
https://textbookfull.com/product/business-process-
management-16th-international-conference-bpm-2018-sydney-nsw-
australia-september-9-14-2018-proceedings-mathias-weske/
https://textbookfull.com/product/machine-learning-and-knowledge-
discovery-in-databases-european-conference-ecml-pkdd-2018-dublin-
ireland-september-10-14-2018-proceedings-part-iii-ulf-brefeld/
Patrice Bellot · Chiraz Trabelsi · Josiane Mothe
Fionn Murtagh · Jian Yun Nie · Laure Soulier
Eric SanJuan · Linda Cappellato · Nicola Ferro (Eds.)
Experimental IR Meets
Multilinguality,
LNCS 11018
Multimodality,
and Interaction
9th International Conference of the CLEF Association, CLEF 2018
Avignon, France, September 10–14, 2018
Proceedings
123
Lecture Notes in Computer Science 11018
Commenced Publication in 1973
Founding and Former Series Editors:
Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen
Editorial Board
David Hutchison
Lancaster University, Lancaster, UK
Takeo Kanade
Carnegie Mellon University, Pittsburgh, PA, USA
Josef Kittler
University of Surrey, Guildford, UK
Jon M. Kleinberg
Cornell University, Ithaca, NY, USA
Friedemann Mattern
ETH Zurich, Zurich, Switzerland
John C. Mitchell
Stanford University, Stanford, CA, USA
Moni Naor
Weizmann Institute of Science, Rehovot, Israel
C. Pandu Rangan
Indian Institute of Technology Madras, Chennai, India
Bernhard Steffen
TU Dortmund University, Dortmund, Germany
Demetri Terzopoulos
University of California, Los Angeles, CA, USA
Doug Tygar
University of California, Berkeley, CA, USA
Gerhard Weikum
Max Planck Institute for Informatics, Saarbrücken, Germany
More information about this series at http://www.springer.com/series/7409
Patrice Bellot Chiraz Trabelsi
•
Experimental IR Meets
Multilinguality,
Multimodality,
and Interaction
9th International Conference
of the CLEF Association, CLEF 2018
Avignon, France, September 10–14, 2018
Proceedings
123
Editors
Patrice Bellot Laure Soulier
Aix-Marseille University Pierre and Marie Curie University
Marseille Cedex 20 Paris Cedex 05
France France
Chiraz Trabelsi Eric SanJuan
Virtual University of Tunis Université d’Avignon et des Pays de
Tunis Avignon
Tunisia France
Josiane Mothe Linda Cappellato
Systèmes d’informations, Big Data et Rec Department of Information Engineering
Institut de Recherche en Informatique de University of Padua
Toulouse Cedex 04 Padua, Padova
France Italy
Fionn Murtagh Nicola Ferro
Department of Computer Science University of Padua
University of Huddersfield Padua
Huddersfield Italy
UK
Jian Yun Nie
DIRO
Universite de Montreal
Montreal, QC
Canada
LNCS Sublibrary: SL3 – Information Systems and Applications, incl. Internet/Web, and HCI
This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Preface
Since 2000, the Conference and Labs of the Evaluation Forum (CLEF) has played a
leading role in stimulating research and innovation in the domain of multimodal and
multilingual information access. Initially founded as the Cross-Language Evaluation
Forum and running in conjunction with the European Conference on Digital Libraries
(ECDL/TPDL), CLEF became a standalone event in 2010 combining a peer-reviewed
conference with a multi-track evaluation forum. The combination of the scientific
program and the track-based evaluations at the CLEF conference creates a unique
platform to explore information access from different perspectives, in any modality and
language.
The CLEF conference has a clear focus on experimental information retrieval
(IR) as seen in evaluation forums (CLEF Labs, TREC, NTCIR, FIRE, MediaEval,
RomIP, TAC, etc.) with special attention to the challenges of multimodality, multi-
linguality, and interactive search ranging from unstructured, to semi-structured and
structured data. CLEF invites submissions on significant new insights demonstrated by
the use of innovative IR evaluation tasks or in the analysis of IR test collections and
evaluation measures, as well as on concrete proposals to push the boundaries of the
Cranfield/TREC/CLEF paradigm.
CLEF 20181 was jointly organized by Avignon, Marseille and Toulon Universities
and was hosted by the University of Avignon, France, during September 10–14, 2018.
The conference format consisted of keynotes, contributed papers, lab sessions, and
poster sessions, including reports from other benchmarking initiatives from around the
world.
The following scholars were invited to give a keynote talk at CLEF 2018: Gabriella
Pasi (University of Milano-Bicocca, Italia), Nicholas Belkin (Rutgers University, NJ,
USA), and Julio Gonzalo (UNED, Spain).
CLEF 2018 received a total of 39 submissions, of which a total of 13 papers (nine
long, four short) were accepted. Each submission was reviewed by three Program
Committee (PC) members, and the program chairs oversaw the reviewing and
follow-up discussions. In all, 13 different countries are represented in the accepted
papers. Many contributions this year tackle the medical e-Health and e-Health multi-
media retrieval challenges in different ways: from medical image analysis to query
suggestion. However, there are many other topics of research in the accepted papers
such as document clustering, social biases in IR, social book search, personality pro-
filing, to cite a few. As in previous editions since 2015, CLEF 2018 continued inviting
CLEF lab organizers to nominate a “best of the labs” paper that was reviewed as a full
paper submission to the CLEF 2018 conference according to the same review criteria
and PC. Among the nine invited papers, six were accepted as long and three as short.
Finally, eight posters were also accepted. Although they are not included in the LNCS
1
http://clef2018.clef-initiative.eu/.
VI Preface
volume, posters give the opportunity to their authors to discuss their research during the
conference and are accessible through the Web pages of the conference.
The conference integrated a series of workshops presenting the results of lab-based
comparative evaluations. CLEF 2018 was the ninth year of the CLEF Conference and
the 19th year of the CLEF initiative as a forum for IR Evaluation. The labs were
selected in peer review based on their innovation potential and the quality of the
resources created. The labs represented scientific challenges based on new data sets and
real-world problems in multimodal and multilingual information access. These data
sets provide unique opportunities for scientists to explore collections, to develop
solutions for these problems, to receive feedback on the performance of their solutions,
and to discuss the issues with peers at the workshops.
In addition to these workshops, the ten benchmarking labs reported results of their
year-long activities in overview talks and lab sessions. Overview papers describing
each of these labs are provided in this volume. The full details for each lab are
contained in a separate publication, the Working Notes, which are available online2.
The ten labs running as part of CLEF 2018 were as follows:
CENTRE@CLEF 2018 -CLEF/NTCIR/TREC Reproducibility3 aims to run a joint
CLEF/NTCIR/TREC task on challenging participants: (1) to reproduce the best results
of the best/most interesting systems in previous editions of CLEF/NTCIR/TREC by
using standard open source IR systems; (2) to contribute back to the community the
additional components and resources developed to reproduce the results in order to
improve existing open source systems.
CheckThat!4 aims to foster the development of technology capable of both spotting
and verifying check-worthy claims in political debates in English and Arabic.
Dynamic Search for Complex Tasks5: The lab strives to answer one key question:
How can we evaluate, and consequently build, dynamic search algorithms? The 2018
Lab focuses on the development of an evaluation framework, where participants submit
“querying agents” that generate queries to be submitted to a static retrieval system.
Effective “querying agents” can then simulate users toward developing dynamic search
systems.
CLEFeHealth6 provides scenarios that aim to ease patients, and nurses, under-
standing and accessing of e-Health information. The goals of the lab are to develop
processing methods and resources in a multilingual setting to enrich difficult-to-
understand e-Health texts, and provide valuable documentation. The tasks are: multi-
lingual information extraction; technologically assisted reviews in empirical medicine;
and patient-centered information retrieval.
ImageCLEF7 organizes three main tasks and a pilot task: (1) a caption prediction
task that aims at predicting the caption of a figure from the biomedical literature based
2
http://ceur-ws.org/Vol-2125/.
3
http://www.centre-eval.org/clef2018/.
4
http://alt.qcri.org/clef2018-factcheck/.
5
https://ekanou.github.io/dynamicsearch/.
6
https://sites.google.com/view/clef-ehealth-2018/.
7
http://www.imageclef.org/2018.
Preface VII
only on the figure image; (2) a tuberculosis task that aims at detecting the tuberculosis
type, severity, and drug resistance from CT (computed tomography) volumes of the
lung; (3) a lifelog task (videos, images, and other sources) about daily activities
understanding and moment retrieval; and (4) a pilot task on visual question answering
where systems are tasked with answering medical questions.
LifeCLEF8 aims at boosting research on the identification of living organisms and
on the production of biodiversity data in general. Through its biodiversity
informatics-related challenges, LifeCLEF is intended to push the boundaries of the
state of the art in several research directions at the frontier of multimedia information
retrieval, machine learning, and knowledge engineering.
MC29 mainly focuses on developing processing methods and resources to mine the
social media (SM) sphere surrounding cultural events such as festivals, music, books,
movies, and museums. Following previous editions (CMC 2016 and MC2 2017), the
2018 edition focused on argumentative mining and multilingual cross SM search.
PAN10 is a networking initiative for digital text forensics, where researchers and
practitioners study technologies that analyze texts with regard to originality, authorship,
and trustworthiness. PAN offered three tasks at CLEF 2018 with new evaluation
resources consisting of large-scale corpora, performance measures, and Web services
that allow for meaningful evaluations. The main goal is to provide for sustainable and
reproducible evaluations, to get a clear view of the capabilities of state-of-the-art-
algorithms. The tasks are: author identification; author profiling; and, author obfuscation.
Early Risk Prediction on the Internet (eRisk)11 explores issues of evaluation
methodology, effectiveness metrics, and other processes related to early risk detection.
Early detection technologies can be employed in different areas, particularly those
related to health and safety. For instance, early alerts could be sent when a predator
starts interacting with a child for sexual purposes, or when a potential offender starts
publishing antisocial threats on a blog, forum, or social network. Our main goal is to
pioneer a new interdisciplinary research area that would be potentially applicable to a
wide variety of situations and to many different personal profiles. eRisk 2018 had two
campaign-style tasks: early detection of signs of depression and early detection of signs
of anorexia.
Personalized Information Retrieval at CLEF (PIR-CLEF)12 provides a framework
for the evaluation of personalized information retrieval (PIR). Current approaches to
the evaluation of PIR are user-centric, mostly based on user studies, i.e., they rely on
experiments that involve real users in a supervised environment. PIR-CLEF aims to
develop and demonstrate a methodology for the evaluation of personalized search that
enables repeatable experiments. The main aim is to enable research groups working on
PIR to both experiment with and provide feedback on the proposed PIR evaluation
methodology.
8
http://www.lifeclef.org/.
9
https://mc2.talne.eu/.
10
http://pan.webis.de/.
11
http://early.irlab.org/.
12
http://www.ir.disco.unimib.it/pir-clef2018/.
VIII Preface
Avignon is famous for its medieval architecture and its international theater festival.
The social program of CLEF 2018 set up a Science and Music Festival in medieval
downtown at Theâtre des Halles13 and surrounding gardens from Tuesday to Thursday.
Music is a very popular hobby among members of the scientific community. Evenings
were a mix of music and participatory science around PlantNet, OpenStreetMaps, and
Wikipedia. Tuesday was especially devoted to welcoming students at CLEF. On
Wednesday the focus was on IR scientific societies around the world mixing all CLEF
languages in one evening. Finally, science outreach activities were carried out on
Thursday; local musicians and students looking for a good time were invited to come
and meet the participants of the CLEF conference.
The success of CLEF 2018 would not have been possible without the huge effort of
several people and organizations, including the CLEF Association14, the PC, the Lab
Organizing Committee, the local organization committee in Avignon, the reviewers,
and the many students and volunteers who contributed.
13
http://www.theatredeshalles.com/.
14
http://www.clef-initiative.eu/association.
Organization
CLEF 2018, Conference and Labs of the Evaluation Forum – Experimental IR meets
Multilinguality, Multimodality, and Interaction, was hosted by the University of
Avignon and jointly co-organized by Avignon, Marseille and Toulon Universities,
France.
General Chairs
Patrice Bellot Aix-Marseille Université - CNRS LSIS, France
Chiraz Trabelsi University of Tunis El Manar, Tunisia
Program Chairs
Josiane Mothe SIG, IRIT, France
Fionn Murtagh University of Huddersfield, UK
Lab Chairs
Jian Yun Nie DIRO, Université de Montréal, Canada
Laure Soulier LIP6, UPMC, France
Proceedings Chairs
Linda Cappellato University of Padua, Italy
Nicola Ferro University of Padua, Italy
Publicity Chair
Adrian Chifu Aix-Marseille Université - CNRS LSIS, France
Sponsoring Chair
Malek Hajjem UAPV, France
X Organization
Local Organization
Eric SanJuan (Chair) LIA, UAPV, France
Tania Jimenez (Co-chair) LIA, UAPV, France
Sebastien Fournier Aix-Marseille Université - CNRS LIS, France
Hervé Glotin Université de Toulon - CNRS LIS, France
Vincent Labatut LIA, UAPV, France
Elisabeth Murisasco Université de Toulon - CNRS LIS, France
Magalie Ochs Aix-Marseille Université - CNRS LIS, France
Juan-Manuel LIA, UAPV, France
Torres-Moreno
Organization XI
Sponsors
Members
Khalid Choukri Evaluations and Language resources Distribution
Agency (ELDA), France
Paul Clough University of Sheffield, UK
Norbert Fuhr University of Duisburg-Essen, Germany
Lorraine Goeuriot Université Grenoble Alpes, France
Julio Gonzalo National Distance Education University (UNED),
Spain
Donna Harman National Institute for Standards and Technology
(NIST), USA
Djoerd Hiemstra University of Twente, The Netherlands
Evangelos Kanoulas University of Amsterdam, The Netherlands
Birger Larsen University of Aalborg, Denmark
Séamus Lawless Trinity College Dublin, Ireland
Mihai Lupu Vienna University of Technology, Austria
Josiane Mothe IRIT, Université de Toulouse, France
Henning Müller University of Applied Sciences Western Switzerland
(HES-SO), Switzerland
Maarten de Rijke University of Amsterdam UvA, The Netherlands
Giuseppe Santucci Sapienza University of Rome, Italy
Jacques Savoy University of Neuchatel, Switzerland
Christa Womser-Hacker University of Hildesheim, Germany
CLEF Steering Committee XIII
Past Members
Jaana Kekäläinen University of Tampere, Finland
Carol Peters ISTI, National Council of Research (CNR), Italy
(Steering Committee Chair 2000–2009)
Emanuele Pianta Centre for the Evaluation of Language
and Communication Technologies (CELCT), Italy
Alan Smeaton Dublin City University, Ireland
Contents
Full Papers
Short Papers
1 Introduction
The information contained in an image and the methods employed to extract
it largely differ depending on its modality, making the latter a crucial aspect
of medical image analysis and retrieval. An image type classification is, there-
fore, a useful preliminary filtering step prior to further analysis [2,16]. Besides
this, the modality is a relevant information to be determined for medical image
or document retrieval, allowing clinicians to filter their search by a particular
modality, often specific to a pathology or organ of interest. Various modality
classification tasks, among others, have been released through the ImageCLEF
challenges [7,10]. We focus this work on the 2013 and 2016 ImageCLEF modality
classification tasks, as they offer multimodal text and image data. The database
is publicly available and the results are fully reproducible as a consequence. The
database also originates from the PubMed Central database (it is a small sub-
set of PubMed Central), allowing us to classify this large database for further
processing and analysis. Much of the medical knowledge is stored in the medical
literature, for example in the form of images and text, although the image type
information is not available. Making this content accessible for research can help
in many other tasks, such as retrieval or classification.
Multimodal analysis is commonly used to extract and fuse information from
multiple modalities [11,24]. In this work, images and captions contain comple-
mentary information fused to boost the classification accuracy. Many methods
c Springer Nature Switzerland AG 2018
P. Bellot et al. (Eds.): CLEF 2018, LNCS 11018, pp. 3–14, 2018.
https://doi.org/10.1007/978-3-319-98932-7_1
4 V. Andrearczyk and H. Müller
have been used to extract high-level features from text and images independently
and to fuse them. Convolutional Neural Networks (CNNs) have obtained state
of the art performance in most computer vision [5,9] and biomedical image anal-
ysis [17] tasks. It is also well suited for text analysis [12]. This paper introduces
several late fusion methods to combine powerful visual and textual CNNs.
2 Related Work
Multimodal textual and visual analysis has been widely studied for applications
including annotation and captioning [11], image generation from text [24], text
and image feature fusion for retrieval and classification [7]. A total of 51 runs
from eight groups were presented in [7] for the ImageCLEF 2013 modality classi-
fication challenge. The best results (81.7% classification accuracy) were obtained
by visual and textual fusion from the IBM Multimedia Analytics group [1] (see
Table 1). A set of color and texture, local and global descriptors (including a color
histogram, moments, wavelets, Local Binary Patterns (LPB) and Scale-Invariant
Feature Transform (SIFT)) was extracted as visual descriptors and fused with
multiple textual descriptors. The best results were obtained using a maximum
late fusion with a classifier built on top of modality tailored keywords (with a
hand-selected vocabulary that likely improved the performance) and a two-level
Support Vector Machine (SVM) classification. The methods developed by other
teams reported in [7] include various types of similar hand-crafted visual and
textual descriptors combined by multiple fusion methods.
In [4], the authors build upon [1] to develop a more complex system. An
ensemble of SVM models is trained on top of similar visual features, while the
text is analyzed by scoring based on the detection of manually-selected patterns
from the captions and from sentences in the body of the article. A weighted score
average trained on a subset of the training data was used for fusing the visual
and textual information. The best current system reached an accuracy of 83.5%
on ImageCLEF 2013 modality classification.
Another set of hand-crafted visual and textual features are combined in [19].
The visual features include local and global texture and color features, while
Bag-of-Words (BoW) features are used to analyze the captions. Multiple CNNs
are combined in [23] by late fusion (average, maximum, majority and median),
yet the resulting accuracy is lower than the shallow hand-crafted methods in [7].
More recently in [16,22], pre-trained deep visual CNNs are finetuned and their
outputs are combined in an ensemble classifier with basic voting fusion meth-
ods. Besides this, deep pre-trained CNNs obtained better performance than shal-
lower ones, motivating the use of very deep pre-trained networks even on these
small datasets. A major drawback of combining multiple CNNs is the increase
of computational complexity and redundancy of features to obtain only a lim-
ited accuracy improvement (less than 1% in [22]). A multimodal approach based
on ensemble learning of various textual and visual features was proposed in
[15], obtaining the state of the art results on the ImageCLEF 2016 subfigure
modality classification task [8]. BoW textual features extracted from the arti-
cle’s text and captions are combined with visual features including hand-crafted
Deep Multimodal Classification of Image Types 5
shallow texture and color features, Bag-of-Visual-Words (BoVW) and deep fea-
tures (ResNet152 [5]). The best results were obtained with an ensemble of SVMs
trained on all the features (see Table 3).
3 Methods
3.1 ImageCLEF 2013 Modality Dataset
The goal of this task is to classify the images into medical modalities and
other images types. Three main categories, namely compound figures, diagnostic
images and generic illustrations are divided into 31 sub-categories [7,18]. The
modality hierarchy and more details on the dataset can be found in [7]. A total
of 2879 training and 2570 test images are provided. The classes are highly imbal-
anced, reflecting the distribution of the images in the data (PubMed Central1 )
containing a large proportion of compound figures.
Fig. 1. Overview of the proposed deep learning visual and textual fusion method.
The decision-level fusion combines the visual and textual predictions. We first
train the visual and textual networks independently, then combine the class prob-
abilities, i.e. outputs of the softmax layers. Simple fusions are used including (a)
a weighted sum, (b) a maximum probability decision and (c) a product of proba-
bilities (elementwise product of probability vectors). Equation 1 summarizes the
class prediction of these three fusion methods.
where csum , cmax , cprod are the class predictions from (a), (b) and (c), yv and yt
are the probability vectors of the visual and textual networks respectively. The
weight α ∈ [0, 1] is used to balance the importance of the visual and textual parts
2
https://dumps.wikimedia.org.
8 V. Andrearczyk and H. Müller
4 Experimental Results
4.1 Network Setups
The networks are trained with an Adam [13] optimizer. The textual, visual and
fusion MLP networks are trained for N = 100, N = 25 and N = 50 epochs
respectively. The initial learning rate is set to 10−4 for finetuning the visual
network and 10−3 for the textual network and MLP from scratch, average decays
β1 and β2 are 0.9 and 0.999 respectively, the learning rate decay is 0.1
N and the
batch size 32. Due to the high class imbalance in the training set, class weights
are used during training for weighting the loss function as: wi = nmax /ni , where
nmax and ni are the number of training samples of the most represented class
and of class i respectively. The most represented class is the one with most
training samples, i.e. “compound figures” in ImageCLEF 2013 and “GFIG” in
ImageCLEF 2016. For the visual network, class weights are not needed when
artificial data augmentation is used.
Table 1. Comparison of our methods with the best runs in ImageCLEF 2013.
ImageCLEF 2013 training data. For the same reason, the MLPs are also trained
without extra training data in order to maintain pairs of visual and textual
inputs. However, the visual network is first trained with artificial data augmen-
tation and extra training data before being fused with the textual network.
The best fusion results are obtained with the feature-level MLP (93.7%).
The previously best results (IBM [4]) on ImageCLEF 2013 were obtained with-
out using extra data, yet our approach without extra data also outperforms
them (91.9% vs. 83.5%, not reported in Table 1). The weighted loss described in
Sect. 4.1 considerably improves the performance of our approach since the best
performance obtained without weighted loss is 92.7% (not reported in the table)
vs. 93.7% with. The confusion matrix of the best results (Feature-level MLP w/
extra training) is shown in Fig. 2.
The most relevant classes are the diagnostic images as they offer more poten-
tial in clinical applications such as retrieval. The confusion matrices for the
three main categories (compound, diagnostic and generic illustrations) are illus-
trated in Fig. 3. It shows that our approach performs an excellent discrimination
between diagnostic (e.g. MRI, CT, histopathology) and other images with lower
relevance (e.g. compound figures, diagrams and maps), which was of critical
importance for the development of the datasets in [6].
In order to evaluate the complementarity of the visual and textual infor-
mation, we measured the overlap of correct classification. With the best MLP
method previously described, the percentage of images correctly classified by
both the visual and textual networks is 64.3%. 22.5% of the test set is cor-
rectly classified by the visual network but incorrectly classified by the textual
10 V. Andrearczyk and H. Müller
Fig. 2. Normalized confusion matrix (%) of the feature-level fusion method on Image-
CLEF 2013. COMP stands for compound figures. Labels starting with D are diagnostic
modalities, those with G are generic illustrations. The complete list of labels can be
found in [7].
Fig. 3. Normalized confusion matrices for the three main categories in ImageCLEF
2013: compound, diagnostic and generic illustrations.
Deep Multimodal Classification of Image Types 11
one and, vice-versa, 7.6% is correctly classified using the caption but incorrectly
classified using visual information. These results suggest, as confirmed by the
fusion results in Table 1, that the visual and textual analyses offer some degree
of complementarity to boost the final classification accuracy.
The accuracy obtained with multiple values of α in Eq. 1 is illustrated in
Fig. 4. The best results with this weighted sum fusion are obtained with a con-
tribution of the visual analysis slightly larger than the textual one (α = 0.51),
although a gradually reducing, yet neat, improvement from the single modality
results is obtained with α values in the range [0.51, 0.99].
Fig. 4. Accuracy of the sum fusion method for various weights α on ImageCLEF 2013.
The networks are implemented in Keras with TensorFlow backend and writ-
ten in Python. The computational training and test times are reported in Table 2
using a Titan Xp GPU.
Table 3. Comparison of our methods with the best runs in ImageCLEF 2016.
visual analysis as shown by the fusion results. BCSG [15] makes use of the text
and caption for the textual analysis. A class distribution of the test set is also
learned based on the ImageCLEF 2015 dataset. We decided not to implement
such approaches for a better generalization to unknown data without prior on
the class distribution. Despite this simplified setup, our textual classifier is on a
par with [15] (72.2%). Extra training data are also used in [15].
The best fusion method is again obtained with a feature-level MLP (89.0%).
Slightly lower accuracy is obtained with the basic product fusion and decision-
level MLP (88.7% and 88.5%).
En ikke liden Vægt lagde jeg paa, at det System, der i de nærmeste
Aar før min Indtrædelse i Ministeriet ved den almindelige Skolefonds
forøgede Indtægter havde udviklet sig, nemlig i større Omfang end
før at bøde paa den meget utilstrækkelige Løn for den yngste
Lærerklasse, Adjunkterne, ved Gratialer efter vilkaarligt Skjøn,
afløstes af en fast Regel for Oprykning igjennem forskjellige
Lønningsklasser efter Anciennitet, hvilket skete ved kongelig
Resolution af 10de Marts 1849. Denne Resolution blev en Forløber
for den nogle Aar efter tilveiebragte Lønningslov for hele
Lærerpersonalet ved de lærde Skoler af 28de Marts 1855, forbunden
med Avancementsbestemmelser, til hvilken Lovs Tilveiebringelse jeg
som Undervisningsinspektør medvirkede og gjorde Forslag, omend
den endelige Lov fjernede sig endel fra Forslaget.
Det ligger saa klart for Dagen, at endel af de saaledes ved kongelige
Resolutioner efter Ministeriets Indstilling afgjorte vigtige Sager,
navnlig om Sorø Akademis og Slagelse og Rønne Skolers
Nedlæggelse og om de lærde Skolers Undervisningsplan, under
almindelige Forhold vilde have krævet Rigsdagens Medvirkning og
Afgjørelse ved Lov, at det ikke paa nogen Maade kunde undgaae min
eller Statsraadets Opmærksomhed, i hvilket naturligvis Sagerne
forhandledes, førend de forelagdes Kongen. Det var saaledes ganske
i sin Orden, at Grundtvig i Rigsdagens 1ste Session 1850 den 27de
Juni stillede en Forespørgsel til mig om den Berettigelse, hvormed
Ministeriet havde handlet. Naar jeg imidlertid havde paataget mig
Ansvaret for den valgte Afgjørelsesmaade og derfor vandt
Rigsdagens stiltiende Anerkjendelse, idet ingen videre Anke eller
Forslag fremkom og selv Grundtvig beroligede sig ved de af mig
givne Erklæringer, hvorved Rigsdagens Kompetence under
almindelige Forhold ganske indrømmedes, da laae Grunden hertil i
den hele anomale Stilling, hvori disse Anliggender vare komne ved,
hvad der var sket og foretaget før Regjeringsforandringen og
endvidere før Grundlovens Vedtagelse, og i den stærke
Nødvendighed af at komme til en Afslutning af de svævende
Spørgsmaal, medens Rigsdagen, da den først i 1850
sammentraadte, selv uden Hensyn til Vanskeligheden ved samtlige
disse Gjenstandes Behandling i den Skikkelse, i hvilken de stykkevis
og betinget af mange Slags Forudsætninger forelaae, var fuldt
optagen af Lovudkast om materielle Sager, der ikke taalte
Opsættelse, samt endelig deri, at den i de enkelte Sager trufne
Afgjørelse i det Hele havde den offentlige Mening for sig og derfor
ogsaa grundlagde en Tilstand, der længe holdt sig med ringe
Modifikationer.
Hvad Kirkevæsenet angik, var det fra det første Øieblik af, da jeg fik
dermed at gjøre, min Overbevisning, at Staten og dens Tjener:
Ministeren skulde yde Folkekirken den Understøttelse og det Værn,
der i Grundloven tilsagdes den, men saa lidet som muligt og
allerhelst slet ikke befatte sig med dens indre Anliggender, hvilke
Kirken imidlertid da maatte sættes istand til selv at varetage. Dette
skulde skee ved den kirkelige Forfatningslov, som Grundlovens § 80
stillede i Udsigt. Men, medens det stod mig fast, at Staten kun
kunde staae i et Understøttelsesforhold til en virkelig ordnet og paa
et bestemt, omend ikke i alle Enkeltheder fastslaaet og uforanderligt
Læregrundlag bygget Kirke, og jeg derfor var en ligesaa bestemt
Modstander af den Grundtvigske Præstefrihed, der lod
Kirkesamfundet forsvinde, som af Retninger, der frygtede alt Kirkeligt
og Præsteligt, kunde jeg ikke være blind for den Vanskelighed, der
laae deri, at den hidtil i sine Anliggenders Styrelse aldeles
uselvstændige Kirke skulde modtage sin Forfatning ved en borgerlig
Lov. Der maatte altsaa efter min Mening arbeides hen til, at denne
Forfatningslovs Indhold saavidt muligt fremgik af Kirkens egen Trang
og Overbevisning og kun fik sin ved Tilsagnet om Understøttelse
betingede Sanktion af Statslovgivningen. Jeg maatte derfor uden
Hensyn til, at den ved Forhandlingen og Vedtagelsen af
Grundlovsparagrafen i Forslag bragte udtrykkelige Bestemmelse om
et forberedende Kirkemøde var udgaaet, fastholde den af min
Formand i Ministeriet, Monrad, opstillede Tanke om et saadant
Kirkemøde som nødvendigt til Forberedelsen af en Kirkeforfatning og
forlangte i det første Udkast til en Finantslov, der forelagdes (i den
første Rigsdagssession 1850), en Sum af 10,000 Rdl. til Afholdelse af
et saadant Møde. Efterat jeg baade ved første og anden Behandling
havde udtalt mig i den her antydede Retning, især ligeoverfor
Grundtvig og Tscherning, mod hvilken sidste jeg gjorde gjældende,
at den danske Kirke hidtil i sin Styrelse slet ikke var, hvad han kaldte
den, en Menighedskirke, bevilgedes der 5000 Rdl. Jeg modtog denne
Bevilling med en dyb Tvivl om, hvorvidt jeg enten i det Aar eller i det
næste vilde være istand til at benytte den og til alvorlig at fremme
Sagen, og derfor med stor Bekymring. Jeg saae ikke blot, hvilke
stærke og skærende Modsætninger der i det Øieblik fandt Sted
imellem de fremragende Ordførere i den danske protestantiske Kirke,
saasom J. P. Mynster, Clausen og Grundtvig, men jeg nærede derhos
Tvivl om, at der da overhovedet var et saadant Menighedsliv og en
saadan hengiven Tilslutning til Kirken tilstede hos Folket, at
Bygningen lod sig opføre paa denne Grundvold, og, hvad der for mig
personlig var det Tungeste og maaske bestyrkede og skærpede hin
Tvivl om Andre: jeg følte, at jeg selv, skjøndt paa min Maade inderlig
religiøs stemt, dog ikke stod saaledes paa hele Aabenbaringstroens
Standpunkt, at jeg tillidsfuld, med Fasthed og uskrømtet Alvor kunde
deltage i, endsige lede Forhandlingen om en Forfatning for den
evangelisk-lutherske Kirke i Danmark; jeg vilde ikke nedrive eller
svække, men skyldte at befæste og opbygge; jeg frygtede for at
vakle eller at hykle; jeg led derved stærkt og lod derfor Sagen, som
heller ingen Anden dengang ytrede ret Mod eller Lyst til at tage fat
paa, hvile. — Men udenfor Hovedspørgsmaalet om Folkekirkens egen
Forfatning paatrængte der sig efter Grundlovens Vedtagelse
forskjellige særlige Spørgsmaal, der angik Berøringen imellem
Folkekirken og Personer, der befandt sig udenfor den, og den disse
Personer nu tilkommende Uafhængighed af Folkekirkens Forskrifter
og Former. Efterat jeg allerede i Rigsdagens første Session 1850 i
Anledning af et Andragende fra den mosaiske Menighed i
Kjøbenhavn og en derved fremkaldt Forespørgsel i Folkethinget
havde forelagt et Udkast til Lov om den religiøse Opdragelse af Børn,
der vare fødte i Ægteskab mellem Personer af forskjellig
Trosbekjendelse, og, da Udkastet paa Grund af dets Fremkomst mod
Sessionens Slutning ikke var kommet til Behandling, havde forelagt
det paany i anden Session samme Aar, fremkom der i denne fra et
Medlem af Folkethinget: Overretsassessor Spandet et Forslag til "Lov
om Trosfrihed", der meget mindre handlede om, hvad Titlen angav,
end om alle de Forhold, i hvilke hidtil kirkelige Handlinger og Former
vare satte i Forbindelse med borgerlige Rettigheder, og som gik ud
paa overalt at ophæve denne Forbindelse paa en Maade, der baade
unødvendig og forstyrrende greb ind i Folkekirkens Opfattelse og
Vedtægt og lidet tilfredsstillede den borgerlige Ordens og
Retssikkerheds Krav. Idet jeg nu under første Behandling af dette
Lovforslag paaviste og bekæmpede denne dobbelte Mangel
ligeoverfor den selv meget alvorlig religiøse Forslagsstiller og hans to
fra høist forskjellige Standpunkter udgaaende Kampfæller: Grundtvig
og Tscherning, erkjendte og udtalte jeg ligefremt og aabent, at der
var Punkter og Tilfælde, for hvilke der nødvendig maatte skaffes
nyere og friere Former og Bestemmelser, navnlig med Hensyn til
Ægteskabs Indgaaelse imellem Medlemmer af forskjellige
Trossamfund eller Personer udenfor ethvert anerkjendt Trossamfund
og med Hensyn til Opdragelsen af Børn fødte i slige Ægteskaber, og,
da et Udvalg var nedsat efter første Behandling, meddelte jeg dette
et Udkast til Lov om den saaledes begrændsede Gjenstand, hvori da
ogsaa Indholdet af det af mig forinden forelagte Forslag om
Børneopdragelsen indgik, hvorfor dette bortfaldt som særskilt. Det
nævnte Udkast tiltraadtes i alt Væsentligt af Udvalget og førtes
derpaa til endelig Vedtagelse gjennem en Forhandling, som jeg
saameget mere maatte lede baade med Kraft og Forsigtighed, som
den i Udkastet ligefrem foreskrevne Form af rent borgerligt
Ægteskab fandt ringe Sympathi hos Justitsministeren og aabenbart
bekæmpedes af min Kollega Clausen, idet han sluttede sig til et
Forslag, der søgte at bevare et tomt Skin af præstelig Medvirkning.
Medens jeg i Folkethinget havde maattet kæmpe for Lovens
Indskrænkning til det Nødvendige og for Skaansel af det
Bestaaende, hvor ingen Forandring var fuldt nødvendig, maatte jeg
omvendt i Landsthinget føre en meget skarp Kamp imod Ørsted og
Blechingberg, der stædigen afviste enhver Reform. Erfaringen har
vist, at den aabnede Udvei, som jeg siden administrativt maatte
supplere ved en af mig selv affattet Formular for den borgerlige
Vielse, da var aldeles tilstrækkelig for Formaalet og ikke har beredet
nogensomhelst Vanskelighed eller Uorden. Den heftige Bevægelse,
som det Spandetske Lovforslag havde fremkaldt i og udenfor
Geistligheden, lagde sig ogsaa snart[66]. Et af Pastor (senere
Biskop) Kirkegaard i Landsthinget i første Session 1850 indbragt
Forslag om Menighedernes Ret til Indførelse af nye Psalmebøger
bortfaldt ved anden Behandling i Henhold til en af mig afgiven
Erklæring om, hvilken Myndighed jeg troede, at Administrationen i
denne Retning havde, og om hvorledes jeg for min Del vilde benytte
denne Ret. Hele Forhandlingen var ført i velvillige Former[67].
Faa Dage efter Slaget ved Idsted undertegnedes den 2den August i
London den Protokol, i hvilken Stormagterne, af hvilke dog Østerrig
først lidt senere tiltraadte med et vist Forbehold, erklærede
Opretholdelsen af det danske Monarki i dets hidtilværende
Udstrækning for at være vigtig for den europæiske Ligevægt, et
Resultat, der væsentlig skyldtes Rusland og England og svarede til
deres Politik, og hvorved Kongen af Danmarks dynastiske Krav
tilfredsstilledes og seirede over Insurrektionen, forsaavidt denne gik
ud paa en fuldkommen Løsrivning. Men tilbage stod Hovedopgaven,
nemlig indenfor den saaledes givne Ramme at ordne Forholdet
imellem Monarkiets Dele paa en baade i sig selv billig Maade og
saaledes, at Rammen ikke udsattes for paany at sprænges, eller at,
hvis en saadan Sprængning indtraadte (hvad jeg for min Del ganske
vist ventede, men vel vogtede mig for at antyde eller udtale som
Forudsætning), det egentlig danske Riges Interesser muligst
sikredes. Til Opgavens Løsning gjaldt det at benytte den vundne
Stilling: den seirrig hævdede Besiddelse af saagodtsom hele Slesvig.
Derhen vendte sig da, medens den danske Bestyrelse af Slesvig
foreløbig ordnedes og de sidste Kampe kæmpedes, min hele Tanke,
og deraf fremgik da mit med den inderligste Overbevisning
nedskrevne Andragende til Statsraadet af 13de Oktober 1850, der nu
er trykt i dansk historisk Tidsskrifts femte Rækkes 4de Bind (for
1883), hvori jeg anbefalede, at vi snarest muligt selv skulde
fremtræde med Forslag om det tydske Sydslesvigs legislative og
administrative Forbindelse med Holsten og det øvrige Slesvigs med
Kongeriget som Grundlag for Underhandlingerne om Monarkiets
Ordning. Det lykkedes mig ikke at vinde, jeg vil ikke sige
Samstemning med, men end ikke blot alvorlig Opmærksomhed for
Forslaget; alle mine Kolleger skrækkedes tilbage fra en Løsning, der
havde saavel den rigtignok de bittreste Erfaringer indeholdende
Tradition som Folkemeningen imod sig; selv Clausen forlod mig, nu
stolende paa at fastholde hele Slesvig. — Med Grev Sponnecks
Sendelse over Berlin til Wien indlededes istedet
Notabelforhandlingerne i Flensborg. — Det havde maaske været
rigtigt, hvis jeg, da den Hovedtanke, der fra Begyndelsen af havde
været ledende for mig med Hensyn til Statens Sammensætning og
Begrændsning, og til hvilken jeg ene havde nogen Tillid, saaledes
tilbagetrængtes og opgaves, paa det Tidspunkt var traadt ud af
Ministeriet. Jeg lod mig imidlertid bevæge til at blive dels af et,
omend ikke stærkt, Haab om at se Tanken fremkomme paany efter
resultatløse Forsøg i andre Retninger, dels af Trangen til at fremme
Bearbeidelsen af en af Hovedgjenstandene for det mig betroede
specielle Ministerium: Almueskolevæsenet i Lovsform saavidt, at der
ingen Tvivl blev om min Iver derfor og Redeligheden i de af mig i saa
Henseende givne Løfter[77].
Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.
textbookfull.com